Dr Tony Weston was pleased to accept the British Association for Counselling and Psychotherapy (BACP) 2011 award for Outstanding Research Project from Ruby Wax. This was for his research on effective therapy for depression, anxiety and distress e.g. following bereavement, relationship problems, addiction, abuse etc. Participants included 321 clients – including 137 of Dr Weston’s own clients – and 27 fellow therapists.
You can read a summary of Dr Weston’s award winning research, below, followed by the full BACP submission, plus references.
You can contact Dr Tony Weston on 01223-894896 (phone is answered by a receptionist 24/7) or email him at email@example.com to make an initial appointment.
BACP Research Award – Summary of The clinical effectiveness of the person-centred psychotherapies: The impact of the therapeutic relationship.
1. Objective/innovation: 1) To measure clinical effectiveness of person-centred therapy (PCT) in the author’s private practice and at a University Counselling Service; 2) To assess the therapeutic relationship as a predictor of outcome; 3) To address criticisms of naturalistic research (cf. randomised controlled trials).
2. Methods: This was primarily an uncontrolled naturalistic experiment. Outcome measures were completed at first therapy session and subsequently (outcome study), with a measure of the relationship (predictor study) plus other data to address criticisms of naturalistic research (rival hypothesis testing).
3. Findings: PCT was an effective intervention for clients who completed subsequent questionnaires with depression (ES(d) =1.48, n=111), anxiety (ES(d) = 1.15, n=91) and distress (ES(d) = 1.80, n=79). Outcomes were comparable with other schools of therapy (e.g. CBT). Criticisms of naturalistic experiments were addressed, supporting the validity of these findings. The therapeutic relationship had an effect on depression outcome, r =.22, p
4. Impact: 1) This research showed PCT can have outcomes comparable with other therapies; 2) showed how criticisms of naturalistic research can be addressed; 3) was the first to show the therapeutic relationship as defined in PCT predicted outcome.
5. Dissemination: This research was written up for the author’s PhD (confirmed June 2011) and other means of disseminating are underway.
back to top
BACP Research Award – The clinical effectiveness of the person-centred psychotherapies: The impact of the therapeutic relationship.
Person-centred therapy (PCT) is in crisis. In the UK the National Institute for Health and Clinical Excellence (NICE) has made recommendations on what treatments should be offered for what diagnoses, including depression and anxiety (NICE, 2002a), schizophrenia (2002b), panic disorder and generalised anxiety disorder (2004a), depression (2004b), eating disorders (2004c), self harm (2004d), PTSD (2005a), depression in children and young people (2005b), OCD (2005c), borderline personality disorder (2008) and revised depression guidelines (2009a). Whilst cognitive-behavioural therapies (CBT) were recommended for all diagnoses, PCT was only recommended for mild to moderate depression (2004b) and this recommendation was changed so that patients should be given warnings about the absence of an evidence base for counselling for depression (2009a). PCT has fared poorly in American Psychological Association (APA) reviews too e.g. Chambless and Hollon 1998.
Lots of researchers have suggested different schools of therapies have approximately equivalent outcomes, e.g. Rosenzweig (1936), Luborsky, Singer and Luborsky (1975), Smith and Glass (1977), Stiles, Shapiro and Elliott (1986), Elliott, Greenberg and Lietaer (2004), Stiles, Barkham, Mellor-Clark and Connell (2007), Minami, Wampold, Serlin and Brown (2007). Although some do not accept this ‘equivalence’ idea e.g. Siev and Chambless (2007).
NICE and APA favour randomised controlled trials (RCTs) for outcomes research. Although some authors argue in favour of naturalistic research e.g. Westen, Novotny and Thompson-Brenner (2004), Stiles et al. (2006), Stiles et al. (2007). RCTs favour therapies that can afford the relatively high costs of RCTs.
As a recently qualified (2004) person-centred therapist the author wanted to contribute to the outcomes research for PCT – Research Question 1: What was the clinical effectiveness of PCT (outcomes study) and how does this compare with other therapies (benchmarking study).
Being mainly self-funded the author was unable to conduct an RCT and needed to use naturalistic methods. In response to Stiles et al. (2006, 2007), Clark, Fairburn and Wessely (2007) made a number of criticisms of naturalistic research, i.e. 1) missing cases could have been treatment failures, 2) findings could have been restricted to the ‘easiest’ clients, 3) findings could have been restricted to the ‘best outcomes’, 4) outcomes could have simply been regression to the mean, 5) outcomes could have been simply ‘natural recovery’, and 6) outcomes could have been attributable to concurrently administered medications. The findings therefore needed to address these criticisms of naturalistic research – Research Question 2: What was the evidence that it was the therapy that led to the outcomes, compared with a rival hypothesis (addressing the criticisms of naturalistic research).
The literature suggested it was insufficient simply to show the existence of a causal relationship between therapy and client change without a plausible explanation to link cause and effect (e.g. Elliott, 2010). In PCT the proposed causal explanation was that the therapeutic relationship (congruent empathy and unconditional positive regard as defined by Rogers) caused client change in PCT (Rogers, 1959) and in therapy more generally (1957). Elliott (2010) drew together seven criteria in the literature required to determine a link between cause and effect, three of which can be addressed by an appropriately conducted process-outcome correlation. Since Rogers (1957) there have been many attempts to find a process-outcome correlation for the therapeutic relationship, e.g Barrett-Lennard (1962), Lesser (1961), Rogers (1967). Reviewing research to date, Gurman (1977) pointed out that only one study (Lesser) had used an appropriate methodology (partial correlation) and this had failed to support a link between therapeutic relationship and outcome. (In fact Lesser used interim rather than ‘end’ scores for 50% of his sample to fit in with the deadline for his PhD).
Reviews by Lambert, DeJulio and Stein (1978) and Watson (1984) pointed to further shortcomings in the therapeutic relationship-outcome research to date. Stiles et al. (1986), Stiles (1988) and Stiles and Shapiro (1994) questioned the validity of therapeutic relationship-outcome research, eventually concluding that there was merit in this form of inquiry e.g. Stiles, Agnew-Davies, Hardy, Barkham and Shapiro (1998).
Burns and Nolen-Hoeksema (1992) found more empathy led to better outcome in CBT (r =.26) and Martin, Garske and Davis (2000) performed a meta-analysis of 79 studies concluding the therapeutic alliance (Horvath & Greenberg, 1989) was moderately correlated with outcome (r=.22). Zuroff and Blatt (2006) reanalysed data from the National Institute for Mental Health Treatment of Depression Collaborative Research Programme (Elkin et al., 1989) and reported that a Rogerian measure of therapeutic relationship (the Barrett-Lennard Relationship Inventory, BLRI, Barrett-Lennard, 1962) was linked to outcome (better relationship, better outcome). In fact a composite score derived from summing the empathy, regard and congruence components of the relationship inventory were used (D. C. Zuroff, personal communication, 29th January 2008), and the unconditionality component was not used in the analysis.
To date there was no evidence in the literature of a link between the therapeutic relationship as defined by Rogers (congruent empathy and unconditional positive regard) and outcome. This evidence was needed to contribute to a plausible explanation linking cause (good relationship) and effect (good outcome) to establish PCT as an evidence-based therapy. A well controlled naturalistic study could contribute to causal inference by 1) documenting temporal precedence, the good relationship preceded good outcome and vice-versa, 2) showing covariation between quality of relationship and quality of outcome, and 3) considering alternate causes (rival hypotheses) – Research question 3 – What was the impact of the therapeutic relationship (process-outcome correlation).
– There was a need to research the clinical effectiveness of PCT.
– Given this was naturalistic research, perhaps making the case for an RCT, there was a need to consider rival hypotheses about the cause of outcome.
– There was a need to research the link between therapeutic relationship and outcome as this is the hypothesised causal relationship for which there was no correlation evidence in the literature.
This research was important in contributing to the re-establishment of PCT as a credible form of treatment for depression, anxiety and general distress; a large proportion of the UK ‘therapy work force’ are not trained in CBT (A. Couchman, personal communication, 3rd January 2008). Furthermore it was hoped that this research would serve to inspire others to conduct quantitative research in PCT and to inspire others to consider PCT as an approach that contributes to the current pan-theoretical interest in relationship/alliance-outcome links.
In addition to the relevant literature, some of which was referred to above, methodological literature was also consulted e.g. Cook and Campbell (1979), Snow and Wiley (1991), Campbell and Russo (1999), Bockman (2000), Mcleod (2003), etc. Furthermore BACP and APA guidelines were consulted. Some researchers favour naturalistic research over RCT e.g. Des Jarlais, Lyles, Crepaz and TREND Group (2004). The TREND guidelines were developed to complement the CONSORT guidelines for RCTs. This research was conducted and reported to comply with the TREND guidelines.
As described, this was primarily an uncontrolled naturalistic experiment, rather than an RCT. Outcome measures were completed at first therapy session and subsequently (outcome study), with a measure of the relationship (process-outcome correlation study) plus other data to address criticisms of naturalistic research and to consider other plausible causes of outcome (rival hypothesis testing).
2.1 Participants – This was intended to be a naturalistic study of bona fide clients requesting therapy from therapists so there were no inclusion or exclusion criteria. All therapists had received some formal training as a person-centred therapist (some were trainees). Whilst 27 therapists signed up to the research, 18 had clients complete subsequent outcome questionnaires, 12 were studying for a person-centred diploma, 6 were post-qualification with an average of 7 years post qualification experience (range 2-20 years). 321 clients, 137 at the private practice (PP) and 184 at the UEA Counselling Service (UCS) consented to take part. The ways of working at the PP and UCS were slightly different, e.g. PP clients only re-completed outcome measures if their start measures had clinical scores, and UCS clients had an exploratory session before their first ‘proper’ session. Of the 137 PP, 100 completed at least one subsequent outcome measure and 37 had no subsequent measure. For these 37, 9 started with non-clinical scores and received no subsequent measure, 28 started with at least one clinical score. Of these 28, 23 did only one session and the remaining 5 did two sessions (1) or three sessions (4). Of the 184 UCS, 38 did not return after their exploratory and 146 entered therapy. Of the 146, 59 did not complete first session paperwork (48 were allocated to a therapist not taking part in the research, 11 reason unknown). So 87 were ‘in the research’ (146 minus 59) of which 40 clients did not complete a subsequent outcome measure, the 47 who did included at least 11 whose therapy came to a premature end because they or their therapist were leaving UEA.
2.2 Measures – Outcome measures were the Beck Depression Inventory Second Edition (BDI-II, Beck, Steer & Brown 1996), Beck Anxiety Inventory (BAI, Beck & Steer, 1993) and the Clinical Outcomes in Routine Evaluation Outcome Measure (CORE-OM, Barkham, Mellor-Clark, Connell & Cahill, 2006). Reliable change criteria were calculated (Jacobson & Truax, 1991) and these were found to be 7.8, 14.1 and 5.3 respectively. To measure the therapeutic relationship clients completed the 40 item BLRI (Barrett-Lennard, 1962) at mean session 4.2 (SD 2.4, median session 5). These are highly regarded outcome measures in regular research use; the BLRI is the definitive PCT process measure and used by Rogers (1967).
2.3 Protocol – the precise methodology evolved during the course of the study in response to findings, with five distinct phases described together with the numbers of clients starting the phase and the intended questionnaire completion: (1) 12 clients at PP completed CORE-OM, (2) 20 clients at PP completed CORE-OM, BDI-II and BAI, (3) 35 clients at PP completed CORE-OM, BDI-II, BAI and BLRI, (4) 184 clients at UCS completed CORE-OM, BDI-II, BAI and BLRI, and (5) 70 clients at PP completed BDI-II, BAI and BLRI. The five phases were due to (1) adding additional outcome measures to CORE-OM with a concern that NICE would not see this as diagnostic-specific, a concern confirmed by NICE (2009a), (2) adding in a measure of therapeutic relationship once the research protocol was working to address research question three, (3) taking up the opportunity to conduct research at a multi-therapist centre, and (4) continuing the research at the PP once the UCS research was completed.
2.4 Treatment duration – overall mean 7.1 sessions (SD 10.2), UCS mean 5.5 (SD 6.4) and PP mean 9.2 (SD 13.4). At the UCS ‘time sensitive’ counselling was conducted with an expected mean of six sessions per client. At the PP 93 clients could vary their attendance (mean 10.6, SD 15.9) and 44 clients were paid for by their employer with an agreed number of sessions (mean 6.3, SD 3.8). One of the weaknesses in Lesser’s (1961) research was prematurely measuring outcome and this research sought to measure ‘outcome’ as accurately as possible.
2.5 Resulting sample – For the 321 clients who started in the research mean age was 31.2 years (SD 12.0) with 65.7% female, Marital status was 62.3% single, 26.2% married, 10.6% separated/divorced and 0.9% widowed. Parental status was 66.7% non-parent, 9.3% with one child, 17.8% with two children and 6.3% with three or more children. Ethnic status was requested in free-format and the most consistent interpretation of this data was nationality, with 88.2% British, 3.1% European, 2.8% American plus clients from Asia, Africa and Australia. Medication information was interpreted by a Member of the Royal College of General Practitioners (MRCGP) and allocated to different groups of psychologically relevant medication. Most clients (83.5%) were taking no relevant medication, 11.2% were taking antidepressants only, with the remaining 5.3% taking anxiolytics, sedatives or anti-psychotics, some in combination with anti-depressants. Characteristics of samples analysed vary, in general terms a lot of younger people (students) start in the research, only some of whom complete the research for various reasons due to the nature of the UCS.
2.6 Participant flow and demographic characteristics for each sample analysed – these are available as per TREND guidelines.
2.7 Analytical approach – SPSS 15.0 and 18.0 for Windows was used for the analysis. Analysis of outcomes for clients with clinical scores, by definition excludes clients starting with sub-clinical scores, whereas the predictor sample includes clients with sub-clinical scores. This is because a test of PCT theory must include clients who are ‘well’ (congruent clients in addition to incongruent clients) within the sample (Watson, 1984, p. 37).
Fieldwork took place 23/9/4 to 14/8/9.
Research Question 1: What was the clinical effectiveness of PCT (outcomes study) and how does this compare with other therapies (benchmarking study).
PCT was an effective intervention for clients who completed subsequent questionnaires with depression (ES(d) =1.48, n=111), anxiety (ES(d) = 1.15, n=91) and distress (ES(d) = 1.80, n=79).
In terms of reliable change for depression (n=111) 29.7% had no reliable change, none reliably deteriorated, and of the 70.3% with reliable change 17.1% reliably improved and 53.2% recovered (reliable change and clinically significant change). For anxiety (n=91), 61.1% had no reliable change, none reliably deteriorated and of the 38.9% with reliable change, 7.4% improved and 31.5% recovered. For distress (n=79) 25.3% had no reliable change, none reliably deteriorated and 74.7% reliably improved, of which 20.3% improved and 54.4% recovered. Anxiety reliable changes were perhaps hampered by the low test-retest stability of the BAI, .75, compared with BDI-II .93 and CORE-OM .90, a more stable anxiety measure would perhaps help future research.
Outcomes were comparable with other schools of therapy (e.g. CBT). This was established by a detailed benchmarking study of the results from this research compared with a selection of findings from the literature, including some of the literature cited by NICE for their anxiety (2004a) and depression (2004b) recommendations. Differences in ES of .4 or greater were considered ‘interesting’ (Elliott et al., 2004).
For depression there was no ‘interesting’ difference comparing these findings with those of Ward, et al. (2000) for CBT and PCT, Watson, et al. (2003) for CBT and PE, Elliott, et al. (2004) meta-analysis (23 studies), Minami, et al. (2007) meta-analysis (29 studies, including CBT). The study of
Missirlian, et al. (2005) appeared to have a large difference compared with this study (ES -1.053), however, this could be accounted for by a number of factors (discussed in Weston 2007/2011) including setting a higher inclusion criteria than the present study (minimum equivalent score of 19 BDI-II points cf. minimum 14 points in this study), on this basis any difference in outcomes between the studies was likely trivial.
The study of Dimidjian, et al. (2006) appeared to have large differences compared with this study (Behavioural activation arm ES -2.375, Cognitive therapy ES -1.060 and anti-depressant arm ES – 2.021) their study was longer (16 weeks cf. 8.5 sessions mean for those with subsequent measure in this study), set a number of exclusion criteria (this study had none) and suffered with a high level of attrition (44.0% in the antidepressant arm did not complete cf. 31.5% of clients in this study who started with a clinical level of depression had no subsequent measure, although some of these still completed therapy). Whilst there were some apparent advantages in this comparator study to some extent these could be set against methodological and statistical artefacts which perhaps explained differences, although there could have been non-trivial differences in the anti-depressant and especially the behavioural activation arms compared with the present study.
For anxiety there were no ‘interesting’ differences comparing these findings with those of the Borkovec and Whisman (1996) meta-analysis, the Gould, et al., (1997) meta-analysis, Bryant, et al. (1998), Bryant, et al. (1999) and the anxiety arm of the Elliott, et al. (2004) meta-analysis.
The study by Barrowclough, et al. (2001) pointed to an advantage for CBT (ES -.523) and this may have been due to the higher inclusion criteria in that study which disappeared when a similar inclusion criteria was applied to the present research and in fact pointed to an advantage for the present study (ES +.542).
The Westen and Morrison (2001) meta-analysis pointed to an advantage for CBT (ES -.933) compared with the present study although these authors questioned the validity of the studies upon which their analysis was based and this perhaps led to the CBT-critical paper by Westen, et al. (2004). On this basis it was not clear that there was any real advantage from the CBT studies in this comparator meta-analysis compared with the present study, although there might have been.
For distress there were ‘interesting’ advantages comparing these findings with those of Elliott, et al. (2004) meta-analysis of 127 treatment groups (ES +.810), Stiles, et al. (2006) (ES +.440) and Stiles, et al. (2007) (ES +.410).
In summary ES for depression, anxiety and distress were compared with other studies and found to be broadly comparable, although without a formal statistical comparison (e.g. non-central t). Whilst there was no randomisation to different treatments in this research, clients in this research were of severities broadly comparable with other studies on the most important predictor of outcome; pre-treatment scores.
Research Question 2: What was the evidence that it was the therapy that led to the outcomes, compared with a rival hypothesis (addressing the criticisms of naturalistic research).
Whilst there was no direct evidence that the PCT was appropriately delivered in compliance with a treatment manual, for a sub-group of clients, client perceptions of congruent empathy and unconditional positive regard were measured and the BLRI data suggested that to varying degrees clients perceived that the proposed therapeutic conditions were indeed present.
In terms of the Clark, et al. (2007) criticisms of naturalistic research the analysis supported the validity of these findings:
1) Missing cases could have been treatment failures
Anxiety and distress outcomes were mainly subsets of the larger depression outcomes sample. For the larger depression sample those clients who started with a clinical level of depression (n=162) led to a sample of 111 clients with a subsequent measure of their depression and there were 51 ‘missing cases’. Each of these missing cases was examined and classified with the information available into one of four groups, to attempt to account for their ‘missing’ status, i.e. attended only one session (n=21), probably did improve according to another measure (n=11), reasons for stopping therapy probably not related to lack of progress (n=3) and possible treatment failures, although no direct evidence of this (n=16). On this basis estimates were made for the impact on effect size and reliable change percentages of ‘missing cases’ and inclusion or exclusion of these groups meant mean estimates for the ES ranged from 1,02 to 1.48 and reliable change percentage estimates ranged from 48.1% to 70.3%. The best available evidence was that missing cases were probably not treatment failures.
2) Findings could have been restricted to the ‘easiest’ clients
Depression – clients who had a subsequent measurement of their depression were significantly more depressed at the start than those who did not have a subsequent measurement of their depression.
Anxiety – clients who had a subsequent measurement of their anxiety were significantly more anxious at the start than those who did not have a subsequent measurement of their anxiety.
Distress – clients who had no subsequent measurement of their distress were not significantly more distressed at the start than those who had a subsequent measurement of their distress.
There was no evidence that the findings were restricted to the ‘easiest’ clients, in fact the evidence pointed to the findings coming from the ‘harder’ clients.
3) Findings could have been restricted to the ‘best outcomes’
See 1 and 2 above, and 5 below.
4) Outcomes could have simply been regression to the mean
Using the method described by Barnett, van der Pols and Dobson (2005) it was found that for depression 7.2% of the observed improvement could be accounted for by regression to the mean effects (approximately 1 BDI-II unit), for anxiety this was 9.6% (approximately 1 BAI unit) and for distress 10.6% (approximately 1 CORE-OM unit). There was no evidence that outcomes were simply regression to the mean, in fact the evidence suggested there was an effect that was not simply regression to the mean.
5) Outcomes could have been simply ‘natural recovery’
Depression – a subset of 36 UCS clients had wait controlled outcomes and this showed there was no significant improvement during waiting and a significant improvement during treatment, suggesting for these clients a wait controlled ES=1.26. Treatment (median 49 days) was significantly longer than wait (median 21 days), so the dynamics of change were examined and there was no significant relationship between change in time and change in depression found during either wait or treatment conditions for this subset. Comparing the wait controlled subset of clients with those clients for whom only waiting data or treatment data were available found no significant differences on start of period or end of period outcome measures, i.e. in terms of outcome measure the wait controlled clients were representative of those other clients who only waited or were only treated.
Anxiety – a subset of 36 UCS clients had wait controlled outcomes and this showed there was a small and significant improvement during waiting (ES=.18) and a significant improvement during treatment (ES=.57), suggesting that for these clients a wait controlled ES=.38. Treatment (median 49 days) was significantly longer than wait (median 20.5 days), so the dynamics of change were examined. This suggested a significant improvement of 5.3 BAI units after the exploratory session and a gradual and significant deterioration as waiting progressed. After the first treatment session there was a significant improvement (4.7 BAI units) and there was no significant relationship between changes in time (days) and change in anxiety found during treatment for this subset. Comparing the wait controlled subset of clients with those clients for whom only waiting data or treatment data were available found no significant differences on start of period or end of period outcome measures, i.e. in terms of outcome measure the wait controlled clients were representative of those other clients who only waited or were only treated.
Distress – a subset of 37 UCS clients had wait controlled outcomes and this showed there was a small and significant improvement during waiting (ES=.22) and a significant improvement during treatment (ES=1.45), suggesting for these clients a wait controlled ES=1.22. Treatment (median 49 days) was significantly longer than wait (median 20 days) and the dynamics of change were examined. This suggested a significant improvement of 1.9 CORE-OM units after the exploratory session and no significant change as waiting progressed. After the first treatment session there was a significant improvement (5.4 CORE-OM units) and there was a significant relationship between time (days) and improvements in distress found during treatment for this subset. Comparing the wait controlled subset of clients with those clients for whom only waiting data or treatment data were available found no significant difference on start of period or end of period outcome measures, i.e. in terms of outcome measures the wait controlled clients were representative of those other clients who only waited or were only treated.
There was no evidence that the findings were simply ‘natural recovery’, in fact the evidence pointed to the opposite – that this was a treatment effect.
6) Outcomes could have been attributable to concurrently administered medications
Depression – clients who were taking relevant medication had no significant difference in outcome compared with those who were not taking medication. There was a significant interaction for 21 clients taking anti-depressants, compared with 86 clients who were not, suggesting a small effect (r=.19) for those on antidepressants. It was not clear that it was the antidepressants themselves that were the cause of this small effect, this apparent effect may have confounded those on antidepressants who tended to have higher depression scores (Stiles, et al, 2008).
Anxiety – clients who were taking relevant medication had no significant difference in outcome compared with those who were not.
Distress – clients who were taking relevant medication had no significant difference in outcome compared with those who were not.
There was no evidence that the findings were wholly attributable to concurrently administered medications. In fact the evidence pointed to clients improving irrespective of medication status and only for depression was there a small effect that may or may not have been due to antidepressants for a subset of the sample.
The BLRI scores provided some evidence that congruent empathy and unconditional positive regard were judged by clients to be to some extent provided by way of treatment. Taken together these findings suggested it was unlikely cf. some criticisms that have been made of naturalistic research (Clark, et al., 2007) that 1) missing cases could have been treatment failures, 2) findings could have been restricted to the ‘easiest’ clients, 3) findings could have been restricted to the ‘best outcomes’, 4) outcomes could have simply been regression to the mean, 5) outcomes could have been simply ‘natural recovery’, and 6) outcomes could have been attributable to concurrently administered medications.
Research question 3 – What was the impact of the therapeutic relationship (process-outcome correlation).
Once outlier and influential cases were removed in accordance with the literature (Field, 2005, 2009), the therapeutic relationship had an effect on depression outcome, r =.22, p
back to top
Barkham, M., Mellor-Clark, J., Connell, J., & Cahill, J. (2006). A core approach to practice-based evidence: A brief history of the origins and applications of the CORE-OM and CORE system. Counselling and Psychotherapy Research , (1) 3-15.
Barnett, A. G., van der Pols, J. C., & Dobson, A. J. (2005). Regression to the mean: What it is and how to deal with it. International Journal of Epidemiology , 215-220.
Barrett-Lennard, G. T. (1962). Dimensions of therapist response as causal factors in therapeutic change. Psychological Monographs , 76 (43, Whole No. 562).
Barrowclough, C., King, P., Colville, J., Russell, E., Burns, A., & Tarrier, N. (2001). A randomised trial of the effectiveness of cognitive-behavioural therapy and supportive counselling for anxiety symptoms in older adults. Journal of Consulting and Clinical Psychology , 69 (5) 756-762.
Beck, A. T., & Steer, R. A. (1993). Beck Anxiety Inventory Manual. San Antonio, USA: The Psychological Corporation Harcourt Brace & Company.
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). The Beck Depression Inventory – Second Edition (BDI-II) Manual. San Antonio: The Psychological Corporation.
Bockman (Editor), L. (2000). Validity and social experimentation. London: Sage.
Borkovec, T. D., & Whisman, M. A. (1996). Psychosocial treatment for generalised anxiety disorder. In M. Mavissakalian, & R. Prien (Editors), Long-term treatment of anxiety disorders. Washington DC: American Psychiatric Association.
British Association for Counselling and Psychotherapy. (2004a). Ethical guidelines for good practice in counselling and psychotherapy. Rugby: British Assocation for Counselling and Psychotherapy.
British Association for Counselling and Psychotherapy. (2004b). Ethical guidelines for researching counselling and psychotherapy. Rugby: British Association for Counselling and Psychotherapy.
Bryant, R. A., Harvey, A. G., Dang, S. T., Sackville, T., & Basten, C. (1998). Treatment of acute stress disorder: A comparison of cognitive-behavioural therapy and supportive counselling. Journal of Consulting and Clinical Psychology , 66 (5) 862-866.
Bryant, R. A., Sackville, T., Dang, S. T., Moulds, M., & Guthrie, R. (1999). Treating acute stress disorder: An evaluation of cognitive behavioural therapy and supportive counselling techniques. American Journal of Psychiatry , 156 (11) 1780-1786.
Burns, D. D., & Nolen-Hoeksema, S. (1992). Therapeutic empathy and recovery from depression in cognitive-behavioural therapy: A structural equation model. Journal of Consulting and Clinical Psychology , 60 (3) 441-449.
Cambell, D. T., & Russo, M. J. (1999). Social Experimentation. London: Sage.
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology , 66 7-18.
Clark, D. M., Fairburn, C. G., & Wessely, S. (2007). Psychological treatment outcomes in routine NHS services a commentary on Stiles et al (2007). Psychological Medicine , Published online: 9 October. doi: 10.1017 S0033291707001869.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.
Crits-Christoph, P., Connolly Gibbons, M. B., Hamilton, J., Ring-Kurtz, S., & Gallop, R. (2011). The dependability of alliance assessments: The alliance-outcome correlation is larger than you might think. Journal of Consulting and Clinical Psychology , 79 (3) 267-278.
Des Jarlais, D. C., Lyles, C., Crepaz, N., & TREND Group. (2004). Improving the reporting quality of nonrandomised evaluations of behavioural and public health interventions: The TREND statement. American Journal of Public Health , 94 (3) 361-366.
Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B., Kohlenberg, R. J., Addis, M. E., et al. (2006). Randomised trial of behavioural activation, cognitive therapy and antidepressant medication in the acute treatment of adults with major depression. Journal of Consulting and Clinical Psychology , 74 (4) 658-670.
Elkin, I., Shea, M. T., Watkins, J. T., & Imber, S. D. (1989). National Institute of Mental Health Treatment of Depression Collaborative Research Programme: General Effectiveness of Treatments. Archives of General Psychiatry , 46 (11) 971-982.
Elliott, R. (2010). Psychotherapy change process research: Realising the promise. Psychotherapy Research , 20 (2) 123-135.
Elliott, R., Freire, E., & Cooper, M. (2008). Empirical Support for Person-Centred/ Experiential Psychotherapies: Meta-analysis Update 2008. 14th BACP Research Conference Research and Regulation – Towards an Evidence-Based Profession (p. 23). Cardiff: British Association for Counselling and Psychotherapy.
Elliott, R., Greenberg, L. S., & Lietaer, G. (2004). Research on Experiential Psychotherapies. In M. J. Lambert (Editor), Bergin and Garfield’s Handbook of Psychotherapy and Behaviour Change (pp. 493-539). New York: John Wiley & Sons.
Field, A. (2005). Discovering statistics using SPSS (and sex, drugs and rock ‘n’ roll) (Second Edition). London: Sage.
Field, A. (2009). Discovering statistics using SPSS Third Edition. London: Sage.
Gould, R. A., Otto, M. W., Pollack, M. H., & Yap, L. (1997). Cognitive behavioural and pharmacological treatment of generalised anxiety disorder: A preliminary meta-analysis. Behaviour Therapy , 28 285-305.
Gurman, A. S. (1977). The patient’s perception of the therapeutic relationship. In A. S. Gurman, & A. M. Razin (Editors), Effective Psychotherapy: A handbook of research (pp. 503-543). Oxford: Pergamon Press.
Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the working alliance inventory. Journal of Counselling Psychology , 36 (2) 223-233.
Lambert, M. J., DeJulio, S. S., & Stein, D. M. (1978). Therapist interpersonal skills: Process, outcome, methodological considerations and recommendations for future research. Psychological Bulletin , 85 (3) 467-489.
Lesser, W. M. (1961). The relationship between counselling progress and empathic understanding. Journal of Counselling Psychology , 8 330-336.
Luborsky, L., Singer, J., & Luborsky, L. (1975). Comparative studies of psychotherapy. Archives of General Psychiatry , 32 995-1008.
Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology , 68 (3) 438-450.
Mcleod, J. (2003). Doing counselling research (Second edition). London: Sage.
Minami, T., Wampold, B. E., Serlin, R. C., Kircher, J. C., & Brown, G. S. (2007). Benchmarks for psychotherapy efficacy in adult major depression. Journal of Consulting and Clinical Psychology , 75 (2) 232-243.
Missirlian, T. M., Toukmanian, S. G., Warwar, S. H., & Greenberg, L. S. (2005). Emotional arousal, client perceptual processing and the working alliance in experiential psychotherapy for depression. Journal of Consulting and Clinical Psychology , 73 (5) 861-871.
NICE. (2004a). Anxiety: management of anxiety (panic disorder, with or without agoraphobia, and generalized anxiety disorder) in adults in primary, secondary and community care CG022. London: http://www.nice.org.uk/page.aspx?o=cg022&c=mental downloaded on 10th May 2006.
NICE. (2005a). Anxiety: Management of post-traumatic stress disorder in adults in primary, secondary and community care CG026. London: http://www.nice.org.uk/page.aspx?o=CG026&c=mental downloaded on 10th May 2006.
NICE. (2008). Borderline Personality Disorder (draft). London.
NICE. (2004b). Depression – management of depression in primary and secondary care CG022. London: http://www.nice.org.uk/page.aspx?o=cg023&c=mental downloaded on 10th May 2006.
NICE. (2002a). Depression and anxiety – computerised cognitive behavioural therapy (CCBT) CG051. London: http://www.nice.org.uk/page.aspx?o=appraisals.completed downloaded on 10th May 2006.
NICE. (2009b). Depression in adults (update): consultation comments and table of responses. London: NICE downloaded from http://www.nice.org.uk/guidance/index.jsp?action=download&o=44991 on 22 July 2009.
NICE. (2009a). Depression in Adults (Update): Draft full guideline for consultation. London: http://www.nice.org.uk/guidance/index.jsp?action=download&o=43311 downloaded on Tuesday 3rd March 2009.
NICE. (2005b). Depression in children and young people – identification and treatment in primary, community and secondary care CG028. London: http://www.nice.org.uk/page.aspx?o=cg028&c=mental downloaded on 10th May 2006.
NICE. (2004c). Eating disorders – core interventions in the treatment and management of anorexia nervosa, bulimia nervosa and related eating disorders CG009. London: http://www.nice.org.uk/page.aspx?o=cg009&c=mental downloaded on 10th May 2006.
NICE. (2005c). Obsessive-compulsive disorder – core interventions in the treatment of obsessive compulsive disorder and body dysmorphic disorder CG031. London: http://www.nice.org.uk/page.aspx?o=cg031&c=mental downloaded on 10th May 2006.
NICE. (2002b). Schizophrenia: Core interventions in the treatment and management of schizophrenia in primary and secondary care CG001. London: http://www.nice.org.uk/page.aspx?o=CG001&c=mental downloaded on 10th May 2006.
NICE. (2004d). Self harm: the short term physical and psychological management and secondary prevention of self harm in primary and secondary care CG016. London: http://www.nice.org.uk/page.aspx?o=cg016&c=mental downloaded on 10th May 2006.
Norcross, J. C. (2011). Psychotherapy relationships that work (2nd Edition). New York: Oxford University Press.
Rogers (Editor), C. R. (1967). The therapeutic relationship and its impact: A study of psychotherapy with schizophrenics. Madison, Milwaukee and London: The University of Wisconsin Press.
Rogers, C. R. (1959). A theory of therapy, personality and interpersonal relationships as developed in the client-centred framework. In S. Koch (Editor), Psychology: A study of science Volume 3: Formulations of the person and social context (pp. 184-256).
Rogers, C. R. (1957). The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology , 21( 2) 95-103.
Rogers, C. R. (1985). Toward a more human science of the person. Journal of Humanistic Psychology , 25 (4) 7-24.
Rosenzweig, S. (1936). Some implicit common factors in diverse methods of psychotherapy. American Journal of Orthopsychiatry , 6 412-415.
Siev, J., & Chambless, D. L. (2007). Specificity of treatment effects: Cognitive therapy and relaxation for generalized anxiety and panic disorders. Journal of Consulting and Clinical Psychology , 513-522.
Smith, M. L., & Glass, G. V. (1977). Meta analysis of psychotherapy outcome studies . American Psychologist , 32 752-760.
Snow, R. E., & Wiley (Editors), D. E. (1991). Improving inquiry in social science: A volume in honour of Lee J Cronbach. London: Lawrence Erlbaum Associates.
Stiles, W. B. (1988). Psychotherapy process-outcome correlations may be misleading. Psychotherapy , 25 (1) 27-35.
Stiles, W. B., & Shapiro, D. A. (1994). Disabuse of the drug metaphor: Psychotherapy process-outcome correlations. Journal of Consulting and Clinical Psychology , 62 (5) 942-948.
Stiles, W. B., Agnew-Davies, R., Hardy, G. E., Barkham, M., & Shapiro, D. A. (1998). Relations of the alliance with psychotherapy outcome: Findings in the second Sheffield psychotherapy project. Journal of Consulting and Clinical Psychology , 66 (5) 791-802.
Stiles, W. B., Barkham, M., Twigg, E., Mellor-Clark, J., & Cooper, M. (2006). Effectiveness of cognitive-behavioural, person-centred and psychodynamic therapies as practised in UK National Health Service settings. Psychological Medicine 36 , 555-566.
Stiles, W. B., Shapiro, D. A., & Elliott, R. (1986). Are all psychotherapies equivalent? American Psychologist , 41 165-180.
Stiles, W., Barkham, M., Mellor-Clark, J., & Connell, J. (2007). Effectiveness of Cognitive-Behavioural, Person-Centred and Psychodynamic Therapies in UK Primary Care Routine Practice: Replication in a Larger Sample. Psychological Medicine , doi:10.1017/S0033291707001511.
Stiles, W., Barkham, M., Mellor-Clark, J., & Connell, J. (2008). Routine psychological treatment and the Dodo verdict: A rejoinder to Clark et al 2007. Psychological Medicine , 38 1-6.
Thorne, B. (2003). Carl Rogers (Second edition). London: Sage.
Ward, E., King, M., Lloyd, M., Bower, P., Sibbald, B., Farrelly, S., et al. (2000). Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care for patients with depression. I: Clinical effectiveness. British Medical Journal , 321 1383-1388.
Watson, J. C., Gordon, L. B., Stermac, L., Kalogerakos, F., & Steckley, P. (2003). Comparing the effectiveness of process-experiential with cognitive-behavioural psychotherapy in the treatment of depression. Journal of Consulting and Clinical Psychology , 71 (4) 773-781.
Watson, N. (1984). The empirical status of Rogers’ hypotheses of the necessary and sufficient conditions for effective psychotherapy. In R. F. Levant, & J. M. Shlien (Editors), Client-centred therapy and the person-centred approach: New directions in theory, research and practice (pp. 17-40). Westport, Connecticut: Praeger.
Westen, D., & Morrison, K. (2001). A multidimensional meta-analysis of treatments for depression, panic and generalised anxiety disorder: An empirical examination of the status of empirically supported therapies. Journal of Consulting and Clinical Psychology , 69 875-899.
Westen, D., Novotny, C. M., & Thompson-Brenner, H. (2004). The empirical status of empirically supported psychotherapies: Assumptions, findings and reporting in controlled clinical trials. Psychological Bulletin , 130 (4) 631-663.
Weston, T. (2008b). Clinical effectivenes of the person-centred approach: A preliminary study. PCE 2008: Present strengths and future challenges: 8th world conference for person-centred and experiential psychotherapy and counselling (p. 140). Norwich: University of East Anglia.
Weston, T. (2008a). Clinical effectiveness of the person-centred approach: A preliminary study. 14th BACP Research Conference Research and Regulation – Towards an Evidence-Based Profession (p. 60). Cardiff: British Association for Counselling and Psychotherapy.
Weston, T. (2005). The clinical effectiveness of the person-centred psychotherapies: A preliminary inquiry including literature review, CORE-OM questionnaires, client session recordings and client feedback. Masters dissertation. Norwich: University of East Anglia.
Weston, T. (2007/2011). The clinical effectiveness of the person-centred psychotherapies: The impact of the therapeutic relationship. Doctoral thesis. Norwich: University of East Anglia.
Zuroff, D. C., & Blatt, S. J. (2006). The Therapeutic Relationship in the Brief Treatment of Depression: Contributions to Clinical Improvement and Enhanced Adaptive Capacities. Journal of Consulting and Clinical Psychology , 74 (1) 130-140.