DIANE A. BELLAS
Public Defender
1401 Lakeside Drive, Suite 400
Oakland, California 94612-4305
(510) 272-6600
XXXXXXXX
State Bar No.
Deputy Public Defender
Attorney for Respondent
SUPERIOR COURT OF CALIFORNIA, COUNTY OF ALAMEDA
RENE C. DAVIDSON COURTHOUSE
The People of the Stateof California,
Petitioner,
v.
XXXXXXXXXX
Respondent.
/ / Dept. No. 11
No.
Date: February 14, 2011
______
MOTION TO EXCLUDE TESTIMONY CONCERNING
THE “STRUCTURED RISK ASSESSMENT-FORSENSIC VERSION” (SRA-FV)
Dana Putnam is one of the state’s evaluators in this case. He wrote his first report on Respondent in August 2002 and has written numerous updates and addenda since then, including an Update Report dated November 30, 2010. During the course of his involvement in this case, he has utilized numerous “risk assessment” actuarial tools, including the Static-99, Static-99R, Static-2002, Static-2002R, MnSOST-R, and SORAG. On January 21, 2011, he authored an “Addendum Report,” which was provided to Respondent’s counsel on January 31, 2011, the day set for trial to begin. (The trial was continued to February 14, 2011 at the request of the Petitioner.) In this report, for the first time, Dr. Putnam utilized another instrument, called the “Structured Risk Assessment – Forensic Version (SRA-FV).” (A copy of the Addendum is attached to this motion as Exhibit A.) Respondent objects to any evidence concerning this item as being unduly late discovery and being unreliable.
I.
EVIDENCE CONCERNING THE SRA-FV SHOULD
BE EXCLUDED AS LATE DISCOVERY
As indicated above, Dr. Putnam has filed numerous reports concerning Respondent over the past eight and one-half years. During that time, he has used various “risk assessment” instruments in his analysis of Respondent’s risk of reoffense. For example, prior to the first trial in this matter, he wrote a report dated June 29, 2009, in which he scored Respondent on the following instruments: Static-99, Static-2002, MnSOST-R, and SORAG. On October 20, 2009, Dr. Putnam wrote an Addendum, using the above-mentioned four instruments, as well as the Static-99R and Static-2002R. This was followed by an email dated November 30, 2009, in which he corrected the Static-2002R data. The following day, December 1, 2009, he re-corrected this data. Then, on December 9, 2009, he revised the data for his Static-99R and Static-2002R scores. In preparation for the upcoming trial, Dr. Putnam issued another report, dated November 30, 2010, again using Static-99R, Static-2002R, MnSOST-R, and SORAG. To counsel’s knowledge, none of these reports, updates, addenda, or emails mentions SRA-FV.
Aware of Dr. Putnam’s use of the risk assessment instruments in his various communications, counsel has had time to become at least somewhat familiar with them and is prepared to cross-examine Dr. Putnam on these items. At the last minute, however, Dr. Putnam has introduced a new technique, the SRA-FV, via an addendum dated January 21, 2011. According to the addendum, the SRA-FV “provides a systematic way for reviewing the extent to which adult male sexual offenders have the kind of long-term vulnerabilities that are known to contribute to sexual recidivism.” (p. 2.) Dr. Putnam indicated that SRA-FV can be used in two ways: “to provide a more accurate indication of recidivism than the Static-99R used alone and [to] provide guidance for selection of the appropriate norm reference group in relation to the Static-99R and the Static-2202R.” (Ibid.)[1] The instrument allegedly “is scored based on a review of long-term vulnerabilities in relation to the areas of sexual interests (sexual preference for children, sexualized violence, and sexual preoccupation), relational style (emotional congruence with children, lack of emotionally intimate relationships with adults, callousness, grievance thinking), and self-management (lifestyle impulsivity, resistance to rules and supervision, and dysfunctional coping).” (Ibid.) Dr. Putnam gave Respondent a score of 3.6, although he did not include a scoring sheet or other explanation of how he arrived at this figure.
Civil discovery rules apply to SVP proceedings. (Leake v. Superior Court (2001) 87 Cal.App.4th 675.) California Code of Civil Procedure section 2024.020 provides, “ (a)Except as otherwise provided in this chapter, any party shall be entitled as a matter of right to complete discovery proceedings on or before the 30th day, and to have motions concerning discovery heard on or before the 15th day, before the date initially set for the trial of the action. (b)Except as provided in Section 2024.050, a continuance or postponement of the trial date does not operate to reopen discovery proceedings.” (See also Cal. Penal Code section 1054.7.) As a matter of practice, attorneys in the offices of the Alameda County District Attorney and the Public Defender do not follow the rigid guidelines of section 2024.020 and rely on the more informal give-and-take of criminal procedure. Nonetheless, section 2024.020 indicates the Legislature’s intent that counsel not be caught unawares by last-minute revelations.
In order to prepare for this trial, counsel has become familiar with the details of Respondent’s entire life history, from the day of his birth until the present. Indeed, given the nature of SVP proceedings, Respondent’s future conduct will also be a subject of substantial testimony. Such preparation requires familiarization with thousands of pages of documents concerning Respondent’s criminal history, his behavior in institutions, his contact with various mental health professionals, and other sources of information. Further, counsel must be familiar with recondite areas of psychology and statistical analysis. This includes numerous risk assessment instruments mentioned above. As the Court can imagine, these matters are the subject of substantial controversy and professional interest, with countless studies, articles, and emails written about the strengths and weaknesses of various diagnoses (particularly paraphilia not otherwise specified, the major diagnosis to be presented by the state’s evaluators) and risk assessment tools. It is simply not reasonable to expect counsel to become familiar with still another risk assessment instrument (and its discontents) at the last possible moment.
In People v. Lamb(2006), 136 Cal. App. 4th 575, defense counsel failed to disclose any statements or reports prepared by his accident reconstruction expert, Mr. Todd. As a sanction for the discovery violation, the trial court prohibited Todd from testifying in surrebuttal. In response to defendant’s allegation that this was inappropriate exclusion of a witness under Penal Code section 1054.5(c), the Court of Appeal noted, “Here, however, the court did not exclude the testimony of Todd. Todd and his associate testified at great length to explain their investigation and conclusions. The court’s sanction was limited in scope, and extended onlyto refusing to permit additional surrebuttal testimony.” (Id., at p. 582.) (The Court also upheld the ruling under Evidence Code section 352; see Argument II, below.)
Similarly, in the present case, Respondent does not seek to prevent Dr. Putnam from testifying (undoubtedly at great length). Rather, he seeks only to prevent testimony on the SRA-FV, presented to Respondent as part of this case for the first time on the day set for trial. For this reason, the Court should prohibit Dr. Putnam (and any other expert witnesses) from testifying regarding the SRA-FV.
//
//
//
//
//
II.
THE EVIDENCE SHOULD BE EXCLUDED AS UNRELIABLE
The brief investigation counsel has been able to do concerning SRA-FV reveals serious issues with its reliability, such that the Court should exclude it entirely from the trial or, at a minimum, hold a Kelly(People v. Kelly (1976) 17 Cal.3d 24) and/or Evidence Code section 402 hearing on the proposed testimony.
SRA-FV was developed by Dr. David Thornton and first presented in incipient form in late 2009; it was apparently not utilized by SVP evaluators until a presentation he gave in December, 2010. It is based on what’s known as the “Bridgewater sample,”a group of 599sex offenders in Massachusetts who were studied from 1959 to 1984. Dr. Brian Abbott, a well-recognized expert in SVP risk assessment protocols, has reviewed Dr. Thornton’s Powerpoint presentations on SRA-FV and subsequent developments. He has noted the following limitations with the instrument: “One study does not mean the method is ready for use forensically. Moreover, Dr. Thornton has not published for peer review anything about how the SRA-FV was developed andwhat are its validity statistics. The inter-rater reliability in the initial developmental study for the SRA-FV was .71 (Knight and Thornton, 2007). This is considered as moderate reliability and well below that which is considered acceptable (.90) when making decision of significant importance (Marshall, 2006), such as to whether an individual meets the statutory risk criterion under the SVP act or which reference group to compare the individual being assessed. Knight and Thornton (2010) reported several cautions in utilizing the SRA-FV due to its less than optimal reliability: (1) ‘The role of Need factors should be studied in future samples, and the robustness of their ability to add materially to predictive accuracy investigated. The limited reliability with which Need factors were rated suggests that a preliminary step should involve further specification of the Need rating scales’ (p. 82); (2) ‘Structured assessments of Need factors should be used cautiously in combination with established empirically derived actuarial instruments. The need for caution arises from two sources. First, that the volume of studies supporting the predictive value of Need ratings is quite small relative to that supporting the value of actuarial instruments. Second, the limited reliability of Need ratings is a serious impediment to effective practical use. Although the promise of Need ratings is great, they will mislead as often as they help if clinicians cannot apply their criteria reliably’ (p.84). Dr. Thornton failed to disclosed these limitations to DMH evaluators who attended the training about the SRA-FV. DMH evaluators likely rely upon this training when applying the SRA-FV in SVP assessments and testifying about it. Since the SRA-FV was fitted to the sample upon which it was developed, it is reasonable to conclude that the validity and reliability of the instrument will be lower and, possibly, unacceptable for forensic use when applied to other populations of sexual offenders. Consequently, it is incumbent upon the SRA-FV developer to conduct appropriate replication studies (Schmidt, 2009) to examine the reliability, error rates, validity, and predictive accuracy of the SRA-FV before releasing it for use by clinicians.
“Thornton (2010) presented data suggesting that three of the ten items on SRA-FV versionmay not be predictive of sexual recidivism (cf. slides 80 and 177). He did not discuss this at the training but viewing these two slides suggest this is the case. These items in question require clinicians to score the PCL-R. The three items total 30.8% of the maximum score (6 points) on the SRA-FV. If the three items are not predictive of sexual recidivism, this means an individual’s score on the SRA-FV is inflated to the extent the individual is scored on these apparently invalid items. This finding raises concerns as to what other items on the SRA-FV may not be predictive of sexual recidivism. The SRA-FV developer should conduct an appropriate statistical analysis showing that each item on the instrument achieves incremental predictive validity over an actuarial instrument.
”Thornton states the SRA-FV can be used to sort offenders into one of the four Static-99R reference groups. The Bridgewater sample upon which the SRA-FV was developed and fitted is one of 23 Static-99R reference groups. Thornton (2010) has performed statistical wizardry to support that the results from the Bridgewater study can be relied upon ipso facto to develop scoring rules that discriminates membership in one of the four Static-99R reference groups. While his statistical methods to develop cut-off scores may be correct (although this is not known since it has not been peer reviewed), the logic of his methods is unsupported conceptually or rationally. Thornton (2010) presumes that the data analysis on a sample of 599 high risk sexual offenders from Massachusetts, who were followed up for recidivism data between 1957 and 1986, is representative not only of the other subjects in the High Risk reference group (from 5 other studies) but also the other three reference groups (consisting of 17 studies). The major problem with this thesis is that the SRA-FV was not administered to any of the subjects of the other 22 studies comprising the Static-99R data set so there is no way to use it for the purpose he asserted. For example, Thornton (2010) instructs users of the SRA-FV to compare sexual offenders who score less than 1.7 on the SRA-FV to the routine corrections group. To accept his logic means that no members in the actual Routine Corrections group (consisting of 8 studies) scored greater than 1.7 on the SRA-FV. It is highly unlikely the range of scores on the SRA-FV for members of the routine corrections sample are restricted to less than 1.7 out of a total of 6 points. Thornton’s designation of the SRA-FV cut-off score of less than 1.7 as being associated with membership in the routine corrections group is akin to saying that no members of the routine corrections group scored above +1 on the Static-99R. This is obviously an incorrect conclusion since the members of the routine corrections group exhibited scores between -3 and +9, which is consistent with the range of scores on the Static-99R observed in members of the other three reference groups. In order to develop cut-off scores on the SRA-FV to distinguish the Static-99R reference group membership, Dr. Thornton must apply the SRA-FV to all members of the other 22 studies from the Static-99R (total of 5,766 individuals) and then statistically analyze whether SRA-FV total scores can discriminate Static-99R reference group membership.” (Email 2/8/11.)
Dr. Thornton’s 2009 presentation acknowledged the preliminary nature of his conclusions: “So far we have a research manual for scoring FSRA [Forensic Structured Risk Assessment]. We need an operational manual (more user friendly). We need to demonstrate that forensic evaluators can score FSRA reliably. We need data on the distribution of FSRA scores in an SVP population other than Bridgewater. We need adjustment tables for Static-99R and Static-2002R.” (Slide No. 54.) He also stated the “three major weaknesses of SRA-FV”: “Not yet published. Only tested in one sample. Requires significant skills to apply reliably. E.G. requires being able to score PCL-R.” (Slide No. 56.)
In a 2007 article entitled, “Evaluating and Improving Risk Assessment Schemes for Sexual Recidivism: A Long-Term Follow-Up of Convicted Sexual Offenders,” Dr. Thornton and Dr. Raymond Knight issued the following caveat regarding the use of such “Need factors”: “Any actuarial that we develop for subsample prediction using the current sample must remain simply a speculative scale until it is replicated on an independent sample. Until that time, it should not be used in decision-making. An extensive literature comparing clinical and actuarial prediction indicates that the clinical adjustment of extant actuarials is substantially more likely to reduce accuracy than to enhance it (Grove & Meehl, 1996; Grove, Zald, Lebow,Snitz, & Nelson, 2000). Even if the additional subgroup measures that we have identified continue to account for significant additional predictive variance, they should be incorporated into decision-making by the creation of a new actuarial. Clinicians are not likely to be able to incorporate them by making adjustments (Faust, 1986; Garb, 1998).” (Id., at pp. 84-85.)
In a more recent article on such “third generation” tools, Thornton and his co-authors indicated, “The research on these measures is still sufficiently underdeveloped that important questions remain concerning the conceptual foundations of these scales, whether they target the most relevant factors and the extent to which it is possible to associate recidivism rates with specific scores.” (Mann, Hanson, Thornton: Assessing Risk for Sexual Recidivism: Some Proposals on the Nature of Psychologically Meaningful Risk Factors, Sexual Abuse: A Journal of Research and Treatment 2010, 22 at p. 193.)
Given these severe limitations, including those acknowledged by its author, the Court should exclude any mention of the SRA-FV from this trial. Even if the Court disagrees that unreliability should result in the exclusion of the proposed testimony, the Court should exclude the evidence under Evidence Code section 352. The amount of time necessary to explain, explore, and criticize this new procedure will greatly outweigh any probative value it may have.[2] At the very least, the Court should hold a hearing under Evidence Code section 402 to determine whether the evidence establishes SRA-FV as a preliminary “fact” meeting minimal standards of relevance and reliability. Finally, while Kelly, supra, generally does not apply to expert psychiatric testimony(People v. Ward (1999) 71 Cal.App.4th 368), since it is not “scientific,” this is the rare instance in which the Court should make a determination in advance of the admissibility of the evidence.
//
//
Dated: ______Respectfully Submitted:
DIANE A. BELLAS
Public Defender
By:______
XXXXXXX
Deputy Public Defender
Cal. Bar No.
DIANE A. BELLAS
Public Defender
1401 Lakeside Drive, Suite 400
Oakland, California 94612-4305
(510) 272-6600
David R. Feld
State Bar No. 88325
Deputy Public Defender
Attorney for Respondent
SUPERIOR COURT OF CALIFORNIA, COUNTY OF ALAMEDA
RENE C. DAVIDSON COURTHOUSE
The People of the Stateof California,
Petitioner,
v.
ATOA ATUALEVAO,
Respondent.
/ / Dept. No. 11
No. 142942
Date: February 14, 2011
______
EXHIBIT A
Attached as Exhibit A is a correct copy of the “WIC 6600 Evaluation Addendum Report” of Dr. Dana Putnam, dated January 21, 2011.
Dated: ______Respectfully Submitted:
DIANE A. BELLAS
Public Defender
By:______
DAVID R. FELD
Deputy Public Defender
Cal. Bar No. 88325
1
[1] This refers to the four “norm reference groups” promulgated by the developers of the Static-99 which purport to distinguish among sex offenders in regard to risk of recidivism.
[2] Indeed, when counsel contacted Dr. Abbott concerning this issue, he offered to testify pro bono to contradict Dr. Putnam’s use and interpretation of the SRA-FV.