Chapter 12 Cookie-Cutter Corrections The Appearance of Scientific Rigour, the Assumption of Homogeneity, and the Fallacy of Division

Jeff Ewert

On 12 October 2017, in Ewert v. Canada, 2018 (SCC 30), the Supreme Court of Canada (SCC) heard the appeal from the decision in Canada v. Ewert, 2016 (FCC 203) where the Federal Court of Appeal overturned the Interim Order that prohibited Correctional Service Canada (CSC) from utilizing impugned actuarial and non-actuarial risk assessment instruments on Indigenous prisoners until such time as a Remedies Hearing to refine the issues had occurred.1 On 12 June 2018, the Supreme Court of Canada overturned the decision of the Federal Court of Appeal and ruled that the CSC had not validated the impugned risk assessment tools on Indigenous peoples. In this chapter, I outline the court processes that led to this final Supreme Court decision.

I embarked on this risk assessment challenge in April of 2000, after reading the user’s manual for the Statistical Information on Recidivism (SIR) Scale—one of the first actuarial risk assessment tools ever adopted for use by the CSC. Within the first pages of that users’ guide was the caution “Not for use on young offenders, female offenders or Aboriginal offenders.” Curious as to why this scale was not intended for use on Indigenous prisoners when, by 2000, a myriad of other tools were already being routinely used to assess the future risk of Indigenous prisoners, I set out to learn more. When I could not get a satisfactory response after inquiring with the CSC, I simply submitted a formal written complaint on an CSC Complaint form, stating “For the same reason the SIR is not to be used on Aboriginal offenders, neither should the VRAG, PCL-R or any other such tools be used.”

In terms of modern risk assessment used by corrections, there are two kinds of tools in current use, distinguished from one another as “actuarial” and “non-actuarial” tools. An actuarial tool is a future risk prediction tool that incorporates post-release follow-up results in order to see how the predictions measure up against the commission of actual future violence or general recidivism. A non-actuarial tool does not incorporate this follow-up.

“Psychopathy,” also a correctional measurement, is a clinical construct, but with important criminal justice implications. The Psychopathy Checklist Revised (PCL-R) is a non-actuarial checklist made up of a 20-item inventory of various personality traits or behaviours believed to be associated with the presence of psychopathy. As testified to by Dr. Stephen Hart in Ewert v. Canada 2015 (FCC 1093) before the Honourable Justice Michael Phelan, the PCL-R was not initially designed as a risk prediction tool, although it has become widely used as such in corrections as it is believed there is a strong correlation between psychopathy and future violence risk. In particular, the PCL-R’s twenty items are divided into two factors. Factor 1 of the PCL-R contains eight factors that are sub-divided into two facets described as “Interpersonal” and “Affective” facets (e.g., “glibness and superficial charm” or “shallow affect”). Individuals are scored 0, 1, or 2 on the absence, potential presence, or actual presence of each of the listed personality features. Factor 2 also has two facets, which are described as “Lifestyle” and “Antisocial” facets (e.g., “early behavioural problems” or “juvenile delinquency”). Factor 2 of the PCL-R contains nine personality features and scores the individual on a value of 0–2 based on the presence or absence of various past types of behaviours. Factor 1 and Factor 2 variables are weighted equally. PCL-R then generates a total score by separately summing the items under each of Factor 1 and Factor 2. Percentages are assigned to each Factor and then an average of the percentages is taken. In addition, a total numeric score is generated by summing Factor 1 and Factor 2 item scores and three items that are not included in either Factor 1 or Factor 2.

The PCL-R was developed by Dr. Robert Hare in the 1970s with the intention of assessing the presence of the construct of psychopathy, but it is used “off-label,” meaning for something other than the purpose of predicting recidivism risk. Thus far, there has not been anything objectionable in using such “off-label” assessments for either diagnostic or risk assessment purposes on white people in prison. This tool, however, has never been demonstrated to reliably assess the risk of an Indigenous prison population. In other words, no sufficiently substantive empirical studies have been conducted on the use of the PCL-R, or any other actuarial or non-actuarial risk assessment instruments, on the Indigenous sub-population in prisons. There have only been a few studies (such as the 2013 Olver Study which I discuss further below) that have analyzed the validity of the PCL-R’s use with reference to Indigenous people, but the very small subject sample does not make the study generalizable. As such, it is the lack of evidence in support of the continued use of the impugned risk assessment tools used on Indigenous people in prison that shaped the Supreme Court of Canada’s decision.

As the Plaintiff, it first took five years to exhaust the CSC’s Offender Complaint and Grievance System (OCGS) with respect to Correction Services Canada’s reliance on tools that have not yet been validated when assessing the risk posed by Indigenous people in prison. Upon receipt of the Final Grievance Decision, I filed an action in August of 2005 as an unrepresented lay-litigant, which had to be first amended as an application for judicial review in Ewert v. Her Majesty the Queen (2007 FCC 13) and I later filed an appeal in Ewert v. Her Majesty the Queen (2008 FCA 285). The Respondent’s Certified Tribunal Record (CTR) contained within the CSC’s OCGS file granted me access to copies of interdepartmental emails I would have otherwise never seen, one of which read from Randy Mason (NHQ-AC) to Paul Sonnichsen (NHQ-AC) on 20 January 2003. The subject line was “Inmate Grievance re use of PCL-R and VRAG instruments”:

Paul: This is timely in that we have already flagged this issue as a concern. In fact, the Research Branch (NHQ) has already began [sic] some work on this—if only in a preliminary capacity. I suspect the inmate will win his case and that this will force our hand as a Service. And rightly so! It has always been our position that the inappropriate use of actuarial scales and measures adversely affects our Aboriginal offender population. In fact, we contend that the use of these measures artificially inflates need and risk ratings. (Signed) Randy Mason

At my judicial review hearing in January 2007, I argued, in part, that it is not demonstrably justifiable in a free and democratic society to permit the use of assessment instruments that bear out higher risk/needs outcomes for Indigenous people due to their inherent and immutable characteristics (or disadvantage, primarily ensuing from colonization). I argued that the inappropriate use of these tools violates ss 7 and 15 of the Charter of Rights and Freedoms (The Constitution Act) because, when used on an Indigenous person, the risk tools may not be actually measuring the risk a subject poses, but something else altogether (such as the impacts of colonialism). I also argued that since these risk tools were first employed the general recidivism rate had remained relatively static and, therefore, merely predicting recidivism did not improve public safety. I argued that risk prediction in the absence of improving public safety is not a sufficiently important objective. I argued that while these instruments categorize offenders in groups, such as where 75 out of 100 will violently reoffend following release, detaining all 100 in this group as high-risk results in the arbitrary detainment of the 25 people who will not reoffend. This, I argued, is contrary to Section 9 of the Charter. This practice is known as the fallacy of division (or the logical error of division) that says what is true for a group might not necessarily be true for every member of the group. CSC detains individuals who could be safely released to the community based on this fallacy as a result of their categorization in a group deemed to be high risk.

My judicial review was quickly dismissed within days on the basis that the Respondent, CSC, claimed it had not had enough time to complete their promised review of CSC intake tools. I appealed on the basis that I was challenging the use of all actuarial and non-actuarial risk assessment instruments on Indigenous peoples for whom the use of these tools had not been validated, not just the CSC intake tools. In September 2008, my appeal was dismissed, but the court did set a timeline for the CSC to complete its promised review of these risk assessment tools and invited me to come back before the Court if the review was unsatisfactory or not completed. The CSC exceeded the timeframe, and the Federal Court of Canada issued a Notice of Status Review with respect to the then five-year-old file and ordered me to amend my appeal and file the originating statement of claim by 30 April 2010. At that point, the BC Legal Services Society got involved and hired two lawyers to take the matter to trial.

At the time of my first PCL-R assessment, I had the curious anomaly of having been assessed in the 100^th percentile (meaning that 100 percent of others would test lower than me) on the PCL-R’s Factor 1 and in the 51^st percentile on the PCL-R’s Factor 2. The second time I was assessed, I was similarly scored in the 98^th percentile on Factor 1 and at the 61^st percentile on Factor 2. As a result, case management personnel and other CSC decision makers were dissuaded from recommending or approving me for any form of conditional release or cascade to lesser security. From the written submission of the Plaintiff presented in Ewert v. Canada (2018 SCC 30):

Correctional decision-making and reliance on the risk assessment tools

1. 63. In making a host of decisions affecting Mr. Ewert’s liberty, the correctional service has consistently relied, in part, either explicitly on the PCL-R findings themselves, or else on the psychological risk assessments which have relied on the PCL-R.
2. 64. The Plaintiff does not say that the PCL-R assessments are the sole basis for certain decisions taken by the correctional service over the past twenty years. Rather, the Plaintiff submits that the high risk assessments yielded by the PCL-R is a contributing cause of certain decisions made by the service.

In 2013, Mark E. Olver, Stephen C. P. Wong, Craig S. Neumann, and Robert D. Hare wrote a paper called “The Structural and Predictive Properties of the Psychopathy Checklist—Revised in Canadian Aboriginal and Non-Aboriginal Offenders.” The paper concluded in part that “[w]hen disaggregated into its constituent factors, for both ancestral groups, the Lifestyle and Antisocial factors consistently and significantly predicted all recidivism outcomes, whereas the Interpersonal and Affective factors did not significantly predict any of the recidivism outcomes” (Olver, Newman, Wong and Hare 2013, 167). In other words, my Factor 1 scores cannot be relied upon with respect to risk of violence prediction and my Factor 2 scores place me right in the middle of the risk of violence continuum. If used in accordance with the findings of the Olver Study, the PCL-R would assess me as the average prisoner in terms of risk of future violence.

The trial, Ewert v. Canada (2015 FC 1093), took place before the Honourable Justice Michael Phelan in Vancouver. Appearing as an expert witness for the Defence was Dr. Marnie Rice, psychologist and co-author of the Violence Risk Appraisal Guide (VRAG), and as a fact witness was Dr. Larry Motiuk, Assistant Commissioner Policy and Programs for CSC. Appearing as an expert witness for the Plaintiff was Dr. Stephen Hart:

1. 41. The Plaintiff says that the actuarial tests applied to Mr. Ewert were not reliable because they were more likely than not to be subject to cross-cultural bias. There is no or insufficient evidence to displace the inference that the actuarial tests are biased.
2. 42. In Mr. Ewert’s case, there is a particular skew arising from the conclusion of the Olver 2013 study demonstrating that PCL-R Factor 1 is not reliable in respect of Aboriginal inmates. Mr. Ewert tested at 100% and 98% on Factor 1, and his Factor 2 tested at 51% and 61%. If Factor 2 is considered on its own, Ewert tests as a normal or average inmate in the middle of the pack. If Factor 1 alone is considered, Mr. Ewert tests at the most psychopathic level. If both Factor 1 and Factor 2 are averaged, Mr. Ewert tests at moderate-high risk and is above the cut-off grade for psychopathic designation.
3. 43. If the Court relies on the Olver study to eliminate Mr. Ewert’s Factor 1 score from the equation, the only available conclusion is that Mr. Ewert’s risk to reoffend has been dramatically overestimated and that the PCL-R tests have wrongly labelled Mr. Ewert as a psychopath.
4. 44. Mr. Ewert would qualify this result to ensure that this conclusion does not detract from the systemic nature of Dr. Hart’s evidence regarding cross-cultural bias. Dr. Hart testified that the Olver study used a smaller sample and did not conduct some of the appropriate reliability analysis. Mr. Ewert’s objective in this lawsuit is systemic—he wishes to challenge the use of these actuarial tools on all inmates. There was no available evidence regarding the average spread between Aboriginal scores on Factor 1 and Factor 2 of the PCL-R. (Submission of the Plaintiff in Ewert v. Canada, 2015 FC 1093)

Plaintiff’s lawyers, Jason Gratl and Eric Purtzki, set the case out as consisting of breaches of the governing legislation (ss 4(g) and 24(1) of the Corrections and Conditional Release Act [CCRA]) and set out that the Defendant, CSC, was aware of the false or otherwise unreliable scores resulting from the use of the questionable tools in respect of Indigenous prisoners. Subsection 4(g) of the CCRA sets out, inter alia, that correctional policies, programs, and practices respect the special needs of Indigenous peoples. One argument we presented was that the use of risk assessment tools on sub-populations of offenders, for whom the tools have not been standardized/validated for use, does not respect ethnic or cultural differences and is not responsive to the special needs of Indigenous peoples.

Another argument we presented was that the Defendant was aware of the existence of information that, if incorporated into the design and development of the risk assessment instrumentation in question, would mitigate the risk assessments produced on Indigenous peoples. Subsection 24(1) of the CCRA sets out that the “Service shall take all reasonable steps to ensure that any information about an offender that it uses is as accurate, up to date and complete as possible.” At trial, the Defendant acknowledged the existence of a study showing that Indigenous prisoners who follow the traditional teachings of their Elders are in a category where only two percent (2%) recidivate, yet this knowledge is not incorporated into the design of any risk assessment tools and, therefore, the Service has not taken all reasonable steps to ensure that all information about an offender that it uses is accurate, up to date, and complete as possible.

Dr. Stephen Hart further provided his expert opinion as to the limited reliability and validity of using risk assessment tools on subjects where the personal characteristics, including culture, are not accounted for by the tools. Dr. Hart opined on the nature of bias due to culture, referred to as cross-cultural bias, or more simply, culture bias:

In Canada, one of the most important cultural characteristics that is likely to cause cross-cultural bias is status as an Aboriginal person. Status as an Aboriginal person is a general or higher-order characteristic that encompasses numerous specific characteristics, including such things as language. Religion or spirituality, self-concept, and fundamental social norms and attitudes . . . there is no research evaluating the cross-cultural bias in reliability or validity of assessment instruments such as the VRAG, SORAG. STATIC-99, VRS-SO, and PCL-R due to status as an Aboriginal person using scientifically adequate procedures. It cannot be safely assumed that research findings derived from a heterogeneous group apply equally to its constituent elements: this would be making the logical error of division. What holds true for a group does not necessarily hold true for the individuals or sub-groups within that group. (Written Submissions of the Plaintiff, Ewert v. Canada 2015 FC 1093)

On 18 September 2015, the Honourable Justice Phelan issued an Interim Oder concluding:

[113] a) that the use of these assessment tools is both inconsistent with the principles in 5.4(9) of the Act by not being responsive to the special needs of Aboriginal people and further such use breaches s. 24(1) of the Act; and further

b) violates the Plaintiff’s s 7 Charter rights without 3.1 justification. (Ewert v. Canada 2015 FC 1093, summary)

Justice Phelan enjoined the Defendant from using the results of the assessment tools until a study could confirm the reliability of those tools in respect to adult Indigenous people in prison.

There were also, however, some issues that were not put before the Court with respect to the use of the risk assessment tools that I include here (see also Cremin et al., 2009). First, while I had initially challenged the use of “any and all actuarial and non-actuarial risk assessment instrumentation on sub-populations of offender for who said instruments have not been validated for use on,” the Court only heard of a few.2 With respect to the PCL-R, the experts who testified at trial agreed that the greater the divergence between Factors 1 and 2, the less reliance should be placed on the scores:

1. 34. In Dr. Rice’s opinion, there was significant doubt about whether Factor 1 scores were reliable in respect of Aboriginal inmates, and acknowledged that in some circumstances, when Factor 1 scores were higher than Factor 2 scores, it would not be appropriate to use the PCL-R to assess the recidivism risk of Aboriginal inmates. She also acknowledged that it would not be appropriate to use the VRAG or SORAG on those Aboriginal inmates, because VRAG and SORAG incorporate the PCL-R scores. (Written Submissions of the Plaintiff, Ewert v. Canada 2015 FC 1093)

During the trial, it was evident there was no available evidence regarding the average spread between Aboriginal scores on Factor 1 and Factor 2 of the PCL-R, but during cross-examination when the Plaintiff’s Factor 1 and Factor 2 scores were presented to Dr. Marnie Rice, who had over forty years of experience in the field, she balked at the suggestion that such a score could exist and stated that she had never heard of such divergence between these factors. I can attest to having seen the raw PCL-R scores of two other Indigenous prisoners that were more divergent than my own.

A third issue that escaped judicial scrutiny during this trial, due in part to the legal strategy of staying focused on the issue of culture bias as the reason for the disastrous overrepresentation of Indigenous peoples within Canadian prisons, is that Canada is not the only settler country that suffers from this ailment. Deborah Dawson (1999) identified a relationship between the use of Westernized risk assessment tools and the overrepresentation of Indigenous Australians and Torres Strait Islander people in the Australian prison system. The same problem was identified by Dawson (1999) with respect to the Māori people of New Zealand. It is a similar relationship as the one between the use of such tools and the overrepresentation of Indigenous people within Canadian prisons.

A fourth issue not scrutinized by the Court in my case is revealed in an article by Quinsey et al. (2006, 218), which states that the authors propose a standardized predictive scheme as a means, in part, of avoiding liability. What this means is that a psychologist or psychiatrist who clinically assesses, for conditional release purposes, the risk of future violence of anyone with past violent conduct, could be liable for any harm suffered by a victim of the subject’s future violence should that subject reoffend. While publicly purporting that the main purpose is to accurately predict risk of future violence in the interests of public safety, amongst their peers the authors admit that part of the purpose of the Violence Prediction Scheme is to avoid liability. Avoiding liability is not a sufficiently important objective when there is either a risk to public safety or a risk of unnecessarily and improperly prolonging the carceral experience of minoritized sub-populations of prisoner due to culture bias. As attested to by the expert witness, Dr. Marnie Rice, there is no evidence that CSC programming does anything to reduce risk.3 Indigenous prisoners are discriminatorily assigned higher risk/needs scores by tools that were not designed to assess their risk.

The Remedies Hearing was held on 25 April 2016 and Justice Phelan went out on reserved decision. The Defendant’s appeal was heard on 13 June 2016 and the judges went out on reserved decision. On 3 August 2016, the Federal Court of Appeal rendered their decision to overturn Justice Phelan’s 18 September 2015 ruling before his decision on the Remedies Hearing was rendered. On 29 September 2016, lawyers Jason Gratl and Eric Purtzki filed our brief for leave to appeal (Memorandum of Argument) to the Supreme Court of Canada. On 9 March 2017, the Supreme Court of Canada granted us leave to appeal the decision of the Federal Court of Appeal. On 12 October 2017 that appeal was heard. Ten interveners joined the Plaintiff in arguing that the tools are inappropriate in their use with respect to Indigenous peoples.4

On 12 June 2018, the Supreme Court of Canada found that the Plaintiff had not proved the tools were invalid when used on Indigenous peoples; it did find that the CSC had not proved that they were valid prior to using them, to the possible detriment of the Indigenous sub-population of prisoner. The CSC is legislatively required to ensure that it takes all reasonable steps to ensure that any information that it uses about an offender is as accurate, up to date, and complete as possible.

[33] On its face, the obligation imposed by s. 24(1) of the CCRA appears to apply to information derived from the impugned tools. Section 24(1) provides that the obligation applies to “any information about an offender that [the CSC] uses” The fact that s. 24(1) applies to “any” such information confirms that, if its words are read in their grammatical and ordinary sense, it applies to the information at issue in this case. (Ewert v. Canada, 2018 at para. 33)

The Supreme Court of Canada also examined the scope of s. 4(g) of the CCRA for the first time:

[53] In my view, the application of that approach leads to the conclusion that the principle set out in s. 4(g) of the CCRA can only be understood as a direction from Parliament to the CSC to advance substantive equality in correctional outcomes for, among others, Indigenous offenders. Section 4(g) represents an acknowledgement of the systemic discrimination faced by Indigenous persons in the Canadian correctional system. This is a long-standing concern, and one that has become more, not less, pressing since s. 4(g) was enacted. In these circumstances, it is critical that the CSC give meaningful effects to s. 4(g) in performing all of its functions. (Ewert v. Canada, 2018, at para. 53)

In concluding that the CSC could no longer utilize the impugned risk assessment instruments on Indigenous prisoners the Court said that, at minimum, the CSC “must conduct research into whether and to what extent they are subject to cross-cultural variance when applied to Indigenous offenders” (para. 67). The Court also stated in part that “Numerous government commissions and reports, as well as decisions of this Court have recognized that discrimination experienced by Indigenous persons, whether as a result of overtly racist attitudes or culturally inappropriate practices, extends to all parts of the criminal justice system, including the prison system” (para. 57).

What will we use to assess the risk of future dangerousness if we do not have actuarial and non-actuarial assessments? First, as argued above, the recidivism rate does not appear to have improved since the introduction of actuarialism/non-actuarialism in the assessment process. The use of the tools has not shown to improve public safety. It is important to not forget that the fallacy of division results in significant numbers of prisoners being held back from conditional release because they are assessed as being in groups of offenders of which a majority percentage are deemed “high risk” to reoffend. This practice constitutes arbitrary detainment, contrary to section 9 of the Charter. What may be statistically true for the group is not necessarily true for all the individuals within that group. Second, as opined by Dr. Stephen Hart at the 2015 trial before Justice Michael Phelan, the tools could still be used to inform a clinical assessment in what is called a “structured clinical assessment.” This would entail utilizing the tools to better identify personality profiles, possible mental disorders, among other things, but without assigning a numerical value or percentile to the assessed risk of future violence prediction.

As also evidenced above, the CSC has admitted in emails from as far back as 2003 that they know Indigenous peoples bear out higher risk/needs assessment scores by way of these impugned tools. This is likely the result of a culture bias scenario in which higher numbers, while not equating to higher risk, equate to incarceration at higher rates, at higher levels of security, and for longer periods of time for Indigenous peoples when compared to their non-Indigenous counterparts. Fifteen years later, the CSC has not yet addressed this. In addition to this, Dr. Larry Motiuk, the CSC’s witness, misled the court when he stated that weight is not necessarily given to such scores by decision makers in the annual security classification.5 Motiuk testified that the CSC knows from its research where the weight is put on those items, but when asked about the research, he replied that the study was completed in late 1992 through early 2003, and was at the time on the CSC’s website. Later in the cross-examination Motiuk testified that, in fact, there were no studies completed by his department that assess the weight CSC decision makers attribute to actuarial test results.6 It is clear that changes needed to the problem of improper risk assessment for Indigenous peoples must come from outside the CSC.