Check out this new Campbell systematic review by Brouillette Alarie and a large international team that takes a hard look at a question many practitioners quietly ask. Do the risk assessment tools used in preventing violent extremism actually work?
The aim of the paper is clear and narrow. The authors set out to assess the reliability and validity of tools used to assess an individual’s risk of engaging in violent extremism. In plain terms, they ask whether these tools produce consistent results and whether they measure what they claim to measure. This matters because these tools are already used to inform decisions about supervision, intervention, and sometimes liberty.
Methodologically, this is a full Campbell systematic review. The team searched 17 academic databases, 45 repositories, Google, prior reviews, and reference lists. Searches covered studies published up to 31 December 2021, in any language. To be included, studies had to report primary quantitative data on the reliability or validity of a violent extremism risk tool used by practitioners at the individual level. There were no restrictions on population, design, or setting. Risk of bias was assessed using a modified COSMIN checklist, and meta analysis was used where possible.
Out of more than 10,800 records, only 20 studies met the inclusion criteria. These studies examined six tools, mostly structured professional judgement instruments such as TRAP 18, ERG22+, VERA, MLG V2, and IVP guidance. Samples were small, heavily male, and often built from open source material like media reports and court documents rather than real world assessments.
So what did they find?
There is some good news. In research settings, different assessors often reached similar conclusions using the same tools, with high inter rater agreement. Content validity was also generally strong, meaning the tools include risk factors that appear relevant to people who later committed extremist violence.
But the problems are serious. Internal consistency of subscales was weak. Construct validity was poorly supported. Most importantly, there is no evidence of true predictive validity. Not a single study tested whether these tools can predict future violent extremism using prospective data. All studies worked backwards, assessing cases where outcomes were already known. Even the strongest results, mainly for TRAP 18, showed high variability and relied on retrospective designs that are known to inflate performance.
The policy implications are straightforward. These tools should not be used as standalone decision making devices, especially in high stakes legal or security contexts. They can help structure professional judgement and guide intervention planning, but they do not meet the standards expected in correctional psychology.
For research, the message is blunt. The field does not need more reviews. It needs access to real data, prospective studies, and independent validation. Until that happens, confidence in violent extremism risk tools should remain cautious and evidence led.