Does 91% Expert Agreement Mean 91% Accuracy in Diagnosing Methamphetamine Psychosis?

Written by

in

Twenty of twenty-two experts agreeing on proposed diagnostic criteria does not mean the criteria correctly diagnose ninety-one percent of patients. Camfield and colleagues’ 2024 methamphetamine-psychosis paper reports expert consensus. It is an important form of evidence, but it is different from a patient-based diagnostic-validation study.

What process produced the criteria?

The investigators recruited experienced clinicians and used an iterative process informed by Delphi techniques. Web-based surveys, discussion and electronic communication progressively refined the proposed wording. The predetermined threshold for consensus was endorsement by at least 80% of respondents.

The final panel included 22 physicians. All voted in the final round, and 20/22 endorsed both proposed diagnostic frameworks, approximately 91%. That denominator consists of voting clinicians, not patients whose diagnoses were checked against an independent reference.

What did the panel propose?

The panel developed an acute methamphetamine-induced psychotic-disorder category and a persisting category, with subacute and chronic timeframes. The latter framework distinguished symptoms continuing up to one month after last use from those extending beyond a month of sobriety.

These were proposed criteria published in 2024. Their appearance in a paper should not be rewritten as proof of universal adoption, a replacement for every existing diagnostic framework or a current treatment recommendation.

The panel emphasized symptom chronology and prior exposure history, particularly for persisting presentations. Its discussion acknowledged that persistent psychotic content may be difficult to distinguish from primary psychosis based on symptoms alone.

Which validation questions remained unanswered?

The consensus process did not establish sensitivity or specificity against a patient-level reference standard. It did not test whether independent clinicians applying the criteria to the same patients would reliably agree, or whether the proposed categories predicted clinical outcomes.

Those are separate questions. Experts can agree that particular wording is useful while uncertainty remains about how it performs when histories are incomplete, exposure is uncertain or alternative diagnoses are plausible. The authors identified inter-rater reliability, validity and clinical usefulness as subjects for further study.

How was scientific literature incorporated?

A PubMed search through May 2022 identified 1,923 unduplicated records. After exclusions, 1,043 were made available in an electronic reference library. The investigators did not formally assess the quality of those articles or conduct a full-text review of all records before making them available.

Panelists could consult full text and other sources, and most reported accessing scientific literature. The study did not establish that everyone reviewed the same papers or assessed them using one standardized appraisal method. Availability of a literature library is not equivalent to a completed systematic evidence assessment.

Does strong agreement eliminate disagreement or selection effects?

Two panelists did not fully endorse the final criteria and raised substantive concerns, including wording about distress, particular hallucinations and exclusion of other disorders. Their disagreement should not disappear behind the rounded consensus percentage.

The recruitment criteria favored clinicians with substantial experience treating methamphetamine-related problems. That expertise was relevant to the task, but shared clinical backgrounds could also promote agreement. English-language clinician and literature restrictions further limited representation.

How should the paper be used?

It provides a clearly described expert proposal and a framework for research questions. It does not supply a numerical probability that a particular person’s psychosis was caused by methamphetamine, nor does a positive toxicology result independently settle that attribution.

A careful description therefore reports what was agreed, who voted and what still required validation. The 91% figure measures endorsement within this panel. It is not a diagnostic accuracy estimate.

Source

Camfield K, Reedy A, Wolf C, Al-Tayyib A, Rinehart D, Simpson SA, for the BEAT Meth MIPD Expert Panel. Diagnosis of methamphetamine-induced psychotic disorder: Findings of an expert consensus panel. Early Intervention in Psychiatry. 2024;18(12):1015–1021. doi:10.1111/eip.13547.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *