Restricting an analysis to people who took more study medication can increase an observed treatment difference. It can also change which people are being compared. Those two possibilities must be considered together.
A pilot trial within opioid treatment
Schottenfeld and colleagues’ 2018 Malaysian pilot randomized men with opioid and amphetamine-type stimulant dependence to atomoxetine or placebo alongside buprenorphine/naloxone and counseling. Sixty-six were randomized; one participant without follow-up data was excluded from the reported intention-to-treat analysis. After inpatient stabilization, outpatient follow-up lasted 16 weeks. Oral atomoxetine began at 40 mg/day and increased to 80 mg/day after seven days.
During weeks 9–16, estimated stimulant-negative urine proportions were 0.77 versus 0.67, with Cohen’s d=0.26. The overall medication comparison was nonsignificant (p=0.41). Among participants reporting at least 60 days of study-medication use, the proportions were 0.90 versus 0.64 and d=0.56; that subgroup comprised 12 atomoxetine and 16 placebo participants. Self-reported stimulant-abstinent days did not differ overall (p=0.42).
What changed when adherence defined the sample?
Adherence was self-reported, urine testing was weekly, and the study population was all male and receiving opioid treatment. The outcome concerned amphetamine-type stimulants, not a validated methamphetamine-only consumed dose.
The adherence subgroup was defined using behavior occurring after assignment. Randomization of treatment does not automatically randomize adherence. Participants who continued medication may differ from those who did not in motivation, attendance, symptoms or other determinants of the outcome.
A larger subgroup estimate therefore cannot, by itself, establish how much benefit every patient would receive if adherence improved. That causal question requires a design or analysis capable of addressing the reasons adherence differed. Simply removing less adherent people does not solve it.
Effect size, significance and the specimen outcome
Cohen’s d describes a standardized difference. It is not the percentage of participants cured, the probability that an individual will benefit or a direct measure of milligrams avoided. A pilot estimate can inform the planning of a larger trial while remaining too uncertain for a definitive efficacy claim.
The urine outcome also needs its original name. A proportion of negative collected tests describes observations under the study’s schedule. It should not be restated as the proportion of all days biologically verified drug-free, especially when testing is intermittent.
For a research-based opinion, the defensible account places the small overall estimate, the nonsignificant overall comparison and the selected adherence result together. This preserves the reason for further investigation without treating the subgroup as an independent confirmation. The historical study does not establish a current medication recommendation or generalize automatically to women, people without opioid dependence or settings without the accompanying treatment.
Source
Schottenfeld RS, et al. Atomoxetine for amphetamine-type stimulant dependence during buprenorphine treatment: A randomized controlled trial. Drug Alcohol Depend. 2018;186:130–137. doi:10.1016/j.drugalcdep.2018.01.017. Full text.