Sanjay Ahuja, PhD
Reltronic, Inc.
Where a condition is rare enough that randomization is infeasible or unethical, sponsors increasingly propose an external control arm. Regulatory receptivity has grown, but it remains conditional, and the conditions are frequently misunderstood.
The regulatory frame is guidance, not rule
Two documents govern. ICH E10, on the choice of control group, establishes that the control must be considered in the context of available standard therapies, the adequacy of the evidence supporting the chosen design, and the applicable ethical considerations. In the United States, the Food and Drug Administration issued Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products on 1 February 2023, with the comment period closing on 2 May 2023.
One feature of that document is routinely overlooked in commercial discussion. It was issued in draft, states on its face that it is not for implementation, and contains non-binding recommendations. Sponsors designing against it are therefore designing against a stated direction of travel rather than a settled requirement. The practical consequence is that early engagement with the review division carries more weight in this design than in a conventional randomized study, because the acceptability of the comparison group is a matter for discussion rather than a matter already determined by rule.
The three conditions, which must hold together
External control is generally considered acceptable where randomization is not feasible or ethical and three circumstances hold together. The conjunction matters. Sponsors sometimes present a strong case on one condition and treat the remaining two as satisfied by implication, which is the most common structural weakness in these submissions.
| Condition | What must be demonstrated |
|---|---|
| Disease severity and unmet need | The condition is life-threatening or seriously debilitating and no satisfactory alternative therapy exists. |
| Predictable natural history | The untreated course is well characterized and highly predictable, so the counterfactual outcome can be established with confidence. |
| Effect size and temporality | The anticipated effect is substantial and self-evident, and closely related in time to administration, such that it is implausibly attributable to confounding. |
The second condition carries most of the weight and is the one most frequently overstated. A natural history described in the literature is not the same as a natural history characterized well enough to serve as a counterfactual for an individual patient. Where the condition is heterogeneous, or where the rate of progression varies substantially between patients, the requirement is not met by aggregate description however extensive.
Where external controls come from in practice
Published surveys of external controls used in regulatory decision making show a distribution that is informative in itself.
| Source | Share | Principal limitation |
|---|---|---|
| Retrospective natural history, including record review | 44% | Ascertainment differs from trial conditions |
| Baseline control, patient serving as own comparator | 33% | Confounded by regression to the mean and secular trend |
| Published data | 11% | Patient-level covariates unavailable |
| Data from a previous clinical study | 11% | Eligibility criteria and endpoints rarely align |
The largest category, retrospective review of medical records, is also the category in which the comparability problem is most acute and most tractable. It is acute because records are generated for clinical rather than research purposes, so the timing, frequency and definition of measurement differ systematically from the trial. It is tractable because the underlying source data exist and can, in principle, be interrogated to establish how far they differ.
Six failure modes
Most regulatory objections to an external control arm reduce to one of the following. None is primarily a question of statistical technique.
- Differential ascertainment. Outcomes in the trial arm are measured on protocol at fixed intervals; outcomes in the external arm are measured when a patient happened to present. Apparent differences in event timing may reflect observation schedule rather than biology.
- Outcome definition drift. The endpoint as defined in the protocol does not correspond to what was recorded in the source data, particularly where the external data span a period in which diagnostic criteria or coding practice changed.
- Unmeasured confounding at baseline. Covariates known to affect prognosis are absent from the external source, so no adjustment can be made and none can be shown to be unnecessary.
- Secular trend in standard of care. Supportive management improved between the period from which the external control is drawn and the trial period, so part of any observed benefit is attributable to era rather than to treatment.
- Selection into the external cohort. Patients appearing in a registry or specialist center record differ systematically from the trial population, frequently by severity, and the direction of that difference is not always predictable.
- Immortal time. The external control accrues follow-up from a point that is not equivalent to randomization, creating survival time in one arm with no counterpart in the other.
What to establish before construction, not after
The recurring error in these programs is sequential rather than analytical. A comparison group is constructed, differences are discovered, and statistical adjustment is applied to close them. Adjustment can correct for measured imbalance. It cannot correct for imbalance in what was measured, or when, or by whom, and it is that class of difference which generates most objections.
The following eight questions are answerable before a comparison group is built, and considerably more expensive to answer afterward.
- Is the untreated course of this condition characterized at the level of the individual patient, or only in aggregate?
- Over what period were the external data generated, and what changed in supportive care across that period?
- How was the endpoint ascertained in the external source, at what frequency, and by whom?
- Which baseline covariates known to affect prognosis are present in the external source, and which are absent?
- What proportion of the external cohort would have met the trial’s eligibility criteria had they been applied?
- From what index point does follow-up accrue in each arm, and are those points equivalent?
- Can the derivation of every value in the comparison group be traced to a source record and reproduced independently?
- Has the review division been consulted on the proposed comparison group, and at what stage?
The question is provenance, not statistics
External control is a legitimate design in the circumstances the guidance describes, and in a sufficiently rare condition it is frequently the only design available. Its acceptability turns on a narrower question than is generally supposed. Not whether the comparison group can be adjusted to resemble the trial arm, but whether what was measured in each, and when, and under what definition, is comparable in the first place.
That question is one of provenance rather than of statistics. It is settled by documentation of how each value came to exist, which is available at the point the comparison group is designed and progressively less available thereafter.
Sources
- United States Food and Drug Administration. Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products. Draft guidance for industry, issued 1 February 2023; comment period closed 2 May 2023. Draft status, non-binding recommendations.
- International Council for Harmonisation. ICH E10, Choice of Control Group and Related Issues in Clinical Trials.
- The Use of External Controls in FDA Regulatory Decision Making. Therapeutic Innovation and Regulatory Science, 2021.
Disclosure: the author is affiliated with Reltronic, Inc. This paper makes no claim regarding any Reltronic product and describes no proprietary method. It is not regulatory advice; sponsors should engage the relevant review division directly on the acceptability of any proposed comparison group. The principal United States guidance cited was in draft at the time of writing and its status should be verified before reliance.
Leave a Reply