Design considerations and best practices in clinical research

The increased focus on precision medicine in oncology and rare diseases has led to a marked increase of single-arm trials (SATs). While randomised clinical trials (RCTs) are still the gold standard for drug approval studies, there is consensus that SATs have their place in clinical drug development as well.

However, the primary disadvantage of SATs is that, unlike RCTs, they do not include results from an internal control group. To supplement this absence of a control group, researchers can incorporate data gathered from external sources (eg, real-world data [RWD] like registries and electronic health records [EHRs]) to act as an external comparator arm ([ECA]).

How ECA studies work
An ECA typically uses available patient-level data to establish a cohort of patients who are comparable to those in the SAT group. It attempts to approximate the function of an RCT control arm (or augments an existing randomly allocated control arm to strengthen the generated evidence).

Researchers employing ECAs can utilise prospective or retrospective data for the comparator arm. One option is to gather data from an existing prospective natural history study or another RWD source. Another option is to plan a new prospective study, potentially even after the SAT data has already been collected. The former option allows for faster data collection; the latter provides the ability to fully customise data collection.

However, because ECA studies join data from two different sources (ie, the comparator arm and the SAT), there are several considerations prior to initiating a study to ensure scientifically valid findings.

Best ECA practices
A successful ECA study requires researchers to include methodologies that support impeccable conduct and minimise potential bias. Some of the five best practices include:

1. Meticulous sample size determination
   Determining the correct sample size is critical in RCTs for methodological and ethical reasons and for allocating human and financial resources. An appropriate sample size increases the likelihood of validity of efficacy findings and that the trial delivers a positive return on sponsor investment.

However, when it comes to ECA studies, researchers must also consider additional factors, such as:

• Finding out whether data already exists for one treatment arm. In instances where SAT results are already available, conditional power arguments might apply. Researchers may then use percentiles to control error probabilities instead of means due to skewed distributions. If individual SAT patient data exists, they can be used directly to perform customised power simulations for the ECA study
• Determining whether to increase conservativeness during calculation. When researchers calculate the required ECA sample size, they can apply conservativeness to combat the variable nature of RWD. Patients in an ECA may be less readily available, data quality could be inadequate, or the RWD might show a higher level of missing data in outcomes and covariates. Also, the application of causal inference methods typically lead to sample size increase.
2. Astute use of estimands
   An estimand precisely defines what needs to be estimated to address a specific scientific question of interest. It includes the specifics of the treatment, population, endpoints, population-level summary and handling of intercurrent events.

Implementing best ECA practices requires researchers to deploy best estimand strategies for identifying the treatment effect. Treatment specifics include a potential requirement of a minimum treatment exposure in the ECA. Regarding populations, many RWD sources document comparator patients who actually received the treatment, while those who were intended to be treated (ITT) are excluded. If a study uses the ITT population in the treatment group but not in the comparator arm, potential bias is introduced.

Endpoints must be suitable for the indication but also for the concrete set-up of the SAT-ECA comparison. The population-level summary may be specifically tailored for the ECA study, for example preferring restricted mean survival times over the Cox proportional hazards model. Intercurrent events like subsequent treatments being differently distributed across cohorts can be handled within the estimand framework as well.

3. Consistent variable data definitions
   To achieve accurate and unbiased results, researchers should establish baseline information that indicates near-identical variable definitions of both cohorts. To establish consistent definitions across data sources, the index date or the official trial commencement date should typically correlate with the start of treatment, not with the SAT enrolment date.

Inclusion and exclusion criteria must be defined as consistently as possible for both the treatment and comparator group. For example, when lines of treatment in site-based RWD sources are numbered based on a clinician’s judgment, the applied counting algorithm might differ across sites. To achieve consistency across all data sets, researchers must monitor and potentially reclassify the lines of treatment to harmonise definitions. This situation could also arise from other baseline data having differential definitions/measurement techniques.

4. Coherent differential measurement timing
   Even when data is derived from diverse sources using the same measurement methods, it’s important for researchers to verify whether the time points for the measurements are consistent. Unless patients in an ECA study are monitored at the same intervals as those in the clinical setting, endpoint data could differ substantially.

Employment of best practices in an ECA include the application of interval-censored data analyses for time-to-event endpoints, such as survival, progression-free survival, duration of response, or time to next treatment or death. For other types of endpoints, visiting windows enable researchers to harmonise measurement timings and compare like with like.

5. Successful avoidance of anomalies
   To ensure best practices in ECA studies, the presence of missing values, unmeasured covariates, and other anomalies must already be considered at the study design phase. Statistically, much can be done in ECA studies, but missing values and unmeasured confounding variables are always hard to tackle and may lead to general limitations that cannot be saved by applying sophisticated statistical methods.

It is critical that a data set that is utilised for the ECA has important covariates to be non- missing or at least available with the lowest possible amount of missing data. Upfront feasibility should investigate and report the amount of anticipated missing data to provide an informed basis for decision-making about whether a data source is fit for purpose. ECAs should always apply a series of sensitivity analyses to check on the robustness of study results, especially when it comes to missing data handling.

A helpful alternative
RCTs remain the gold standard for drug development, where feasible, and SATs remain an important component of clinical research as well. Given the limitation of an absent control group in SATs, ECA studies provide a helpful way to contextualise findings, especially in oncology and rare diseases. However, ECA studies are complex and following best practices as described above is necessary to pave the way for successful and accepted evidence generation.

Gerd Rippin is Director Biostatistics, Real World Solutions at IQVIA

Digital inspiration for pharma on mobile, social media, strategy, best practice, regulations and more