Implementing Adaptive Designs

Implementing Adaptive Designs: Logistical and Operational Considerations

1  Introduction

The successful conduct of any clinical trial requires cross-departmental coordination. This is true regardless of the trial design. However, it could be argued that the successful implementation of adaptive designs relies even more on integrating input from a number of different line functions, including biometrics, clinical research, data management, drug supply, and clinical operations. This provides both a challenge and a unique opportunity to improve the way we work.

The logistical infrastructure supporting the conduct of a trial obviously reflects the needs of the underlying trial design. The long and successful tradition of “non-flexible” double-blind randomized parallel group designs has led to the development of systems and processes as they are now established across the industry. Supporting adaptive designs with the currently available infrastructure is not impossible, but may be viewed as challenging. Adaptive designs would benefit from building (1) flexibility, and (2) the capability for high speed data acquisition / analysis / reporting into the trial supporting infrastructure. The full benefits of integrated flexibility across the broader infrastructures of trial support will become visible once they are deployed on a portfolio level. Importantly, some of the proposed changes will yield benefits irrespective of whether the design contains elements of adaptation. But systems and processes with enhanced flexibility and speed will clearly also act as an enabler for the deployment of adaptive designs.

While the implementation of integrated platform technology is desirable and beneficial to the conduct of adaptive designs, it is NOT a necessary requirement. We want to stress this, to avoid the misperception that implementing adaptive designs is only possible in a high-tech environment. Nevertheless, it should be acknowledged that advances in technology may potentially hold the key to promoting transformational change to the clinical development paradigm. It is from within this environment that we may see adaptive designs being levered from being a minor player, as they are today, to a major player within early AND late phase clinical development programs.

Advances in technology today, where electronic data capture is increasingly becoming commonplace, is perhaps just the beginning. It is not too difficult to imagine a future where systems are emerging that integrate data capture, monitor recruitment, monitor and trigger the dispensing of drug supplies, and disseminate trial data to inbuilt decision making modules facilitating pre-planned adaptations of certain aspects of the study design. Under such systems, changes to, say, dosage would be triggered in a manner that is not only seamless, but also virtually invisible to the sponsor, patient and investigator. With this in mind, the following must be interpreted within the framework of the rapidly changing world of technology.

2  Scope

The objective of this paper is to facilitate the discussion around the feasibility of using and implementing adaptive designs in practice. The material covered here is relevant for studies in early AND late clinical development. However, the most amenable arena for wider-scale implementation of adaptive designs may initially lie within early development. The motto of “fortiter in re, suaviter in modo” may be amenable to early phase II projects to some degree, whereas later registration type phase II/III studies will require particularly stringent adherence to the recommendations laid out in this paper. Our challenge lies in building credibility around the conduct of adaptive designs, so that this paradigm can gain the acceptance with sponsors AND health authorities that it deserves. Sloppiness in implementation could easily bring the concept into disrepute.

3  Change management

As with any disruptive technology, internal resistance to proposed changes may be significant. Anyone whose routine is disturbed by adaptive methods will have no trouble finding reasons not to pursue adaptive protocols. To manage such resistance and to effectively develop the necessary “can do attitude” within teams we recommend the following:

Education

At a broad departmental level, educating and raising the awareness of all affected stakeholder groups, with an emphasis on laying out the risk/benefits of doing, or not doing adaptive designs. Affected line functions will include statisticians, clinicians, drug supply managers, data managers, clinical operations, internal regulatory groups, trainers and project managers.

Involvement of senior management should be part of this effort, as early adopters of adaptive methodologies are likely to require the unambiguous support of their organization’s senior leaders.

Early identification of opportunities

Adaptive designs require more time for upfront planning and can present a challenge to cross-functional coordination. Early identification of opportunities for adaptive designs within development plans is key to be successful. We recommend identifying the opportunity, and setting the stage for the approach within the first writing of Early Clinical Development Plans, i.e., well before a First-in-human study is conducted. This provides the window to allow for at least 12 months of planning time.

Early Planning

The need for early planning will result in bringing together a cross-functional team at a much earlier timepoint than would otherwise be the case. Building strong interactions between departments will facilitate effective project management and will yield benefits beyond the initial aim of supporting the implementation of adaptive designs. By moving preparatory activities to earlier timepoints in the development process, speedier and streamlined decision making will eventually become possible.

The following departments should be involved in the planning: Clinical Research, Biostatistics, Clinical Pharmacology, Modeling and Simulation (M&S), IT, and, importantly, project management. In our experience the process should be driven by a project statistician, who is, or can be supported by, an expert in clinical trial simulation. With increasing experience there should be economy of scale, resulting in reduced planning time.

4  Identifying opportunities - feasibility checklist

The following issues should be considered during the planning phase: recruitment rate, treatment duration, timing of treatment readouts, endpoints, number of centers, study drug formulation, route of drug administration, positioning of the planned experiment within the development plan. These questions are not independent from each other, and collectively they build a case for or against the possible application of an adaptive design. Ultimately the question is: “What is the most appropriate design at hand to address the research question?” rather than “How can adaptive design elements be integrated into our study at all cost?”

Where are we in the development plan?

Generally, in early development we know less about our compounds resulting in more opportunities for adaptive designs. Conversely, as we progress through the development plan there are fewer research questions left to be answered, resulting in fewer possibilities for adaptations.

Will the study need to be regulatory acceptable?

Regulators at this point in time may be more comfortable with exploring adaptive designs in a phase I/II setting. Including adaptations within a phase III trial will require good and early interactions with health authorities to ensure regulatory support for the chosen approach. Rather than thinking about any one experiment in isolation, we propose to approach the discussion from the perspective of the overall development plan. This is particularly important when considering seamless phase II/III designs (Maca et al., 2006).

Stable population and treatment effects

Ideally the study population and treatment effects should be stable throughout the duration of the trial. However, if we cannot fully achieve exchangeability of patients, there are analytical approaches to mitigate the risk of potential time, center, or other factor effects. Indeed, some adaptive design statistical techniques may have advantages over other statistical approaches, with regard to validity in the presence of non-homogeneity of data caused by unknown factors. These techniques combine results of analyses produced during separate stages of the trial, during which data would naturally be expected to be relatively more homogeneous compared to the trial as a whole. Interpretability of overall results in the absence of exchangeability would remain a concern, however.

Recruitment rate relative to treatment data availability

Before we can adapt, we need to gather the information on which the adaptation will be based. Very fast recruitment may preclude us from the opportunity to learn. There is a tension between our traditional desire to complete recruitment into a study at the earliest possible timepoint, and the recruitment speed which would be optimal for learning within an adaptive design: slower may be better. As a rule of thumb we propose to establish whether the overall recruitment duration is at least four times the observational period required before the primary endpoint reads out in any one patient. For example, in a stroke trial where each patient is observed over a period of 3 months, an adaptive design could be considered if the trial is open for recruitment for 12 months or longer. This view may need to be modified, should there be a good early predictor of final outcome, allowing for the deployment of a longitudinal model. For example, in acute stroke it might be possible to use early measurements of the stroke scale to predict final outcome (Grieve and Krams, 2005). Should the early observation be a good predictor of the final outcome, we may consider using it in our assessment of weighing recruitment speed versus time needed before endpoint readout. To formally establish the “optimal” recruitment speed, we propose to conduct clinical trial simulations, mimicking the potential real life environment of the trial and exploring the impact of longitudinal models.

Adaptive designs in early development, where there is greater uncertainty about design parameters, may consider modulating the recruitment rate by initially opening up fewer centers during a learning phase of the trial.

Potential delays between observing the patient and capturing this information into the database should be added onto the observational time between start of treatment and readout of primary endpoint.

Data capture and the impact on timeliness of data availability

Timely (if not real-time electronic) data capture is an important enabler for adaptive designs. Very fast timelines for data analysis, decision making, and implementation of changes are equally critical and need to be carefully coordinated. The faster data becomes available, the better. Electronic Data Capture (eDC) is an important enabler, and should be considered at least for decision-critical data (e.g. primary endpoint).

Primary endpoint.

How well is the endpoint understood, and is it accepted by regulators? For a confirmatory study a regulatory acceptable endpoint is a must.

Endpoints with delayed readout (e.g. mortality in oncology trials) may appear to preclude considering an adaptive design strategy. However, there may be early predictors of final outcome (e.g., in oncology, assessing tumor size; Berry, 2003).

At least in principle it is conceivable to conduct earlier phase trials on a short term endpoint, but then switch to a more long term endpoint in a later phase, particularly if the correlation between the short and long term endpoint is well understood and strong. In any case we recommend incorporating long and short term endpoints within the same study and across clinical development to assist in building valuable knowledge about the disease and its relevant endpoints, even if this will only benefit future projects.

Study drug

Drug supply management issues at times are perceived as an insurmountable hurdle to the implementation of adaptive designs. In our opinion, the opposite is the case: making use of the supporting infrastructure facilitating adaptive designs can also facilitate the implementation of flexible real-time supply chain management of study drug. This opens up an opportunity to minimize drug supply wastage (see below).

The following questions should be addressed:

1. What is the cost of goods?
2. Will central randomization be available? (a necessary requirement)
3. Will the treatment be acute or chronic? If chronic, will the dose level be fixed?
4. How many dose levels will we require? What is the route of administration and how easy is it to constitute different dose strengths?
5. How can we ensure optimal blinding?
6. Should the formulation of the study drug represent the eventually marketed product (for confirmatory phase III trials) or could the formulation used in the study be different from the eventually marketed product?
7. How many centers/drug supply depots will be involved? Where are they based?

Identifying the most appropriate formulation, dosing regimen, or route of administration may be a research question in its own right during early development, and is open to potential adaptive learning procedures.

Producing different dose-levels is easily possible by offering several dose strengths and combining two or more of them. For instance, tablets of dose strengths 0x, 1x, 3x and 4x allow for equidistant dose progression from 0x to 8x when two tablets are combined at a time. Or for intravenous study drugs, vials of equal volume in dose strengths 0x, 1x and 3x, together with an instruction on how much volume to take out of each vial, allow the preparation of a very large number of different dose levels. Doing all of this in a blinded fashion is easily achieved by concealing the dose-strength of the tablet/vial used to the pharmacist who prepares the study medication. Dispensing labeled blister packs may be a convenient and cost-effective alternative.

The quality of existing information from clinical trials in the indication

How well do we understand the disease and the endpoints we propose to use for decision-making purposes in our study? The better the quantity and quality of existing information, the easier it will be to run meaningful clinical trial simulations evaluating different scenarios. For Bayesian approaches there needs to be a discussion on which prior distributions to use and which information value to attribute to them.

Relative values of efficacy and toxicity for the indication and compound

Adaptive designs have great potential in settings where there is concern over toxicity. When there is a risk of toxicity, generally more doses need to be explored to find an efficacious dose that also has tolerable toxicity.

5  If an adaptive design is feasible . . . what next?

Simulation guided clinical trial design – scenario planning

Simulation is a key part of early planning, and requires time and resources. Simulating different design scenarios, including comparisons to standard designs, helps improve the operational design characteristics and will facilitate a decision as to which design to deploy. Large scale simulations will allow us to describe the type I and type II errors. Other questions which can be addressed through simulation include: