Single-arm designs with a performance goal

Several other vignettes focus on two-arm randomized designs, although the Bayesian binary outcome vignette also includes a single-arm example. Single-arm trials – in which every subject receives the experimental therapy and the comparator is an external benchmark, often called a performance goal (PG) or objective performance criterion (OPC) – are common in early-phase oncology, rare-disease, and proof-of-concept studies. This vignette shows how to set up a Goldilocks single-arm design with survival_adapt().

The decision rule

In a single-arm trial there is no concurrent control, so the estimand is the cumulative event probability in the treatment arm. Let \(\tau\) denote end_of_study:

\[p_{\text{treatment}} = \Pr(\text{event by } \tau \mid \text{data}).\]

The argument h0 plays the role of a benchmark on this scale: a target failure probability (or, equivalently, \(1 - h_0\) is a target survival probability) drawn from external evidence such as a published rate, registry, or historical cohort. In clinical-trial terminology this benchmark may be referred to as a performance goal (PG) or objective performance criterion (OPC). With alternative = "less" and prob_ha, the trial declares success when

\[\Pr(p_{\text{treatment}} < h_0 \mid \text{data}) \;>\; \texttt{prob\_ha},\]

i.e. when the posterior assigns enough mass to “the experimental therapy has a lower failure rate than the benchmark”. Choosing alternative = "greater" reverses the direction; alternative = "two.sided" is not allowed for method = "bayes-surv".

At each interim look, prior_surv is updated with observed events and exposure to predict remaining follow-up. Each hypothetical completed dataset is then analyzed with prior_surv_final, the prior used for the actual final Bayesian survival analysis. The fraction that passes the posterior success threshold drives the futility (Fn) and expected-success (Sn) stopping rules. The default Qn = 1 disables the optional immediate-success rule in these examples.

The default prior_surv_final = prior_surv uses the same prior for prediction and analysis. To bring external evidence into prediction while using a weak analysis prior, specify both explicitly. h0 remains the fixed performance goal; it is separate from either prior. See the observed-interim example for a worked specification of the two prior roles.

Setting up the design

Suppose the existing standard of care has a 30% event probability by 24 months, and we are testing a new agent that we hope will reduce this to 20%. We use an interim look at 50 of 80 enrolled subjects:

end_of_study <- 24
benchmark <- 0.30                       # external standard-of-care failure rate
target    <- 0.20                       # rate we hope the new therapy achieves

# Convert the target failure rate into a constant hazard (so we can simulate)
ht <- prop_to_haz(probs = target, endtime = end_of_study)
ht
#> [1] 0.009297648

We set prop_loss = 0.05: independently of event time, dropout time is exponential with rate \(-\log(0.95)/24\) per month, giving a 5% dropout CDF at 24 months. The observed proportion censored by dropout can be lower because events can occur first; it is not a fixed count of four losses among 80 participants. Survival analyses retain the follow-up observed before dropout.

Now we run survival_adapt():

out <- survival_adapt(
  hazard_treatment = ht,
  hazard_control   = NULL,              # single-arm
  cutpoints        = NULL,
  N_total          = 80,
  lambda           = 5,                 # enrollments per month (constant)
  lambda_time      = NULL,
  interim_look     = 50,
  end_of_study     = end_of_study,
  prior_surv            = c(0.1, 0.1),       # Gamma(0.1, 0.1) on the hazard
  prop_loss        = 0.05,
  alternative      = "less",
  h0               = benchmark,         # benchmark failure probability
  Fn               = 0.05,
  Sn               = 0.95,
  prob_ha          = 0.95,
  N_impute         = 50,
  N_mcmc           = 2000,
  method           = "bayes-surv")

out
#>   prob_threshold margin alternative N_treatment N_control N_enrolled N_max
#> 1           0.95    0.3        less          80         0         80    80
#>   post_prob_ha est_final ppp_success stop_futility stop_immediate_success
#> 1       0.9875 0.1910975        0.08             0                      0
#>   stop_expected_success trial_success     stopping_reason decision_time
#> 1                     0          TRUE maximum_sample_size      39.05894
#>   accrual_stop_time analysis_ready_time planned_completion_time
#> 1          15.05894            39.05894                39.05894
#>   followup_person_time peak_active_followup
#> 1             1702.955                   74

There is no need to supply block or rand_ratio: they are redundant in a single-arm design because no randomization is performed.

The principal trial-level quantities are:

N_control = 0: no concurrent control was simulated.
margin = 0.30: this is the value of h0 that the trial is testing against. Note that it is on the cumulative-failure scale, not the survival scale.
est_final is the posterior mean of \(p_{\text{treatment}}\) at end_of_study, not a treatment effect relative to control.
post_prob_ha is the posterior probability that \(p_{\text{treatment}} < h_0\).

Operating characteristics

A single trial replicate does not establish whether the design is well calibrated. Power and type I error are estimated by repeated simulation under alternative and null scenarios, respectively. The following analyses are not evaluated in the vignette because they require several minutes:

# Power: simulate under the alternative (true rate = 0.20)
out_power <- sim_trials(
  N_trials         = 1000,
  hazard_treatment = ht,
  hazard_control   = NULL,
  cutpoints        = NULL,
  N_total          = 80,
  lambda           = 5,
  lambda_time      = NULL,
  interim_look     = 50,
  end_of_study     = end_of_study,
  prior_surv            = c(0.1, 0.1),
  prop_loss        = 0.05,
  alternative      = "less",
  h0               = benchmark,
  Fn               = 0.05,
  Sn               = 0.95,
  prob_ha          = 0.95,
  N_impute         = 50,
  N_mcmc           = 2000,
  method           = "bayes-surv",
  return_trace     = TRUE,
  seed             = 3082)

# Type I error: simulate under the null (true rate = benchmark/PG/OPC = 0.30)
ht_null <- prop_to_haz(probs = benchmark, endtime = end_of_study)
out_t1error <- sim_trials(
  N_trials         = 1000,
  hazard_treatment = ht_null,
  hazard_control   = NULL,
  cutpoints        = NULL,
  N_total          = 80,
  lambda           = 5,
  lambda_time      = NULL,
  interim_look     = 50,
  end_of_study     = end_of_study,
  prior_surv            = c(0.1, 0.1),
  prop_loss        = 0.05,
  alternative      = "less",
  h0               = benchmark,
  Fn               = 0.05,
  Sn               = 0.95,
  prob_ha          = 0.95,
  N_impute         = 50,
  N_mcmc           = 2000,
  method           = "bayes-surv",
  seed             = 3083)

oc <- summarise_sims(list(
  "target event probability" = out_power,
  "benchmark event probability" = out_t1error
))
effect_by_scenario <- c(
  "target event probability" = target,
  "benchmark event probability" = benchmark
)
oc$true_event_probability <- unname(effect_by_scenario[oc$scenario])

oc
plot_sim_ocs(
  oc,
  effect = "true_event_probability",
  xlab = "True treatment event probability"
)
plot_sim_stopping(out_power)
plot_sim_decisions(out_power)

The operating-characteristic plot compares scenarios on the clinically natural event-probability scale. The stopping plot then expands one scenario to show where enrollment ends and why. Finally, the decision map uses the retained traces to show how interim predictive probabilities fall relative to the expected-success and futility thresholds. For large calibration grids, retain traces only for scenarios whose interim behavior needs closer inspection.

Calibration proceeds as for two-arm designs. Screen candidate prob_ha values under the null, where the true event probability equals the benchmark, and validate the selected design with a fresh seed and adequate Monte Carlo precision. If sample size or the stopping thresholds change to improve power, reassess type I error for the revised design. The calibration vignette demonstrates this workflow; its log-rank threshold is specific to that example.

A practical caveat on benchmarks

The validity of a single-arm Goldilocks trial rests entirely on the benchmark h0 (the PG or OPC) being a fair representation of the population the trial is enrolling. Drift in standard of care, differences in patient mix, and unmeasured confounding all bias the comparison in a way that randomization would otherwise neutralize. A Bayesian framework can incorporate uncertainty about the benchmark itself – e.g. by replacing a fixed h0 with a prior distribution informed by historical data – but this is outside the scope of the simple h0 scalar that survival_adapt() exposes, and would require a custom analysis. When in doubt, simulating the design under several plausible values of the true rate (including ones near the benchmark) is a useful way to characterize its sensitivity.

Single-arm designs with a performance goal

The decision rule

Setting up the design

Operating characteristics

A practical caveat on benchmarks

See also