Adaptive Subgroup Selection in Sequential Trials

Introduction

ASSISTant is an R package for Adaptive Subgroup Selection In Sequential Trials. This vignette reproduces all the simulations in the original paper of Lai, Lavori and Liao (2014).

NOTE The number of simulations has been drastically reduced in this vignette in order to avoid taxing CRAN servers. The full_doc sources contain the complete sources and output; see files in the directory

system.file("full_doc", package="ASSISTant")

library(ASSISTant)
data(LLL.SETTINGS)
str(LLL.SETTINGS)

## List of 3
##  $ trialParameters:List of 4
##   ..$ N         : num [1:3] 300 400 500
##   ..$ type1Error: num 0.05
##   ..$ eps       : num 0.5
##   ..$ type2Error: num 0.2
##  $ scenarios      :List of 11
##   ..$ S0 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S1 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0.3 0 0.3 0 0.3 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S2 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0.3 0 0.3 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S3 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.3 0 0.3 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S4 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0.3 0 0.3 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S5 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.6 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S6 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0 0 0 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S7 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##   ..$ S8 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 2 1 1.5 1 1 1 0.5 1 0.5 ...
##   ..$ S9 :List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0.5 0 0.4 0 0.3 0 0 0 0 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 0.5 1 1 1 1.5 1 2 1 2 ...
##   ..$ S10:List of 2
##   .. ..$ mean: num [1:2, 1:6] 0 0 0 0 0 0 0 0.3 0 0.4 ...
##   .. ..$ sd  : num [1:2, 1:6] 1 1 1 1 1 1 1 1 1 1 ...
##  $ prevalences    :List of 2
##   ..$ table1: num [1:6] 0.167 0.167 0.167 0.167 0.167 ...
##   ..$ table2: num [1:6] 0.2 0.1 0.3 0.1 0.1 0.2

The LLL.SETTINGS list contains all the scenarios described in the paper.

Table 1 Results

Scenario S0

This is the null setting.

scenario <- LLL.SETTINGS$scenarios$S0
designParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
designA <- ASSISTDesign$new(trialParameters = LLL.SETTINGS$trialParameters,
                            designParameters = designParameters)
print(designA)

## Design Parameters:
##  Number of Groups: 6
##  Prevalence:
## 
## |    Group1|    Group2|    Group3|    Group4|    Group5|    Group6|
## |---------:|---------:|---------:|---------:|---------:|---------:|
## | 0.1666667| 0.1666667| 0.1666667| 0.1666667| 0.1666667| 0.1666667|
## 
##  Using Discrete Rankin scores? FALSE
## 
##  Normal Rankin Distribution means (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      0|      0|      0|      0|      0|      0|
## |Alt  |      0|      0|      0|      0|      0|      0|
## 
##  Normal Rankin Distribution SDs (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      1|      1|      1|      1|      1|      1|
## |Alt  |      1|      1|      1|      1|      1|      1|
## 
## Trial Parameters:
## List of 5
##  $ N         : num [1:3] 300 400 500
##  $ type1Error: num 0.05
##  $ eps       : num 0.5
##  $ type2Error: num 0.2
##  $ effectSize: num 0.0642
## 
## Boundaries:
## 
## 
## |    btilde|        b|        c|
## |---------:|--------:|--------:|
## | -1.460993| 2.390404| 2.491775|

result <- designA$explore(numberOfSimulations = 50, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.020000; P(Reject H0_subgp) = 0.000000; P(Reject H0) = 0.020000
## P(Early stop for efficacy [futility]) = 0.020000 [0.640000]
## Mean [SD] Randomized N = 414.000000 [72.871147]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.20|
## |         2|       0.46|
## |         3|       0.34|
## 
## Mean [SD] Lost N = 183.880000 [85.293644]
## Mean [SD] Analyzed N = 230.120000 [92.989716]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 250.94118|  4.955686|
## |             1|             2| 201.42857|  6.241184|
## |             1|             3| 152.33333|  3.511885|
## |             1|             4| 107.66667|  6.027714|
## |             1|             5|  52.80000|  4.266146|
## |             2|             1| 328.00000| 12.727922|
## |             2|             2| 267.33333| 25.026653|
## |             2|             5|  71.66667| 15.143756|
## |             3|             3| 232.00000|        NA|
## |             3|             4| 161.50000|  9.192388|
## |             3|             5|  82.00000| 12.529964|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     6|     1|       0.02|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 17|  7|  3|  3|  5|
## |             2|  2|  3|  0|  0|  3|
## |             3|  0|  0|  1|  2|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.98|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |        1|            19|             0|
## |        1|            10|             0|
## |        1|             4|             0|
## |        1|             5|             0|
## |        1|            11|             0|
## |        0|             1|             1|
## NULL

Alternative Scenario S1

scenario <- LLL.SETTINGS$scenarios$S1
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.780000; P(Reject H0_subgp) = 0.040000; P(Reject H0) = 0.820000
## P(Early stop for efficacy [futility]) = 0.700000 [0.000000]
## Mean [SD] Randomized N = 372.000000 [90.441096]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.58|
## |         2|       0.12|
## |         3|       0.30|
## 
## Mean [SD] Lost N = 45.500000 [112.449762]
## Mean [SD] Analyzed N = 326.500000 [94.545368]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|       sd|
## |-------------:|-------------:|--------:|--------:|
## |             1|             5|  48.0000|       NA|
## |             2|             1| 330.0000|       NA|
## |             3|             1| 415.6667| 8.504900|
## |             3|             3| 246.0000|       NA|
## |             3|             5|  80.8000| 8.228001|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     3|     1|       0.02|
## |     6|    39|       0.78|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  0|  0|  0|  0|  1|
## |             2|  1|  0|  0|  0|  0|
## |             3|  3|  0|  1|  0|  5|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.18|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |     0.75|             4|             1|
## |     0.00|             1|             1|
## |     1.00|             6|             0|
## |     0.00|            39|            39|
## NULL

Alternative Scenario S2

scenario <- LLL.SETTINGS$scenarios$S2
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.400000; P(Reject H0_subgp) = 0.320000; P(Reject H0) = 0.720000
## P(Early stop for efficacy [futility]) = 0.420000 [0.000000]
## Mean [SD] Randomized N = 436.000000 [82.709228]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.22|
## |         2|       0.20|
## |         3|       0.58|
## 
## Mean [SD] Lost N = 122.160000 [123.479381]
## Mean [SD] Analyzed N = 313.840000 [100.538014]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 254.0000|  2.828427|
## |             1|             2| 199.0000|  8.485281|
## |             1|             3| 141.5000| 21.920310|
## |             1|             4|  96.0000| 11.313709|
## |             1|             5|  50.0000|        NA|
## |             2|             4| 133.0000|  5.656854|
## |             3|             1| 423.5000|  7.778175|
## |             3|             2| 334.3333|  7.505554|
## |             3|             3| 241.2500|  6.701990|
## |             3|             4| 167.6667|  5.267827|
## |             3|             5|  87.0000|        NA|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     2|       0.04|
## |     2|     4|       0.08|
## |     3|     4|       0.08|
## |     4|     6|       0.12|
## |     6|    20|       0.40|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  2|  2|  2|  2|  1|
## |             2|  0|  0|  0|  2|  0|
## |             3|  2|  3|  4|  9|  1|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.28|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.5000000|             4|             2|
## | 0.2000000|             5|             4|
## | 0.3333333|             6|             4|
## | 0.5384615|            13|             6|
## | 1.0000000|             2|             0|
## | 0.0000000|            20|            20|
## NULL

Alternative Scenario S3

scenario <- LLL.SETTINGS$scenarios$S3
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.120000; P(Reject H0_subgp) = 0.420000; P(Reject H0) = 0.540000
## P(Early stop for efficacy [futility]) = 0.260000 [0.020000]
## Mean [SD] Randomized N = 460.000000 [69.985421]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.12|
## |         2|       0.16|
## |         3|       0.72|
## 
## Mean [SD] Lost N = 223.540000 [126.875837]
## Mean [SD] Analyzed N = 236.460000 [122.725546]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|         sd|
## |-------------:|-------------:|--------:|----------:|
## |             1|             1| 251.6667|  4.3204938|
## |             1|             2| 199.7500|  7.8898669|
## |             1|             3| 142.0000|  8.4852814|
## |             1|             4| 108.0000|         NA|
## |             1|             5|  49.0000|         NA|
## |             2|             1| 335.7500|  6.7515430|
## |             2|             2| 267.0000|         NA|
## |             2|             3| 209.0000|         NA|
## |             2|             5|  65.0000|         NA|
## |             3|             1| 416.8000|  5.8480766|
## |             3|             2| 334.3750|  9.8841504|
## |             3|             3| 261.6667| 11.5902258|
## |             3|             4| 166.8000|  9.8843310|
## |             3|             5|  82.5000|  0.7071068|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    12|       0.24|
## |     2|     6|       0.12|
## |     3|     2|       0.04|
## |     5|     1|       0.02|
## |     6|     6|       0.12|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  6|  4|  2|  1|  1|
## |             2|  4|  1|  1|  0|  1|
## |             3|  5|  8|  3|  5|  2|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.46|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.2000000|            15|            12|
## | 0.5384615|            13|             6|
## | 0.6666667|             6|             2|
## | 1.0000000|             6|             0|
## | 0.7500000|             4|             1|
## | 0.0000000|             6|             6|
## NULL

Alternative Scenario S4

scenario <- LLL.SETTINGS$scenarios$S4
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.060000; P(Reject H0_subgp) = 0.100000; P(Reject H0) = 0.160000
## P(Early stop for efficacy [futility]) = 0.040000 [0.180000]
## Mean [SD] Randomized N = 466.000000 [68.838765]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.12|
## |         2|       0.10|
## |         3|       0.78|
## 
## Mean [SD] Lost N = 146.480000 [84.999457]
## Mean [SD] Analyzed N = 319.520000 [101.819015]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 251.80000|  1.483240|
## |             1|             4| 100.25000|  8.811518|
## |             1|             5|  52.66667|  1.527525|
## |             2|             4| 130.00000|  8.366600|
## |             2|             5|  66.50000| 24.748737|
## |             3|             1| 416.00000|        NA|
## |             3|             3| 251.66667|  8.524475|
## |             3|             4| 170.91667| 11.712141|
## |             3|             5|  79.16667|  5.980524|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     3|     1|       0.02|
## |     4|     1|       0.02|
## |     5|     2|       0.04|
## |     6|     3|       0.06|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  5|  0|  0|  8|  3|
## |             2|  0|  0|  0|  4|  2|
## |             3|  1|  0|  6| 12|  6|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.84|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.8333333|             6|             1|
## | 0.8333333|             6|             1|
## | 0.9583333|            24|             1|
## | 0.8181818|            11|             2|
## | 0.0000000|             3|             3|
## NULL

Alternative Scenario S5

scenario <- LLL.SETTINGS$scenarios$S5
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.140000; P(Reject H0_subgp) = 0.660000; P(Reject H0) = 0.800000
## P(Early stop for efficacy [futility]) = 0.460000 [0.000000]
## Mean [SD] Randomized N = 444.000000 [67.491496]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.10|
## |         2|       0.36|
## |         3|       0.54|
## 
## Mean [SD] Lost N = 270.120000 [141.724314]
## Mean [SD] Analyzed N = 173.880000 [126.469427]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 248.6923|  5.807820|
## |             1|             2| 206.5000|  2.121320|
## |             1|             3| 157.0000|  4.242641|
## |             1|             4| 104.5000|  4.949747|
## |             2|             1| 333.3333|  4.457204|
## |             3|             1| 417.3125|  5.313113|
## |             3|             2| 330.0000| 12.727922|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    30|       0.60|
## |     2|     1|       0.02|
## |     3|     1|       0.02|
## |     4|     1|       0.02|
## |     6|     7|       0.14|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 13|  2|  2|  2|  0|
## |             2|  6|  0|  0|  0|  0|
## |             3| 16|  2|  0|  0|  0|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |     0.2|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1428571|            35|            30|
## | 0.7500000|             4|             1|
## | 0.5000000|             2|             1|
## | 0.5000000|             2|             1|
## | 0.0000000|             7|             7|
## NULL

Alternative Scenario S6

scenario <- LLL.SETTINGS$scenarios$S6
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.060000; P(Reject H0_subgp) = 0.720000; P(Reject H0) = 0.780000
## P(Early stop for efficacy [futility]) = 0.480000 [0.040000]
## Mean [SD] Randomized N = 436.000000 [69.282032]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.12|
## |         2|       0.40|
## |         3|       0.48|
## 
## Mean [SD] Lost N = 254.060000 [123.124196]
## Mean [SD] Analyzed N = 181.940000 [115.743808]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 249.85714|  7.336496|
## |             1|             2| 189.66667|  5.507570|
## |             1|             4| 102.66667| 11.718931|
## |             1|             5|  49.00000|  8.485281|
## |             2|             1| 330.42857|  6.106203|
## |             2|             2| 267.00000| 14.142136|
## |             3|             1| 416.55556|  3.574602|
## |             3|             2| 336.75000| 12.685293|
## |             3|             5|  95.66667|  7.023769|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    26|       0.52|
## |     2|     6|       0.12|
## |     4|     1|       0.02|
## |     5|     3|       0.06|
## |     6|     3|       0.06|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 14|  3|  0|  3|  2|
## |             2|  7|  2|  0|  0|  0|
## |             3|  9|  4|  0|  0|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.22|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1333333|            30|            26|
## | 0.3333333|             9|             6|
## | 0.6666667|             3|             1|
## | 0.4000000|             5|             3|
## | 0.0000000|             3|             3|
## NULL

Alternative Scenario S7

scenario <- LLL.SETTINGS$scenarios$S7
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.420000; P(Reject H0_subgp) = 0.480000; P(Reject H0) = 0.900000
## P(Early stop for efficacy [futility]) = 0.460000 [0.000000]
## Mean [SD] Randomized N = 428.000000 [85.809471]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.26|
## |         2|       0.20|
## |         3|       0.54|
## 
## Mean [SD] Lost N = 137.180000 [132.051735]
## Mean [SD] Analyzed N = 290.820000 [102.494338]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|         sd|
## |-------------:|-------------:|--------:|----------:|
## |             1|             1| 243.5000| 12.0208153|
## |             1|             2| 198.7500|  5.5000000|
## |             1|             3| 153.5000|  0.7071068|
## |             1|             4|  95.0000|         NA|
## |             2|             3| 206.0000|  8.4852814|
## |             3|             1| 422.0000|         NA|
## |             3|             2| 341.0000|  6.1644140|
## |             3|             3| 257.0000|  5.0709255|
## |             3|             4| 165.6667|  1.5275252|
## |             3|             5|  83.0000|         NA|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     3|       0.06|
## |     2|     9|       0.18|
## |     3|     9|       0.18|
## |     4|     3|       0.06|
## |     6|    21|       0.42|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  2|  4|  2|  1|  0|
## |             2|  0|  0|  2|  0|  0|
## |             3|  1|  5|  8|  3|  1|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |     0.1|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |     0.00|             3|             3|
## |     0.00|             9|             9|
## |     0.25|            12|             9|
## |     0.25|             4|             3|
## |     1.00|             1|             0|
## |     0.00|            21|            21|
## NULL

Alternative Scenario S8

scenario <- LLL.SETTINGS$scenarios$S8
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.180000; P(Reject H0_subgp) = 0.500000; P(Reject H0) = 0.680000
## P(Early stop for efficacy [futility]) = 0.260000 [0.000000]
## Mean [SD] Randomized N = 460.000000 [72.843136]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.14|
## |         2|       0.12|
## |         3|       0.74|
## 
## Mean [SD] Lost N = 170.720000 [104.977556]
## Mean [SD] Analyzed N = 289.280000 [85.082682]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 250.0000|  5.291503|
## |             1|             3| 147.6000|  2.509980|
## |             1|             4| 106.0000|  6.055301|
## |             2|             1| 322.0000|        NA|
## |             2|             3| 207.0000|        NA|
## |             3|             2| 331.3333| 11.590226|
## |             3|             3| 250.7647| 10.195155|
## |             3|             4| 160.0000|  1.732051|
## |             3|             5|  89.5000|  3.109126|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     4|       0.08|
## |     2|     3|       0.06|
## |     3|    14|       0.28|
## |     4|     3|       0.06|
## |     5|     1|       0.02|
## |     6|     9|       0.18|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  3|  0|  5|  4|  0|
## |             2|  1|  0|  1|  0|  0|
## |             3|  0|  3| 17|  3|  4|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.32|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.0000000|             4|             4|
## | 0.0000000|             3|             3|
## | 0.3913043|            23|            14|
## | 0.5714286|             7|             3|
## | 0.7500000|             4|             1|
## | 0.0000000|             9|             9|
## NULL

Alternative Scenario S9

scenario <- LLL.SETTINGS$scenarios$S9
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.560000; P(Reject H0_subgp) = 0.360000; P(Reject H0) = 0.920000
## P(Early stop for efficacy [futility]) = 0.480000 [0.000000]
## Mean [SD] Randomized N = 426.000000 [85.260705]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.26|
## |         2|       0.22|
## |         3|       0.52|
## 
## Mean [SD] Lost N = 125.840000 [153.421968]
## Mean [SD] Analyzed N = 300.160000 [127.468341]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 235.0000|        NA|
## |             1|             2| 194.0000|        NA|
## |             2|             1| 329.6667|  2.081666|
## |             3|             1| 418.5000|  4.949747|
## |             3|             2| 330.6250| 13.394429|
## |             3|             3| 243.2500|  7.365460|
## |             3|             4| 168.0000|  1.414214|
## |             3|             5|  83.0000|        NA|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     5|       0.10|
## |     2|     8|       0.16|
## |     3|     3|       0.06|
## |     4|     1|       0.02|
## |     5|     1|       0.02|
## |     6|    28|       0.56|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  1|  0|  0|  0|
## |             2|  3|  0|  0|  0|  0|
## |             3|  2|  8|  4|  2|  1|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.08|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1666667|             6|             5|
## | 0.1111111|             9|             8|
## | 0.2500000|             4|             3|
## | 0.5000000|             2|             1|
## | 0.0000000|             1|             1|
## | 0.0000000|            28|            28|
## NULL

Alternative Scenario S10

scenario <- LLL.SETTINGS$scenarios$S10
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table1,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50,
                          trueParameters = trueParameters, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.420000; P(Reject H0_subgp) = 0.000000; P(Reject H0) = 0.420000
## P(Early stop for efficacy [futility]) = 0.300000 [0.160000]
## Mean [SD] Randomized N = 422.000000 [91.003476]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.32|
## |         2|       0.14|
## |         3|       0.54|
## 
## Mean [SD] Lost N = 78.100000 [118.835206]
## Mean [SD] Analyzed N = 343.900000 [117.819345]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|       sd|
## |-------------:|-------------:|---------:|--------:|
## |             1|             1| 252.00000|       NA|
## |             1|             5|  50.42857| 4.035556|
## |             3|             1| 418.25000| 9.979145|
## |             3|             2| 331.00000|       NA|
## |             3|             5|  81.00000| 8.074652|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     6|    21|       0.42|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  0|  0|  0|  7|
## |             2|  0|  0|  0|  0|  0|
## |             3|  4|  1|  0|  0| 16|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.58|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |        1|             5|             0|
## |        1|             1|             0|
## |        1|            23|             0|
## |        0|            21|            21|
## NULL

Table 2 Results

Scenario S0

scenario <- LLL.SETTINGS$scenarios$S0
designParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
designA <- ASSISTDesign$new(trialParameters = LLL.SETTINGS$trialParameters,
                            designParameters = designParameters)
print(designA)

## Design Parameters:
##  Number of Groups: 6
##  Prevalence:
## 
## | Group1| Group2| Group3| Group4| Group5| Group6|
## |------:|------:|------:|------:|------:|------:|
## |    0.2|    0.1|    0.3|    0.1|    0.1|    0.2|
## 
##  Using Discrete Rankin scores? FALSE
## 
##  Normal Rankin Distribution means (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      0|      0|      0|      0|      0|      0|
## |Alt  |      0|      0|      0|      0|      0|      0|
## 
##  Normal Rankin Distribution SDs (null row, alt. row):
## 
## 
## |     | Group1| Group2| Group3| Group4| Group5| Group6|
## |:----|------:|------:|------:|------:|------:|------:|
## |Null |      1|      1|      1|      1|      1|      1|
## |Alt  |      1|      1|      1|      1|      1|      1|
## 
## Trial Parameters:
## List of 5
##  $ N         : num [1:3] 300 400 500
##  $ type1Error: num 0.05
##  $ eps       : num 0.5
##  $ type2Error: num 0.2
##  $ effectSize: num 0.0642
## 
## Boundaries:
## 
## 
## |    btilde|        b|        c|
## |---------:|--------:|--------:|
## | -1.460993| 2.371573| 2.467677|

result <- designA$explore(numberOfSimulations = 50, showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.020000; P(Reject H0_subgp) = 0.020000; P(Reject H0) = 0.040000
## P(Early stop for efficacy [futility]) = 0.040000 [0.620000]
## Mean [SD] Randomized N = 406.000000 [79.308234]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.28|
## |         2|       0.38|
## |         3|       0.34|
## 
## Mean [SD] Lost N = 175.400000 [80.752835]
## Mean [SD] Analyzed N = 230.600000 [105.285811]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 240.0000|  9.518403|
## |             1|             2| 211.3333|  7.364946|
## |             1|             3| 116.0000| 16.970563|
## |             1|             4|  86.0000|  7.071068|
## |             1|             5|  59.0000|  3.162278|
## |             2|             1| 325.0000|        NA|
## |             2|             2| 283.0000|  4.242641|
## |             2|             3| 158.5000| 14.849242|
## |             2|             4| 111.0000|        NA|
## |             2|             5|  94.0000|        NA|
## |             3|             1| 369.0000|        NA|
## |             3|             3| 192.0000|  7.071068|
## |             3|             4| 156.0000|        NA|
## |             3|             5| 105.0000|  7.937254|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     2|     1|       0.02|
## |     6|     1|       0.02|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 11| 12|  2|  5|  5|
## |             2|  1|  2|  2|  1|  1|
## |             3|  1|  0|  2|  1|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.96|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 1.0000000|            13|             0|
## | 0.9285714|            14|             1|
## | 1.0000000|             6|             0|
## | 1.0000000|             7|             0|
## | 1.0000000|             9|             0|
## | 0.0000000|             1|             1|
## NULL

Alternative Scenario S1

scenario <- LLL.SETTINGS$scenarios$S1
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.780000; P(Reject H0_subgp) = 0.060000; P(Reject H0) = 0.840000
## P(Early stop for efficacy [futility]) = 0.760000 [0.000000]
## Mean [SD] Randomized N = 366.000000 [84.780629]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.58|
## |         2|       0.18|
## |         3|       0.24|
## 
## Mean [SD] Lost N = 44.200000 [97.398403]
## Mean [SD] Analyzed N = 321.800000 [67.726340]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|       sd|
## |-------------:|-------------:|--------:|--------:|
## |             2|             2| 282.0000|       NA|
## |             3|             2| 347.6667| 4.725816|
## |             3|             3| 205.0000|       NA|
## |             3|             4| 133.5000| 3.535534|
## |             3|             5| 103.2500| 1.707825|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     2|     1|       0.02|
## |     3|     1|       0.02|
## |     5|     1|       0.02|
## |     6|    39|       0.78|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  0|  0|  0|  0|  0|
## |             2|  0|  1|  0|  0|  0|
## |             3|  0|  3|  1|  2|  4|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.16|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |     0.75|             4|             1|
## |     0.00|             1|             1|
## |     1.00|             2|             0|
## |     0.75|             4|             1|
## |     0.00|            39|            39|
## NULL

Alternative Scenario S2

scenario <- LLL.SETTINGS$scenarios$S2
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.520000; P(Reject H0_subgp) = 0.220000; P(Reject H0) = 0.740000
## P(Early stop for efficacy [futility]) = 0.520000 [0.020000]
## Mean [SD] Randomized N = 424.000000 [79.693290]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.22|
## |         2|       0.32|
## |         3|       0.46|
## 
## Mean [SD] Lost N = 79.800000 [98.886866]
## Mean [SD] Analyzed N = 344.200000 [80.416264]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 237.00000|        NA|
## |             1|             2| 222.00000|        NA|
## |             1|             3| 122.00000|        NA|
## |             1|             4|  87.00000|  5.567764|
## |             2|             2| 281.00000|        NA|
## |             2|             4| 137.00000|        NA|
## |             3|             2| 353.00000|  4.242641|
## |             3|             3| 198.00000| 26.870058|
## |             3|             4| 149.00000| 11.302655|
## |             3|             5|  95.66667|  5.686241|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     2|     1|       0.02|
## |     4|     8|       0.16|
## |     5|     1|       0.02|
## |     6|    26|       0.52|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  1|  1|  3|  0|
## |             2|  0|  1|  0|  1|  0|
## |             3|  0|  2|  2|  9|  3|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.26|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.0000000|             1|             1|
## | 0.7500000|             4|             1|
## | 1.0000000|             3|             0|
## | 0.3846154|            13|             8|
## | 0.6666667|             3|             1|
## | 0.0000000|            26|            26|
## NULL

Alternative Scenario S3

scenario <- LLL.SETTINGS$scenarios$S3
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.060000; P(Reject H0_subgp) = 0.560000; P(Reject H0) = 0.620000
## P(Early stop for efficacy [futility]) = 0.300000 [0.040000]
## Mean [SD] Randomized N = 458.000000 [64.174507]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.08|
## |         2|       0.26|
## |         3|       0.66|
## 
## Mean [SD] Lost N = 244.860000 [104.019035]
## Mean [SD] Analyzed N = 213.140000 [90.550022]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|         sd|
## |-------------:|-------------:|--------:|----------:|
## |             1|             1| 240.1250|  7.7170960|
## |             1|             2| 209.8000|  7.5099933|
## |             1|             4| 100.0000|         NA|
## |             2|             1| 323.0000|  4.2426407|
## |             2|             2| 277.5000|  6.3639610|
## |             2|             4| 116.0000|         NA|
## |             2|             5|  75.0000|         NA|
## |             3|             1| 401.5000|  2.1213203|
## |             3|             2| 345.6667| 12.6923299|
## |             3|             3| 187.5000|  0.7071068|
## |             3|             4| 151.0000|         NA|
## |             3|             5| 109.0000|  1.4142136|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     9|       0.18|
## |     2|    18|       0.36|
## |     4|     1|       0.02|
## |     6|     3|       0.06|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  8| 10|  0|  1|  0|
## |             2|  2|  2|  0|  1|  1|
## |             3|  2| 15|  2|  1|  2|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.38|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.2500000|            12|             9|
## | 0.3333333|            27|            18|
## | 1.0000000|             2|             0|
## | 0.6666667|             3|             1|
## | 1.0000000|             3|             0|
## | 0.0000000|             3|             3|
## NULL

Alternative Scenario S4

scenario <- LLL.SETTINGS$scenarios$S4
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.160000; P(Reject H0_subgp) = 0.120000; P(Reject H0) = 0.280000
## P(Early stop for efficacy [futility]) = 0.140000 [0.160000]
## Mean [SD] Randomized N = 452.000000 [78.869564]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.18|
## |         2|       0.12|
## |         3|       0.70|
## 
## Mean [SD] Lost N = 103.560000 [60.185143]
## Mean [SD] Analyzed N = 348.440000 [65.372140]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|      mean|        sd|
## |-------------:|-------------:|---------:|---------:|
## |             1|             1| 238.00000|        NA|
## |             1|             3| 127.00000|  8.485281|
## |             1|             4|  96.33333|  8.594572|
## |             1|             5|  61.50000|  9.071935|
## |             3|             3| 196.00000| 22.538855|
## |             3|             4| 151.64286|  7.601692|
## |             3|             5| 102.80000| 10.347302|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     4|     5|       0.10|
## |     5|     1|       0.02|
## |     6|     8|       0.16|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  0|  2|  6|  6|
## |             2|  0|  0|  0|  0|  0|
## |             3|  0|  0|  3| 14| 10|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.72|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |   1.0000|             1|             0|
## |   1.0000|             5|             0|
## |   0.7500|            20|             5|
## |   0.9375|            16|             1|
## |   0.0000|             8|             8|
## NULL

Alternative Scenario S5

scenario <- LLL.SETTINGS$scenarios$S5
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.100000; P(Reject H0_subgp) = 0.740000; P(Reject H0) = 0.840000
## P(Early stop for efficacy [futility]) = 0.400000 [0.000000]
## Mean [SD] Randomized N = 446.000000 [73.429127]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.14|
## |         2|       0.26|
## |         3|       0.60|
## 
## Mean [SD] Lost N = 282.580000 [122.120230]
## Mean [SD] Analyzed N = 163.420000 [104.629783]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 238.8182|  7.769404|
## |             1|             2| 200.3333| 10.503967|
## |             2|             1| 330.0000|  9.183318|
## |             2|             2| 278.0000|        NA|
## |             2|             4| 113.0000|        NA|
## |             3|             1| 399.2667| 10.746871|
## |             3|             2| 357.6000|  6.655824|
## |             3|             3| 211.5000|  3.535534|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    28|       0.56|
## |     2|     8|       0.16|
## |     3|     1|       0.02|
## |     6|     5|       0.10|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1| 11|  3|  0|  0|  0|
## |             2|  7|  1|  0|  1|  0|
## |             3| 15|  5|  2|  0|  0|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.16|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.1515152|            33|            28|
## | 0.1111111|             9|             8|
## | 0.5000000|             2|             1|
## | 1.0000000|             1|             0|
## | 0.0000000|             5|             5|
## NULL

Alternative Scenario S6

scenario <- LLL.SETTINGS$scenarios$S6
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.080000; P(Reject H0_subgp) = 0.680000; P(Reject H0) = 0.760000
## P(Early stop for efficacy [futility]) = 0.320000 [0.000000]
## Mean [SD] Randomized N = 456.000000 [70.450446]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.12|
## |         2|       0.20|
## |         3|       0.68|
## 
## Mean [SD] Lost N = 275.360000 [129.742722]
## Mean [SD] Analyzed N = 180.640000 [114.948215]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 236.8750|  7.453427|
## |             1|             2| 199.5000| 13.435029|
## |             1|             3| 126.0000|        NA|
## |             1|             4| 106.0000|        NA|
## |             1|             5|  61.0000|        NA|
## |             2|             1| 326.8000|  6.648308|
## |             2|             4| 134.0000|        NA|
## |             3|             1| 395.0526|  9.507153|
## |             3|             2| 359.0000| 14.000000|
## |             3|             3| 204.3333|  8.504900|
## |             3|             4| 132.0000|        NA|
## |             3|             5|  85.0000|        NA|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|    26|       0.52|
## |     2|     5|       0.10|
## |     3|     3|       0.06|
## |     6|     4|       0.08|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  8|  2|  1|  1|  1|
## |             2|  5|  0|  0|  1|  0|
## |             3| 19|  3|  3|  1|  1|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.24|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |   0.1875|            32|            26|
## |   0.0000|             5|             5|
## |   0.2500|             4|             3|
## |   1.0000|             3|             0|
## |   1.0000|             2|             0|
## |   0.0000|             4|             4|
## NULL

Alternative Scenario S7

scenario <- LLL.SETTINGS$scenarios$S7
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.580000; P(Reject H0_subgp) = 0.380000; P(Reject H0) = 0.960000
## P(Early stop for efficacy [futility]) = 0.540000 [0.000000]
## Mean [SD] Randomized N = 416.000000 [86.567185]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.30|
## |         2|       0.24|
## |         3|       0.46|
## 
## Mean [SD] Lost N = 104.820000 [132.987446]
## Mean [SD] Analyzed N = 311.180000 [105.744144]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|       sd|
## |-------------:|-------------:|--------:|--------:|
## |             1|             2| 212.5000|  5.00000|
## |             1|             3| 123.0000|       NA|
## |             2|             2| 290.0000|       NA|
## |             3|             1| 393.0000|       NA|
## |             3|             2| 345.2000| 10.94075|
## |             3|             3| 206.5556| 12.74864|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     2|    10|       0.20|
## |     3|     8|       0.16|
## |     6|    29|       0.58|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  0|  4|  1|  0|  0|
## |             2|  0|  1|  0|  0|  0|
## |             3|  1|  5|  9|  0|  0|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.04|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## | coverage| selectedCount| rejectedCount|
## |--------:|-------------:|-------------:|
## |      0.0|             1|             1|
## |      0.0|            10|            10|
## |      0.2|            10|             8|
## |      0.0|            29|            29|
## NULL

Alternative Scenario S8

scenario <- LLL.SETTINGS$scenarios$S8
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.360000; P(Reject H0_subgp) = 0.480000; P(Reject H0) = 0.840000
## P(Early stop for efficacy [futility]) = 0.400000 [0.000000]
## Mean [SD] Randomized N = 438.000000 [83.029378]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.22|
## |         2|       0.18|
## |         3|       0.60|
## 
## Mean [SD] Lost N = 121.880000 [114.118563]
## Mean [SD] Analyzed N = 316.120000 [89.724440]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|  mean|        sd|
## |-------------:|-------------:|-----:|---------:|
## |             1|             2| 209.0| 12.727922|
## |             1|             3| 120.0|  5.385165|
## |             1|             4|  87.0|        NA|
## |             3|             1| 393.0|        NA|
## |             3|             2| 350.5| 11.030261|
## |             3|             3| 198.0| 11.162437|
## |             3|             4| 153.5|  4.203173|
## |             3|             5| 100.5|  9.110434|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     1|       0.02|
## |     2|     5|       0.10|
## |     3|    11|       0.22|
## |     4|     3|       0.06|
## |     5|     4|       0.08|
## |     6|    18|       0.36|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  0|  2|  5|  1|  0|
## |             2|  0|  0|  0|  0|  0|
## |             3|  1|  4| 11|  4|  4|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.16|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.0000000|             1|             1|
## | 0.1666667|             6|             5|
## | 0.3125000|            16|            11|
## | 0.4000000|             5|             3|
## | 0.0000000|             4|             4|
## | 0.0000000|            18|            18|
## NULL

Alternative Scenario S9

scenario <- LLL.SETTINGS$scenarios$S9
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.540000; P(Reject H0_subgp) = 0.420000; P(Reject H0) = 0.960000
## P(Early stop for efficacy [futility]) = 0.480000 [0.000000]
## Mean [SD] Randomized N = 426.000000 [85.260705]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.26|
## |         2|       0.22|
## |         3|       0.52|
## 
## Mean [SD] Lost N = 124.200000 [148.980207]
## Mean [SD] Analyzed N = 301.800000 [115.994370]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 234.0000|        NA|
## |             1|             2| 219.5000|  2.121320|
## |             1|             5|  65.0000|        NA|
## |             2|             2| 288.0000|        NA|
## |             3|             1| 384.0000|  2.828427|
## |             3|             2| 346.6667| 15.491933|
## |             3|             3| 195.4000|  7.300685|
## |             3|             4| 159.5000| 10.606602|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     1|     3|       0.06|
## |     2|    11|       0.22|
## |     3|     5|       0.10|
## |     4|     2|       0.04|
## |     6|    27|       0.54|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  1|  2|  0|  0|  1|
## |             2|  0|  1|  0|  0|  0|
## |             3|  2|  9|  5|  2|  0|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |    0.04|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 0.0000000|             3|             3|
## | 0.0833333|            12|            11|
## | 0.0000000|             5|             5|
## | 0.0000000|             2|             2|
## | 1.0000000|             1|             0|
## | 0.0000000|            27|            27|
## NULL

Alternative Scenario S10

scenario <- LLL.SETTINGS$scenarios$S10
trueParameters <- list(prevalence = LLL.SETTINGS$prevalences$table2,
                       mean = scenario$mean,
                       sd = scenario$sd)
result <- designA$explore(numberOfSimulations = 50, trueParameters = trueParameters,
                          showProgress = FALSE)
analysis <- designA$analyze(result)
print(designA$summary(analysis))

## P(Reject H0_ITT) = 0.280000; P(Reject H0_subgp) = 0.020000; P(Reject H0) = 0.300000
## P(Early stop for efficacy [futility]) = 0.260000 [0.280000]
## Mean [SD] Randomized N = 420.000000 [83.299313]
## 
## Stage at exit (proportion)
## 
## 
## | exitStage| proportion|
## |---------:|----------:|
## |         1|       0.26|
## |         2|       0.28|
## |         3|       0.46|
## 
## Mean [SD] Lost N = 105.280000 [101.708876]
## Mean [SD] Analyzed N = 314.720000 [98.752430]
## 
## Mean loss by futility stage and subgroup
## 
## 
## | FutilityStage| selectedGroup|     mean|        sd|
## |-------------:|-------------:|--------:|---------:|
## |             1|             1| 237.0000|  5.244044|
## |             1|             2| 215.0000|  8.869423|
## |             1|             5|  56.5000|  7.148426|
## |             2|             1| 325.0000|        NA|
## |             2|             5|  84.0000|        NA|
## |             3|             1| 400.0000|        NA|
## |             3|             2| 359.0000|        NA|
## |             3|             5| 100.7059| 10.042190|
## 
## Chance of each subpopulation rejected
## 
## 
## | group| count| proportion|
## |-----:|-----:|----------:|
## |     5|     1|       0.02|
## |     6|    14|       0.28|
## 
## Counts by futility stage and subgroup choice
## 
## 
## | FutilityStage| G1| G2| G3| G4| G5|
## |-------------:|--:|--:|--:|--:|--:|
## |             1|  5|  4|  0|  0|  6|
## |             2|  1|  0|  0|  0|  1|
## |             3|  1|  1|  0|  0| 17|
## 
## CI Statistics:
## Overall coverage and coverage for rejections:
## 
## | overall| rejection|
## |-------:|---------:|
## |     0.7|         0|
## 
## P(theta_test is in the confidence interval)
## 
## 
## |  coverage| selectedCount| rejectedCount|
## |---------:|-------------:|-------------:|
## | 1.0000000|             7|             0|
## | 1.0000000|             5|             0|
## | 0.9583333|            24|             1|
## | 0.0000000|            14|            14|
## NULL

References

Lai, Tze Leung, Philip W. Lavori, and Olivia Yueh-Wen Liao. 2014. “Adaptive Choice of Patient Subgroup for Comparing Two Treatments.” Contemporary Clinical Trials 39 (2): 191–200. https://doi.org/10.1016/j.cct.2014.09.001.