Version:	0.42
Date:	2023-05-03
Author:	Samuel Pawel [aut, cre]
Maintainer:	Samuel Pawel <samuel.pawel@uzh.ch>
Title:	Bayesian Design of Replication Studies
Description:	Provides functionality for determining the sample size of replication studies using Bayesian design approaches in the normal-normal hierarchical model (Pawel et al., 2022) <doi:10.48550/arXiv.2211.02552>.
License:	GPL-3
Encoding:	UTF-8
Imports:	methods, lamW
Suggests:	roxygen2, tinytest
RoxygenNote:	7.2.3
NeedsCompilation:	no
URL:	https://github.com/SamCH93/BayesRepDesign
BugReports:	https://github.com/SamCH93/BayesRepDesign/issues
Packaged:	2023-05-03 07:42:49 UTC; sam
Repository:	CRAN
Date/Publication:	2023-05-03 18:55:43 UTC

Density method for class "designPrior"

Description

Density method for class "designPrior"

Usage

## S3 method for class 'designPrior'
density(x, ...)

Arguments

Value

Returns the density function of the design prior

Author(s)

Samuel Pawel

Examples

dp <- designPrior(to = 2.3123, so = 0.1, mu = 1.1, tau = 0.2)
f <- density(dp)
tseq <- seq(1, 3.5, 0.01)
plot(tseq, f(theta = tseq), type = "l", xlab = "theta", ylab = "Design prior density")

Design prior for effect size

Description

Creates a design prior for the effect size which can then be used for power and sample size calculations of a replication study. The design prior is obtained from updating an initial prior for the effect size by the data from the original study. A normal-normal hierarchical model is assumed, see Pawel et al. (2022) for details.

Usage

designPrior(
  to,
  so,
  mu = 0,
  sp = Inf,
  tau = 0,
  g = sp^2/(tau^2 + so^2),
  h = tau^2/so^2,
  type = c(NA, "conditional", "predictive", "EB")
)

Arguments

Details

The "conditional" design prior corresponds to a point mass at the original effect estimate, i.e., assuming that the true effect size is equal to the original effect estimate. The "predictive" design prior is obtained from updating a uniform initial prior by the likelihood of the original data. The "EB" design prior is obtained by empirical Bayes estimation of the variance of the normal prior and induces adaptive shrinkage that depends on the p-value of the original effect estimate.

Value

Returns an object of class "designPrior" which is a list containing:

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

See Also

pors, ssd

Examples

designPrior(to = 1.1, so = 1)

Plot method for class "designPrior"

Description

Plot method for class "designPrior"

Usage

## S3 method for class 'designPrior'
plot(x, ...)

Arguments

Value

Plots the density of the design prior

Author(s)

Samuel Pawel

Examples

dp <- designPrior(to = 2.3123, so = 0.1, mu = 1.1, tau = 0.2)
plot(dp)
plot(dp, xlim = c(0, 5), length.out = 500)

Compute probability of replication success

Description

This function computes the probabiliy of replication success based on a success region for the replication effect estimate, a design prior, and a replication standard error. If the specified number of sites is larger than 1, the supplied success region has to be formulated in terms of the meta-analytic replication effect estimate across sites.

Usage

pors(sregion, dprior, sr, nsites = 1)

Arguments

Value

The probability of replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

dprior <- designPrior(to = 1.1, so = 1)
sregion <- successRegion(intervals = cbind(1.96, Inf))
pors(sregion = sregion, dprior = dprior, sr = 1)

Probability of replication success based on Bayes factor

Description

This function computes the probability to achieve replication success based on a Bayes factor. The Bayes factor is oriented so that values above one indicate evidence for the null hypothesis of the effect size being zero, whereas values below one indicate evidence for the hypothesis of the effect size being non-zero (with normal prior assigned to it).

Usage

porsBF01(level, dprior, sr, priormean = 0, priorvar = 1)

Arguments

Value

The probability to achieve replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

## specify design prior
to1 <- 2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.03)
porsBF01(level = 1/10, dprior = dprior, sr = c(0.05, 0.04))

Probability of replication success based on replication Bayes factor

Description

This function computes the probability to achieve replication success based on the replication Bayes factor. The replication Bayes factor is assumed to be oriented so that values below one indicate replication success, whereas values above one indicate evidence for the null hypothesis.

Usage

porsBFr(level, dprior, sr, paradox = TRUE)

Arguments

Value

The probability to achieve replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Verhagen, J. and Wagenmakers, E. J. (2014). Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General, 145:1457-1475. doi:10.1037/a0036731

Ly, A., Etz, A., Marsman, M., & Wagenmakers, E.-J. (2018). Replication Bayes factors from evidence updating. Behavior Research Methods, 51(6), 2498-2508. doi:10.3758/s13428-018-1092-x

Examples

## specify design prior
to1 <- 0.2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.03)
porsBFr(level = 1/10, dprior = dprior, sr = c(0.05, 0.04))

Probability of replication success based on the sceptical Bayes factor

Description

This function computes the probability to achieve replication success based on the sceptical Bayes factor. The sceptical Bayes factor is assumed to be oriented so that values below one indicate replication success.

Usage

porsBFs(level, dprior, sr, paradox = TRUE)

Arguments

Value

The probability to achieve replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Pawel, S. and Held, L. (2020). The sceptical Bayes factor for the assessement of replication success. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(3), 879-911. doi:10.1111/rssb.12491

Examples

## specify design prior
to1 <- 0.2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1)
porsBFs(level = 1/3, dprior = dprior, sr = 0.05)

Probability of replication success based on effect size equivalence

Description

This function computes the probability to achieve replication success on equivalence of original and replication effect size. Effect size equivalence is defined by the confidence interval for the difference between the original and replication effect sizes falling within an equivalence region around zero defined by the specified margin.

Usage

porsEqu(level, dprior, margin, sr)

Arguments

Value

The probability to achieve replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Anderson, S. F. and Maxwell, S. E. (2016). There's more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21(1), 1-12. doi:10.1037/met0000051

Examples

## specify design prior
to1 <- 2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.1)
porsEqu(level = 0.1, dprior = dprior, margin = 0.3, sr = c(0.05, 0.03))

Probability of replication success based on meta-analytic significance

Description

This function computes the probability to achieve replication success on statistical significance of the fixed-effects meta-analytic effect estimate obtained from combining original and replication effect estimates.

Usage

porsMeta(level, dprior, sr)

Arguments

Value

The probability to achieve replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

## specify design prior
to1 <- 2
so1 <- 1
dprior <- designPrior(to = to1, so = so1, tau = 0.1)
porsMeta(level = 0.025^2, dprior = dprior, sr = c(0.2, 0.1))

Probability of replication success based on the sceptical p-value

Description

This function computes the probability to achieve replication success based on the sceptical p-value.

Usage

porsPs(level, dprior, sr)

Arguments

Details

The sceptical p-value is assumed to be uncalibrated as in Held (2020). The package ReplicationSuccess allows for sample size and power calculations with the recalibrated sceptical p-value (https://CRAN.R-project.org/package=ReplicationSuccess).

Value

The probability to achieve replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Held, L. (2020). A new standard for the analysis and design of replication studies (with discussion). Journal of the Royal Statistical Society: Series A (Statistics in Society), 183(2), 431-448. doi:10.1111/rssa.12493

Examples

## specify design prior
to1 <- 0.2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1)
porsPs(level = 0.025, dprior = dprior, sr = c(0.05, 0.01))

Probability of replication success based on significance

Description

This function computes the probability to achieve replication success on statistical significance of the replication effect estimate.

Usage

porsSig(level, dprior, sr)

Arguments

Value

The probability to achieve replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

## specify design prior
to1 <- 2
so1 <- 1
dprior <- designPrior(to = to1, so = so1, tau = 0.1)
porsSig(level = 0.025, dprior = dprior, sr = c(0.5, 0.3))

Probability of replication success based on TOST equivalence

Description

This function computes the probability to achieve replication success based on establishing the absence of a practically relevant effect size with the Two One-Sided Tests (TOST) procedure in the replication study.

Usage

porsTOST(level, dprior, margin, sr)

Arguments

Value

The probability to achieve replication success

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Anderson, S. F. and Maxwell, S. E. (2016). There's more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21(1), 1-12. doi:10.1037/met0000051

Examples

## specify design prior
to1 <- 2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.1)
porsTOST(level = 0.1, dprior = dprior, margin = 0.3, sr = c(0.05, 0.03))

Print method for class "designPrior"

Description

Print method for class "designPrior"

Usage

## S3 method for class 'designPrior'
print(x, ...)

Arguments

Value

Prints text summary in the console and invisibly returns the "designPrior" object

Author(s)

Samuel Pawel

Examples

dp <- designPrior(to = 0.5, so = 0.05, sp = 0.2, tau = 0.1)
print(dp)

Print method for class "ssdRS"

Description

Print method for class "ssdRS"

Usage

## S3 method for class 'ssdRS'
print(x, ...)

Arguments

Value

Prints text summary in the console and invisibly returns the "ssdRS" object

Author(s)

Samuel Pawel

Examples

## specify design prior
to1 <- 2
so1 <- 1
dprior <- designPrior(to = to1, so = so1)

## compute required standard error for significance at one-sided 2.5%
sregionfunSig <- function(sr, alpha = 0.025) {
    successRegion(intervals = cbind(stats::qnorm(p = 1- alpha)*sr, Inf))
}
ssd1 <- ssd(sregionfun = sregionfunSig, dprior = dprior, power = 0.8)
print(ssd1)

Print method for class "successRegion"

Description

Print method for class "successRegion"

Usage

## S3 method for class 'successRegion'
print(x, ...)

Arguments

Value

Prints text summary in the console and invisibly returns the "successRegion" object

Author(s)

Samuel Pawel

Examples

## success region for two-sided significance test
successRegion(intervals = rbind(c(1.96, Inf), c(-Inf, -1.96)))
## success region for one-sided significance test
successRegion(intervals = rbind(c(1.96, Inf)))

Sample size related to standard error and unit standard deviation

Description

This function computes the sample size related to a specified standard error \sigma and unit standard deviation unitSD, which is the standard deviation of one effective unit (one measurement, one pair of measurements, one event, etc.). The relationship \sigma = \code{unitSD}/\sqrt{n} is assumed. The unit standard deviation depends on the parameter type and the assumptions underlying the standard error calculation. The default is unitSD = 2 which is, under some assumptions, a reasonable approximation to the unit standard deviation for standardized mean differences and log odds/hazard/rate ratios, see Section 2.4 in Spiegelhalter et al. (2004).

Usage

se2n(se, unitSD = 2)

Arguments

Value

The sample size corresponding to the specified standard error and unit standard deviation

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Spiegelhalter, D.J., Abrams, K.R., Myles, J.P. (2004). Bayesian approaches to clinical trials and health care evaluation. Wiley. doi:10.1002/0470092602

Examples

smd1 <- 0.3
so1 <- 0.05
dprior <- designPrior(to = smd1, so = so1)
ssd1 <- ssdSig(level = 0.025, dprior = dprior, power = 0.8)
se2n(se = ssd1$sr, unitSD = 2) # required n

Sample size determination for replication success

Description

This function computes the standard error of the replication effect estimate required to achieve replication success with a certain probability and based on a certain type of success region.

Usage

ssd(
  sregionfun,
  dprior,
  power,
  nsites = 1,
  searchInt = c(.Machine$double.eps^0.5, 4),
  ...
)

Arguments

Value

Returns an object of class "ssdRS" which is a list containing:

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

## specify design prior
to1 <- 2
so1 <- 1
dprior <- designPrior(to = to1, so = so1)

## compute required standard error for significance at one-sided 2.5%
sregionfunSig <- function(sr, alpha = 0.025) {
    successRegion(intervals = cbind(stats::qnorm(p = 1- alpha)*sr, Inf))
}
ssd(sregionfun = sregionfunSig, dprior = dprior, power = 0.8)

Sample size determination for replication success based on Bayes factor

Description

This function computes the standard error required to achieve replication success with a certain probability and based on the Bayes factor under normality. The Bayes factor is oriented so that values above one indicate evidence for the null hypothesis of the effect size being zero, whereas values below one indicate evidence for the hypothesis of the effect size being non-zero (with normal prior assigned to it).

Usage

ssdBF01(
  level,
  dprior,
  power,
  priormean = 0,
  priorvar = 1,
  searchInt = c(.Machine$double.eps^0.5, 2)
)

Arguments

Value

Returns an object of class "ssdRS". See ssd for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

## specify design prior
to1 <- 0.2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.03)
ssdBF01(level = 1/10, dprior = dprior, power = 0.8)

Sample size determination for replication success based on replication Bayes factor

Description

This function computes the standard error required to achieve replication success with a certain probability and based on the replication Bayes factor under normality. The replication Bayes factor is assumed to be oriented so that values below one indicate replication success, whereas values above one indicate evidence for the null hypothesis.

Usage

ssdBFr(
  level,
  dprior,
  power,
  searchInt = c(.Machine$double.eps^0.5, 2),
  paradox = TRUE
)

Arguments

Value

Returns an object of class "ssdRS". See ssd for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Verhagen, J. and Wagenmakers, E. J. (2014). Bayesian tests to quantify the result of a replication attempt. Journal of Experimental Psychology: General, 145:1457-1475. doi:10.1037/a0036731

Ly, A., Etz, A., Marsman, M., and Wagenmakers, E.-J. (2018). Replication Bayes factors from evidence updating. Behavior Research Methods, 51(6), 2498-2508. doi:10.3758/s13428-018-1092-x

Examples

## specify design prior
to1 <- 0.2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.03)
ssdBFr(level = 1/10, dprior = dprior, power = 0.8)

Sample size determination for replication success based on the sceptical Bayes factor

Description

This function computes the standard error required to achieve replication success with a certain probability and based on the sceptical Bayes factor. The sceptical Bayes factor is assumed to be oriented so that values below one indicate replication success.

Usage

ssdBFs(
  level,
  dprior,
  power,
  searchInt = c(.Machine$double.eps^0.5, 2),
  paradox = TRUE
)

Arguments

Value

Returns an object of class "ssdRS". See ssd for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Pawel, S. and Held, L. (2020). The sceptical Bayes factor for the assessement of replication success. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(3), 879-911. doi:10.1111/rssb.12491

Examples

## specify design prior
to1 <- 0.2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.03)
ssdBFs(level = 1/10, dprior = dprior, power = 0.9)

Sample size determination for replication success based on effect size equivalence

Description

This function computes the standard error required to achieve replication success with a certain probability and based on effect size equivalence of original and replication effect size. Effect size equivalence is defined by the confidence interval for the difference between the original and replication effect sizes falling within an equivalence region around zero defined by the specified margin.

Usage

ssdEqu(level, dprior, power, margin, searchInt = c(0, 2))

Arguments

Value

Returns an object of class "ssdRS". See ssd for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Anderson, S. F. and Maxwell, S. E. (2016). There's more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21(1), 1-12. doi:10.1037/met0000051

Examples

## specify design prior
to1 <- 0.2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.05)
ssdEqu(level = 0.1, dprior = dprior, power = 0.8, margin = 0.2)

Sample size determination for replication success based on meta-analytic significance

Description

This function computes the standard error required to achieve replication success with a certain probability and based on statistical significance of the fixed-effects meta-analytic effect estimate obtained from combining original and replication effect estimates.

Usage

ssdMeta(level, dprior, power, searchInt = c(0, 10))

Arguments

Value

Returns an object of class "ssdRS". See ssd for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

## specify design prior
to1 <- 2
so1 <- 1
dprior <- designPrior(to = to1, so = so1, tau = 0.25, sp = Inf)
ssdMeta(level = 0.025^2, dprior = dprior, power = 0.95)

Sample size determination for replication success based on the sceptical p-value

Description

This function computes the standard error required to achieve replication success with a certain probability and based on the sceptical p-value.

Usage

ssdPs(level, dprior, power)

Arguments

Details

The sceptical p-value is assumed to be uncalibrated as in Held (2020). The package ReplicationSuccess allows for sample size and power calculations with the recalibrated sceptical p-value (https://CRAN.R-project.org/package=ReplicationSuccess).

Value

Returns an object of class "ssdRS". See ssd for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Held, L. (2020). A new standard for the analysis and design of replication studies (with discussion). Journal of the Royal Statistical Society: Series A (Statistics in Society), 183(2), 431-448. doi:10.1111/rssa.12493

Examples

## specify design prior
to1 <- 0.2
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.03)
ssdPs(level = 0.05, dprior = dprior, power = 0.9)

Sample size determination for replication success based on significance

Description

This function computes the standard error required to achieve replication success with a certain probability and based on statistical significance of the replication effect estimate.

Usage

ssdSig(level, dprior, power)

Arguments

Value

Returns an object of class "ssdRS". See ssd for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

## specify design prior
to1 <- 2
so1 <- 0.5
dprior <- designPrior(to = to1, so = so1, tau = 0.1)
ssdSig(level = 0.025, dprior = dprior, power = 0.9)

Sample size determination for replication success based on TOST equivalence

Description

This function computes the standard error required to achieve replication success with a certain probability and based on establishing the absence of a practically relevant effect size with the Two One-Sided Tests (TOST) procedure in the replication study.

Usage

ssdTOST(level, dprior, power, margin, searchInt = c(0, 2))

Arguments

Value

Returns an object of class "ssdRS". See ssd for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Anderson, S. F. and Maxwell, S. E. (2016). There's more than one way to conduct a replication study: Beyond statistical significance. Psychological Methods, 21(1), 1-12. doi:10.1037/met0000051

Examples

## specify design prior
to1 <- 0.05
so1 <- 0.05
dprior <- designPrior(to = to1, so = so1, tau = 0.05)
ssdTOST(level = 0.05, dprior = dprior, power = 0.9, margin = 0.3)

Success region for replication effect estimate

Description

Creates a success region object which can then be used for computing the probability of replication success with pors.

Usage

successRegion(intervals)

Arguments

Value

Returns an object of class "successRegion" which is a matrix containing the success intervals sorted in ascending order

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

See Also

pors, ssd

Examples

successRegion(intervals = rbind(c(1.96, Inf), c(-Inf, -1.96)))
successRegion(intervals = cbind(1.96, Inf))

Success region based on significance

Description

This function returns the success region for the (meta-analytic) replication effect estimate to achieve significance

Usage

successRegionSig(sr, to, tau = 0, nsites = 1, level)

Arguments

Value

An object of class "successRegion". See successRegion for details.

Author(s)

Samuel Pawel

References

Pawel, S., Consonni, G., and Held, L. (2022). Bayesian approaches to designing replication studies. arXiv preprint. doi:10.48550/arXiv.2211.02552

Examples

successRegionSig(sr = 0.05, to = 0.2, tau = 0.01, nsites = 3, level = 0.025)