Title:	Power for Meta-Analysis of Dependent Effects
Version:	0.2.0
BugReports:	https://github.com/MikkelVembye/POMADE/issues
Description:	Provides functions to compute and plot power levels, minimum detectable effect sizes, and minimum required sample sizes for the test of the overall average effect size in meta-analysis of dependent effect sizes.
Depends:	R (≥ 4.1.0)
License:	MIT + file LICENSE
Encoding:	UTF-8
LazyData:	true
RoxygenNote:	7.2.3
Imports:	ggplot2, dplyr, magrittr, purrr, future, furrr, stats, stringr, utils, tidyr, tibble
Suggests:	covr, roxygen2, testthat (≥ 3.0.0)
Config/testthat/edition:	3
Language:	en-US
URL:	https://mikkelvembye.github.io/POMADE/
NeedsCompilation:	no
Packaged:	2024-02-13 18:45:25 UTC; B199526
Author:	Mikkel H. Vembye [aut, cre], James E. Pustejovsky [aut]
Maintainer:	Mikkel H. Vembye <mikkel.vembye@gmail.com>
Repository:	CRAN
Date/Publication:	2024-02-13 20:41:20 UTC

Co-Teaching Dataset

Description

Data from a meta-analysis on the effects of collaborative models of instruction on student achievement from Vembye, Weiss, and Bhat (2023).

Usage

VWB23_pilot

Format

A tibble with 76 rows/studies and 9 variables

study_year

Study author and year of publication

studyid

Unique study ID

esid

Unique effect size ID

kj

Number of effect sizes per study

N_meanj

Average sample size of study

Nt_meanj

Average sample size of treatment group within study

Nc_meanj

Average sample size of control group within study

ESS_meanj

Roughly approximated effective sample sizes

vg_ms_mean

Average cluster bias corrected sampling variance estimates

Source

Find background material on Vembye's OSF page, and the preprint at https://osf.io/preprints/metaarxiv/mq5v7/.

References

Vembye, M. H., Weiss, F., & Bhat, B. H. (2023). The Effects Co-Teaching and Related Collaborative Models of Instruction on Student Achievement: A Systematic Review and Meta-Analysis. Review of Educational Research, doi:10.3102/00346543231186588

Cluster Bias Correction

Description

Function to conduct cluster bias correction of sampling variance estimates obtained from cluster-randomized studies in which the reported variance does not account for clustering.

Usage

cluster_bias_adjustment(sigma2js, cluster_size = 22, icc = 0.2)

Arguments

Value

Returns a vector of cluster bias adjusted variance estimates

Examples

cbc_var <- cluster_bias_adjustment(
  sigma2js = c(0.04, 0.06, 0.08, 0.1),
  cluster_size = 15,
  icc = 0.15
)

cbc_var

Approximate Effective Sample Sizes

Description

Approximate Effective Sample Sizes

Usage

effective_sample_sizes(
  sample_sizes_raw = NULL,
  Nt_raw = NULL,
  Nc_raw = NULL,
  cluster_size = 22,
  icc = 0.22
)

Arguments

Details

N_j/DE

Value

A vector of effective sample sizes, adjusted for cluster-dependence.

Examples

sample_sizes <- sample(50:1000, 50, replace = TRUE)
effective_sample_sizes(
  sample_sizes_raw = sample_sizes,
  cluster_size = 20,
  icc = 0.15
)

Minimum Detectable Effect Size (MDES) for Meta-Analysis With Dependent Effect Sizes

Description

Compute the minimum detectable effect size in a meta-analysis of dependent effect size estimates, given a specified number of studies, power level, estimation method, and further assumptions about the distribution of studies.

Usage

mdes_MADE(
  J,
  tau,
  omega,
  rho,
  alpha = 0.05,
  target_power = 0.8,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  upper = 2,
  show_lower = FALSE
)

Arguments

Value

Returns a tibble with information about the expectation of the number of studies, the between-study and within-study variance components, the sample correlation, the contrast effect, the level of statistical significance, the target power value(s), the minimum detectable effect size, the number of iterations, the model to handle dependent effect sizes, and the methods used to obtain sampling variance estimates as well as the number effect sizes per study.

Examples

mdes_MADE(
  J = 30,
  tau = 0.05,
  omega = 0.02,
  rho = 0.2,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = 4 / 100,
  n_ES_dist = 6,
  seed = 10052510
)

Finding the Number of Studies Needed to Obtain a Certain Amount of Power

Description

Compute the minimum number of studies needed to obtain a specified power level in a meta-analysis of dependent effect size estimates, given an effect size of practical concern, estimation method, and further assumptions about the distribution of studies.

Usage

min_studies_MADE(
  mu,
  tau,
  omega,
  rho,
  alpha = 0.05,
  target_power = 0.8,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  upper = 100,
  show_lower = FALSE
)

Arguments

Value

Returns a tibble with information about the expectation of the effect size of practical concern, the between-study and within-study variance components, the sample correlation, the contrast effect, the level of statistical significance, the target power value(s), the number of studies needed, the number of iterations, the model to handle dependent effect sizes, and the methods used to obtain sampling variance estimates as well as the number effect sizes per study.

Examples

min_studies_MADE(
  mu = 0.3,
  tau = 0.05,
  omega = 0.01,
  rho = 0.2,
  target_power = .7,
  alpha = 0.05,
  model = "CE",
  var_df = "RVE",
  sigma2_dist = 4 / 200,
  n_ES_dist = 5.5,
  seed = 10052510
)

Generic plot function for 'MADE' objects

Description

Create a faceted plot displaying the results of a set of power analyses. This is a generic function to make facet_grid plots, with specific methods defined for power_MADE, mdes_MADE, and min_studies_MADE objects.

Usage

plot_MADE(
  data,
  v_lines,
  legend_position,
  color,
  numbers,
  number_size,
  numbers_ynudge,
  caption,
  x_lab,
  x_breaks,
  x_limits,
  y_breaks,
  y_limits,
  y_expand = NULL,
  warning,
  traffic_light_assumptions,
  ...
)

Arguments

Value

A ggplot object

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (In preparation). Conducting power analysis for meta-analysis of dependent effect sizes: Common guidelines and an introduction to the POMADE R package.

See Also

plot_MADE.power, plot_MADE.mdes, plot_MADE.min_studies

Examples

power_dat <-
  power_MADE(
    J = c(50, 56),
    mu = 0.15,
    tau = 0.1,
    omega = 0.05,
    rho = 0,
    sigma2_dist = 4 / 200,
    n_ES_dist = 6
  )

power_example <-
  plot_MADE(
   data = power_dat,
   power_min = 0.8,
   expected_studies = c(52, 54),
   warning = FALSE,
   caption = TRUE,
   color = TRUE,
   model_comparison = FALSE,
   numbers = FALSE
   )

power_example

Plot function for a 'mdes' object

Description

Creates a faceted plot for minimum detectable effect size (mdes) analyses calculated using mdes_MADE.

Usage

## S3 method for class 'mdes'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = NULL,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = ggplot2::waiver(),
  y_limits = NULL,
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  es_min = NULL,
  expected_studies = NULL,
  ...
)

Arguments

Details

In general, it can be rather difficult to guess/approximate the true model parameters and sample characteristics a priori. Calculating the minimum detectable effect size under just a single set of assumptions can easily be misleading even if the true model and data structure only slightly diverge from the yielded data and model assumptions. To maximize the informativeness of the analysis, Vembye, Pustejovsky, & Pigott (In preparation) suggest accommodating the uncertainty of the power approximations by reporting or plotting minimum detectable effect size estimates across a range of possible scenarios, which can be done using plot_MADE.mdes.

Value

A ggplot plot showing the minimum detectable effect size across the expected number of studies, faceted by the between-study and within-study SDs, with different colors, lines, and shapes corresponding to different values of the assumed sample correlation.

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (In preparation). Conducting power analysis for meta-analysis of dependent effect sizes: Common guidelines and an introduction to the POMADE R package.

See Also

plot_MADE

Examples

mdes_MADE(
  J = c(25, 35),
  tau = 0.05,
  omega = 0,
  rho = 0,
  target_power = .6,
  alpha = 0.1,
  sigma2_dist = 4 / 200,
  n_ES_dist = 8,
  seed = 10052510
) |>
  plot_MADE(expected_studies = c(28, 32), numbers = FALSE)

Plot function for a 'min_studies' object

Description

Creates a faceted plot with analyses of the minimum number of studies needed to obtained a given effect size with specified levels of power, as calculated using min_studies_MADE.

Usage

## S3 method for class 'min_studies'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = NULL,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = ggplot2::waiver(),
  y_limits = NULL,
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  v_shade = NULL,
  h_lines = NULL,
  ...
)

Arguments

Details

In general, it can be rather difficult to guess/approximate the true model parameters and sample characteristics a priori. Calculating the minimum number of studies needed under just a single set of assumptions can easily be misleading even if the true model and data structure only slightly diverge from the yielded data and model assumptions. To maximize the informativeness of the analysis, Vembye, Pustejovsky, & Pigott (In preparation) suggest accommodating the uncertainty of the power approximations by reporting or plotting power estimates across a range of possible scenarios, which can be done using plot_MADE.power.

Value

A ggplot plot showing the minimum number of studies needed to obtain a given effect size with a certain amount of power and level-alpha, faceted across levels of the within-study SD and the between-study SD, with different colors, lines, and shapes corresponding to different values of the assumed sample correlation. If length(unique(data$mu)) > 1, it returns a ggplot plot showing the minimum studies needed to obtained a given effect size with a certain amount of power and level-alpha across effect sizes of practical concern, faceted by the between-study and within-study SDs, with different colors, lines, and shapes corresponding to different values of the assumed sample correlation.

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (In preparation). Conducting power analysis for meta-analysis of dependent effect sizes: Common guidelines and an introduction to the POMADE R package.

See Also

plot_MADE

Examples

min_studies_MADE(
  mu = c(0.25, 0.35),
  tau = 0.05,
  omega = 0.02,
  rho = 0.2,
  target_power = .7,
  sigma2_dist = 4 / 200,
  n_ES_dist = 6,
  seed = 10052510
) |>
  plot_MADE(y_breaks = seq(0, 10, 2), numbers = FALSE)

Plot function for a 'power' object

Description

Creates a faceted plot or plots for power analyses conducted with power_MADE.

Usage

## S3 method for class 'power'
plot_MADE(
  data,
  v_lines = NULL,
  legend_position = "bottom",
  color = TRUE,
  numbers = TRUE,
  number_size = 2.5,
  numbers_ynudge = 0,
  caption = TRUE,
  x_lab = NULL,
  x_breaks = NULL,
  x_limits = NULL,
  y_breaks = seq(0, 1, 0.2),
  y_limits = c(0, 1),
  y_expand = NULL,
  warning = TRUE,
  traffic_light_assumptions = NULL,
  power_min = NULL,
  expected_studies = NULL,
  model_comparison = FALSE,
  ...
)

Arguments

Details

In general, it can be rather difficult to guess/approximate the true model parameters and sample characteristics a priori. Calculating power under only a single set of assumptions can easily be misleading even if the true model and data structure only slightly diverge from the yielded data and model assumptions. To maximize the informativeness of the power approximations, Vembye, Pustejovsky, & Pigott (In preparation) suggest accommodating the uncertainty of the power approximations by reporting or plotting power estimates across a range of possible scenarios, which can be done using plot_MADE.power.

Value

A ggplot plot showing power across the expected number of studies, faceted by the between-study and within-study SDs, with different colors, lines, and shapes corresponding to different values of the assumed sample correlation. If model_comparison = TRUE, it returns a ggplot plot showing power across the expected number of studies, faceted by the between-study and within-study SDs, with different colors, lines, and shapes corresponding to different working models for dependent effect size estimates

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (In preparation). Conducting power analysis for meta-analysis of dependent effect sizes: Common guidelines and an introduction to the POMADE R package.

See Also

plot_MADE

Examples

power_dat <-
  power_MADE(
    J = c(50, 56),
    mu = 0.15,
    tau = 0.1,
    omega = 0.05,
    rho = 0,
    sigma2_dist = 4 / 200,
    n_ES_dist = 6
  )

power_example <-
  plot_MADE(
   data = power_dat,
   power_min = 0.8,
   expected_studies = c(52, 54),
   warning = FALSE,
   caption = TRUE,
   color = TRUE,
   model_comparison = FALSE,
   numbers = FALSE
   )

power_example

Power Approximation for Overall Average Effects in Meta-Analysis With Dependent Effect Sizes

Description

Compute power of the test of the overall average effect size in a meta-analysis of dependent effect size estimates, given a specified number of studies, effect size of practical concern, estimation method, and further assumptions about the distribution of studies.

Usage

power_MADE(
  J,
  mu,
  tau,
  omega,
  rho,
  alpha = 0.05,
  d = 0,
  model = "CHE",
  var_df = "RVE",
  sigma2_dist = NULL,
  n_ES_dist = NULL,
  iterations = 100,
  seed = NULL,
  warning = TRUE,
  average_power = TRUE
)

Arguments

Details

Find all background material behind the power approximations in Vembye, Pustejovsky, & Pigott (2022), including arguments for why it is suggested neither to conduct power analysis based on balanced assumptions about the number of effects per study and the study variance nor to use the original power approximation assuming independence among effect sizes (Hedges & Pigott, 2001).

Value

Returns a tibble with information about the expectation of the number of studies, the effect size of practical concern, the between-study and within-study variance components, the sample correlation, the contrast effect, the level of statistical significance, the sampling variance of overall average effect size of practical concern, the degrees of freedom, the power, the mcse, the number of iterations, the model to handle dependent effect sizes, and the methods used to obtain sampling variance estimates as well as the number effect sizes per study.

References

Vembye, M. H., Pustejovsky, J. E., & Pigott, T. D. (2022). Power approximations for overall average effects in meta-analysis with dependent effect sizes. Journal of Educational and Behavioral Statistics, 1–33. doi:10.3102/10769986221127379

Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6(3), 203–217. doi:10.1037/1082-989X.6.3.203

Examples

power <- power_MADE(
   J = c(40, 60),
   mu = 0.2,
   tau = 0.2,
   omega = 0.1,
   rho = 0.7,
   sigma2_dist = \(x) rgamma(x, shape = 5, rate = 10),
   n_ES_dist = \(x) 1 + stats::rpois(x, 5.5 - 1),
   model = c("CHE", "MLMA", "CE"),
   var_df = c("Model", "Satt", "RVE"),
   alpha = .05,
   seed = 10052510,
   iterations = 5
 )

power

Between-Study Variance Approximation Function

Description

Rough approximation of the between-study variance based on assumption about the typical sample size of studies included in the synthesis

Usage

tau2_approximation(sample_size = 100, es, df_minus2 = TRUE)

Arguments

Value

A tibble with small, medium, and large magnitudes of tau2

Examples

tau2_approximation(
sample_size = 50,
es = 0.1,
df_minus2 = TRUE
)