NO.PING.PONG

Description

This package contains two functions, NO.PING.PONG and PLOT_NO.PING.PONG, for revealing what happens when effect size estimates from previous studies are taken into account when evaluating each new dataset in a study sequence. The analyses can be conducted for cumulative meta-analyses and for Bayesian data analyses. The findings from these methods can be contrasted with those from null hypothesis statistical sigificance testing, which does not take previous findings into account.

The package contains sample data for a wide selection of research topics that can be used to illustrate why previous findings should be taken into account when attempting to reach a conclusion about an effect when evaluating a new dataset. Jointly considering previous findings along with new data is more likely to result in correct conclusions than does the traditional practice of not incorporating previous findings and which often results in a back and forth ping-pong of conclusions when evaluating a sequence of studies.

References

O'Connor, B. P., & Ermacora, D. (2021). Unnecessary ping-pong: Illustrations of why previous findings should be taken into account when evaluating new datasets. Canadian Journal of Behavioural Science, 53(3), 328-341. https://doi.org/10.1037/cbs0000259

O'Connor, B. P., & Khattar, N. (2022). Controversies regarding null hypothesis testing. In W. O'Donohue, A. Masuda, & S. O. Lilienfeld (Eds.). Avoiding Questionable Research Practices in Applied Psychology (pp. 147-174). Cham, Switzerland: Springer Nature Switzerland.

Converts between r, d, g, and OR effect sizes

Description

Converts effect sizes, including (a) r to d, g, or OR, (b) d or g to r, d, g or OR, and (c) OR to r, d, or g.

Usage

CONVERT_ES(donnes, ES, ES_type_IN='r', ES_var = NULL, 
           totalN = NULL, grp1_n = NULL, grp2_n = NULL,
           gvar_type_OUT = 'd', 
           CI_level_out = 95,
           CI_level_in = 95, CI_in_lb = NULL, CI_in_ub = NULL, 
           verbose = TRUE)

Arguments

Details

This function converts r, z (Fishers z transformation of r), d, g, and OR effect sizes to r, z, d, g, and OR effect sizes using conventional formulas (Borenstein & Hedges, 2019; Borenstein, Hedges, Higgins, & Rothstein, 2009). The effect size variances and confidence intervals are also computed if sufficient data are provided as input.

When the input effect sizes are d or g values, it is helpful if the group Ns are also provided whenever possible.

The gvar_type_OUT argument provides a choice between d or g effect size variances whenever ES_type_OUT is set to g. The reason for this option is that authors of published meta-analyses sometimes report d variances when their analyses were conducted on g effect sizes. This is presumably not a wise practice, but it also does not make much difference in the computed values.

Value

A matrix containing the following statistics, if they can be computed using the input informaiton: the r, z, d, g, and OR effect sizes and their corresponding variances, confidence intervals, and the totalNs used in the analyses.

Author(s)

Brian P. O'Connor

References

Borenstein, M., & Hedges, L. V. (2019). Effect sizes for meta-analysis. In H. Cooper, L. V., Hedges, & J. C. Valentine (Eds). The handbook of research synthesis and meta-analysis (pp. 207-244). (3rd. edition). New York, NY: Russell Sage Foundation.

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Converting among effect sizes. In, Introduction to meta-analysis pp. 45-49. Chichester, UK: John Wiley & Sons.

Valentine, J. C. & Cooper, H. (2003). Effect size substantive interpretation guidelines: Issues in the interpretation of effect sizes. Washington, DC: What Works Clearinghouse.

Examples

# convert d effect sizes to r	
head(data_NPP$Omega3_Depression)
CONVERT_ES(donnes = data_NPP$Omega3_Depression, ES = 'SMD', ES_type_IN='d', ES_var = NULL, 
           grp1_n = 'CN', grp2_n = 'EN')

 
# convert r effect sizes to g	
head(data_NPP$Math_Performance)
CONVERT_ES(donnes = data_NPP$Math_Performance, ES = 'r', ES_type_IN='r', ES_var = NULL,
           totalN = 'N')

Incorporating previous findings when evaluating new data

Description

A function for revealing what happens when previous findings are taken into account when analyzing new data.

Usage

NO.PING.PONG(donnes, 
             ES_type_IN=NULL, ES_type_OUT='r',
             rawdata_type = 'for_correl',
             ma_method = 'HS',
             cor_stat = 'ZCOR',
             Bayes_type = c('Schmidt_Raju'), 
             prior_type='META',
             ES = NULL, N = NULL, ES_var = NULL,
             CI_level_out = 95, CI_level_in = 95, CI_in_lb=NULL, CI_in_ub=NULL,
             grp1_mn = NULL, grp1_sd = NULL, grp1_n = NULL, 
             grp2_mn = NULL, grp2_sd = NULL, grp2_n = NULL, 
             gvar_type_OUT = 'd',
             paired_samples_ES_type = NULL,
             funnel_plot=FALSE, funnel_plot_type='png', funnel_plot_title=NULL,
             nitt = 53000, burnin = 3000, thin = 10, 
             verbose=TRUE)

Arguments

Details

This function reveals what happens when effect size estimates from previous studies are taken into account when evaluating each new dataset in a study sequence. The analyses are conducted for cumulative meta-analyses and for Bayesian data analyses. The findings from these methods can be contrasted with those from null hypothesis statistical sigificance testing (NHST), which does not take previous findings into account and which often results in a back and forth ping-pong of conclusions about a phenomenon.

The function relies on the metafor package for the meta-analyses and on the MCMCglmm package for the Bayesian analyses.

The function input can be (1) a list with raw datasets, (2) a matrix with study effect size information, or (3) a matrix with study group means, standard deviations, and Ns. See the Examples below.

The function ouput includes, whenever possible, the results of meta-analyses and Bayesian analyses in four possible effect size metrics: r, Fisher's z, d, and g. Conversions between effect sizes are conducted using conventional formulas (see Borenstein, 2009).

Input data:

When the input is a list with raw datasets and the focus is on group mean differences in scores (re: rawdata_type = 'indep_groups'), then it is assumed that the grouping variable is in the first column of the input data ('donnes') and that there are only two levels of the grouping variable.

When the input is a matrix with r effect sizes (ES_type_IN = 'r'), then N should also be provided whenever possible. If r (ES) and the variance of r (ES_var) are provided but not N, then a formula is used to determine what each study N must have been: (1 - r^2) / r_var + 2.

When the input is a matrix with Fisher's z effect sizes (ES_type_IN = 'z'), then either ES_var or N must also be provided. If ES_var is provided but not N, then meta-analyses can be conducted but not the Bayesian analyses.

When the input is a matrix with d or g effect sizes (ES_type_IN = 'd' or ES_type_IN = 'g'), then it is best if grp1_N and grp2_N are also provided. If only d or g (ES) and the corresponding effect size variances (ES_var) are provided, but without the group Ns, then meta-analyses can be conducted but not the Bayesian analyses.

When the input is a matrix with odds ratios (ES_type_IN = 'OR'), then one of 'CI_in_lb' or 'CI_in_ub' must also be provided, and it is helpful if N is also provided. If one of 'CI_in_lb' or 'CI_in_ub' is provided but not N, then meta-analyses can be conducted but not the Bayesian analyses.

Bayesian analyses:

When the input is a matrix with study effect size information, then N must be provided or N must be internally computable from the provided information for the Bayesian analyses to be possible.

The Bayesian analyses are always conducted using correlation coefficient effect sizes. When the input data are not raw data nor correlation coefficients, then the correlation coefficient equivalents are computed from the input data. Conventional conversion formulae are also used when the requested output effect size metric is z, d, or g.

For the "Bayes_type" argument: The Schmidt_Raju method is the computationally fastest method of conducting the Bayesian analyses, but it is based on correlation coefficient effect sizes, or the correlation coefficient equivalents. The Bayesian analyses can also be conducted using raw data for the sequence of studies, or using data that are generated to have the same effect size as those in a raw dataset. The raw data option is generally the better approach but it is only feasible when the raw data are available. By default, the function will run the analyses for the Schmidt_Raju method, for generated data, and for raw data whenever possible.

For the "prior_type" argument: The priors used in the updating analyses (both cumulative and Bayesian) can be the effect size and standard error from previous studies based on a meta-analysis or based on Bayesian analyses. The options are 'BAYES' or 'META', and 'META' is the default.

Consistency and agreement rates are computed for the NHST analyses, for the updating MAs, and for the Bayesian analyses. The consistency rate is the proportion of times that the most common conclusion is reached for a pool of effect sizes. Three conclusions are possible for each effect size: a positive effect, a negative effect, and no effect. The signs of the effect sizes and the possible inclusion of a zero value in a confidence interval are used to make these categorizations (e.g., a negative effect conclusion is when a negative effect size has a confidence interval that does not include zero). The number of times each of the three possible conclusions occurs for a pool of effect sizes is counted, and the consistency rate is based on the most common conclusion. The agreement rate for a pool of effect sizes is the proportion of times that the conclusions for individual studies are identical to the conclusion (re: the same three categories) of the final, all-studies-combined MA. More detailed descriptions of the analytic methods were provided by O'Connor and Ermacora (2021).

The output from this function can be entered into the PLOT_NO.PING.PONG function in order to obtain a graphical display of the findings across a sequence of studies.

Value

An object of class "NO.PING.PONG". The object is a list containing the following possible components:

Author(s)

Brian P. O'Connor

References

OConnor, B. P., & Ermacora, D. (2021). Unnecessary ping-pong: Illustrations of why previous findings should be taken into account when evaluating new datasets. Canadian Journal of Behavioural Science, 53(3), 328-341. https://doi.org/10.1037/cbs0000259

O'Connor, B. P., & Khattar, N. (2022). Controversies regarding null hypothesis testing. In W. O'Donohue, A. Masuda, & S. O. Lilienfeld (Eds.). Avoiding Questionable Research Practices in Applied Psychology (pp. 147-174). Cham, Switzerland: Springer Nature Switzerland.

Examples

# data from SchmidtRaju (2007, p. 303)
data_Schmidt_Raju <- '
1    60   .44 
2    75   .20 
3    85   .60 
4   110   .32 
5    50   .41 
6    90   .25 
7   100   .12 
8    65   .35 
9    80   .35 
10   65   .19 '
data_Schmidt_Raju <- data.frame(read.table(text=data_Schmidt_Raju, fill=TRUE))
colnames(data_Schmidt_Raju) <- c('Study','N','r')

NO.PING.PONG(data_Schmidt_Raju, ES_type_IN='r', ES_type_OUT='r', 
             ma_method='REML', cor_stat = 'COR',
             Bayes_type = c('Schmidt_Raju', 'generated'), 
             prior_type='META', CI_level_out = 95,
             ES = 'r', N = 'N', ES_var = NULL,
             nitt=13000, burnin=3000, thin=10)      
                          	

	
# using only ES & ES_var (the effect size & the effect size variance for each study)
# Anxiety_Therapy
# input data = Hedges.g & ES_var for each study
# Kampmanna (2016). Meta-analysis of technology-assisted interventions for social anxiety disorder
NO.PING.PONG(data_NPP$Anxiety_Therapy, 
             ES = 'Hedges.g', N = 'N', ES_var = 'Var', ES_type_IN = 'g') 


             





# Alcohol_Intake
# input data = Hedges g & the Ns for the experimental & control groups
# Prestwich (2016). Does Changing Social Influence Engender Changes in Alcohol Intake  Table 2
NO.PING.PONG(data_NPP$Alcohol_Intake, ES_type_IN='g', ES_type_OUT='g', 
             ES = 'g.Alcohol.Intake', grp1_n = 'Exp.n', grp2_n = 'Ctrl.n') 


# Anxiety_Therapy   
# input data = Hedges g & the N for each study
# Kampmanna (2016). Meta-analysis of technology-assisted interventions for social anxiety disorder
NO.PING.PONG(data_NPP$Anxiety_Therapy, ES_type_IN='g', ES_type_OUT='g',
             ES = 'Hedges.g', N = 'N',  ma_method='FE') 


# Cannabis_Psychosis   
# input data = Cohen's d & the N for each study
# Large (2001). Cannabis Use and Earlier Onset of Psychosis - A Systematic Meta-analysis
# Supplementary Online Content - The data are from the "eFigure".
NO.PING.PONG(data_NPP$Cannabis_Psychosis, ES_type_IN='d',
             ES = 'Std_diff_in_mean', N = 'N') 


# CBT_Autism
# input data = group means, SDs, & Ns
# Weston (2016). Effectiveness of cognitive behavioural therapy with people who have autistic 
# spectrum disorders  A systematic review and meta-analysis
NO.PING.PONG(data_NPP$CBT_Autism,   
             grp1_mn = 'Con_Mean', grp1_sd = 'Con_SD', grp1_n = 'Con_N', 
             grp2_mn = 'CBT_Mean', grp2_sd = 'CBT_SD', grp2_n = 'CBT_N') 


# CBT_Social_Anxiety    
# input data = Hedges g & the Ns for the experimental & control groups
# Kampmanna (2016). Meta-analysis of technology-assisted interventions for social anxiety  fig 4
NO.PING.PONG(data_NPP$CBT_Social_Anxiety, ES_type_IN='g', 
             ES = 'Hedgesg', grp1_n = 'N.exp', grp2_n = 'N.ctrl') 


# Ego_Depletion
# input data = group means, SDs, & Ns
# Hagger (2016). A multilab preregistered replication of the ego-depletion effect
NO.PING.PONG(data_NPP$Ego_Depletion,  
             grp1_mn = 'E.Mean', grp1_sd = 'E.SD', grp1_n = 'E.N', 
             grp2_mn = 'H.Mean', grp2_sd = 'H.SD', grp2_n = 'H.N') 


# Hypomanic_BIS
# input data = group means, SDs, & Ns
# Katz (2021). The Dual-System Theory of Bipolar Spectrum Disorders   r (BIS) = -.04
NO.PING.PONG(data_NPP$Hypomanic_BIS,  
             grp1_mn = 'BIS_B_MClin', grp1_sd = 'BIS_B_SDClin', grp1_n = 'Nclin', 
             grp2_mn = 'BIS_B_MHC',   grp2_sd = 'BIS_B_SDHC',   grp2_n = 'Nhealthy') 


# IAT_Discrimination
# input data = r & N for each study
# Oswald (2013). Predicting Ethnic and Racial Discrimination - A Meta-Analysis of IAT Studies
NO.PING.PONG(data_NPP$IAT_Discrimination, ES_type_IN='r', ES = 'R', N = 'N') 


# Many_Labs$Anchoring
# input data = group means, SDs, & Ns
# Klein et al. (2014). Investigating variation in replicability: A many labs replication project
NO.PING.PONG(data_NPP$Many_Labs$Anchoring,  
             grp1_mn = 'grp1_mn', grp1_sd = 'grp1_sd', grp1_n = 'grp1_N', 
             grp2_mn = 'grp2_mn', grp2_sd = 'grp2_sd', grp2_n = 'grp2_N') 


# Many_Labs$Gamblers_Fallacy
# input data = group means, SDs, & Ns
# Klein et al. (2014). Investigating variation in replicability: A many labs replication project
NO.PING.PONG(data_NPP$Many_Labs$Gamblers_Fallacy,  
             grp1_mn = 'grp1_mn', grp1_sd = 'grp1_sd', grp1_n = 'grp1_N', 
             grp2_mn = 'grp2_mn', grp2_sd = 'grp2_sd', grp2_n = 'grp2_N') 


# Many_Labs$Math_Attitudes
# input data = group means, SDs, & Ns
# Klein et al. (2014). Investigating variation in replicability: A many labs replication project
NO.PING.PONG(data_NPP$Many_Labs$Math_Attitudes,  
             grp1_mn = 'grp1_mn', grp1_sd = 'grp1_sd', grp1_n = 'grp1_N', 
             grp2_mn = 'grp2_mn', grp2_sd = 'grp2_sd', grp2_n = 'grp2_N') 


# Many_Labs$Sunk_Costs
# input data = group means, SDs, & Ns
# Klein et al. (2014). Investigating variation in replicability: A many labs replication project
NO.PING.PONG(data_NPP$Many_Labs$Sunk_Costs,  
             grp1_mn = 'grp1_mn', grp1_sd = 'grp1_sd', grp1_n = 'grp1_N', 
             grp2_mn = 'grp2_mn', grp2_sd = 'grp2_sd', grp2_n = 'grp2_N') 


# Many_Labs$Quote_Attribution
# input data = group means, SDs, & Ns
# Klein et al. (2014). Investigating variation in replicability: A many labs replication project
NO.PING.PONG(data_NPP$Many_Labs$Quote_Attribution,  
             grp1_mn = 'grp1_mn', grp1_sd = 'grp1_sd', grp1_n = 'grp1_N', 
             grp2_mn = 'grp2_mn', grp2_sd = 'grp2_sd', grp2_n = 'grp2_N') 


# Many_Labs$Flag_Priming
# input data = group means, SDs, & Ns
# Klein et al. (2014). Investigating variation in replicability: A many labs replication project
NO.PING.PONG(data_NPP$Many_Labs$Flag_Priming,  
             grp1_mn = 'grp1_mn', grp1_sd = 'grp1_sd', grp1_n = 'grp1_N', 
             grp2_mn = 'grp2_mn', grp2_sd = 'grp2_sd', grp2_n = 'grp2_N') 


# Math_Performance
# input data = r & N for each study
# Chen (2014). Association between individual differences in non-symbolic number acuity and 
# math performance - A meta-analysis Table 1
NO.PING.PONG(data_NPP$Math_Performance, ES_type_IN='r', ES = 'r', N = 'N') 


# Omega3_Depression
# Grosso (2014).  Role of omega-3 fatty acids in the treatment of depressive disorders
# Findings are for Omega 3 predicting DSM major depression improvement - Fig 2, top portion, p. 9
# input data = group means, SDs, & Ns
NO.PING.PONG(data_NPP$Omega3_Depression,  
             grp1_mn = 'Cmn', grp1_sd = 'Csd', grp1_n = 'CN', 
             grp2_mn = 'Emn', grp2_sd = 'Esd', grp2_n = 'EN') 


# paired samples             
# input data = raw data
NO.PING.PONG(donnes=data_NPP$Paired_Samples, 
             rawdata_type = 'paired_samples',
             paired_samples_ES_type = 'SMCRH') 


# PopulationR.02 - generated data for 2 variables with a population r = .02
# input data = raw data
# O'Connor & Ermacora (2012). Unnecessary ping-pong
NO.PING.PONG(data_NPP$PopulationR.02) 


# PopulationR.10 - generated data for 2 variables with a population r = .10
# input data = raw data
# O'Connor & Ermacora (2012). Unnecessary ping-pong
NO.PING.PONG(data_NPP$PopulationR.10) 


# PopulationR.11 - generated data for 2 variables with a population r = .11
# input data = raw data
# O'Connor & Khattar (2022). Controversies regarding null hypothesis testing
NO.PING.PONG(data_NPP$PopulationR.11) 


# PopulationR.20 - generated data for 2 variables with a population r = .20
# input data = raw data
# O'Connor & Ermacora (2012). Unnecessary ping-pong
NO.PING.PONG(data_NPP$PopulationR.20) 


head(data_NPP$PopulationR.32)
# PopulationR.32 - generated data for 2 variables with a population r = .32
# input data = raw data
# O'Connor & Khattar (2022). Controversies regarding null hypothesis testing
NO.PING.PONG(data_NPP$PopulationR.32) 


head(data_NPP$PopulationR.57)
# PopulationR.57 - generated data for 2 variables with a population r = .57
# input data = raw data
# O'Connor & Khattar (2022). Controversies regarding null hypothesis testing
NO.PING.PONG(data_NPP$PopulationR.57) 


head(data_NPP$PopulationR.077)
# PopulationR.077 - generated data for 2 variables with a population r = .077
# input data = raw data
# O'Connor & Khattar (2022). Controversies regarding null hypothesis testing
NO.PING.PONG(data_NPP$PopulationR.077) 


head(data_NPP$PopulationRneg.04)
# PopulationRneg.04 - generated data for 2 variables with a population r = -04
# input data = raw data
# O'Connor & Khattar (2022). Controversies regarding null hypothesis testing
NO.PING.PONG(data_NPP$PopulationRneg.04) 


# SelfEsteem_Depression
# input data = r & N for each study
# 2013 Sowislo - Does Low Self-Esteem Predict Depression and Anxiety  r = .57
NO.PING.PONG(data_NPP$SelfEsteem_Depression, ES_type_IN='r', 
             ES = 'rSED', N = 'N') 
            
             

Plots of output from the NO.PING.PONG function

Description

Provides plots of the output from the NO.PING.PONG function for a sequence of studies

Usage

PLOT_NO.PING.PONG(nopingpongOutput,
	                  plot_this = c('NHST','CUM_META','BAYES_SR'),
	                  plot_save = FALSE, plot_save_type = 'png', 
	                  plot_title=NULL, Xrange=NULL)

Arguments

Details

This function provides plots of the output from the NO.PING.PONG function for a sequence of studies, with options for specifying the kind of results to be plotted (via the plot_this argument), whether to save the plot to disc, the file type of the saved plot (via the plot_save_type argument), the plot title, and the x axis range for the plot.

Value

A plot is produced, but there are no returned values.

Author(s)

Brian P. O'Connor

References

O'Connor, B. P., & Ermacora, D. (2021). Unnecessary ping-pong: Illustrations of why previous findings should be taken into account when evaluating new datasets. Canadian Journal of Behavioural Science, 53(3), 328-341. https://doi.org/10.1037/cbs0000259

O'Connor, B. P., & Khattar, N. (2022). Controversies regarding null hypothesis testing. In W. O'Donohue, A. Masuda, & S. O. Lilienfeld (Eds.). Avoiding Questionable Research Practices in Applied Psychology (pp. 147-174). Cham, Switzerland: Springer Nature Switzerland.

Examples

# data from SchmidtRaju (2007, p. 303)
data_Schmidt_Raju <- '
1    60   .44 
2    75   .20 
3    85   .60 
4   110   .32 
5    50   .41 
6    90   .25 
7   100   .12 
8    65   .35 
9    80   .35 
10   65   .19 '
data_Schmidt_Raju <- data.frame(read.table(text=data_Schmidt_Raju, fill=TRUE))
colnames(data_Schmidt_Raju) <- c('Study','N','r')
data_Schmidt_Raju <- data_Schmidt_Raju[,2:3]  

nppOutput <- NO.PING.PONG(data_Schmidt_Raju, ES_type_IN='r', ES_type_OUT='r', 
                          ma_method='REML',
                          Bayes_type = c('Schmidt_Raju', 'generated'), 
                          prior_type='META', CI_level_in = 95,
                          ES = 'r', N = 'N', ES_var = NULL,
                          nitt=13000, burnin=3000, thin=10, verbose=TRUE)      

PLOT_NO.PING.PONG(nppOutput, plot_this = c('NHST','CUM_META','BAYES_SR','BAYES_GEN'))


 
# Cannabis Psychosis data
nppOutput <- NO.PING.PONG(data_NPP$Cannabis_Psychosis, ES_type_IN='d', ma_method='REML',
                          Bayes_type = c('Schmidt_Raju', 'generated'), prior_type='META',
                          ES = 'Std_diff_in_mean', N = 'N', ES_var = NULL, #ES_var = 'Variance',
                          nitt=13000, burnin=3000, thin=10, verbose=TRUE) 

PLOT_NO.PING.PONG(nppOutput, plot_this = c('NHST','CUM_META'))
PLOT_NO.PING.PONG(nppOutput, plot_this = c('NHST','CUM_META','BAYES_SR','BAYES_GEN'))
PLOT_NO.PING.PONG(nppOutput, plot_this = c('NHST','CUM_META','BAYES_GEN','BAYES_RAW'))


# PopulationR.20 data (has raw data)
nppOutput <- NO.PING.PONG(data_NPP$PopulationR.20, ES_type_OUT='r',
                          rawdata_type = 'for_correl',
                          ma_method='REML',
                          Bayes_type = c('generated', 'Schmidt_Raju'), 
                          prior_type='META', CI_level_in = 95,
                          ES = 'r', N = 'N', ES_var = NULL,
                          nitt=13000, burnin=3000, thin=10, verbose=TRUE) 
                         
PLOT_NO.PING.PONG(nppOutput, plot_this = c('NHST','CUM_META','BAYES_GEN','BAYES_RAW'))



# raw data for paired samples             
nppOutput <- 
NO.PING.PONG(donnes=data_NPP$Paired_Samples, 
             rawdata_type = 'paired_samples',
             ES_type_OUT = 'd',
             paired_samples_ES_type = 'SMCRH') 

PLOT_NO.PING.PONG(nppOutput, plot_this = c('NHST','CUM_META'))
PLOT_NO.PING.PONG(nppOutput, plot_this = c('NHST','CUM_META','BAYES_GEN'))
PLOT_NO.PING.PONG(nppOutput, plot_this = c('NHST','CUM_META','BAYES_GEN','BAYES_RAW'))

data_NPP

Description

A list with variety of example datasets

Usage

data(data_NPP)

Details

A list with the following elements:

Alcohol_Intake is data from a meta-analysis on 36 effect sizes (Hedges g values) for the effect of manipulations of social influences on perceived norms regarding alcohol intake and on reductions in alcohol intake (Prestwich, Kellar, Conner, Lawton, Gardner,& Turgut, 2016, Table 2, p. 850).

Anxiety_Therapy is data from a meta-analysis on 81 effect sizes (Hedges g values) for the pre-post difference between transdiagnostic cognitive behavior therapy and anxiety (Pearl & Norton, 2017, Figure 2, p. 14).

Cannabis_Psychosis is data from a meta-analysis on 38 effect sizes (standardized mean differences) from comparisons of the age at onset of psychosis in cannabis-using groups with the age at onset of psychosis in noncannabis-using groups (Large et al., 2001). The data are from the "eFigure" in the Supplementary Material.

CBT_Autism is data from Weston, Hodgekins, and Langdon (2016), who reported findings from a meta-analysis of 17 studies on the effectiveness of CBT on affective symptoms for people with autistic spectrum disorders (median N = 36). The final effect size, in correlation coefficient metric, was .11.

CBT_Social_Anxiety is data from Kampmanna, Emmelkamp, and Morina (2016, Figure 4), who reported findings from a meta-analysis of 24 studies on internet delivered cognitive behavior therapy (vs. control conditions) for social anxiety disorder (median N = 65). The final effect size, in correlation coefficient metric, was .32.

Ego_Depletion is data from the pre-registered, replication studies for the ego-depletion effect (Hagger et al., 2016). These data were from 23 laboratories (N = 2141).

Hypomanic_BIS is data from Katz, Naftalovich, Matanky, and Yovel (2021), who reported findings from a meta-analysis of 19 studies on hypomanic personality tendencies and behavioral inhibition system sensitivity (median N = 230). The final effect size, in correlation coefficient metric, was -.04.

IAT_Discrimination is data from a meta-analysis on 298 effect sizes (r values) for the association between scores on the Implicit Association Test (IAT) and ethnic and racial discrimination (Oswald, Mitchell, Blanton, Jaccard, & Tetlock (2013).

Many_Labs is a list with summary data for eight pre-registered datasets described by Klein et al. (2014; https://osf.io/wx7ck/). The datasets were from the Many Labs Replication Project. For all eight projects, there were attempts to replicate the original effect across 36 samples (studies), involving over 6,000 participants. The data are for: (1) Sunk Costs (Oppenheimer, Meyvis, & Davidenko, 2009), wherein participants are supposedly more likely to go see their favorite football team play an important game on a freezing cold day if they had paid for the ticket than if the ticket had been free; (2) Anchoring (Jacowitz & Kahneman, 1995), wherein participants subsequent judgments are supposedly affected by anchoring information provided by researchers; (3) Gamblers Fallacy (Oppenheimer & Monin, 2009), wherein the rarity of an independent, chance observation supposedly influences beliefs about what occurred before the event; (4) Quote Attribution (Lorge & Curtiss, 1936), wherein participants opinions are supposedly more influenced by whether they are exposed to the opinions of a highly regarded than a lowly regarded other person; (5) Flag Priming (Carter, Ferguson, & Hassin, 2011; Study 2), wherein subtle exposure to the American flag supposedly increases conservative opinions among US participants; (6) Currency Priming (Caruso, Vohs, Baxter, & Waytz, 2013), wherein mere exposure to money symbols increases participants endorsements of the current social system; (7) Imagined Contact (Husnu & Crisp, 2010; Study 1), wherein merely imagining contact with members of ethnic outgroups is supposedly sufficient to reduce prejudice toward those groups; and (8) Math Attitudes (Nosek, Banaji, & Greenwald, 2002), wherein women supposedly have more negative implicit attitudes toward math compared to arts than do men.

Math_Performance is data from a meta-analysis on 35 effect sizes (r values) for the association between individual differences in non-symbolic number acuity and math performance (Chen & Li, 2014, Table 1, p. 167).

Omega3_Depression is data from a meta-analysis on 12 effect sizes for the role of omega-3 fatty acids in the treatment of major depression (Grosso et al., 2014, Figure 2, top portion, p. 9).

Paired_Samples is a list with raw data for two variables from 50 random samples, N = 100 each, that were randomly drawn from a population of 100,000 cases in which the difference in mean scores was .50 and the correlation between the two variables was .70.

PopulationR.02 is a list with raw data for two variables from 50 random samples, N = 100 each, that were randomly drawn from a population of 100,000 cases in which the correlation between the two variables was .02. The data are from O'Connor and Ermacora (2021).

PopulationR.10 is a list with raw data for two variables from 50 random samples, N = 100 each, that were randomly drawn from a population of 100,000 cases in which the correlation between the two variables was .10. The data are from O'Connor and Ermacora (2021).

PopulationR.11 is a list with raw data for two variables from 50 random samples, N = 36 each, that were randomly drawn from a population of 100,000 cases in which the correlation between the two variables was .11. The data are from O'Connor and Khattar (2022).

PopulationR.20 is a list with raw data for two variables from 50 random samples, N = 100 each, that were randomly drawn from a population of 100,000 cases in which the correlation between the two variables was .20. The data are from O'Connor and Ermacora (2021).

PopulationR.32 is a list with raw data for two variables from 50 random samples, N = 65 each, that were randomly drawn from a population of 100,000 cases in which the correlation between the two variables was .32. The data are from O'Connor and Khattar (2022).

PopulationR.57 is a list with raw data for two variables from 50 random samples, N = 225 each, that were randomly drawn from a population of 100,000 cases in which the correlation between the two variables was .57. The data are from O'Connor and Khattar (2022).

PopulationR.077 is a list with raw data for two variables from 50 random samples, N = 50 each, that were randomly drawn from a population of 100,000 cases in which the correlation between the two variables was .077. The data are from O'Connor and Khattar (2022).

PopulationRneg.04 is a list with raw data for two variables from 50 random samples, N = 230 each, that were randomly drawn from a population of 100,000 cases in which the correlation between the two variables was -.04. The data are from O'Connor and Khattar (2022).

SelfEsteem_Depression is data from Sowislo and Orth (2013, Table 2), who reported findings from a meta-analysis of 77 longitudinal studies that provided effect sizes for self-esteem predicting future depressive symptoms (median N = 224). The final effect size, in correlation coefficient metric, was .57.

References

Chen, Q., & Li, J. (2014). Association between individual differences in non-symbolic number acuity and math performance: A meta-analysis. Acta Psychologica, 148, 163-172.

Grosso, G., Pajak, A., Marventano, S., Castellano, S., Galvano, F., Bucolo, C., & Caraci, F. (2014). Role of omega-3 fatty acids in the treatment of depressive disorders: A comprehensive meta-analysis of randomized clinical trials. PLoS ONE, 9(5), e96905.

Hagger, M. S., Chatzisarantis, N. L., Alberts, H. A., Anggono, C. O., Batailler, C. B., Birt, A. R., et al. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11(4), 546-573.

Kampmann, I. L., Emmelkamp, P. M. G., & Morina, N. (2016). Meta-analysis of technology-assisted interventions for social anxiety disorder. Journal of Anxiety Disorders, 42, 7184.

Katz, B. A., Naftalovich, H., Matanky, K., & Yovel, I. (2021). The dual-system theory of bipolar spectrum disorders: A meta-analysis. Clinical Psychology Review, 83, Article 101945.

Klein, R. A., et al. (2014). Investigating variation in replicability: A many labs replication project. Social Psychology, 45, 142152.

Large, M., Sharma, S., Compton, M. T., Slade, T., Nielssen, O. (2011). Cannabis use and earlier onset of psychosis: a systematic meta-analysis. Archives of General Psychiatry, 68(6), 555-561.

O'Connor, B. P., & Ermacora, D. (2021). Unnecessary ping-pong: Illustrations of why previous findings should be taken into account when evaluating new datasets. Canadian Journal of Behavioural Science, 53(3), 328-341. https://doi.org/10.1037/cbs0000259

O'Connor, B. P., & Khattar, N. (2022). Controversies regarding null hypothesis testing. In W. O'Donohue, A. Masuda, & S. O. Lilienfeld (Eds.). Avoiding Questionable Research Practices in Applied Psychology (pp. 147-174). Cham, Switzerland: Springer Nature Switzerland.

Oswald, F. L., Mitchell, G., Blanton, H., Jaccard, J., & Tetlock, P. E. (2013). Predicting ethnic and racial discrimination: A meta-analysis of IAT criterion studies. Journal of Personality and Social Psychology, 105, 171-192.

Pearl, S. B., & Norton, P. J. (2017). Transdiagnostic versus diagnosis specific cognitive behavioural therapies for anxiety: A meta-analysis. Journal of Anxiety Disorders, 46, 11-24.

Prestwich, A., Kellar, I., Conner, M., Lawton, R., Gardner, P., & Turgut, L. (2016). Does changing social influence engender changes in alcohol intake? A meta-analysis. Journal of Consulting & Clinical Psychology, 84, 845-860.

Sowislo, J. F., & Orth, U. (2013). Does low self-esteem predict depression and anxiety? A meta-analysis of longitudinal studies. Psychological Bulletin, 139(1), 213240. https://doi.org/10.1037/a0028931

Weston, L., Hodgekins, J., & Langdon, P. E. (2016). Effectiveness of cognitive behavioural therapy with people who have autistic spectrum disorders: A systematic review and meta-analysis. Clinical Psychology Review, 49, 4154. https://doi.org/10.1016/j.cpr.2016.08.001

Examples

names(data_NPP)
 
head(data_NPP$Alcohol_Intake)

head(data_NPP$Anxiety_Therapy)

head(data_NPP$Cannabis_Psychosis)

head(data_NPP$CBT_Autism)

head(data_NPP$CBT_Social_Anxiety)

head(data_NPP$Ego_Depletion)

head(data_NPP$Hypomanic_BIS)

head(data_NPP$IAT_Discrimination)

head(data_NPP$Many_Labs)

head(data_NPP$Math_Performance)

head(data_NPP$Omega3_Depression)

head(data_NPP$Paired_Samples)

head(data_NPP$PopulationR.02)

head(data_NPP$PopulationR.10)

head(data_NPP$PopulationR.11)

head(data_NPP$PopulationR.20)

head(data_NPP$PopulationR.32)

head(data_NPP$PopulationR.57)

head(data_NPP$PopulationR.077)

head(data_NPP$PopulationRneg.04)

head(data_NPP$SelfEsteem_Depression)