Type: | Package |
Title: | Tidy Multiverse Analysis Made Simple |
Version: | 0.2.2 |
Maintainer: | Michael Jongho Moon <michael.moon@utoronto.ca> |
Description: | Extends 'multiverse' package (Sarma A., Kale A., Moon M., Taback N., Chevalier F., Hullman J., Kay M., 2021) <doi:10.31219/osf.io/yfbwm>, which allows users perform to create explorable multiverse analysis in R. This extension provides an additional level of abstraction to the 'multiverse' package with the aim of creating user friendly syntax to researchers, educators, and students in statistics. The 'mverse' syntax is designed to allow piping and takes hints from the 'tidyverse' grammar. The package allows users to define and inspect multiverse analysis using familiar syntax in R. |
License: | GPL (≥ 3) |
Encoding: | UTF-8 |
LazyData: | true |
Depends: | R (≥ 3.6), multiverse (≥ 0.6.2) |
Imports: | Rdpack, magrittr (≥ 1.5), rlang, dplyr (≥ 1.1), tidyr, tidyselect, stringr, stats, utils, broom, igraph, ggraph, ggplot2 (≥ 3.5.2), ggupset (≥ 0.4.1) |
Suggests: | methods, tibble, purrr, scales, MASS, testthat (≥ 3.0.0), pkgdown (≥ 1.5.1), covr, knitr, rmarkdown, kableExtra, spelling |
RoxygenNote: | 7.3.2 |
RdMacros: | Rdpack |
VignetteBuilder: | knitr |
URL: | https://github.com/mverseanalysis/mverse/, https://mverseanalysis.github.io/mverse/ |
Config/testthat/edition: | 3 |
Language: | en-US |
NeedsCompilation: | no |
Packaged: | 2025-06-20 16:29:40 UTC; moon |
Author: | Michael Jongho Moon [aut, cre], Haoda Li [aut], Mingwei Xu [aut], Nathan Taback [aut], Fanny Chevalier [aut], Alison Gibbs [ctb] |
Repository: | CRAN |
Date/Publication: | 2025-06-21 10:40:06 UTC |
Display the AIC and BIC score of the fitted models across the multiverse
Description
Display the AIC and BIC score of glm
regression
results across the multiverse.
Usage
## S3 method for class 'mverse'
AIC(object, ..., k = 2)
## S3 method for class 'mverse'
BIC(object, ...)
AIC(object, ..., k = 2)
BIC(object, ...)
Arguments
object |
a |
... |
ignored. for compatibility only. |
k |
ignored. for compatibility only. |
Value
a multiverse table as a tibble
Add branch conditions to a mverse
object.
Description
This method adds one or more branch conditions to
an existing mverse
object. Branch conditions
are used to specify an option in one branch dependent
on an option in another branch.
Usage
add_branch_condition(.mverse, ...)
## S3 method for class 'mverse'
add_branch_condition(.mverse, ...)
Arguments
.mverse |
a |
... |
branch conditions. |
Value
a mverse
object.
See Also
Other branch condition functions:
branch_condition()
Examples
# Define branches and add them to an \code{mverse} object.
y <- mutate_branch(alldeaths, log(alldeaths + 1))
distribution <- family_branch(poisson, gaussian)
# You can match branching options by providing the options
# the way provide them when defining branches.
match_poisson <- branch_condition(alldeaths, poisson)
mv <- mverse(hurricane) %>%
add_mutate_branch(y) %>%
add_family_branch(distribution) %>%
add_branch_condition(match_poisson)
summary(mv)
# You can also condition to reject a pair of options by
# setting reject = TRUE.
match_log_lin <- branch_condition(log(alldeaths + 1), poisson, reject = TRUE)
mv <- add_branch_condition(mv, match_log_lin)
summary(mv)
Add family branches to a mverse
object.
Description
This method adds a family branch to an existing mverse
object.
A family branch is used to define options for the analysis distributions
when using glm_mverse()
.
Usage
add_family_branch(.mverse, br)
Arguments
.mverse |
a |
br |
a |
Value
The resulting mverse
object.
See Also
Other family branch functions:
family_branch
Examples
# Define a family branch.
model_distributions <- family_branch(
gaussian, poisson(link = "log")
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
add_family_branch(model_distributions)
Add filter branches to a mverse
object.
Description
This method adds one or more filter branches to
an existing mverse
object. Filter branches
are used to define options for conditions
for selecting subsets of data rows.
Usage
add_filter_branch(.mverse, ...)
Arguments
.mverse |
a |
... |
|
Value
The resulting mverse
object.
See Also
Other filter branch functions:
filter_branch
Examples
# Define a filter branch.
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
!Name %in% c("Katrina"),
!Name %in% c("Katrina"),
TRUE # include all
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
add_filter_branch(hurricane_outliers)
Add formula branches to a mverse
object.
Description
This method adds a formula branch to an existing mverse
object.
A formula branch is used to specify model structure options for the
analysis.
Usage
add_formula_branch(.mverse, br)
Arguments
.mverse |
a |
br |
a |
Value
The resulting mverse
object.
See Also
Other formula branch functions:
formula_branch
Examples
# Define a formula branch.
model_specifications <- formula_branch(
y ~ MasFem,
y ~ MasFem + hurricane_strength,
y ~ MasFem * hurricane_strength
)
# Create a mverse, add the branch.
mv <- create_multiverse(hurricane) %>%
add_formula_branch(model_specifications)
Add mutate branches to a mverse
object.
Description
This method adds one or more mutate branches to
an existing mverse
object. Mutate branches
are used to define options for adding a new
column to the analysis dataset.
Usage
add_mutate_branch(.mverse, ...)
Arguments
.mverse |
a |
... |
|
Value
The resulting mverse
object.
See Also
Other mutate branch functions:
mutate_branch
Examples
# Define mutate branches.
hurricane_strength <- mutate_branch(
# damage vs. wind speed vs.pressure
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014,
# Standardized versions
scale(NDAM),
scale(HighestWindSpeed),
-scale(Minpressure_Updated_2014),
)
y <- mutate_branch(
alldeaths, log(alldeaths + 1)
)
# Create a mverse and add the branches.
mv <- create_multiverse(hurricane) %>%
add_mutate_branch(hurricane_strength) %>%
add_mutate_branch(y)
# You can also add multiple branches with a single call.
mv <- create_multiverse(hurricane) %>%
add_mutate_branch(hurricane_strength, y)
Create a new branch condition.
Description
A branch condition conditions option x
to
depend on option y
. When the branch condition
is added to a mverse
object, option x
is executed only when y
is. Use reject = TRUE
,
to negate the condition.
Usage
branch_condition(x, y, reject = FALSE)
Arguments
x |
option 1 |
y |
option 2 |
reject |
if TRUE, the condition rejects universes with option 1 and option 2 |
Value
A branch_condition
object.
See Also
Other branch condition functions:
add_branch_condition()
Examples
# Example branches.
y <- mutate_branch(alldeaths, log(alldeaths + 1))
model <- formula_branch(y ~ MasFem * strength, y ~ MasFem + strength)
# Define a new branch condition.
match_poisson <- branch_condition(alldeaths, poisson)
# Define a branch condition that reject an option dependent on another.
match_log_lin <- branch_condition(log(alldeaths + 1), poisson, reject = TRUE)
Execute the entire multiverse.
Description
This method executes the analysis steps
defined in the mverse
objected
across the entire multiverse.
Usage
execute_multiverse(.mverse, parallel = FALSE, progress = FALSE)
## S3 method for class 'mverse'
execute_multiverse(.mverse, parallel = FALSE, progress = FALSE)
Arguments
.mverse |
a |
parallel |
passed to |
progress |
passed to |
Value
The resulting mverse
object.
Examples
# Define a mutate branch.
hurricane_strength <- mutate_branch(
# damage vs. wind speed vs.pressure
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014,
# Standardized versions
scale(NDAM),
scale(HighestWindSpeed),
-scale(Minpressure_Updated_2014),
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
add_mutate_branch(hurricane_strength)
# The branched variables are not populated across the multiverse yet.
# Execute the multiverse; the variables are populated after the execution.
execute_multiverse(mv)
Extract branched values.
Description
extract
returns a tibble of selected values
across the multiverse in a long format.
Usage
extract(...)
## S3 method for class 'mverse'
extract(
.mverse,
columns = NULL,
nuni = NULL,
frow = NULL,
include_branch_options = TRUE,
...
)
Arguments
... |
Ignored. |
.mverse |
a |
columns |
a character vector of column names to extract. |
nuni |
a positive integer for the number of universes to extract. |
frow |
proportion of rows to extract from each universe. |
include_branch_options |
when |
Details
This method extracts data values across
the multiverse. You can specify a subset of data
to extract using columns
, universe
,
nuni
, and frow
.
You can specify the columns to extract from each
universe by passing the column names as a character
vector to columns
. The default values is
NULL
extracting all columns with branches.
Use universe
to specify a set of universes
by their integer ids. Use nuni
to specify
the number of universes to extract data from. The
method then selects the subset randomly. Specifying
universe
manually will override nuni
value.
By default, they are both set to NULL
and
the method returns data from all universes.
Use frow
to randomly extract a fraction of
data from each universe. The default value is NULL
and all rows are returned as they are. Note if you select
1 the method will return shuffle rows in each universe
before returning them. If frow
is greater than 1,
the method randomly samples rows with replacement.
Value
a tibble containing the selected columns across the multiverse.
Examples
# Define mutate branches.
hurricane_strength <- mutate_branch(
# damage vs. wind speed vs.pressure
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014,
# Standardized versions
scale(NDAM),
scale(HighestWindSpeed),
-scale(Minpressure_Updated_2014),
)
y <- mutate_branch(
alldeaths, log(alldeaths + 1)
)
# Create a mverse and add the branches.
mv <- create_multiverse(hurricane) %>%
add_mutate_branch(hurricane_strength, y)
execute_multiverse(mv)
# Extract all branched columns from all universes
extract(mv)
# Specify the columns to extract from each universe using columns
# You can select both branched and non-branched columns
extract(mv, columns = c("hurricane_strength", "NDAM"))
# Specify the universe to extract from using universe
extract(mv, universe = 1)
# Specify the number of universes to extract from using nuni
# The universes are randomly selected
extract(mv, nuni = 3)
# Specify the proportion of data to extract from each universe using
# frow. The rows are randomly selected
extract(mv, frow = 0.7)
Create a new family branch.
Description
Create a new family branch.
Usage
family_branch(..., name = NULL)
Arguments
... |
branch definition expressions. |
name |
(optional) Name for the new family. |
Value
a family_branch
object.
See Also
Other family branch functions:
add_family_branch()
Examples
# Define a family branch.
model_distributions <- family_branch(
gaussian, poisson(link = "log")
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
add_family_branch(model_distributions)
Create a new filter branch.
Description
Create a new filter branch.
Usage
filter_branch(..., name = NULL)
Arguments
... |
branch definition expressions. |
name |
(optional) Name for the new filter. |
Value
a filter_branch object.
See Also
Other filter branch functions:
add_filter_branch()
Examples
# Define a filter branch.
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
!Name %in% c("Katrina"),
!Name %in% c("Katrina"),
TRUE # include all
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
add_filter_branch(hurricane_outliers)
Create a new formula branch.
Description
The function specifies the model formula for fitting 'lm_mverse()' and
'glm_mverse()'. You can list the model specification formulae individually
or use covariates
option paired with one or more formulae.
Usage
formula_branch(..., covariates = NULL, name = NULL)
Arguments
... |
branch definition expressions. |
covariates |
(optional) A character vector of optional covariates. Each unique combination of the supplied covariates is translated into a unique branch option. See Details. |
name |
(optional) Name for the new formula. |
Details
The optional argument covariates
is allows you to specify a set of
optional covariates in addition to other independent variable such as
treatment variables and blocking variables which are specified using formula.
For each covariate provided, a branch is added to the multiverse with the
option to include or exclude the covariate in the model.
For example, formula_branch(y ~ x, covariates = c("c1", "c2"))
creates
the following 4 model specifications:
y ~ x
y ~ x + c1
y ~ x + c2
y ~ x + c1 + c2
Here, y
is the outcome variable and x
may be a treatment
variable in an experiment setting. c1
and c2
may be additional
covariates about the experiment units that may or may not be relevant.
Value
a formula_branch
object.
See Also
Other formula branch functions:
add_formula_branch()
Examples
# Define a formula branch.
model_specifications <- formula_branch(
y ~ MasFem,
y ~ MasFem + hurricane_strength,
y ~ MasFem * hurricane_strength
)
# Create a mverse, add the branch.
mv <- create_multiverse(hurricane) %>%
add_formula_branch(model_specifications)
# Specify the covariates separately.
model_specifications <- formula_branch(
y ~ MasFem,
covariates = c("hurricane_strength", "Year", "Category", "NDAM")
)
model_specifications
Fit negative binomial regression models across the multiverse
Description
glm.nb_mverse
fits MASS::glm.nb
across the multiverse
according to model specifications provided by formula_branch
.
At least one formula_branch
must have been added.
Usage
glm.nb_mverse(.mverse, parallel = FALSE, progress = FALSE)
Arguments
.mverse |
a |
parallel |
passed to |
progress |
passed to |
Value
A mverse
object with glm.nb
fitted.
See Also
Other model fitting functions:
glm_mverse()
,
lm_mverse()
Examples
# Displaying the multiverse table with \code{glm.nb} models fitted.
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
TRUE # include all
)
model_specifications <- formula_branch(alldeaths ~ MasFem)
mv <- create_multiverse(hurricane) %>%
add_filter_branch(hurricane_outliers) %>%
add_formula_branch(model_specifications) %>%
glm.nb_mverse()
summary(mv)
Fit generalized linear regression models across the multiverse.
Description
glm_mverse
fits glm
across the multiverse
according to model specifications provided by formula_branch
.
At least one formula_branch
must have been added.
You can also specify the underlying error distribution and
the link function by adding a family_branch
. If no
family_branch
has been provided, it follows
the default behaviour of glm
using the Gaussian
distribution with an identity link.
Usage
glm_mverse(.mverse, parallel = FALSE, progress = FALSE)
Arguments
.mverse |
a |
parallel |
passed to |
progress |
passed to |
Value
A mverse
object with glm
fitted.
See Also
Other model fitting functions:
glm.nb_mverse()
,
lm_mverse()
Examples
# Fitting \code{glm} models across a multiverse.
hurricane_strength <- mutate_branch(
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014
)
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
TRUE # include all
)
model_specifications <- formula_branch(
alldeaths ~ MasFem,
alldeaths ~ MasFem + hurricane_strength
)
model_distributions <- family_branch(poisson)
mv <- create_multiverse(hurricane) %>%
add_filter_branch(hurricane_outliers) %>%
add_mutate_branch(hurricane_strength) %>%
add_formula_branch(model_specifications) %>%
add_family_branch(model_distributions) %>%
glm_mverse()
Data on Atlantic hurricanes in the U.S. between 1950 and 2012.
Description
A dataset for the study conducted by Jung et al. (2014) in Female hurricanes are deadlier than male hurricanes.
Usage
hurricane
Format
A data frame with 94 rows and 12 variables:
- Year
Year in which the hurricane landed on U.S.
- Name
Name of the hurricane.
- MasFem
Femininity index of the hurricane name collected by Jung et al. (1 - very masculine; 11 - very feminine).
- MinPressure_before
Minimum pressure of the hurricane at the time of landfall in the U.S. (original).
- Minpressure_Updated_2014
Minimum pressure of the hurricane at the time of landfall in the U.S. (updated).
- Gender_MF
Gender indicator for the hurricane name based on
MasFem
index (1 -MasFem
> 6; 0 otherwise).- Category
Hurricane category on a scale of 1 to 5, with 5 being the most severe.
- alldeaths
Number of fatalities.
- NDAM
Normalized damage in 2013 U.S. million dollars.
- Elapsed.Yrs
Time since hurricane.
- Source
Source from where the data was gathered.
- HighestWindSpeed
Maximum wind speed.
- MasFem_MTUrk
Femininity index of the hurricane name collected by Simonsohn et al.
- NDAM15
Normalized damage in 2015 U.S. million dollars.
Details
The dataset was collected by Jung et al. in their study Female hurricanes are deadlier than male hurricanes. Their study didn't include hurricanes Katrina and Audrey which were deemed as outliers. Simonsohn et al. collected the extra data for Specification curve analysis including an additional femininity index based on an MTUrk survey and updated normalized damage amount in 2015 U.S. dollars.
This dataset includes data prepared by Jung et al. (2014) as well as those prepared by Simonsohn et al. (2020). Specifically, all data on Katrina and Audrey are from Simonsohn et al. (2020) except minimum pressure updated in 2014. They were retrieved from Continental United States Hurricane Impacts/Landfalls 1851-2021 table maintained by U.S. National Oceanic and Atmospheric Administration. Maximum wind speed, femininity index from MTUrk survey, and 2015 damage amounts are also from Simonsohn et al. (2020).
Source
Kiju Jung, Sharon Shavitt, Madhu Viswanathan, and Joseph M. Hilbe. (2014). "Female hurricanes are deadlier than male hurricanes." Proceedings of the National Academy of Sciences, 111(24), 8782-8787. doi:10.1073/pnas.1402786111
Uri Simonsohn, Joseph P. Simmons, and Leif D. Nelson. (2020). “Specification curve analysis” Nature Human Behaviour, 4, 1208–14. doi:10.1038/s41562-020-0912-z
Fit linear regression models across the multiverse.
Description
lm_mverse
fits lm
across the multiverse
according to model specifications provided by formula_branch
.
At least one formula_branch
must have been added.
Usage
lm_mverse(.mverse, parallel = FALSE, progress = FALSE)
Arguments
.mverse |
a |
parallel |
passed to |
progress |
passed to |
Value
A mverse
object with lm
fitted.
See Also
Other model fitting functions:
glm.nb_mverse()
,
glm_mverse()
Examples
# Fitting \code{lm} models fitted across a multiverse.
hurricane_strength <- mutate_branch(
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014
)
y <- mutate_branch(
alldeaths, log(alldeaths + 1)
)
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
TRUE # include all
)
model_specifications <- formula_branch(
y ~ MasFem,
y ~ MasFem + hurricane_strength
)
mv <- create_multiverse(hurricane) %>%
add_filter_branch(hurricane_outliers) %>%
add_mutate_branch(hurricane_strength, y) %>%
add_formula_branch(model_specifications) %>%
lm_mverse()
Plot a multiverse tree diagram.
Description
A multiverse tree diagram displays the branching combination
of all the branches added to the given mverse
object
taking any branch conditions defined. The method also allows
zooming into a subset of branches using branches
parameter.
Usage
multiverse_tree(
.mverse,
label = "none",
branches = NULL,
label_size = NULL,
label_angle = 0,
label_hjust = 0,
label_vjust = 0
)
Arguments
.mverse |
A |
label |
Display the branch option name when "name" or the definition when "code". No label is displayed when "none" (default). |
branches |
A character vector. Display a subset of branches when specified. Display all when NULL. |
label_size |
A numeric. Set size of option labels. |
label_angle |
A numeric. Rotate option labels. |
label_hjust |
A numeric. Set the horizontal justification of the node labels. |
label_vjust |
A numeric. Set the vertical justification of the node labels. |
Value
A ggplot
object displaying the multiverse tree.
Examples
{
# Display a multiverse tree with multiple branches.
outliers <- filter_branch(!Name %in% c("Katrina", "Audrey"), TRUE)
femininity <- mutate_branch(MasFem, Gender_MF)
strength <- mutate_branch(
NDAM, HighestWindSpeed, Minpressure_Updated_2014, log(NDAM)
)
y <- mutate_branch(alldeaths, log(alldeaths + 1))
model <- formula_branch(y ~ femininity * strength, y ~ femininity + strength)
distribution <- family_branch(poisson, gaussian)
mv <- mverse(hurricane) %>%
add_filter_branch(outliers) %>%
add_mutate_branch(femininity, strength, y) %>%
add_formula_branch(model) %>%
add_family_branch(distribution)
multiverse_tree(mv)
# Display a multiverse tree with branch conditions.
match_poisson <- branch_condition(alldeaths, poisson)
match_log_lin <- branch_condition(log(alldeaths + 1), gaussian)
add_branch_condition(mv, match_poisson)
add_branch_condition(mv, match_log_lin)
multiverse_tree(mv)
# You can adjust colour scale of the edges
# using a ggraph::scale_edge_colour*() function.
multiverse_tree(mv) + ggraph::scale_edge_colour_viridis(
discrete = TRUE,
labels = c("Distribution", "Model", "Strength",
"Femininity", "Outliers", "y")
)
# Display a multiverse tree for a subset of branches
# with name label for each option.
multiverse_tree(mv, branches = c("y", "distribution"), label = "name")
# with code label for each option.
multiverse_tree(mv, branches = c("y", "distribution"), label = "code")
# adjusting size and orientation of the labels
multiverse_tree(mv, branches = c("y", "distribution"),
label = "name", label_size = 4, label_angle = 45)
}
Create a new mutate branch.
Description
Create a new mutate branch.
Usage
mutate_branch(..., name = NULL)
Arguments
... |
branch definition expressions. |
name |
(optional) Name for the new variable. |
Value
a mutate_branch object.
See Also
Other mutate branch functions:
add_mutate_branch()
Examples
# Define mutate branches.
hurricane_strength <- mutate_branch(
# damage vs. wind speed vs.pressure
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014,
# Standardized versions
scale(NDAM),
scale(HighestWindSpeed),
-scale(Minpressure_Updated_2014),
)
# Create a mverse and add the branch.
mv <- create_multiverse(hurricane) %>%
add_mutate_branch(hurricane_strength)
Create a new mverse
object
Description
Constructs a new mverse object which extends
multiverse::multiverse
object.
Usage
mverse(data)
create_multiverse(data)
Arguments
data |
source dataframe. |
Value
A mverse
object with the source dataframe attached.
Examples
# Create a mverse object.
mv <- mverse(hurricane)
# create_multiverse() is an alias of mverse().
mv <- create_multiverse(hurricane)
Print method for *_branch
objects.
Description
Print method for *_branch
objects.
Usage
## S3 method for class 'branch'
print(x, ...)
Arguments
x |
a |
... |
ignored. for compatibility only. |
Value
No return value, called for printing only.
Objects exported from other packages
Description
These objects are imported from other packages. Follow the links below to see their documentation.
- magrittr
Number of cards given for each referee-player pair in soccer.
Description
A dataset containing card counts between 2,053 soccer players playing in the first male divisions of England, Germany, France, and Spain in the 2012-2013 season and 3,147 referees that these players played under in professional matches. The dataset contains other covariates including 2 independent skin tone ratings per player. Each line represents a player-referee pair.
Usage
soccer
Format
A data frame with 146,028 rows and 26 variables:
- playerShort
short player ID
- player
player name
- club
player club
- leagueCountry
country of player club ( England, Germany, France, and Spain)
- birthday
player birthday
- height
player height (in cm)
- weight
player weight (in kg)
- position
detailed player position
- games
number of games in the player-referee dyad
- victories
victories in the player-referee dyad
- ties
ties in the player-referee dyad
- defeats
losses in the player-referee dyad
- goals
goals scored by a player in the player-referee dyad
- yellowCards
number of yellow cards player received from referee
- yellowReds
number of yellow-red cards player received from referee
- redCards
number of red cards player received from referee
- rater1
skin rating of photo by rater 1 (5-point scale ranging from “very light skin” to “very dark skin”)
- rater2
skin rating of photo by rater 2 (5-point scale ranging from “very light skin” to “very dark skin”)
- refNum
unique referee ID number (referee name removed for anonymizing purposes)
- refCountry
unique referee country ID number (country name removed for anonymizing purposes)
- meanIAT
mean implicit bias score (using the race IAT) for referee country, higher values correspond to faster white | good, black | bad associations
- nIAT
sample size for race IAT in that particular country
- seIAT
standard error for mean estimate of race IAT
- meanExp
mean explicit bias score (using a racial thermometer task) for referee country, higher values correspond to greater feelings of warmth toward whites versus blacks
- nExp
sample size for explicit bias in that particular country
- seExp
standard error for mean estimate of explicit bias measure
Details
The skin colour of each player was rated by two independent raters,
rater1
and rater2
, and the 5-point scale values were
scaled to 0 to 1 - i.e., 0, 0.25, 0.5, 0.75, 1.
Source
Silberzahn R, Uhlmann EL, Martin DP, et al. Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science. 2018;1(3):337-356. doi:10.1177/2515245917747646
Display a specification curve across the multiverse.
Description
Returns a ggplot
object that displays
the specification curve as proposed by (Simonsohn et al. 2020).
Note that the order of universes may not correspond to the order
in the summary table.
Usage
spec_curve(
.spec_summary,
label = "name",
order_by = c("estimate", "is_significant"),
colour_by = "is_significant",
palette_common = NULL,
pointsize = 2,
linewidth = 0.5,
spec_matrix_spacing = 10,
theme_common = ggplot2::theme_minimal(),
sep = "---"
)
Arguments
.spec_summary |
A specification table created using
|
label |
If "name", uses the branch option names. If "code", display the codes used to define the branch options. |
order_by |
A character vector by which the curve is sorted. |
colour_by |
The name of the variable to colour the curve. |
palette_common |
A character vector of colours to match the values of
the variable |
pointsize |
Size of the points in the specification curve and the specification matrix. |
linewidth |
Width of confidence interval lines. |
spec_matrix_spacing |
A numeric for adjusting the specification matrix
spacing passed to |
theme_common |
A |
sep |
A string used internally to create the specification matrix. The string must be distinct from all branch names, option names, and option codes. Use a different value if any of them contains the default value. |
Value
a ggplot
object with the specification curve plot for
the estimates passed in the spec_summary()
.
References
Simonsohn U, Simmons JP, Nelson LD (2020). “Specification curve analysis.” Nature Human Behaviour, 4(11), 1208–1214. doi:10.1038/s41562-020-0912-z.
See Also
Other specification curve analysis:
spec_summary()
Examples
femininity <- mutate_branch(
1 * (MasFem > 6), 1 * (MasFem > mean(MasFem))
)
y <- mutate_branch(log(alldeaths + 1), alldeaths)
intensity <- mutate_branch(
Minpressure_Updated_2014,
Category,
NDAM,
HighestWindSpeed
)
model <- formula_branch(
y ~ femininity,
y ~ femininity * intensity
)
family <- family_branch(
gaussian, poisson
)
match_poisson <- branch_condition(alldeaths, poisson)
match_gaussian <- branch_condition(log(alldeaths + 1), gaussian)
stable <- mverse(hurricane) %>%
add_mutate_branch(y, femininity, intensity) %>%
add_formula_branch(model) %>%
add_family_branch(family) %>%
add_branch_condition(match_poisson, match_gaussian) %>%
glm_mverse() %>%
spec_summary("femininity")
# default behaviour
spec_curve(stable)
# coloring and sorting based on other variable
stable %>%
dplyr::mutate(colour_by = y_branch) %>%
spec_curve(order_by = c("estimate", "colour_by"), colour_by = "colour_by")
# Because the output is a \code{ggplot} object, you can
# further modify the asethetics of the specification curve
# using \code{ggplot2::theme()} and the specication matrix
# using \code{ggupset::theme_combmatrix()}
spec_curve(stable) +
ggplot2::labs(y = "Estimates", colour = "Significant at 0.05 level",
title = "Specification curve of femininity") +
ggplot2::theme(legend.position = "bottom") +
ggupset::theme_combmatrix(
combmatrix.label.width = ggplot2::unit(c(25, 100, 0, 0), "pt")
)
Create a specification table for a selected variable.
Description
Returns estimates for a selected variable across the multiverse along with
the universe specification information in a table. The resulting table
can be used for spe_curve()
.
Usage
spec_summary(.mverse, var, conf.int = TRUE, conf.level = 0.95)
Arguments
.mverse |
A |
var |
A character specifying the variable of interest. |
conf.int |
Whether the table should include confidence intervals. |
conf.level |
The confidence level for the confidence level and
|
Value
A spec_summary
object that includes estimates and
specification across the multiverse for the selected term(s). A boolean
column is_significant
indicates whether p.value
for the
universe is less than the specified significance level
(1 - conf.level
).
See Also
Other specification curve analysis:
spec_curve()
Examples
femininity <- mutate_branch(
1 * (MasFem > 6), 1 * (MasFem > mean(MasFem))
)
intensity <- mutate_branch(
Minpressure_Updated_2014,
Category,
NDAM,
HighestWindSpeed
)
model <- formula_branch(
log(alldeaths + 1) ~ femininity,
log(alldeaths + 1) ~ femininity * intensity
)
mv <- mverse(hurricane) %>%
add_mutate_branch(femininity) %>%
add_mutate_branch(intensity) %>%
add_formula_branch(model) %>%
lm_mverse()
spec_summary(mv, "femininity")
Display the multiverse table with results.
Description
This method returns the multiverse table displaying all universes defined by the multiverse. Each row corresponds to a universe and the columns include universe number, branch option name, and branch option definition.
Usage
## S3 method for class 'mverse'
summary(object, ...)
## S3 method for class 'lm_mverse'
summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...)
## S3 method for class 'glm_mverse'
summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...)
## S3 method for class 'glm.nb_mverse'
summary(object, conf.int = TRUE, conf.level = 0.95, output = "estimates", ...)
Arguments
object |
a |
... |
Ignored. |
conf.int |
When |
conf.level |
The confidence level of the confidence interval
returned using |
output |
The output of interest. The possible values are "estimates" ("e"), "df", "deviance" ("de"), and "aic" ("bic"). Alternatively, the first letters may be used. Default value is "estimates". |
Details
When you pass a mverse
objected fitted with model,
the summary table includes results of the fitted models
across the multiverse.
Value
a multiverse table as a tibble.
Examples
# Displaying the multiverse table without any fitted values.
hurricane_strength <- mutate_branch(
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014
)
mv <- create_multiverse(hurricane) %>%
add_mutate_branch(hurricane_strength)
summary(mv)
## Displaying after adding a a filter branch.
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
!Name %in% c("Katrina"),
TRUE # include all
)
mv <- add_filter_branch(mv, hurricane_outliers)
summary(mv)
# Displaying the multiverse table with \code{lm} models fitted.
hurricane_strength <- mutate_branch(
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014
)
y <- mutate_branch(
alldeaths, log(alldeaths + 1)
)
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
TRUE # include all
)
model_specifications <- formula_branch(
y ~ MasFem,
y ~ MasFem + hurricane_strength
)
mv <- create_multiverse(hurricane) %>%
add_filter_branch(hurricane_outliers) %>%
add_mutate_branch(hurricane_strength, y) %>%
add_formula_branch(model_specifications) %>%
lm_mverse()
summary(mv)
# Displaying the multiverse table with \code{glm} models fitted.
hurricane_strength <- mutate_branch(
NDAM,
HighestWindSpeed,
Minpressure_Updated_2014
)
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
TRUE # include all
)
model_specifications <- formula_branch(
alldeaths ~ MasFem,
alldeaths ~ MasFem + hurricane_strength
)
model_distributions <- family_branch(poisson)
mv <- create_multiverse(hurricane) %>%
add_filter_branch(hurricane_outliers) %>%
add_mutate_branch(hurricane_strength) %>%
add_formula_branch(model_specifications) %>%
add_family_branch(model_distributions) %>%
glm_mverse()
summary(mv)
# Displaying the multiverse table with \code{glm.nb} models fitted.
hurricane_outliers <- filter_branch(
!Name %in% c("Katrina", "Audrey", "Andrew"),
TRUE # include all
)
model_specifications <- formula_branch(alldeaths ~ MasFem)
mv <- create_multiverse(hurricane) %>%
add_filter_branch(hurricane_outliers) %>%
add_formula_branch(model_specifications) %>%
glm.nb_mverse()
summary(mv)
Performs one or two sample t-tests on data columns.
Description
t_test_mverse
performs t-tests across the multiverse.
If x or y is specified, then performs one and two sample t-tests
on specified columns of the data. If both x and y are NULL, then
performs t.test based on the formula branches.
Usage
t_test_mverse(
.mverse,
x = NULL,
y = NULL,
alternative = "two.sided",
mu = 0,
paired = FALSE,
var.equal = FALSE,
conf.level = 0.95,
parallel = FALSE,
progress = FALSE
)
Arguments
.mverse |
a |
x |
(optional) column name of data within mverse object |
y |
(optional) column name of data within mverse object |
alternative |
a character string specifying the alternative hypothesis, must be one of "two.sided" (default), "greater" or "less". You can specify just the initial letter. |
mu |
a number indicating the true value of the mean (or difference in means if you are performing a two sample test). |
paired |
a logical indicating whether you want a paired t-test. |
var.equal |
a logical variable indicating whether to treat the two variances as being equal. |
conf.level |
confidence level of the interval. |
parallel |
passed to |
progress |
passed to |
Value
a multiverse table displaying the t-test results as a tibble.
Examples
# Performing a unpaired two sample t-test.
library(dplyr)
mv <- soccer %>%
filter(!is.na(rater1), !is.na(rater2)) %>%
mverse()
x <- mutate_branch(
((rater1 + rater2) / 2) > mean((rater1 + rater2) / 2),
ifelse(rater1 > rater2, rater1 > 0.5, rater2 > 0.5)
)
y <- mutate_branch(
redCards, yellowCards, yellowReds
)
two_sample_form <- formula_branch(y ~ x)
mv <- mv %>%
add_mutate_branch(x, y) %>%
add_formula_branch(two_sample_form)
t_test_mverse(mv)