library(irt)
irt package contains many useful functions commonly used in psychometrics.
Item parameters are defined within three main objects types:
__Item__ object contains all of the information about a test/survey item. It
is the building block of Testlet and Itempool objects. It mainly contains
item parameters and item's psychometric model. But it can also contain many
other item attributes such as item content, item_id. User can also define
additional attributes, for example, key, enemies, word_count,
seed_group, etc.
__Testlet__ object is a set of Item objects. It can also have it's own
psychometric model. User can also define additional attributes.
__Itempool__ object is a collection of Item and Testlet objects.
ItemIn order to create an Item object, the psychometric model and item parameter
values is sufficient. Specifying an item_id field is required if Item will be
used within an Itempool or Testlet.
A three parameter logistic model item (3PL) requires a, b and c
parameters to be specified:
item1 <- item(a = 1.2, b = -.8, c = .33, model = "3PL")
item1
#> A '3PL' item.
#> Model: 3PL (Three-Parameter Logistic Model)
#> Model Parameters:
#> a = 1.2
#> b = -0.8
#> c = 0.33
#> D = 1
#>
#> --------------------------
a is the item discrimination, b is the item difficulty and c is the
pseudo-guessing parameter.
By default, the value of scaling constant D is specified as 1. But it can be
overridden:
item1 <- item(a = 1.2, b = -.8, c = .33, D = 1.7, model = "3PL")
item1
#> A '3PL' item.
#> Model: 3PL (Three-Parameter Logistic Model)
#> Model Parameters:
#> a = 1.2
#> b = -0.8
#> c = 0.33
#> D = 1.7
#>
#> --------------------------
item_id and content field can be specified as well:
item1 <- item(a = 1.2, b = -.8, c = .33, D = 1.7, model = "3PL",
item_id = "ITM384", content = "Quadratic Equations")
item1
#> A '3PL' item.
#> Item ID: ITM384
#> Model: 3PL (Three-Parameter Logistic Model)
#> Content: Quadratic Equations
#> Model Parameters:
#> a = 1.2
#> b = -0.8
#> c = 0.33
#> D = 1.7
#>
#> --------------------------
Additional fields can be added through misc field:
item1 <- item(a = 1.2, b = -.8, c = .33, D = 1.7, model = "3PL",
item_id = "ITM384", content = "Quadratic Equations",
misc = list(key = "A",
enemies = c("ITM664", "ITM964"),
seed_year = 2020,
target_grade = "11")
)
item1
#> A '3PL' item.
#> Item ID: ITM384
#> Model: 3PL (Three-Parameter Logistic Model)
#> Content: Quadratic Equations
#> Model Parameters:
#> a = 1.2
#> b = -0.8
#> c = 0.33
#> D = 1.7
#>
#> Misc:
#> key: "A"
#> enemies: "ITM664", "ITM964"
#> seed_year: 2020
#> target_grade: "11"
#> --------------------------
An item characteristic curve can be plotted using plot function:
plot(item1)
Rasch model item requires b parameter to be specified:
item2 <- item(b = -.8, model = "Rasch")
item2
#> A 'Rasch' item.
#> Model: Rasch (Rasch Model)
#> Model Parameters:
#> b = -0.8
#>
#> --------------------------
For Rasch model, D parameter cannot be specified.
1PL ModelA one-parameter model item requires b parameter to be specified:
item3 <- item(b = -.8, D = 1.7, model = "1PL")
item3
#> A '1PL' item.
#> Model: 1PL (One-Parameter Logistic Model)
#> Model Parameters:
#> b = -0.8
#> D = 1.7
#>
#> --------------------------
2PL ModelA two-parameter model item requires a and b parameters to be specified:
item4 <- item(a = 1.2, b = -.8, D = 1.702, model = "2PL")
item4
#> A '2PL' item.
#> Model: 2PL (Two-Parameter Logistic Model)
#> Model Parameters:
#> a = 1.2
#> b = -0.8
#> D = 1.702
#>
#> --------------------------
4PL ModelA four-parameter model item requires a, b, c and d parameters to be
specified:
item5 <- item(a = 1.06, b = 1.76, c = .13, d = .98, model = "4PL",
item_id = "itm-5")
item5
#> A '4PL' item.
#> Item ID: itm-5
#> Model: 4PL (Four-Parameter Logistic Model)
#> Model Parameters:
#> a = 1.06
#> b = 1.76
#> c = 0.13
#> d = 0.98
#> D = 1
#>
#> --------------------------
d is the upper-asymptote parameter.
GRM)A Graded Response model item requires a and b parameters to be specified.
b parameters is ascending vector of threshold parameters:
item6 <- item(a = 1.22, b = c(-1.9, -0.37, 0.82, 1.68), model = "GRM",
item_id = "itm-6")
item6
#> A 'GRM' item.
#> Item ID: itm-6
#> Model: GRM (Graded Response Model)
#> Model Parameters:
#> a = 1.22
#> b = -1.9; -0.37; 0.82; 1.68
#> D = 1
#>
#> --------------------------
plot(item6)
D parameter can also be specified.
GPCM)A Generalized Partial Credit model item requires a and b parameters to be
specified. b parameters is ascending vector of threshold parameters:
item7 <- item(a = 1.22, b = c(-1.9, -0.37, 0.82, 1.68), D = 1.7, model = "GPCM",
item_id = "itm-7")
item7
#> A 'GPCM' item.
#> Item ID: itm-7
#> Model: GPCM (Generalized Partial Credit Model)
#> Model Parameters:
#> a = 1.22
#> b = -1.9; -0.37; 0.82; 1.68
#> D = 1.7
#>
#> --------------------------
PCM)A Partial Credit model item requires b parameters to be
specified. b parameters is ascending vector of threshold parameters:
item8 <- item(b = c(-1.9, -0.37, 0.82, 1.68), model = "PCM")
item8
#> A 'PCM' item.
#> Model: PCM (Partial Credit Model)
#> Model Parameters:
#> b = -1.9; -0.37; 0.82; 1.68
#>
#> --------------------------
An item with random item parameters can be generated using generate_item
function:
generate_item("3PL")
#> A '3PL' item.
#> Model: 3PL (Three-Parameter Logistic Model)
#> Model Parameters:
#> a = 1.0595
#> b = -0.4464
#> c = 0.2409
#> D = 1
#>
#> Misc:
#> key: "D"
#> possible_options: "A", "B", "C", "D"
#> --------------------------
generate_item("2PL")
#> A '2PL' item.
#> Model: 2PL (Two-Parameter Logistic Model)
#> Model Parameters:
#> a = 1.09
#> b = 1.9824
#> D = 1
#>
#> Misc:
#> key: "B"
#> possible_options: "A", "B", "C", "D"
#> --------------------------
generate_item("Rasch")
#> A 'Rasch' item.
#> Model: Rasch (Rasch Model)
#> Model Parameters:
#> b = 0.1665
#>
#> Misc:
#> key: "A"
#> possible_options: "A", "B", "C", "D"
#> --------------------------
generate_item("GRM")
#> A 'GRM' item.
#> Model: GRM (Graded Response Model)
#> Model Parameters:
#> a = 0.5913
#> b = -0.6408; 0.1115; 1.5018
#> D = 1
#>
#> --------------------------
# The number of categories of polytomous items can be specified:
generate_item("GPCM", n_categories = 5)
#> A 'GPCM' item.
#> Model: GPCM (Generalized Partial Credit Model)
#> Model Parameters:
#> a = 0.6472
#> b = -2.705; -0.6416; -0.0431; 0.3486
#> D = 1
#>
#> --------------------------
TestletA testlet is simply a collection of Item objects:
item1 <- item(a = 1.2, b = -.8, c = .33, D = 1.7, model = "3PL",
item_id = "ITM384", content = "Quadratic Equations")
item2 <- item(a = 0.75, b = 1.8, c = .21, D = 1.7, model = "3PL",
item_id = "ITM722", content = "Quadratic Equations")
item3 <- item(a = 1.06, b = 1.76, c = .13, d = .98, model = "4PL",
item_id = "itm-5")
t1 <- testlet(c(item1, item2, item3))
t1
#> An object of class 'Testlet'.
#> Model: BTM
#>
#> Item List:
#>
#> item_id model a b c d D content
#> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
#> 1 ITM384 3PL 1.2 -0.8 0.33 NA 1.7 Quadratic Equations
#> 2 ITM722 3PL 0.75 1.8 0.21 NA 1.7 Quadratic Equations
#> 3 itm-5 4PL 1.06 1.76 0.13 0.98 1 <NA>
An testlet_id field is required if testlet will be used in an item pool.
t1 <- testlet(item1, item2, item3, testlet_id = "T1")
t1
#> An object of class 'Testlet'.
#> Testlet ID: T1
#> Model: BTM
#>
#> Item List:
#>
#> item_id model a b c d D content
#> <chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl> <chr>
#> 1 ITM384 3PL 1.2 -0.8 0.33 NA 1.7 Quadratic Equations
#> 2 ITM722 3PL 0.75 1.8 0.21 NA 1.7 Quadratic Equations
#> 3 itm-5 4PL 1.06 1.76 0.13 0.98 1 <NA>
ItempoolAn Itempool object is the most frequently used object type in irt package.
It is a collection of Item and Testlet objects.
item1 <- generate_item("3PL", item_id = "I1")
item2 <- generate_item("3PL", item_id = "I2")
item3 <- generate_item("3PL", item_id = "I3")
ip1 <- itempool(item1, item2, item3)
Item pools can be composed of items from different psychometric models and testlets:
item4 <- generate_item("GRM", item_id = "I4")
item5 <- generate_item("3PL", item_id = "T1-I1")
item6 <- generate_item("3PL", item_id = "T1-I2")
t1 <- testlet(item5, item6, item_id = "T1")
ip2 <- itempool(item1, item2, item3, item4, t1)
Most of the time item pools are generated using data frames:
n_item <- 6 # Number of items
ipdf <- data.frame(a = rlnorm(n_item), b = rnorm(n_item),
c = runif(n_item, 0, .3))
ip3 <- itempool(ipdf)
ip3
#> An object of class 'Itempool'.
#> Model of items: 3PL
#> D = 1
#>
#> item_id a b c
#> <chr> <dbl> <dbl> <dbl>
#> 1 Item_1 1.67 0.677 0.195
#> 2 Item_2 0.999 0.325 0.135
#> 3 Item_3 9.22 -1.18 0.186
#> 4 Item_4 3.47 0.169 0.248
#> 5 Item_5 0.283 1.74 0.262
#> 6 Item_6 0.992 0.0354 0.130
# Scaling constant can be specified
ip4 <- itempool(ipdf, D = 1.7)
ip4
#> An object of class 'Itempool'.
#> Model of items: 3PL
#> D = 1.7
#>
#> item_id a b c
#> <chr> <dbl> <dbl> <dbl>
#> 1 Item_1 1.67 0.677 0.195
#> 2 Item_2 0.999 0.325 0.135
#> 3 Item_3 9.22 -1.18 0.186
#> 4 Item_4 3.47 0.169 0.248
#> 5 Item_5 0.283 1.74 0.262
#> 6 Item_6 0.992 0.0354 0.130
ipdf <- data.frame(
item_id = c("Item_1", "Item_2", "Item_3", "Item_4", "Item_5", "Item_6"),
model = c("3PL", "3PL", "3PL", "GPCM", "GPCM", "GPCM"),
a = c(1.0253, 1.3609, 1.6617, 1.096, 0.9654, 1.3995),
b1 = c(NA, NA, NA, -1.112, -0.1709, -1.1324),
b2 = c(NA, NA, NA, -0.4972, 0.2778, -0.5242),
b3 = c(NA, NA, NA, -0.0077, 0.9684, NA),
D = c(1.7, 1.7, 1.7, 1.7, 1.7, 1.7),
b = c(0.7183, -0.4107, -1.5452, NA, NA, NA),
c = c(0.0871, 0.0751, 0.0589, NA, NA, NA),
content = c("Geometry", "Algebra", "Algebra", "Geometry", "Algebra",
"Algebra")
)
ip5 <- itempool(ipdf)
Itempool objects can also be converted to a data frame:
as.data.frame(ip2)
#> item_id testlet_id model a b c b1 b2 b3 D key
#> 1 I1 <NA> 3PL 1.0267 0.6891 0.2299 NA NA NA 1 C
#> 2 I2 <NA> 3PL 0.8968 -0.5595 0.1377 NA NA NA 1 D
#> 3 I3 <NA> 3PL 1.3462 0.1563 0.0341 NA NA NA 1 A
#> 4 I4 <NA> GRM 1.4821 NA NA -0.7702 -0.0635 0.5574 1 <NA>
#> 5 T1-I1 Testlet_1 3PL 1.1838 -0.8271 0.2119 NA NA NA 1 C
#> 6 T1-I2 Testlet_1 3PL 0.8859 1.2291 0.2267 NA NA NA 1 B
#> possible_options
#> 1 A, B, C, D
#> 2 A, B, C, D
#> 3 A, B, C, D
#> 4 NA
#> 5 A, B, C, D
#> 6 A, B, C, D
Probability of correct response (for dichotomous items) and probability of
each category (for polytomous items) can be calculated using prob function:
item1 <- generate_item("3PL")
theta <- 0.84
# The probability of correct and incorrect response for `item1` at theta = 0.84
prob(item1, theta)
#> 0 1
#> [1,] 0.1262267 0.8737733
# Multiple theta values
prob(item1, theta = c(-1, 1))
#> 0 1
#> [1,] 0.2887969 0.7112031
#> [2,] 0.1162770 0.8837230
# Polytomous items:
item2 <- generate_item(model = "GPCM")
prob(item2, theta = 1)
#> 0 1 2 3
#> [1,] 0.02749999 0.2592981 0.3540715 0.3591304
prob(item2, theta = c(-1, 0, 1))
#> 0 1 2 3
#> [1,] 0.38461606 0.5055458 0.0962317 0.01360647
#> [2,] 0.14295920 0.5032828 0.2565882 0.09716975
#> [3,] 0.02749999 0.2592981 0.3540715 0.35913042
Probability of correct response (or category) for each item in an item pool can be calculated as:
ip <- generate_ip(model = "3PL", n = 7)
ip
#> An object of class 'Itempool'.
#> Model of items: 3PL
#> D = 1
#> possible_options = c("A", "B", "C", "D")
#>
#> item_id a b c key
#> <chr> <dbl> <dbl> <dbl> <chr>
#> 1 Item_1 0.562 -0.363 0.0691 A
#> 2 Item_2 1.42 0.564 0.074 C
#> 3 Item_3 0.942 -0.0758 0.231 D
#> 4 Item_4 1.07 -1.70 0.0365 B
#> 5 Item_5 0.808 0.888 0.235 B
#> 6 Item_6 1.00 1.43 0.158 A
#> 7 Item_7 1.44 0.780 0.0289 B
prob(ip, theta = 0)
#> 0 1
#> Item_1 0.4181755 0.5818245
#> Item_2 0.6393804 0.3606196
#> Item_3 0.3708340 0.6291660
#> Item_4 0.1332453 0.8667547
#> Item_5 0.5141995 0.4858005
#> Item_6 0.6805058 0.3194942
#> Item_7 0.7328930 0.2671070
# When there are multiple theta values, a list where each element corresponds
# to a theta value returned.
prob(ip, theta = c(-2, 0, 1))
#> [[1]]
#> 0 1
#> Item_1 0.6656635 0.33433646
#> Item_2 0.9025086 0.09749140
#> Item_3 0.6610870 0.33891296
#> Item_4 0.5577232 0.44227679
#> Item_5 0.6974074 0.30259257
#> Item_6 0.8163113 0.18368871
#> Item_7 0.9537475 0.04625255
#>
#> [[2]]
#> 0 1
#> Item_1 0.4181755 0.5818245
#> Item_2 0.6393804 0.3606196
#> Item_3 0.3708340 0.6291660
#> Item_4 0.1332453 0.8667547
#> Item_5 0.5141995 0.4858005
#> Item_6 0.6805058 0.3194942
#> Item_7 0.7328930 0.2671070
#>
#> [[3]]
#> 0 1
#> Item_1 0.29544927 0.7045507
#> Item_2 0.32373838 0.6762616
#> Item_3 0.20490316 0.7950968
#> Item_4 0.05009109 0.9499089
#> Item_5 0.36522316 0.6347768
#> Item_6 0.51082906 0.4891709
#> Item_7 0.40910043 0.5908996
Item characteristic curves (ICC) can be plotted:
# Plot ICC of each item in the item pool
plot(ip)
# Plot test characteristic curve
plot(ip, type = "tcc")
Information value of an item at a given \(\theta\) value can also be calculated:
item1 <- generate_item("3PL")
info(item1, theta = -2)
#> [1] 0.04714935
# Multiple theta values
info(item1, theta = c(-1, 1))
#> [1] 0.1492103 0.1451877
# Polytomous items:
item2 <- generate_item(model = "GPCM")
info(item2, theta = 1)
#> [1] 0.4666825
info(item2, theta = c(-1, 0, 1))
#> [1] 0.4275631 0.5504794 0.4666825
Information values for each item in an item pool can be calculated as:
ip <- generate_ip(model = "3PL", n = 7)
ip
#> An object of class 'Itempool'.
#> Model of items: 3PL
#> D = 1
#> possible_options = c("A", "B", "C", "D")
#>
#> item_id a b c key
#> <chr> <dbl> <dbl> <dbl> <chr>
#> 1 Item_1 0.826 0.946 0.0109 C
#> 2 Item_2 0.961 0.0229 0.0481 D
#> 3 Item_3 0.715 -0.972 0.108 D
#> 4 Item_4 1.16 0.325 0.195 B
#> 5 Item_5 1.32 -0.347 0.106 B
#> 6 Item_6 0.779 -0.0502 0.124 B
#> 7 Item_7 0.734 -1.20 0.196 B
info(ip, theta = 0)
#> Item_1 Item_2 Item_3 Item_4 Item_5 Item_6 Item_7
#> [1,] 0.1418601 0.2093189 0.09592887 0.2033117 0.34417 0.1185853 0.08301112
info(ip, theta = c(-2, 0, 1))
#> Item_1 Item_2 Item_3 Item_4 Item_5 Item_6 Item_7
#> [1,] 0.04453504 0.07206803 0.08132883 0.0165125 0.0735661 0.04996529 0.07363456
#> [2,] 0.14186012 0.20931894 0.09592887 0.2033117 0.3441700 0.11858530 0.08301112
#> [3,] 0.16668362 0.17428530 0.06999601 0.2138576 0.1887511 0.10698616 0.05768018
Information functions can be plotted:
# Plot information function of each item
plot_info(ip)
# Plot test information function
plot_info(ip, tif = TRUE)
For a given set of item parameters and item responses, the ability (\(\theta\))
estimates can be calculated using est_ability function.
# Generate an item pool
ip <- generate_ip(model = "2PL", n = 10)
true_theta <- rnorm(5)
resp <- sim_resp(ip = ip, theta = true_theta, output = "matrix")
# Calculate raw scores
est_ability(resp = resp, ip = ip, method = "sum_score")
#> $est
#> S1 S2 S3 S4 S5
#> 0 4 8 4 7
#>
#> $se
#> S1 S2 S3 S4 S5
#> NA NA NA NA NA
# Estimate ability using maximum likelihood estimation:
est_ability(resp = resp, ip = ip, method = "ml")
#> $est
#> S1 S2 S3 S4 S5
#> -5.000000 -0.315766 1.039143 -0.277098 0.522099
#>
#> $se
#> S1 S2 S3 S4 S5
#> 3.343795 0.618737 0.703673 0.615753 0.621608
# Estimate ability using EAP estimation:
est_ability(resp = resp, ip = ip, method = "eap")
#> $est
#> S1 S2 S3 S4 S5
#> -1.779985 -0.247671 0.760455 -0.218466 0.399286
#>
#> $se
#> S1 S2 S3 S4 S5
#> 0.664673 0.536863 0.560395 0.535971 0.538794
# Estimate ability using EAP estimation with a different prior
# (prior mean = 0, prior standard deviation = 2):
est_ability(resp = resp, ip = ip, method = "eap", prior_pars = c(0, 2))
#> $est
#> S1 S2 S3 S4 S5
#> -3.110786 -0.323925 1.034657 -0.285509 0.522756
#>
#> $se
#> S1 S2 S3 S4 S5
#> 1.149488 0.616265 0.689011 0.614193 0.625192