| Type: | Package |
| Title: | Receiver Operating Characteristics Surface |
| Version: | 1.4 |
| Date: | 2025-05-23 |
| Author: | Tianwei Yu [aut, cre], Peizhou Liao [ctb] |
| Maintainer: | Tianwei Yu <yutianwei@cuhk.edu.cn> |
| Description: | Plots the Receiver Operating Characteristics Surface for high-throughput class-skewed data, calculates the Volume under the Surface (VUS) and the FDR-Controlled Area Under the Curve (FCAUC), and conducts tests to compare two ROC surfaces. Computes eROC curve and the corresponding AUC for imperfect reference standard. |
| License: | GPL-2 |
| Depends: | rgl, poibin |
| NeedsCompilation: | no |
| Packaged: | 2025-06-01 01:20:41 UTC; yu |
| Repository: | CRAN |
| Date/Publication: | 2025-06-04 12:00:14 UTC |
Receiver Operating Characteristics Surface for class-skewed data
Description
The package constructs Receiver Operating Characteristics Surface (ROCS) using the false positive rate (FPR), true positive rate (TPR) and the false discovery rate (FDR). It computes the volume under the surface (VUS) and FDR-controlled area under the curve (FCAUC) to gauge the separability between two classes while taking into account the proportion of false discoveries.
Details
| Package: | ROCS |
| Type: | Package |
| Version: | 1.0 |
| Date: | 2024-11-13 |
| License: | GPL 2.0 or later |
Major functions are rocs.x and fcauc.x.
Author(s)
Tianwei Yu
Maintainer: Tianwei Yu <yutianwei@cuhk.edu.cn>
Calculating expected Receiver Operating Characteristics Curve in the presence of imperfect reference standards.
Description
This function builds an eROC curve and returns the expected values of TPR, FPR, and TDR. It also calculates the AUC of the eROC curve. The 95% bootstrap percentile confidence interval for the AUC is provided.
Usage
e.roc(x, mu, method='RNA', bt.ci=TRUE, bt.nreps=100, do.plot=TRUE)
Arguments
x |
Vector; the scores yielded by the classifier. |
mu |
Vector; the probabilistic confidence assigned by the imperfect reference standard. |
method |
The method used to compute the cumulative distribution function for the Poisson binomial distribution. “DFT-CF” for the DFT-CF method, “RF” for the recursive formula, “RNA” for the refined normal approximation, “NA” for the normal approximation. |
bt.ci |
Whether to compute the bootstrap confidence interval. |
bt.nreps |
The number of bootstrap replicates. |
do.plot |
Whether to plot the eROC curve. |
Details
The eROC curve is a generalization of ROC curve given the class membership uncertainties. See the reference for the definition of the eROC curve.
Value
Returns the area under the eROC curve, the expected values of TPR, FPR, and TDR.
Author(s)
Peizhou Liao. Email: pliao3@emory.edu.
References
Liao P, Wu H, and Yu T (2016). ROC Curve Analysis in the Presence of Imperfect Reference Standards. Stat Biosci doi:10.1007/s12561-016-9159-7.
Examples
## normal scores
sample.p <- rnorm(100, mean=1, sd=sqrt(2))
sample.n <- rnorm(100, mean=-1, sd=sqrt(2))
## probabilistic confidence
mu.p <- rbeta(100, shape1=5, shape2=1)
mu.n <- rbeta(100, shape1=1, shape2=5)
## combine the sample
x.sample <- c(sample.p, sample.n)
mu.sample <- c(mu.p, mu.n)
## build eROC curve
e.roc.fit <- e.roc(x=x.sample, mu=mu.sample)
Plotting two ROC curves, highlighting the segments of the curves corresponding to acceptable FDR levels.
Description
The function draws two colored ROC curves in one plot. For each curve, the segment that corresponds to acceptable FDR (e.g. <= 0.2) is colored differently than the rest of the curve.
Usage
fcauc.comp.fptp(roc.1, roc.2, FDR.cut = 0.2, lwd = 3,
colors = c("blue", "green", "cyan", "red"),
exp.labels = c("experiment 1", "experiment 2"))
Arguments
roc.1 |
A list object with three components: TP, FP and TDR, each being a vector. |
roc.2 |
A list object with three components: TP, FP and TDR, each being a vector. |
FDR.cut |
The cutoff value of FDR. |
lwd |
The line width parameter to be passed on to the function lines(). |
colors |
A vector of at least four colors. It is used for the coloring of the two ROC curves. |
exp.labels |
Experiment labels in the plot. |
Value
No value is returned.
Author(s)
Tianwei Yu. Email: tianwei.yu@emory.edu.
References
Yu T (2012) ROCS: Receiver Operating Characteristic Surface for Class-Skewed High-Throughput Data. PLoS ONE 7(7): e40598.
See Also
fcauc.fptp()
Generating ROC plot with shading based on FDR using user-provided FPR, TPR, FDR values
Description
The function plots the ROC curve using user-provided FPR, TPR, FDR values. It shades the area corresponding to acceptable FDR level. The FDR-controlled area under the curve (FCAUC) is calculated for the shaded area.
Usage
fcauc.fptp(FP, TP, TDR, FDR.cut = 0.2, do.plot=TRUE)
Arguments
FP |
Vector of false positive rate. |
TP |
Vector of true positive rate. |
TDR |
Vector of true discovery rate. |
FDR.cut |
The cutoff value of FDR, for the purpose of FCAUC cualculation. |
do.plot |
Whether to generate the plot. |
Value
The FCAUC value is returned.
Author(s)
Tianwei Yu. Email: tianwei.yu@emory.edu.
References
Yu T (2012) ROCS: Receiver Operating Characteristic Surface for Class-Skewed High-Throughput Data. PLoS ONE 7(7): e40598.
See Also
fcauc.x
Generating ROC plot with shading based on FDR
Description
The function plots the ROC curve. It shades the area corresponding to acceptable FDR level. The FDR-controlled area under the curve (FCAUC) is calculated for the shaded area.
Usage
fcauc.x(x0, x1, FDR.cut = 0.2)
Arguments
x0 |
Vector; the raw data of the null class. |
x1 |
Vector; the raw data of the non-null class. |
FDR.cut |
The FDR level at which to shade the AUC and calculate the FCAUC. |
Value
The FCAUC value is returned.
Author(s)
Tianwei Yu. Email: tianwei.yu@emory.edu.
References
Yu T (2012) ROCS: Receiver Operating Characteristic Surface for Class-Skewed High-Throughput Data. PLoS ONE 7(7): e40598.
See Also
fcauc.fptp
Examples
#perfect separation
x0<-runif(1000)
x1<-runif(100)+2
fcauc.x(x0, x1)
#partial separation
x0<-rnorm(1000, mean=0, sd=1.5)
x1<-rnorm(100, mean=3, sd=1)
fcauc.x(x0, x1)
Plotting Receiver Operating Characteristics Surface with user-provided FPR, TPR & TDR
Description
The function plots the colored ROCS in 3 dimensions using the rgl utilities. It shades the FDR-controlled AUC in the FPR-TPR plain. The VUS of the surface and the FCAUC area are calculated.
Usage
rocs.fptp(FP, TP, TDR, FDR.cut = 0.2)
Arguments
FP |
Vector of false positive rate. |
TP |
Vector of true positive rate. |
TDR |
Vector of true discovery rate. |
FDR.cut |
The cutoff value of FDR, for the purpose of FCAUC cualculation. |
Details
The ROCS is the surface spanned by the TPR-FPR-TDR and its projection to the TPR-TDR plain. The vulume is between the surface and its projection on the TPR-FDR plain (i.e. the AUC of ROC).
Value
The volume under the surface (VUS) is returned.
Author(s)
Tianwei Yu. Email: tianwei.yu@emory.edu.
References
Yu T (2012) ROCS: Receiver Operating Characteristic Surface for Class-Skewed High-Throughput Data. PLoS ONE 7(7): e40598.
See Also
rocs.x
Calculating Receiver Operating Characteristics Surface from one-dimensional data.
Description
The function plots the colored ROCS in 3 dimensions using the rgl utilities. It shades the FDR-controlled AUC in the FPR-TPR plain. The VUS of the surface and the FCAUC area are calculated. The significance of the VUX is assessed by one-sided permutation test.
Usage
rocs.x(x0, x1, s0=NULL, s1=NULL, n.perm = 1000, do.plot = TRUE, FDR.cut=0.2)
Arguments
x0 |
Vector; the raw data of the null class. |
x1 |
Vector; the raw data of the non-null class. |
s0 |
Vector; the confidence level of the class assignment of the null class observations. The length should be the same as x0. The default is NULL, in which case all observations are considered to be assigned without uncertainty. |
s1 |
Vector; the confidence level of the class assignment of the non-null class observations. The length should be the same as x0. The default is NULL, in which case all observations are considered to be assigned without uncertainty. |
n.perm |
The number of permutations to assess the significance of the VUX. |
do.plot |
Whether to plot the 3D surface, or just return the VUX. |
FDR.cut |
The FDR level at which to shade the AUC and calculate the FCAUC. |
Details
The ROCS is the surface spanned by the TPR-FPR-TDR and its projection to the TPR-TDR plain. The volume is between the surface and its projection on the TPR-FDR plain (i.e. the AUC of ROC).
Value
The volume under the surface (VUS) is returned.
Author(s)
Tianwei Yu. Email: tianwei.yu@emory.edu.
References
Yu T (2012) ROCS: Receiver Operating Characteristic Surface for Class-Skewed High-Throughput Data. PLoS ONE 7(7): e40598. Yu T, Jones DP. (2014) Improving peak detection in high-resolution LC/MS metabolomics data using preexisting knowledge and machine learning approach. Bioinformatics. 30(20): 2941-2948.
See Also
rocs.fptp
Examples
x0<-rnorm(500, mean=0, sd=1.5)
x1<-rnorm(50, mean=4, sd=1)
rocs.x(x0, x1)$vus
Testing for significant difference between two ROCS
Description
The null hypothesis being tested is that the two ROCS are generated from populations with equal class-separating power. This function uses a bootstrap-based test.
Usage
test.rocs(x0, x1, z0, z1, B = 1000, do.plot = TRUE)
Arguments
x0 |
Vector; the raw data of the true-negative class in study 1. |
x1 |
Vector; the raw data of the true-positive class in study 1. |
z0 |
Vector; the raw data of the true-negative class in study 2. |
z1 |
Vector; the raw data of the true-positive class in study 2. |
B |
The number of bootstrap samples to be used in order to estimate the spread of the distribution under the null hypothesis. |
do.plot |
Whether to plot the spread based on bootstrap samples. |
Value
The p-value of the test is returned.
Author(s)
Tianwei Yu. Email: tianwei.yu@emory.edu.
References
Yu T (2012) ROCS: Receiver Operating Characteristic Surface for Class-Skewed High-Throughput Data. PLoS ONE 7(7): e40598.
Examples
x0<-rnorm(1000, mean=0, sd=1.5)
x1<-rnorm(100, mean=3, sd=1)
z0<-rnorm(800, mean=0, sd=1.5)
z1<-rnorm(80, mean=3.25, sd=1)
test.rocs(x0, x1, z0,z1, B=1000)