A Collection of Efficient and Extremely Fast R Functions

Description

A collection of fast (utility) functions for data analysis. Column and row wise means, medians, variances, minimums, maximums, many t, F and G-square tests, many regressions (normal, logistic, Poisson), are some of the many fast functions. References: a) Tsagris M., Papadakis M. (2018). Taking R to its limits: 70+ tips. PeerJ Preprints 6:e26605v1 <doi:10.7287/peerj.preprints.26605v1>. b) Tsagris M. and Papadakis M. (2018). Forward regression in R: from the extreme slow to the extreme fast. Journal of Data Science, 16(4): 771–780. <doi:10.6339/JDS.201810_16(4).00006>. c) Chatzipantsiou C., Dimitriadis M., Papadakis M. and Tsagris M. (2020). Extremely Efficient Permutation and Bootstrap Hypothesis Tests Using Hypothesis Tests Using R. Journal of Modern Applied Statistical Methods, 18(2), eP2898. <doi:10.48550/arXiv.1806.10947>. d) Tsagris M., Papadakis M., Alenazi A. and Alzeley O. (2024). Computationally Efficient Outlier Detection for High-Dimensional Data Using the MDP Algorithm. Computation, 12(9): 185. <doi:10.3390/computation12090185>. e) Tsagris M. and Papadakis M. (2025). Fast and light-weight energy statistics using the R package Rfast. <doi:10.48550/arXiv.2501.02849>.

Details

Maintainers

Manos Papadakis <rfastofficial@gmail.com>

Note

Acknowledgments: We would like to acknowledge:

• Professor Kurt Hornik, Doctor Uwe Ligges (and the rest of R core team) for their invaluable help with this R package.

• Erich Studerus for his invaluable comments.

• Neal Fultz for his suggestions.

• Vassilis Vasdekis for his invaluable help with the random effects models.

• Marios Dimitriadis' work was funded by the Special Account for Research Funds of the University of Crete, Department of Computer Science.

• Phillip Si is greatly acknowledged for his help with the Floyd-Warshal algorithm.

• Keefe Murphy for his invaluable help with NEWS file and for his suggestions.

• Zacharias Papadovassilakis gave us the inspiration for the memory efficient version of the k-NN algorithm.

• Yannis Pantazis explained us how the orhtogonal matching pursuit works.

• Achim Zeileis for his help with the quasi Poisson regression models.

• Pedro J. Aphalo for finding a bug.

• Dimitris Kyriakis for finding a bug.

• Cyrille Conord for finding a bug.

• Aaron Robotham for finding a bug.

• Calvin Pan from the Department of Human Genetics at UCLA found a bug in the function glm_logistic and he is greatly acknowledged for that.

• Adam Muschielok from Rodenstock GmbH found a bug in the function vmf.mle and he is greatly acknowledged for that.

• Bret Presnell for detecting and correcting a bug in the function rvmf.

• Gabriel Hoffman for spotting a mistake in the fuction dirimultinom.mle.

• Lutz von der Burchard for spotting a bug in the function bic.corfsreg.

Author(s)

• Manos Papadakis <papadakm95@gmail.com>

• Michail Tsagris <mtsagris@uoc.gr>

• Marios Dimitriadis <kmdimitriadis@gmail.com>

• Stefanos Fafalios <stefanosfafalios@gmail.com>

• Ioannis Tsamardinos <tsamard@csd.uoc.gr>

• Matteo Fasiolo <matteo.fasiolo@gmail.com>

• Giorgos Borboudakis <borbudak@gmail.com>

• John Burkardt <jburkardt@fsu.edu>

• Changliang Zou <nk.chlzou@gmail.com>

Check if any column or row is fill with values

Description

Check if any column or row is fill with values.

Usage

colrow.value(x,value=0)

Arguments

Details

Check all the column if any has all its elements equal to argument value. If found, return "TRUE". Otherwise continues with rows. If columns and rows hasn't any value vector then return "FALSE". Even if it returns "FALSE" that doesn't mean the determinant can't be value. It might be but if check before and found any value vector then for sure the determinant it'll be value.

Value

A boolean value, "TRUE" if any column OR row is all filled with value. "FALSE" otherwise.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

rowMins, rowFalse, nth, colrange, colMedians, colVars, colSort, rowSort, rowTrue

Examples

x <- matrix(runif(10*10),10,10)
res<-colrow.value(x) 

x<-NULL

Deep copy

Description

Deep copy.

Usage

env.copy(x,all.names=FALSE)

Arguments

Details

Deep copy of the environment object.

Value

A copy of the first argument.

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colShuffle, colVars, colmeans

Examples

x <- new.env()
x$imaginary <- NULL
x$real <- NULL

# you can library the package and just press x and R will understand 
# and search automatically for a function to print the environment
x

y <- env.copy(x)

x$real <- 10

x$real == y$real # FALSE

Design Matrix

Description

Design Matrix.

Usage

design_matrix(x, ones = TRUE)

Arguments

Details

This function implements the R's "model.matrix" function and is used only when the x is a factor/charactervector or Dataframe.

Value

Returns the same matrix with model.matrix.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>

See Also

model.matrix

Examples

a <- design_matrix( iris[, 5] )
b <- model.matrix( ~ iris[,5] )  ## R's built-in function
all.equal(as.vector(a),as.vector(b)) ## true

a<-b<-NULL

Insert/remove function names in/from the NAMESPACE file

Description

Insert/remove function names in/from the NAMESPACE file.

Usage

AddToNamespace(path.namespace,path.rfolder,paths.full = FALSE)
RemoveFromNamespace(path.namespace,files.to.remove)

Arguments

Details

AddToNameSpace: Reads the files that are exported in NAMESPACE and the functions that are inside rfolder (where R files are) and insert every function that is not exported. For that you must add the attribute "#[export]" above every function you wish to export. Also you can use the attribute "#[export s3]" for exporting S3methods. Finally, if you don't want the program to read a file just add at the top of the file the attribute "#[dont read]".

RemoveFromNamespace: Remove every function, from argument "files.to.remove", from NAMESPACE.

Value

AddToNameSpace:

RemoveFromNamespace: Return the files that could not be removed.

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colShuffle, colVars, colmeans

Examples

## Not run: 
	#for example: path.namespace="C:\some_file\NAMESPACE" where is NAMESPACE file
	#path.rfolder="C:\some_file\R\" where is R files are
	#system.time( a<-AddToNamespace(path.namespace,path.rfolder) )
	#if(length(a)==0){
	#	print("all the files are inserted")
	#}else{
	#	print("The new files that inserted are: \n")
	#	a
	#}
	#system.time( a<-RemoveFromNamespace(path.namespace,c("a","b")) )
	#if(length(a)==0){
	#	print("all the files are inserted")
	#}else{
	#	print("The files thatcould not be deleted are: \n")
	#	a
	#}

## End(Not run)

Lower and Upper triangular of a matrix

Description

Lower/upper triangular matrix.

Usage

lower_tri(x, suma = FALSE, diag = FALSE)
upper_tri(x, suma = FALSE, diag = FALSE)
lower_tri.assign(x, v, diag = FALSE)
upper_tri.assign(x, v, diag = FALSE)

Arguments

Value

Get a lower/upper triangular logical matrix with values TRUE/FALSE, a vector with the values of a lower/upper triangular, the sum of the upper/lower triangular if suma is set TRUE or assign to the lower/upper (only for large matrices) triangular. You can also include diagonal with any operation if argument diag is set to "TRUE".

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

rowMins, colFalse, nth, rowrange, rowMedians, rowVars, colTrue

Examples

x <- matrix(runif(10*10),10,10)

all.equal(lower_tri(c(10,10)),lower.tri(x))

all.equal(lower_tri(x),x[lower.tri(x)])

#all.equal(upper_tri(c(10,10)),upper.tri(x))

#all.equal(upper_tri(x),x[upper.tri(x)])



#all.equal(lower_tri(c(10,10),diag = TRUE),lower.tri(x,diag = TRUE))

#all.equal(lower_tri(x,diag = TRUE),x[lower.tri(x,diag = TRUE)])

#all.equal(upper_tri(c(10,10),diag = TRUE),upper.tri(x,diag = TRUE))

#all.equal(upper_tri(x,diag = TRUE),x[upper.tri(x,diag = TRUE)])

all.equal(lower_tri.assign(x,diag = TRUE,v=rep(1,1000)),x[lower.tri(x,diag = TRUE)]<-1)

all.equal(upper_tri.assign(x,diag = TRUE,v=rep(1,1000)),x[upper.tri(x,diag = TRUE)]<-1)

x<-NULL

Match

Description

Return the positions of its first argument that matches in its second.

Usage

Match(x,key=NULL)

Arguments

Details

This function implements the R's \"match\" function. This version basicaly calculates the match(x,sort(unique(x))) for now. Do not use the argument key!

Value

Returns the position/positions of the given key/keys in the x vector.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>

See Also

match

Examples

y <- rnorm(100)
a <- Match(y)
b <-50
all.equal(as.vector(a),as.vector(b))

Permutation

Description

Permute the given vector.

Usage

permutation(x, nperm = gamma(length(x)+1))
permutation.next(x, nperm = gamma(length(x)+1))
permutation.prev(x, nperm = gamma(length(x)+1))
bincomb(n)

Arguments

Details

This function implements "Permutation", which means all the possible combinations. In the permutation.next and permutation.prev if there aren't possible combinations it returns the same vector. "Binary Combinations" for "bincomb", means all the possible combinations for the binary number with length "n".

Value

Returns a matrix with all possible combinations of the given vector or a matrix row with one possible combinations.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>

See Also

combn,comb_n

Examples

y <- rnorm(3)
b <- permutation(y)
b <- permutation.next(y)
b <- permutation.prev(y)
g <- bincomb(3)

Replicate columns/rows

Description

Replicate columns/rows.

Usage

rep_col(x,n)
rep_row(x,n)

Arguments

Value

A matrix where each column/row is equal to "x".

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

rowMins, rowFalse, nth, colrange, colMedians, colVars, colSort, rowSort, rowTrue

Examples

x <- runif(10)
all.equal(rep_col(x,10),matrix(x,nrow=length(x),ncol=10))
all.equal(rep_row(x,10),matrix(x,ncol=length(x),nrow=10,byrow=TRUE))

Represantation of Stack

Description

Represantation of Stack.

Usage

Stack(x,type=NULL)

Arguments

Details

Stack is an abstract data type - data structure based on the principle of last in first out. To access the 3 fields, use operator "$".

Value

An object of class "Stack". This object holds 3 fields:

pop: remove the first element (from the top). top: access the first element (from the top). push: add an element to the top of the Stack.

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colShuffle, colVars, colmeans

Examples

x<-Stack(10,type=integer)

x$push(5)
x$push(10)
x$top()  == 10
x$pop()
x$top() == 5

y<-rnorm(10)
x<-Stack(x)

x$push(5) # length increased to 11
x$top() # access the last element that pushed, 5
x$pop() # pop the last element that pushed

Sort and unique

Description

Sort and unique numbers.

Usage

sort_unique(x)
sort_unique.length(x)
Unique(x, fromLast = FALSE)

Arguments

Details

The "sort_unique" function implements R's "unique" function using C++'s function but also sort the result. The "sort_unique.length" returns the length of the unique numbers only for itegers.

Value

Returns the discrete values or sorted or their length (depending on the function you do).

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>

See Also

colSort, rowSort, sort_cor_vectors

Examples

y <- rnorm(100)
a <- sort_unique(y)
b <- sort.int(unique(y))
all.equal(as.vector(a),as.vector(b))
x <- rpois(1000,10)
sort_unique.length(x)
length(sort_unique(x))

x<-a<-b<-NULL

ANOVA for two quasi Poisson regression models

Description

ANOVA for two quasi Poisson regression models.

Usage

anova_quasipois.reg(mod0, mod1, n) 

Arguments

Details

This is an ANOVA type significance testing for two quasi Poisson models.

Value

A vector with 4 elements, the test statistic value, its associated p-value and the relevant degrees of freedom of the numerator and the denominator.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Papke L. E. & Wooldridge J. (1996). Econometric methods for fractional response variables with an application to 401(K) plan participation rates. Journal of Applied Econometrics, 11(6): 619–632.

McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

See Also

anova_qpois.reg, qpois.reg, univglms, quasipoisson.anova

Examples

y <- rnbinom(200, 10, 0.5)
x <- matrix(rnorm(200 * 3), ncol = 3)
a1 <- qpois.reg(x, y)
a0 <- qpois.reg(x[, 1], y)
res<-anova_quasipois.reg(a0, a1, 200)
b1 <- glm(y ~ x, family = quasipoisson)
b0 <- glm(y ~ x[, 1], family = quasipoisson)
res<-anova(b0, b1, test = "F")
c1 <- glm(y ~ x, family = poisson)
c0 <- glm(y ~ x[, 1], family = poisson)
res<-anova(c0, c1, test = "Chisq")

All k possible combinations from n elements

Description

All k possible combinations from n elements.

Usage

comb_n(n, k,simplify=TRUE)

Arguments

Value

A matrix with k columns and rows equal to the number of possible unique combinations of n with k elements. If simplify is set to TRUE then a list with k values where each value has length equal to the number of possible unique combinations of n with k elements.

Author(s)

Manos Papadakis and Marios Dimitriadis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com> and Marios Dimitriadis <kmdimitriadis@gmail.com>.

References

Nijenhuis A. and Wilf H.S. (1978). Combinatorial Algorithms for Computers and Calculators. Academic Press, NY.

See Also

nth, colMaxs, colMins, colrange

Examples

comb_n(20, 4)
combn(20, 4)
x <- rnorm(5)
res<-comb_n(x, 3)

Analysis of covariance

Description

Analysis of covariance

Usage

ancova1(y, ina, x, logged = FALSE)

Arguments

Details

Analysis of covariance is performed. No interaction between the factor and the covariate is tested. Only the main effects. The design need not be balanced. The values of ina need not have the same frequency. The sums of squares have been adjusted to accept balanced and unbalanced designs.

Value

A matrix with the test statistic and the p-value for the factor variable and the covariate.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

D.C. Montgomery (2001). Design and analysis of experiments (5th Edition). New York: John Wiley & Sons

See Also

ancovas, ftests, ttests, anova1

Examples

y <- rnorm(90)
ina <- rbinom(90, 2, 0.5) + 1
x <- rnorm(90)
a <- ancova1(y, ina, x)

Analysis of variance with a count variable

Description

Analysis of variance with a count variable.

Usage

poisson.anova(y, ina, logged = FALSE)
geom.anova(y, ina, type = 1, logged = FALSE)
quasipoisson.anova(y, ina, logged = FALSE)

Arguments

Details

This is the analysis of variance with Poisson or geometric distributed data. What we do is a log-likelihood ratio test. However, this is exactly the same as Poisson regression with a single predictor variable who happens to be categorical. Needless to say that this is faster function than the glm command in R. For the same purpose with a Bernoulli variable use g2Test. The quasinpoisson.anova is when in the glm function you specify family = quasipoisson. This is suitable for the case of over or under-dispersed data.

Value

A vector with two values, the difference in the deviances (or the scale difference in the case of quasi poisson) and the relevant p-value. The quasipoisson.anova also returns the estimate of the \phi parameter.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

logistic.cat1, g2Test, poisson.anovas, anova, poisson_only, poisson.mle

Examples

y <- rpois(300, 10)
ina <- rbinom(300, 3, 0.5) + 1 
a1 <- poisson.anova(y, ina) 
a2 <- glm(y ~ ina, poisson) 


res<-anova(a2, test = "Chisq")

y <- rgeom(300, 0.7)
res<-geom.anova(y, ina) 

Angular central Gaussian random values simulation

Description

Angular central Gaussian random values simulation.

Usage

racg(n, sigma, seed = NULL)

Arguments

Details

The algorithm uses univariate normal random values and transforms them to multivariate via a spectral decomposition. The vectors are then scaled to have unit length.

Value

A matrix with the simulated data.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Tyler D. E. (1987). Statistical analysis for the angular central Gaussian distribution on the sphere. Biometrika 74(3): 579-589.

See Also

acg.mle, rmvnorm, rmvlaplace, rmvt

Examples

s <- cov( iris[, 1:4] )
#x <- racg(100, s)
#res<-acg.mle(x)  
#res<-vmf.mle(x)  ## the concentration parameter, kappa, is very low, close to zero, as expected.

Apply method to Positive and Negative number

Description

Apply method to Positive and Negative number.

Usage

negative(x,method = "min")
positive(x,method = "min")
positive.negative(x,method = "min")

Arguments

Details

These functions apply the chosen method to the chosen subset (negative, positive, or both) from the vector and return the result.

Value

negative: apply the chosen method to every negative number of the input vector. positive: apply the chosen method to every positive number of the input vector. positive.negative: apply the chosen method to every negative and positive number of the input vector.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

nth, colnth, rownth,sort_unique, Round

Examples

x <- rnorm(1000)

identical(negative(x,"min"), min(x<0))
identical(positive(x,"min"), min(x>0))
identical(positive.negative(x,"min"), c(min(x<0),min(x>0)))

x<-NULL

Apply to each column a method under condition

Description

Apply to each column a method under condition.

Usage

apply.condition(x,method = "+",oper = ">",cond.val = 0)

Arguments

Details

Apply to each col the specified method using the condition.

Value

An integer vector with the coresponding values.

Author(s)

Manos Papadakis and Michail Tsagris

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com> and Michail Tsagris <mtsagris@uoc.gr>.

See Also

colsums, colMedians, colVars

Examples

x <- matrix(rpois(100,6),10, 10)
identical(apply(x,2,function(x){ sum(x[x>0]) }), apply.condition(x,"+",">",0))
x<-NULL

BIC (using partial correlation) forward regression

Description

BIC (using partial correlation) forward regression.

Usage

bic.corfsreg(y, x, tol = 2) 

Arguments

Details

The forward regression tries one by one the variables using the F-test, basically partial F-test every time for the latest variable. This is the same as testing the significance of the coefficient of this latest enetered variable. Alternatively the correlation can be used and this case the partial correlation coefficient. There is a direct relationship between the t-test statistic and the partial correlation coefficient. Now, instead of having to calculate the test statistic, we calculate the partial correlation coefficient. The largest partial correlation indicates the candidate variable to enter the model. If the BIC of the regression model with that variable included, reduces, less than "tol" from the previous model without this variable, the variable enters.

Value

A matrix with two columns, the index of the selected variable(s) and the BIC of each model. The first line is always 0 and the BIC of the model with no predictor variables.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Draper, N.R. and Smith H. (1988). Applied regression analysis. New York, Wiley, 3rd edition.

See Also

cor.fsreg, score.glms, univglms, logistic_only, poisson_only, regression

Examples

## 200 variables, hence 200 univariate regressions are to be fitted
x <- matrix( rnorm(200 * 200), ncol = 200 )
y <- rnorm(200)
a1 <- bic.corfsreg(y, x)
a2 <- cor.fsreg(y, x)
x <- NULL

BIC forward regression with generalised linear models

Description

BIC forward regression with generalised linear models.

Usage

bic.fs.reg(y, x, tol = 2, type = "logistic") 

Arguments

Details

The forward regression tries one by one the variables using the BIC at each step for the latest variable. If the BIC of the regression model with that variable included, is less than "tol" from the previous model without this variable, the variable enters.

Value

A matrix with two columns, the index of the selected variable(s) and the BIC of each model.

Author(s)

Marios Dimitriadis

R implementation and documentation: Marios Dimitriadis <kmdimitriadis@gmail.com>.

References

Draper, N.R. and Smith H. (1988). Applied regression analysis. New York, Wiley, 3rd edition.

See Also

fs.reg, bic.corfsreg, cor.fsreg, score.glms, univglms, logistic_only, poisson_only, regression

Examples

x <- matrix(rnorm(200 * 50), ncol = 50)
## 200 variables, hence 200 univariate regressions are to be fitted
y <- rbinom(200, 1, 0.5)
a <- bic.fs.reg(y, x)
x <- NULL

Backward selection regression

Description

Backward selection regression.

Usage

bs.reg(y, x, alpha = 0.05, type = "logistic")

Arguments

Details

This function currently implements only the binary Logistic and Poisson regressions. If the sample size is less than the number of variables a notification message will appear and no backward regression will be performed.

Value

The output of the algorithm is an S3 object including:

Author(s)

Marios Dimitriadis

R implementation and documentation: Marios Dimitriadis <mtsagris@csd.uoc.gr>

See Also

fs.reg, univglms, cor.fsreg

Examples

y <- rbinom(50, 1, 0.5)
x <- matrnorm(50, 10)
res<-bs.reg(y, x)

Binary search algorithm

Description

Search a value in an ordered vector.

Usage

binary_search(x, v, index=FALSE)

Arguments

Details

The functions is written in C++ in order to be as fast as possible.

Value

Search if the v exists in x. Then returns TRUE/FALSE if the value is been found.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

is_element

Examples

x <- sort(rnorm(1000))
v <- x[50]
b <- binary_search(x,v) 
b1 <- binary_search(x,v,TRUE) 

Binomial coefficient and its logarithm

Description

Binomial coefficient and its logarithm.

Usage

Lchoose(x, k)
Choose(x, k)

Arguments

Details

The binomial coefficient or its logarithm are evaluated.

Value

A vector with the answers.

Author(s)

Manos Papadakis

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

See Also

comb_n, Lbeta, Lgamma

Examples

x <- sample(20:30, 100, replace = TRUE)
res<-Choose(x, 4)
res<-Lchoose(x, 4)

x<-NULL

Bootstrap t-test for 2 independent samples

Description

Bootstrap t-test for 2 independent samples.

Usage

boot.ttest2(x, y, B = 999)

Arguments

Details

Instead of sampling B times from each sample, we sample \sqrt{B} from each of them and then take all pairs. Each bootstrap sample is independent of each other, hence there is no violation of the theory.

Value

A vector with the test statistic and the bootstrap p-value.

Author(s)

Michail Tsagris and Christina Chatzipantsiou

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Christina Chatzipantsiou <chatzipantsiou@gmail.com>.

References

B.L. Welch (1951). On the comparison of several mean values: an alternative approach. Biometrika, 38(3/4), 330-336.

Efron Bradley and Robert J. Tibshirani (1993). An introduction to the bootstrap. New York: Chapman & Hall/CRC.

Chatzipantsiou C., Dimitriadis M., Papadakis M. and Tsagris M. (2019). Extremely efficient permutation and bootstrap hypothesis tests using R. To appear in the Journal of Modern Applied Statistical Methods.

https://arxiv.org/ftp/arxiv/papers/1806/1806.10947.pdf

See Also

ttest2, exact.ttest2, ftest

Examples

tic <- proc.time()
x <- rexp(40, 4)
y <- rbeta(50, 2.5, 7.5)
a <- boot.ttest2(x, y, 9999)
a

Check Namespace and Rd files

Description

Check Namespace/Rd and examples files.

Usage

checkNamespace(path.namespace,path.rfolder,paths.full = FALSE)
checkAliases(path.man,path.rfolder,paths.full = FALSE)
checkTF(path.man,paths.full = FALSE)
checkExamples(path.man,package,each = 1,print.errors = stderr(),
	print.names = FALSE,paths.full = FALSE)
checkUsage(path.man,path.rfolder,paths.full = FALSE)

Arguments

Details

checkNamespace: reads from the NAMESPACE folder all the export R functions, reads from folder R all the R functions and check if all the functions are export.

checkAliases: reads from the man directory all the Rd files, then reads from each file the aliases and check if:

• All the R files has man file or an alias.

• All aliases belongs to functions.

• If there are dublicated aliases.

checkExamples: reads from the man directory all the Rd files, then read from each file the examples and then run each of them. If you want to print the errors in any file then set "print.errors=file_name" or in the standard error "print.errors=stderr()" and then you will see all the errors for every file. Set to argument "package" the name of your package. The argument "print.names" it is very helpful because if any of you function crashes R during running you will never know which one was. So setting it "TRUE", it will print the name of each file before running it's example.It might crash, but you will know which file. Remember that there is always an error timeout so it might didn't crash the current file but one from the previous.

checkTF: reads from the man directory all the Rd files, then read from each file the examples and checks if any examples has the values "T" and "F" instead "TRUE" and "FALSE". The "T","F" is wrong.

checkUsage: reads from the man directory all the Rd files and for each man check if the usage section has the right signature for the functions from the R directory.

checkTF, checkUsage, checkAliases: you can choose which files not to read for both R and Rd. You must add in the first line of the file in comment the "attribute" "[dont read]". Then each function will know which file to read or not. For Rd you add "%[dont read]" and for R "#[dont read]". Finally, these functions will return in the result a list of which files had this attribute.

Value

checkNamespace: a vector with the names of missing R files. (Don't use it for now)

checkAliases: a list with 4 fields.

• Missing Man files: A vector with the names of the missing Rd files or nothing.

• Missing R files: A vector with the names of the missing R files or nothing.

• Duplicate alias: A vector with the names of the dublicate aliases or nothing.

• dont read: A list with 2 fields

  • R: A character vector whith the names of the files that had attribute "#[dont read]" or nothing.

  • Rd: A character vector whith the names of the files that had attribute "%[dont read]" or nothing.

checkExamples: a list with 3 fields

• Errors: A character vector with the names of the Rd files that produced an error.

• Big Examples: A character vector with the names of the Rd files that has big examples per line.

• dont read: A list with 2 fields

  • R: A character vector whith the names of the files that had attribute "#[dont read]" or nothing.

  • Rd: A character vector whith the names of the files that had attribute "%[dont read]" or nothing.

checkTF: a list with 3 fields

• TRUE: A character vector with the names of the Rd files that has "T" or nothing.

• FALSE: A character vector with the names of the Rd files that has "F" or nothing.

• dont read: A list with 2 fields

  • R: A character vector whith the names of the files that had attribute "#[dont read]" or nothing.

  • Rd: A character vector whith the names of the files that had attribute "%[dont read]" or nothing.

checkUsage: a list with 3 fields

• missing functions: A character vector with the name of the file that is missing and the Rd file that is found or nothing.

• missmatch functions: A character vector with the name of the file that has missmatch function and the Rd file that is found or nothing.

• dont read: A list with 2 fields

  • R: A character vector whith the names of the files that had attribute "#[dont read]" or nothing.

  • Rd: A character vector whith the names of the files that had attribute "%[dont read]" or nothing.

• hidden functions: A character vector with the name of the functions thath have been declared as hidden.

• usage lines wider than 90 characters: A list with the Rd's that have usages that are wider than 90 characters.

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

AddToNamespace, sourceR, sourceRd, read.examples

Examples

	#for example: path.namespace="C:\some_file\NAMESPACE"
	#for example: path.rfolder="C:\some_file\R\"
	#for example: path.man="C:\some_file\man\"
	#system.time( a<-checkNamespace(path.namespace,path.rfolder) )
	#system.time( b<-checkAliases(path.man,path.rfolder) )
	#system.time( b<-checkExamples(path.man) )
	#system.time( b<-checkExamples(path.man,2) )
	#system.time( b<-checkTF(path.man) )
	#system.time( b<-checkTF(path.man,path.rfolder) )

Check if values are integers and convert to integer

Description

Check if values are integers and convert to integer.

Usage

is_integer(x)
as_integer(x,result.sort = TRUE,init = 1, parallel = FALSE)

Arguments

Details

The behavior of these functions are different than R's built in.

is_integer: check if all the values are integers in memory. If typeof is double, and the values are integers in range -2^31 : 2^31 then it is better to convert to integer vector for using less memory. Also you can decrease the time complexity.

as_integer: converts the discrete values to integers.

Value

is_integer: A logical value, TRUE if all values are integers and in range -2^31 : 2^31. Otherwise FALSE.

as_integer: By default the function will return the same result with "as.numeric" but the user can change the "init" value not start from 1 like R's. Also the result can be unsorted using "result.sort".

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

as_integer, colVars, colmeans

Examples

x<-runif(10)
y1<-is_integer(x) # y1 is FALSE
x<-as.numeric(rpois(10,10)) # integers but typeof is double
y1<-is_integer(x) # y1 is TRUE so you can convert to integer vector.

as_integer(letters) ## as.numeric(letters) produce errors
x<-y1<-NULL

Check whether a square matrix is symmetric

Description

Check whether a square matrix is symmetric.

Usage

is.symmetric(x)

Arguments

Details

Instead of going through the whole matrix, the function will stop if the first disagreement is met.

Value

A boolean value, TRUE of FALSE.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

cholesky, cora, cova

Examples

x <-matrix( rnorm( 100 * 400), ncol = 400 )
s1 <- cor(x)
is.symmetric(s1)
x <- x[1:100, ]
is.symmetric(x)

x<-s1<-NULL

Chi-square and G-square tests of (unconditional) indepdence

Description

Chi-square and G-square tests of (unconditional) indepdence.

Usage

gchi2Test(x, y, logged = FALSE)

Arguments

Details

The function calculates the test statistic of the \chi^2 and the G^2 tests of unconditional independence between x and y. x and y need not be numerical vectors like in g2Test.

Value

A matrix with two rows. In each row the X2 or G2 test statistic, its p-value and the degrees of freedom are returned.

Author(s)

Manos Papadakis and Michail Tsagris.

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com> and Michail Tsagris <mtsagris@uoc.gr>.

References

Tsagris M. (2017). Conditional independence test for categorical data using Poisson log-linear model. Journal of Data Science, 15(2): 347–356.

See Also

g2Test_univariate, g2Test_univariate_perm, g2Test

Examples

nvalues <- 3
nvars <- 2
nsamples <- 2000
data <- matrix( sample( 0:(nvalues - 1), nvars * nsamples, replace = TRUE ), nsamples, nvars )

res<-gchi2Test(data[, 1], data[, 2])
res<-g2Test_univariate( data, rep(3, 2) )  ## G^2 test
res<-chisq.test(data[, 1], data[, 2])  ## X^2 test from R
  
data<-NULL

Cholesky decomposition of a square matrix

Description

Cholesky decomposition of a square matrix.

Usage

cholesky(x,parallel = FALSE)

Arguments

Details

The Cholesky decomposition of a square positive definite matrix is computed. The use of parallel is suggested for matrices with dimensions of 1000 or more.

Value

An upper triangular matrix.

Author(s)

Manos Papadakis

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>

See Also

is.symmetric

Examples

x = matrix(rnorm(1000 * 50), ncol = 50)
s = cov(x)
a1 <- cholesky(s)
#a2 <- chol(s)
#all.equal(a1[upper.tri(a1)], a2[upper.tri(a2)])
x <- NULL
s <- NULL
a1 <- NULL
a2 <- NULL

Circular or angular regression(s)

Description

Regression with circular dependent variable and Euclidean or categorical independent variables.

Usage

spml.reg(y, x, tol = 1e-07, seb = FALSE, maxiters = 100)
spml.regs(y, x, tol = 1e-07, logged = FALSE, maxiters = 100, parallel = FALSE)

Arguments

Details

The Newton-Raphson algorithm is fitted in this regression as described in Presnell et al. (1998).

For the case of spml.regs(), for each colum of x a circual regression model is fitted and the hypothesis testing of no association between y and this variable is performed.

Value

For the spml.reg() a list including:

For the spml.regs() a matrix with two columns, the test statistics and their associated (log) p-values.

Author(s)

Michail Tsagris and Stefanos Fafalios.

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Stefanos Fafalios <stefanosfafalios@gmail.com>.

References

Presnell Brett, Morrison Scott P. and Littell Ramon C. (1998). Projected multivariate linear models for directional data. Journal of the American Statistical Association, 93(443): 1068-1077.

See Also

spml.mle, iag.mle, acg.mle

Examples

x <- rnorm(100)
z <- cbind(3 + 2 * x, 1 -3 * x)
y <- cbind( rnorm(100,z[ ,1], 1), rnorm(100, z[ ,2], 1) )
y <- y / sqrt( rowsums(y^2) )
a1 <- spml.reg(y, x)
y <- atan( y[, 2] / y[, 1] ) + pi * I(y[, 1] < 0) 
a2 <- spml.reg(y, x)

x <- matrnorm(100, 5)
a3 <- spml.regs(y, x)

Circular-linear correlation

Description

It calculates the squared correlation between a circular and one or more linear variables.

Usage

circlin.cor(theta, x)

Arguments

Details

The squared correlation between a circular and one or more linear variables is calculated.

Value

A matrix with as many rows as linear variables including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Mardia, K. V. and Jupp, P. E. (2000). Directional statistics. Chicester: John Wiley & Sons.

See Also

spml.reg

Examples

#phi <- rvonmises(50, 2, 20, rads = TRUE)
#x <- 2 * phi + rnorm(50)
#y <- matrix(rnorm(50 * 5), ncol = 5)
#res<-circlin.cor(phi, x)
#res<-circlin.cor(phi, y)
y <- NULL

Coefficient matrices

Description

Coefficient matrices.

Usage

coeff(x, method,vector = FALSE)

Arguments

Details

• bhattacharyya : \sum \sqrt(P_i * Q_i)

Value

A square matrix with the pairwise coefficients.

Author(s)

Manos Papadakis.

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

dista, Dist

Examples

x <- matrix(rnorm(50 * 10), ncol = 10)
a1 <- coeff(x,"bhattacharyya")
x<-a1<-NULL

Colum-wise cumulative operations (sum, prod, min, max)

Description

Colum-wise cumulative operations (sum, prod, min, max).

Usage

colCumSums(x)
colCumProds(x)
colCumMins(x)
colCumMaxs(x)

Arguments

Details

Cumulative mins, maxs, sums and prods are returned.

Value

A matrix with the results. It has one row less than the initial matrix.

Author(s)

Manos Papadakis and Michail Tsagris

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com> and Michail Tsagris <mtsagris@uoc.gr>.

See Also

colsums, colMedians, colVars

Examples

x <- matrnorm(10, 10)
res<-colCumSums(x)
res<-colCumMins(x)
res<-colCumMaxs(x)
res<-colCumProds(x)

Column and row wise coefficients of variation

Description

Column and row wise coefficients of variation.

Usage

colcvs(x, ln = FALSE, unbiased = FALSE) 
rowcvs(x, ln = FALSE, unbiased = FALSE) 

Arguments

Details

The colum-wise coefficients of variation are calculated.

Value

A vector with the coefficient of variation for each column or row.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

colsums, colVars

Examples

m <- rnorm(100, 10)
x <- matrix(rnorm(100 * 100, m, 1), ncol = 100)
a1 <- colcvs(x)
a2 <- colcvs(x[1:25, ], unbiased = TRUE)
a3 <- colcvs( exp(x), ln = TRUE)
x <- NULL

Column and row-wise Any

Description

Column and row-wise Any/All of a matrix.

Usage

colAny(x)
rowAny(x)
colAll(x, parallel = FALSE, cores = 0)
rowAll(x, parallel = FALSE, cores = 0)

Arguments

Details

The functions is written in C++ in order to be as fast as possible.

Value

A vector where item "i" is true if found Any/All true in column/row "i". Otherwise false.

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

Median, colMedians, colMeans (buit-in R function)

Examples

x <- matrix(as.logical(rbinom(100*100,1,0.5)),100,100)
a<-colAny(x)
#b<-apply(x,2,any)
#all.equal(a,b)

a<-rowAny(x)
#b<-apply(x,1,any)
#all.equal(a,b)

a<-colAll(x)
#b<-apply(x,2,all)
#all.equal(a,b)

a<-b<-x<-NULL

Column and row-wise Order - Sort Indices

Description

Column and row-wise Order - Sort Indices.

Usage

colOrder(x,stable=FALSE,descending=FALSE, parallel = FALSE, cores = 0)
rowOrder(x,stable=FALSE,descending=FALSE, parallel = FALSE, cores = 0)
Order(x,stable=FALSE,descending=FALSE,partial = NULL,parallel = FALSE)

Arguments

Details

The function applies "order" in a column or row-wise fashion or Order a vector. If you want the same results as R's, then set "stable=TRUE" because "stable=FALSE" uses a sorting algorithm that it is not stable like R's sort. But it is faster to use the default. This version is faster for large data, more than 300.

Value

For "colOrder" and "rowOrder" a matrix with integer numbers. The result is the same as apply(x, 2, order) or apply(x, 1, order).

For "Order" sort the vector and returns the indices of each element that it has before the sorting. The result is the same as order(x) but for the same exactly results set argument "stable" to "TRUE".

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colsums, coldiffs, colMedians, colprods

Examples

x <- matrix( runif(10 * 10), ncol = 10 )
res<-colOrder(x)
res<-apply(x, 2, order)
res<-rowOrder(x)
t(apply(x, 1, order))

y <- rnorm(100)
b <- Order(y)
a <- order(y)
all.equal(a,b) ## false because it is not stable
b <- Order(y,stable=TRUE)
all.equal(a,b) ## true because it is stable

x<-y<-b<-a<-NULL

Column and row-wise Shuffle

Description

Column and row-wise shuffle of a matrix.

Usage

colShuffle(x)
rowShuffle(x)

Arguments

Details

The functions is written in C++ in order to be as fast as possible.

Value

A vector with the column/row Shuffle.

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

Median, colVars, colMeans (buit-in R function)

Examples

x <- matrix( rnorm(100 * 100), ncol = 100 )
colShuffle(x)
rowShuffle(x)

x<-NULL

Column and row-wise means of a matrix

Description

Column and row-wise means of a matrix.

Usage

colmeans(x, parallel = FALSE, cores = 0)
## S3 method for class 'matrix'
colmeans(x, parallel = FALSE, cores = 0)
## S3 method for class 'data.frame'
colmeans(x, parallel = FALSE, cores = 0)
rowmeans(x)
colhameans(x, parallel = FALSE)
rowhameans(x)

Arguments

Value

A vector with the column or row arithmetic or harmonic means.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colsums, rowsums, colMins, colMedians, colMads

Examples

x <- matrix(rpois(100 * 100, 10),ncol = 100)
x1 <- colmeans(x)
#x2 <- colMeans(x)
#all.equal(x1,x2)

x1 <- rowmeans(x)
#x2 <- rowMeans(x)
#all.equal(x1,x2)

colhameans(x)
rowhameans(x) 

x<-x1<-x2<-NULL

Column and row-wise medians of a matrix or median of a vector

Description

Column and row-wise medians of a matrix or median of a vector.

Usage

colMedians(x,na.rm = FALSE, parallel = FALSE, cores = 0)
rowMedians(x,na.rm = FALSE, parallel = FALSE, cores = 0)
Median(x,na.rm=FALSE)
med(x,na.rm=FALSE)

Arguments

Details

The functions is written in C++ in order to be as fast as possible.

Value

A vector with the column medians.

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

Median, colVars, colMeans (buit-in R function)

Examples

x <- matrix( rnorm(100 * 100), ncol = 100 )
a <- apply(x, 2, median) 
b1 <- colMedians(x) 
all.equal(as.vector(a), b1)

x<-a<-b1<-NULL

Column and row-wise nth smallest value of a matrix/vector

Description

Column and row-wise nth smallest value of a matrix/vector.

Usage

colnth(x,elems, num.of.nths = 1,descending = FALSE,na.rm = FALSE,
	index.return = FALSE, parallel = FALSE, cores = 0)
rownth(x,elems, num.of.nths = 1,descending = FALSE,na.rm = FALSE,
	index.return = FALSE, parallel = FALSE, cores = 0)
nth(x, k, num.of.nths = 1,descending = FALSE,index.return = FALSE,na.rm = FALSE)

Arguments

Details

The functions is written in C++ in order to be as fast as possible.

Value

For "colnth" , "rownth": A vector with the column/row nth

For "nth": The nth value.

Author(s)

Manos Papadakis <papadakm95@gmail.com>

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

Median, colMedians, colMeans (buit-in R function)

Examples

x <- matrix( rnorm(100 * 100), ncol = 100 )
elems <- sample(1:100,100,TRUE)
colnth(x,elems)
rownth(x,elems)
x <- rnorm(1000)

nth(x, 500)
#sort(x)[500]

x<-elems<-NULL

Column and row-wise products

Description

Column and row-wise products.

Usage

colprods(x, method = "direct")
rowprods(x)

Arguments

Details

The product of the numbers in a matrix is returned either column-wise or row-wise.

Value

A vector with the column or the row products.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colsums, coldiffs, colMedians

Examples

x <- matrix( runif(100 * 10), ncol = 10 )
res<-colprods(x)
res<-rowprods(x)

x<-NULL

Column and row-wise range of values of a matrix

Description

Column and row-wise range of values of a matrix.

Usage

colrange(x, cont = TRUE, parallel = FALSE,cores = 0)
rowrange(x, cont = TRUE)

Arguments

Value

A vector with the relevant values.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colMins, colMaxs, rowMins, rowMaxs, nth, colMedians, colVars, colSort, rowSort

Examples

x <- matrix( rnorm(100 * 100), ncol = 100 )

a1 <- colrange(x) 
a2 <- apply(x, 2, function(x) diff( range(x)) ) 
all.equal(a1, a2)

a1 <- rowrange(x) 
a2 <- apply(x, 1, function(x) diff( range(x)) ) 
all.equal(a1, a2)

x<-a1<-a2<-NULL

Column and row-wise ranks

Description

Column and row-wise ranks.

Usage

colRanks(x,method = "average",descending = FALSE,
     stable = FALSE, parallel = FALSE, cores = 0)
rowRanks(x,method = "average",descending = FALSE,
     stable = FALSE, parallel = FALSE, cores = 0)
Rank(x,method = "average",descending = FALSE,stable = FALSE, parallel = FALSE)

Arguments

Details

For each column or row of a matrix the ranks are calculated and they are returned.

Value

A matrix with the column or row-wise ranks.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

Rank, correls

Examples

x <- matrnorm(100, 10)
a1 <- colRanks(x)
a2 <- apply(x, 2, rank)
b1 <- rowRanks(x)
b2 <- apply(x, 1, rank)

a1 <- Rank(x[,1])
a1 <- rank(x[,1])

x<-a1<-a2<-b1<-b2<-NULL

Column and row-wise sums of a matrix

Description

Column and row-wise sums of a matrix.

Usage

colsums(x,indices = NULL, parallel = FALSE, na.rm = FALSE, cores = 0)
rowsums(x,indices = NULL, parallel = FALSE, na.rm = FALSE, cores = 0)

Arguments

Value

A vector with sums.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colMedians, colmeans, colVars

Examples

x <- matrix(rpois(500 * 100, 10),ncol = 100)
x1 <- colsums(x)
x2 <- colSums(x)
all.equal(x1,x2)

x1 <- rowsums(x)
x2 <- rowSums(x)
all.equal(x1,x2)

x<-x1<-x2<-NULL

Column and row-wise tabulate

Description

Column and row-wise tabulate of a matrix.

Usage

colTabulate(x, max_number = max(x))
rowTabulate(x, max_number = max(x))

Arguments

Details

The functions is written in C++ in order to be as fast as possible.

Value

A matrix where in each column the command "tabulate" has been performed. The number of rows of the returned matrix will be equal to the max_number if given. Otherwise, the functions will find this number.

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colShuffle, colVars, colmeans

Examples

x <- matrix( rbinom(100 * 100, 4, 0.5), ncol = 100 )
colTabulate(x)
rowTabulate(x)

x<-NULL

Column and row-wise variances and standard deviations of a matrix

Description

Column and row-wise variances and standard deviations of a matrix

Usage

## S3 method for class 'matrix'
colVars(x, std = FALSE, na.rm = FALSE, parallel = FALSE, cores = 0)
## S3 method for class 'data.frame'
colVars(x, std = FALSE, na.rm = FALSE, parallel = FALSE, cores = 0)
colVars(x, std = FALSE, na.rm = FALSE, parallel = FALSE, cores = 0)
rowVars(x, std = FALSE, na.rm = FALSE, parallel = FALSE, cores = 0)

Arguments

Details

We found this on stackoverflow which was created by David Arenburg. We then modified the function to match the sums type formula of the variance, which is faster.

Value

A vector with the column variances or standard deviations.

Author(s)

Michail Tsagris and Manos Papadakis.

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

colmeans, colMedians, colrange

Examples

x <- matrix( rnorm(100 * 100), ncol = 100 )
a2 <- colVars(x)
x<-a2<-NULL

Column and row-wise mean absolute deviations

Description

Column and row-wise mean absolute deviations.

Usage

colMads(x,method = "median",na.rm=FALSE,parallel = FALSE, cores = 0)
rowMads(x,method = "median",na.rm=FALSE,parallel = FALSE, cores = 0)
Mad(x,method = "median",na.rm=FALSE)

Arguments

Details

The functions is written in C++ in order to be as fast as possible.

Value

A vector with the column-wise mean absolute deviations.

Author(s)

Manos Papadakis

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

colMedians, rowMedians, colVars, colmeans, colMeans (buit-in R function)

Examples

x <- matrix( rnorm(100 * 100), ncol = 100 )
a <- colMads(x)

x<-NULL

Column-wise MLE of some univariate distributions

Description

Column-wise MLE of some univariate distributions.

Usage

colexpmle(x)
colexp2.mle(x)
colgammamle(x, tol = 1e-07)
colinvgauss.mle(x)
collaplace.mle(x)
collindley.mle(x)
colmaxboltz.mle(x)
colnormal.mle(x)
colpareto.mle(x)
colrayleigh.mle(x)
colvm.mle(x, tol = 1e-07)
colweibull.mle(x, tol = 1e-09, maxiters = 100, parallel = FALSE)
colnormlog.mle(x)

Arguments

Details

For each column, the same distribution is fitted and its parameter and log-likelihood are computed.

Value

A matrix with two, three or five (for the colnormlog.mle) columns. The first one or the first two contain the parameter(s) of the distribution and the other columns contain the log-likelihood values.

Author(s)

Michail Tsagris and Stefanos Fafalios

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Stefanos Fafalios <stefanosfafalios@gmail.com>

References

Kalimuthu Krishnamoorthy, Meesook Lee and Wang Xiao (2015). Likelihood ratio tests for comparing several gamma distributions. Environmetrics, 26(8):571-583.

N.L. Johnson, S. Kotz and N. Balakrishnan (1994). Continuous Univariate Distributions, Volume 1 (2nd Edition).

N.L. Johnson, S. Kotz and N. Balakrishnan (1970). Distributions in statistics: continuous univariate distributions, Volume 2.

Sharma V. K., Singh S. K., Singh U. and Agiwal V. (2015). The inverse Lindley distribution: a stress-strength reliability model with application to head and neck cancer data. Journal of Industrial and Production Engineering, 32(3): 162-173.

See Also

vm.mle, poisson.mle, normal.mle, gammamle

Examples

x <- matrix(rnorm(1000 * 50), ncol = 50)
a <- colnormal.mle(x)
b <- collaplace.mle(x)
x <- NULL

Column-wise Yule's Y (coefficient of colligation)

Description

Column-wise Yule's Y (coefficient of colligation).

Usage

col.yule(x, y = NULL, ina)

Arguments

Details

Yule's coefficient of colligation is calculated for every column.

Value

A vector with Yule's Y, one for every column of x is returned.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Yule G. Udny (1912). On the Methods of Measuring Association Between Two Attributes. Journal of the Royal Statistical Society, 75(6):579-652.

See Also

yule, odds

Examples

x <- matrix(rbinom(300 * 10, 1, 0.5), ncol = 10)
ina <- rep(1:2, each = 150)
res<-col.yule( x, ina = ina )

Column-wise differences

Description

Column-wise differences.

Usage

coldiffs(x)

Arguments

Details

This function simply does this function x[, -1] - x[, -k], where k is the last column of the matrix x. But it does it a lot faster. That is, 2nd column - 1st column, 3rd column - 2nd column, and so on.

Value

A matrix with one column less containing the differences between the successive columns.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

Dist, dista, colmeans

Examples

x <- matrix( rnorm(50 * 10), ncol = 10 )
res<-coldiffs(x)

x<-NULL

Column-wise kurtosis and skewness coefficients

Description

Column-wise kurtosis and skewness coefficients.

Usage

colkurtosis(x, pvalue = FALSE)

colskewness(x, pvalue = FALSE)

Arguments

Details

The skewness and kurtosis coefficients are calculated. For the skewness coefficient we use the sample unbiased version of the standard deviation. For the kurtosis, we do not subtract 3.

Value

If "pvalue" is FALSE, a vector with the relevant coefficient. Otherwise a matrix with two columns. The kurtosis or skewness coefficient and the p-value from the hypothesis test that they are significantly different from 3 or 0 respectively.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

skew, skew.test2, colMedians, colmeans, colVars, sftests

Examples

## 200 variables, hence 200 F-tests will be performed
x = matrix( rnorm(200 * 50), ncol = 50 )
## 200 observations in total
colkurtosis(x)
colskewness(x)
x <- NULL

Column-wise matching coefficients

Description

Column-wise matching coefficients.

Usage

match.coefs(x, y = NULL, ina, type = "jacc") 

Arguments

Details

Two matrices are given as imput and for each column matching coefficients are calculated, either the Jaccard or the simple matching coefficient or both.

Value

A matrix with one or two columns, depending on the type you have specified. If you specify "both", there will be two columns, if you specify "jacc" or "smc" then just one column.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

odds, colTabulate

Examples

x <- matrix(rbinom(400 * 10, 1, 0.5), ncol = 10)
y <- matrix(rbinom(400 * 10, 1, 0.5), ncol = 10)
a <- match.coefs(x, y, type = "both")
x <- NULL
y <- NULL

Column-wise minimum and maximum of a matrix

Description

Column-wise minimum and maximum of a matrix.

Usage

colMins(x, value = FALSE, parallel = FALSE, cores = 0)
colMaxs(x, value = FALSE, parallel = FALSE, cores = 0)
colMinsMaxs(x, parallel = FALSE, cores = 0)

Arguments

Value

A vector with the relevant values.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

rowMins, rowMaxs, nth, colrange, colMedians, colVars, colSort, rowSort

Examples

x <- matrix( rnorm(100 * 200), ncol = 200 )

s1 <- colMins(x) 
s2 <- apply(x, 2, min) 

s1 <- colMaxs(x) 
s2 <- apply(x, 2, max) 

s1 <- colMinsMaxs(x)
s2 <- c(apply(x, 2, min), apply(x, 2, max)) 

x<-s1<-s2<-NULL

Column-wise true/false value of a matrix

Description

Column-wise true/false value of a matrix.

Usage

colTrue(x)
colFalse(x)
colTrueFalse(x)

Arguments

Value

An integer vector where item "i" is the number of the true/false values of "i" column.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

rowMins, rowFalse, nth, colrange, colMedians, colVars, colSort, rowSort, rowTrue

Examples

x <- matrix(as.logical(rbinom(100*100,1,0.5)),100,100)

s1 <- colTrue(x) 

s1 <- colFalse(x)  

s1 <- colTrueFalse(x)

x<-s1<-NULL

Column-wise uniformity tests for circular data

Description

Column-wise uniformity tests for circular data.

Usage

colwatsons(u)

Arguments

Details

These tests are used to test the hypothesis that the data come from a circular uniform distribution. The Kuiper test is much more time consuming and this is why it not implemented yet. Once we figure out a way to make it fast, we will incldue it.

Value

A matrix with two columns, the value of the test statistic and its associated p-value.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Jammalamadaka, S. Rao and SenGupta, A. (2001). Topics in Circular Statistics, pg. 153-55 (Kuiper's test) & 156-157 (Watson's test).

See Also

watson, vmf.mle, rvonmises

Examples

#x <- matrix( rvonmises(n = 50 * 10, m = 2, k = 0), ncol = 10 )
#res<-colwatsons(x)
x <- NULL

Convert R function to the Rfast's coresponding

Description

Convert R function to the Rfast's coresponding.

Usage

as.Rfast.function(Rfunction.name,margin=NULL)

Arguments

Details

Given the name of R function, it returns the coresponding function's name from Rfast.

Value

The coresponding Rfast function.

Author(s)

Manos Papadakis and Michail Tsagris

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com> and Michail Tsagris <mtsagris@uoc.gr>.

See Also

colsums, colMedians, colVars

Examples

res<-as.Rfast.function("var")

Convert a dataframe to matrix

Description

Convert a dataframe to matrix.

Usage

data.frame.to_matrix(x,col.names = NULL,row.names = NULL)

Arguments

Details

This functions converts a dataframe to matrix. Even if there are factors, the function converts them into numerical values. Attributes are not allowed for now.

Value

A matrix wich has the numrical values from the dataframe.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>

See Also

Match, is.symmetric, permutation

Examples

res<-data.frame.to_matrix(iris)

Correlation based forward regression

Description

Correlation based forward regression.

Usage

cor.fsreg(y, x, ystand = TRUE, xstand = TRUE, threshold = 0.05, 
tolb = 2, tolr = 0.02, stopping = "BIC") 

Arguments

Details

The forward regression tries one by one the variables using the F-test, basically partial F-test every time for the latest variable. This is the same as testing the significance of the coefficient of this latest enetered variable. Alternatively the correlation can be used and this case the partial correlation coefficient. There is a direct relationship between the t-test statistic and the partial correlation coefficient. Now, instead of having to calculate the test statistic, we calculate the partial correlation coefficient. Using Fisher's z-transform we get the variance imediately. The partial correlation coefficient, using Fisher's z-transform, and the partial F-test (or the coefficient's t-test statistic) are not identical. They will be identical for large sample sizes though.

Value

A matrix with three columns, the index of the selected variables, the logged p-value and the the test statistic value and the BIC or adjusted R^2 of each model. In the case of stopping="BICR2" both of these criteria will be returned.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Draper, N.R. and Smith H. (1988). Applied regression analysis. New York, Wiley, 3rd edition.

See Also

score.glms, univglms, logistic_only, poisson_only, regression

Examples

## 200 variables, hence 200 univariate regressions are to be fitted
x <- matrnorm(200,  100)
y <- rnorm(200)
cor.fsreg(y, x)
x <- NULL

Correlation between pairs of variables

Description

Correlations between pairs of variables.

Usage

corpairs(x, y, rho = NULL, logged = FALSE, parallel = FALSE) 

Arguments

Details

The paired correlations are calculated. For each column of the matrices x and y the correlation between them is calculated.

Value

A vector of correlations in the case of "rho" being NULL, or a matrix with two extra columns, the test statistic and the (logged) p-value.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Lambert Diane (1992). Zero-Inflated Poisson Regression, with an Application to Defects in Manufacturing. Technometrics. 34(1):1-14.

Johnson Norman L., Kotz Samuel and Kemp Adrienne W. (1992). Univariate Discrete Distributions (2nd ed.). Wiley

Cohen, A. Clifford (1960). Estimating parameters in a conditional Poisson distribution. Biometrics. 16:203-211.

Johnson, Norman L. Kemp, Adrianne W. Kotz, Samuel (2005). Univariate Discrete Distributions (third edition). Hoboken, NJ: Wiley-Interscience.

See Also

correls, allbetas, mvbetas

Examples

x <- matrnorm(100, 100)
y <- matrnorm(100, 100)
corpairs(x, y)
a <- corpairs(x, y)
x <- NULL
y <- NULL

Correlation between a vector and a set of variables

Description

Correlation between a vector and a set of variables.

Usage

correls(y, x, type = "pearson", a = 0.05, rho = 0)
groupcorrels(y, x, type = "pearson", ina)

Arguments

Details

The functions uses the built-in function "cor" which is very fast and then includes confidence intervals and produces a p-value for the hypothesis test.

Value

For the "correls" a matrix with 5 column; the correlation, the p-value for the hypothesis test that each of them is eaqual to "rho", the test statistic and the $a/2%$ lower and upper confidence limits.

For the "groupcorrels" a matrix with rows equal to the number of groups and columns equal to the number of columns of x. The matrix contains the correlations only, no statistical hypothesis test is performed.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

allbetas, univglms

Examples

x <- matrnorm(60, 100 )
y <- rnorm(60)
r <- cor(y, x)  ## correlation of y with each of the xs
a <- allbetas(y, x)  ## the coefficients of each simple linear regression of y with x
b <- correls(y, x)
ina <- rep(1:2, each = 30)
b2 <- groupcorrels(y, x, ina = ina) 
x <- NULL

Fast covariance and correlation matrix calculation

Description

Fast covariance and correlation matrix calculation.

Usage

cova(x, center = FALSE, large = FALSE)

cora(x, large = FALSE)

Arguments

Details

The calculations take place faster than the built-in functions cor as the number of variables increases. This is true if the number of variables is high, say from 500 and above. The "cova" on the other hand is always faster. For the "cova" in specific, we have an option to center the data prior to the cross product. This can be more stable if you have many tens of thousands of rows due to numerical issues that can arise.

For the correlation matrix we took the code from here

https://stackoverflow.com/questions/18964837/fast-correlation-in-r-using-c-and-parallelization/18965892#18965892

Value

The covariance or the correlation matrix.

Author(s)

Michail Tsagris and Manos Papadakis <papadakm95@gmail.com>.

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

colVars, cor, cov

Examples

x <- matrnorm(100, 40)
s1 <- cov(x) 
s2 <- cova(x)
all.equal(s1, s2)
x <- NULL

Cox confidence interval for the ratio of two Poisson variables

Description

Cox confidence interval for the ratio of two Poisson variables.

Usage

cox.poisrat(x, y, alpha = 0.05)
col.coxpoisrat(x, y, alpha = 0.05)

Arguments

Details

Cox confidence interval for the ratio of two Poisson means is calculated.

Value

For the cox.poisrat a vector with three elements, the ratio and the lower and upper confidence interval limits. For the col.coxpoisrat a matrix with three columns, the ratio and the lower and upper confidence interval limits.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Krishnamoorthy K., Peng J. and Zhang D. (2016). Modified large sample confidence intervals for Poisson distributions: Ratio, weighted average, and product of means. Communications in Statistics-Theory and Methods, 45(1): 83-97.

See Also

correls, Table

Examples

x <- rpois(100, 10)
y <- rpois(100, 10)
res<-cox.poisrat(x, y)

Cross-Validation for the k-NN algorithm

Description

Cross-Validation for the k-NN algorithm.

Usage

knn.cv(folds = NULL, nfolds = 10, stratified = FALSE, seed = NULL, y, x, k, 
dist.type = "euclidean", type = "C", method = "average", freq.option = 0, 
pred.ret = FALSE, mem.eff = FALSE) 

Arguments

Details

The concept behind k-NN is simple. Suppose we have a matrix with predictor variables and a vector with the response variable (numerical or categorical). When a new vector with observations (predictor variables) is available, its corresponding response value, numerical or categorical, is to be predicted. Instead of using a model, parametric or not, one can use this ad hoc algorithm.

The k smallest distances between the new predictor variables and the existing ones are calculated. In the case of regression, the average, median, or harmonic mean of the corresponding response values of these closest predictor values are calculated. In the case of classification, i.e. categorical response value, a voting rule is applied. The most frequent group (response value) is where the new observation is to be allocated.

This function does the cross-validation procedure to select the optimal k, the optimal number of nearest neighbours. The optimal in terms of some accuracy metric. For the classification it is the percentage of correct classification and for the regression the mean squared error.

Value

A list including:

Author(s)

Marios Dimitriadis

R implementation and documentation: Marios Dimitriadis <kmdimitriadis@gmail.com>

References

Friedman J., Hastie T. and Tibshirani R. (2017). The elements of statistical learning. New York: Springer.

Cover TM and Hart PE (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory. 13(1):21-27.

Tsagris Michail, Simon Preston and Andrew T.A. Wood (2016). Improved classification for compositional data using the \alpha-transformation. Journal of classification 33(2): 243-261.

See Also

knn, Dist, dista, dirknn.cv

Examples

x <- as.matrix(iris[, 1:4])
y <- iris[, 5]
mod <- knn.cv(folds = NULL, nfolds = 10, stratified = FALSE, seed = NULL, y = y, x = x, 
k = c(3, 4), dist.type = "euclidean", type = "C", method = "average", 
freq.option = 0, pred.ret = FALSE, mem.eff = FALSE) 

Cross-Validation for the k-NN algorithm using the arc cosinus distance

Description

Cross-Validation for the k-NN algorithm using the arc cosinus distance.

Usage

dirknn.cv(y, x, k = 5:10, type = "C", folds = NULL, nfolds = 10,
stratified = TRUE, seed = NULL, parallel = FALSE, pred.ret = FALSE)

Arguments

Details

The concept behind k-NN is simple. Suppose we have a matrix with predictor variables and a vector with the response variable (numerical or categorical). When a new vector with observations (predictor variables) is available, its corresponding response value, numerical or categorical, is to be predicted. Instead of using a model, parametric or not, one can use this ad hoc algorithm.

The k smallest distances between the new predictor variables and the existing ones are calculated. In the case of regression, the average, median, or harmonic mean of the corresponding response values of these closest predictor values are calculated. In the case of classification, i.e. categorical response value, a voting rule is applied. The most frequent group (response value) is where the new observation is to be allocated.

This function does the cross-validation procedure to select the optimal k, the optimal number of nearest neighbours. The optimal in terms of some accuracy metric. For the classification it is the percentage of correct classification and for the regression the mean squared error.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris mtsagris@uoc.gr.

References

Friedman J., Hastie T. and Tibshirani R. (2017). The elements of statistical learning. New York: Springer.

Cover TM and Hart PE (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory. 13(1):21-27.

See Also

dirknn, knn.cv, knn

Examples

x <- as.matrix(iris[, 1:4])
x <- x / sqrt( Rfast::rowsums(x^2) )
y <- iris[, 5]
mod <- dirknn.cv(y = y, x = x, k = c(3, 4) )

Density of the multivariate normal and t distributions

Description

Density of the multivariate normal and t distributions.

Usage

dmvnorm(x, mu, sigma, logged = FALSE) 
dmvt(x, mu, sigma, nu, logged = FALSE) 

Arguments

Details

The (log) density of the multivariate normal distribution is calculated for given mean vector and covariance matrix.

Value

A numerical vector with the density values calculated at each vector (row of the matrix x).

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Kanti V. Mardia, John T. Kent and John M. Bibby (1979). Multivariate analysis. Academic Press, London.

See Also

rmvnorm, rmvt, mvnorm.mle, iag.mle

Examples

x <- matrnorm(100, 20)
mu <- colmeans(x)
s <- cova(x)
#1 <- dmvnorm(x, mu, s) 
#2 <- dmvt(x, mu, s, 1)
x <- NULL 

Diagonal Matrix

Description

Fill the diagonal of a matrix or create a diagonal and initialize it with a specific value.

Usage

Diag.fill(x,v=0)
Diag.matrix(len,v=0)

Arguments

Value

Diag.fill returns a diagonal matrix where all the elements in the diagonal are equal to "v".

Diag.matrix returns a diagonal matrix where has dimension "len,len" and all the elements in the diagonal are equal to "v". It is fast for huge matrices with dimensions more than [row,col] = [500,500]

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

rowMins, colFalse, nth, rowrange, rowMedians, rowVars, colSort, rowSort, colTrue

Examples

x <- matrix(rbinom(100*100,1,0.5),100,100)

f <- Diag.fill(x,1)
f <- Diag.fill(x,1:100) ##equals to diag(x)<-1:100
f <- Diag.matrix(100,1) ##equals to diag(1,100,100)
f <- Diag.matrix(100,1:100) ##equals to diag(1:100,100,100)

f<-x<-NULL

Distance between vectors and a matrix - Sum of all pairwise distances in a distance matrix

Description

Distance between vectors and a matrix - Sum of all pairwise distances in a distance matrix..

Usage

dista(xnew, x, type = "euclidean", k = 0, index = FALSE, 
 trans = TRUE, square = FALSE, p = 0, result = "matrix", parallel = FALSE)
total.dista(xnew, x, type = "euclidean", k = 0,
 square = FALSE, p = 0, parallel = FALSE)  

Arguments

Details

The target of this function is to calculate the distances between xnew and x without having to calculate the whole distance matrix of xnew and x. The latter does extra calculations, which can be avoided.

• euclidean : \sum \sqrt( \sum | P_i - Q_i |^2)

• manhattan : \sum \sum | P_i - Q_i |

• minimum : \sum \min | P_i - Q_i |

• maximum : \sum \max | P_i - Q_i |

• minkowski : \sum ( \sum | P_i - Q_i |^p)^\frac{1}{p}

• bhattacharyya : \sum - ln \sum \sqrt(P_i * Q_i)

• hellinger : \sum 2 * \sqrt( 1 - \sum \sqrt(P_i * Q_i))

• kullback_leibler : \sum \sum P_i * log(\frac{P_i}{Q_i})

• jensen_shannon : \sum 0.5 * ( \sum P_i * log(2 * \frac{P_i}{Q_i} + Q_i) + \sum Q_i * log(2 * \frac{Q_i}{P_i} + Q_i))

• canberra : \sum \sum \frac{| P_i - Q_i |}{P_i + Q_i}

• chi_square X^2 : \sum \sum (\frac{(P_i - Q_i )^2}{P_i + Q_i})

• soergel : \sum \frac{\sum | P_i - Q_i |}{\sum \max(P_i , Q_i)}

• sorensen : \sum \frac{\sum | P_i - Q_i |}{\sum (P_i + Q_i)}

• cosine : \sum \frac{\sum (P_i * Q_i)}{\sqrt(\sum P_i^2) * \sqrt(\sum Q_i^2)}

• wave_hedges : \sum \frac{\sum | P_i - Q_i |}{\max(P_i , Q_i)}

• motyka : \sum \sum \frac{\min(P_i , Q_i)}{(P_i + Q_i)}

• harmonic_mean : \sum 2 * \frac{\sum P_i * Q_i}{P_i + Q_i}

• jeffries_matusita : \sum \sqrt( 2 - 2 * \sum \sqrt(P_i * Q_i))

• gower : \sum \frac{1}{d} * \sum | P_i - Q_i |

• kulczynski : \sum \frac{\sum | P_i - Q_i |}{\sum \min(P_i , Q_i)}

• itakura_saito : \sum \frac{P_i}{Q_i} - log(\frac{P_i}{Q_i}) - 1

Value

A matrix with the distances of each xnew from each vector of x. The number of rows of the xnew and and the number of columns of xnew are the dimensions of this matrix.

Author(s)

Michail Tsagris.

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

mahala, Dist, total.dist, total.dista

Examples

xnew <- as.matrix( iris[1:10, 1:4] )
x <- as.matrix( iris[-c(1:10), 1:4] )
a <- dista(xnew, x)
b <- as.matrix( dist( rbind(xnew, x) ) )
b <- b[ 1:10, -c(1:10) ]
sum( abs(a - b) )

## see the time
x <- matrix( rnorm(1000 * 4), ncol = 4 )
dista(xnew, x)
as.matrix( dist( rbind(xnew, x) ) )

x<-b<-a<-xnew<-NULL

Distance matrix - Sum of all pairwise distances in a distance matrix

Description

Distance matrix - Sum of all pairwise distances in a distance matrix.

Usage

Dist(x,method = "euclidean", square = FALSE,p=0, 
    result = "matrix" ,vector = FALSE, parallel = FALSE)
total.dist(x, method = "euclidean", square = FALSE, p = 0)
vecdist(x)

Arguments

Details

The distance matrix is compute with an extra argument for the Euclidean distances. The "kullback_leibler" refers to the symmetric Kullback-Leibler divergence.

• euclidean : \sum \sqrt( \sum | P_i - Q_i |^2)

• manhattan : \sum | P_i - Q_i |

• minimum : \sum \min | P_i - Q_i |

• maximum : \sum \max | P_i - Q_i |

• minkowski : ( \sum | P_i - Q_i |^p)^{\frac{1}{p}}

• bhattacharyya : - ln (\sum \sqrt(P_i * Q_i))

• hellinger : 2 * \sqrt( 1 - \sum \sqrt(P_i * Q_i))

• kullback_leibler : \sum P_i * log(\frac{P_i}{Q_i})

• jensen_shannon : 0.5 * ( \sum P_i * log(2 * \frac{P_i}{Q_i + Q_i}) + \sum Q_i * log(2 * \frac{Q_i}{P_i + Q_i}))

• canberra : \sum \frac{| P_i - Q_i |}{P_i + Q_i}

• chi_square X^2 : \sum (\frac{(P_i - Q_i )^2}{P_i + Q_i})

• soergel : \frac{\sum | P_i - Q_i |}{\sum \max(P_i , Q_i)}

• sorensen : \frac{\sum | P_i - Q_i |}{\sum (P_i + Q_i)}

• cosine : \sum \frac{\sum (P_i * Q_i)}{\sqrt(\sum P_i^2) * \sqrt(\sum Q_i^2)}

• wave_hedges : \sum \frac{\sum | P_i - Q_i |}{\max(P_i , Q_i)}

• motyka : \sum \frac{\min(P_i, Q_i)}{(P_i + Q_i)}

• harmonic_mean : 2 * \frac{\sum P_i * Q_i}{P_i + Q_i}

• jeffries_matusita : \sum \sqrt( 2 - 2 * \sum \sqrt(P_i * Q_i))

• gower : \sum \frac{1}{d} * \sum | P_i - Q_i |

• kulczynski : \sum \frac{\sum | P_i - Q_i |}{\sum \min(P_i , Q_i)}

• itakura_saito : \sum \frac{P_i}{Q_i} - log(\frac{P_i}{Q_i}) - 1

• haversine : 2 * R * \arcsin(\sqrt(\sin((lat_2 - lat_1)/2)^2 + \cos(lat_1) * \cos(lat_2) * \sin((lon_2 - lon_1)/2)^2))

Value

A square matrix with the pairwise distances.

Author(s)

Manos Papadakis.

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

References

Mardia K. V., Kent J. T. and Bibby J. M. (1979). Multivariate Analysis. Academic Press.

See Also

dista, colMedians

Examples

x <- matrix(rnorm(50 * 10), ncol = 10)
a1 <- Dist(x)
a2 <- as.matrix( dist(x) )

x<-a1<-a2<-NULL

Distance variance, covariance and correlation

Description

Distance variance, covariance and correlation.

Usage

dvar(x, bc = FALSE)
dcov(x, y, bc = FALSE)
dcor(x, y, bc = FALSE)
bcdcor(x, y)

Arguments

Details

The distance variance of a matrix/vector, the distance covariance or distance correlation of two matrices is calculated. For the distance variance of a vector we use the fast method of Huo and Szekely (2016).

Value

The distance covariance or the distance variance.

For the bias corrected distance correlation its value only.

For the distance correlation a list including:

Author(s)

Michail Tsagris and Manos Papadakis.

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Szekely G.J., Rizzo M.L. and Bakirov N.K.(2007). Measuring and Testing Independence by Correlation of Distances. Annals of Statistics, 35(6):2769–2794.

Szekely G. J. and Rizzo M. L. (2023). The Energy of Data and Distance Correlation. Chapman and Hall/CRC.

Tsagris M. and Papadakis M. (2025). Fast and light-weight energy statistics using the R package Rfast. https://arxiv.org/abs/2501.02849

See Also

pdcor, dcor.ttest, edist

Examples

x <- as.matrix(iris[1:50, 1:4])
y <- as.matrix(iris[51:100, 1:4])
res <- dvar(x[, 1])
#dcor(x, y)

Eigenvalues in high dimensional principal component analysis

Description

Eigenvalues in high dimensional (n<<p) principal component analysis.

Usage

hd.eigen(x, center = TRUE, scale = FALSE, k = NULL, vectors = FALSE, large = FALSE)

Arguments

Details

When n<<p, at most the first n eigenvalues are non zero. Hence, there is no need to calculate the other p-n zero eigenvalues. When center is TRUE, the eigenvalues of the covariance matrix are calculated. When both the center and scale is TRUE the eigenvalues of the correlation matrix are calculated. One or more eigenvectors (towards the end) will be 0. In general the signs might be the opposite than R's, but this makes no difference. We use the Crossprod instead of the relevant built-in function. The higher the dimensions of the matrix are the faster this function becomes.

Value

A list including:

Author(s)

Michail Tsagris.

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>.

See Also

rmdp

Examples

x <- matrnorm(20, 50)
#a <- hd.eigen(x, FALSE, FALSE)
#b <- prcomp(x, center = FALSE, scale = FALSE)
#a
#b$sdev^2
#x <- NULL

Empirical and exponential empirical likelihood tests for one sample

Description

Empirical and exponential empirical likelihood tests for one sample.

Usage

eel.test1(x, mu, tol = 1e-09, logged = FALSE)
el.test1(x, mu, tol = 1e-07, logged = FALSE) 

Arguments

Details

Exponential empirical likelihood is a non parametric method. In this case we use it as the non parametric alternative to the t-test. Newton-Raphson is used to maximise the log-likelihood ratio test statistic. In the case of no solution, NULL is returned. Despite the function having beeen written in R, it is pretty fast. As for the empirical likelihood ratio test, there is a condition for the range of possible values of mu. If mu is outside this range it is rejected immediately.

Value

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Owen A. B. (2001). Empirical likelihood. Chapman and Hall/CRC Press.

See Also

ftest, ttest1

Examples

x <- rnorm(500)
a1 <- eel.test1(x, 0)
a2 <- el.test1(x, 0)

Empirical and exponential empirical likelihood tests for two samples

Description

Empirical and exponential empirical likelihood tests for two samples.

Usage

eel.test2(x, y, tol = 1e-09, logged = FALSE)
el.test2(x, y, tol = 1e-07, logged = FALSE)

Arguments

Details

Empirical and exponential empirical likelihood are two non parametric hypothesis testing methods. We can use them as non parametric alternatives to the t-test. Newton-Raphson is used to maximise the log-likelihood ratio test statistic. In the case of no solution, NULL is returned.

Value

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Owen A. B. (2001). Empirical likelihood. Chapman and Hall/CRC Press.

See Also

ftests, ttests, ttest

Examples

x <- rnorm(200)
y <- rnorm(300)
eel.test2(x, y)
el.test2(x, y)

Energy distance between matrices

Description

Energy distance between matrices.

Usage

edist(x, y = NULL)

Arguments

Details

This calculates the energy distance between two matrices. It will work even for tens of thousands of rows, it will just take some time. See the references for more information. If you have many matrices and want to calculate the distance matrix, then put them in a list and use the function.

Value

If "x" is matrix, a numerical value, the energy distance. If "x" is list, a matrix with all pairwsie distances of the matrices.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

References

Szekely G. J. and Rizzo M. L. (2004) Testing for Equal Distributions in High Dimension, InterStat, November (5).

Szekely G. J. (2000) Technical Report 03-05, E-statistics: Energy of Statistical Samples, Department of Mathematics and Statistics, Bowling Green State University.

Sejdinovic D., Sriperumbudur B., Gretton A. and Fukumizu, K. (2013). Equivalence of distance-based and RKHS-based statistics in hypothesis testing. The Annals of Statistics, 41(5): 2263–2291.

Szekely G. J. and Rizzo M. L. (2023). The Energy of Data and Distance Correlation. Chapman and Hall/CRC.

Tsagris M. and Papadakis M. (2025). Fast and light-weight energy statistics using the R package Rfast. https://arxiv.org/abs/2501.02849

See Also

dcov, Dist, dista

Examples

x <- as.matrix( iris[1:50, 1:4] )
y <- as.matrix( iris[51:100, 1:4] )
res<-edist(x, y)
z <- as.matrix(iris[101:150, 1:4])
a <- list()
a[[ 1 ]] <- x
a[[ 2 ]] <- y
a[[ 3 ]] <- z
res<-edist(a)

x<-y<-z<-a<-NULL

Estimation of an AR(1) model

Description

Estimation of an AR(1) model.

Usage

ar1(y, method = "cmle") 
colar1(y, method = "cmle")

Arguments

Details

Instead of the classical MLE for the AR(1) model which requires numerical optimsation (Newton-Raphson for example) we estimate the parameters of the AR(1) model using conditional maximum likelihood. This procedure is described in Chapter 17 in Lee (2006). In some, it assumes that the first observation is deterministic and hence conditioning on that observation, there is a closed form solution for the parameters. The second alternative is to use the method of moments and hence the Yule-Walker equations.

Value

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

http://econ.nsysu.edu.tw/ezfiles/124/1124/img/Chapter17_MaximumLikelihoodEstimation.pdf

See Also

rm.lines, varcomps.mle, rm.anovas

Examples

y <- as.vector(lh)
ar1(y)
ar(y, FALSE, 1, "ols")

ar1(y, method = "yw")
ar(y, FALSE, 1, "yw")

a1 <- colar1(cbind(y, y) )
b1 <- colar1(cbind(y, y), method = "yw")

Estimation of the Box-Cox transformation

Description

Estimation of the Box-Cox transformation.

Usage

bc(x, low = -1, up = 1)

Arguments

Details

The functions estimates the best \lambda in the Box-Cox power transformation.

Value

The optimal value of \lambda.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Box George E. P. and Cox D. R. (1964). An analysis of transformations. Journal of the Royal Statistical Society, Series B, 26 (2):211-252.

See Also

correls, auc

Examples

x <- exp(rnorm(1000))
res<-bc(x)

Exact t-test for 2 independent samples

Description

Exact t-test for 2 independent samples.

Usage

exact.ttest2(x, y)

Arguments

Details

This function performs an exact t-test. With few observations, permutation or bootstrap calculation of the p-value is advisable. However, with even fewer observations, one can perform all possible permutations and calculate the exact p-value. This is what this function does. BUT, pay attention, as this works with few samples. If for example each sample contains 15 numbers, you will need a lot of memory (more than 17 GB) for this function to work. the reason is that we create the matrix with all possible permutations first and then perform the two-sample t-test.

Value

A vector with the number of permutations, test statistic and the permutation based p-value.

Author(s)

Michail Tsagris and Manos Papadakis

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>

References

B.L. Welch (1951). On the comparison of several mean values: an alternative approach. Biometrika, 38(3/4), 330-336.

See Also

boot.ttest2, ttest2, ftest

Examples

x <- rnorm(7)
y <- rnorm(7)
res<-exact.ttest2(x, y)

Exponential empirical likelihood for a one sample mean vector hypothesis testing

Description

Exponential empirical likelihood for a one sample mean vector hypothesis testing.

Usage

mv.eeltest1(x, mu, tol = 1e-06)

Arguments

Details

Multivariate hypothesis test for a one sample mean vector. This is a non parametric test and it works for univariate and multivariate data. The p-value is currently computed only asymptotically (no bootstrap calibration at the moment).

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>.

References

Jing Bing-Yi and Andrew TA Wood (1996). Exponential empirical likelihood is not Bartlett correctable. Annals of Statistics 24(1): 365-369.

Owen A. B. (2001). Empirical likelihood. Chapman and Hall/CRC Press.

See Also

james, mv.eeltest2

Examples

x <- Rfast::rmvnorm(100, numeric(10), diag( rexp(10, 0.5) ) )
res<-mv.eeltest1(x, numeric(10) )

Exponential empirical likelihood hypothesis testing for two mean vectors

Description

Exponential empirical likelihood hypothesis testing for two mean vectors.

Usage

mv.eeltest2(y1, y2, tol = 1e-07, R = 0)

Arguments

Details

Exponential empirical likelihood is a non parametric hypothesis testing procedure for one sample. The generalisation to two (or more samples) is via searching for the mean vector that minimises the sum of the two test statistics.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>.

References

Jing Bing-Yi and Andrew TA Wood (1996). Exponential empirical likelihood is not Bartlett correctable. Annals of Statistics 24(1): 365-369.

G.S. James (1954). Tests of Linear Hypothese in Univariate and Multivariate Analysis when the Ratios of the Population Variances are Unknown. Biometrika, 41(1/2): 19-43.

Krishnamoorthy K. and Yanping Xia (2006). On Selecting Tests for Equality of Two Normal Mean Vectors. Multivariate Behavioral Research 41(4): 533-548.

Owen A. B. (2001). Empirical likelihood. Chapman and Hall/CRC Press.

Amaral G.J.A., Dryden I.L. and Wood A.T.A. (2007). Pivotal bootstrap methods for k-sample problems in directional statistics and shape analysis. Journal of the American Statistical Association 102(478): 695-707.

Preston S.P. and Wood A.T.A. (2010). Two-Sample Bootstrap Hypothesis Tests for Three-Dimensional Labelled Landmark Data. Scandinavian Journal of Statistics 37(4): 568-587.

Tsagris M., Preston S. and Wood A.T.A. (2017). Nonparametric hypothesis testing for equality of means on the simplex. Journal of Statistical Computation and Simulation, 87(2): 406-422.

See Also

james, mv.eeltest1

Examples

res<-mv.eeltest2( as.matrix(iris[1:25, 1:4]), as.matrix(iris[26:50, 1:4]), R = 0 )
res<-mv.eeltest2( as.matrix(iris[1:25, 1:4]), as.matrix(iris[26:50, 1:4]), R = 1 )

FBED variable selection method using the correlation

Description

FBED variable selection method using the correlation.

Usage

cor.fbed(y, x, ystand = TRUE, xstand = TRUE, alpha = 0.05, K = 0)

Arguments

Details

FBED stands for Forward Backward with Earcly Dropping. It is a variation of the classical forward selection, where at each step, only the statistically significant variables carry on. The rest are dropped. The process stops when no other variables can be selected. If K = 1, the process is repeated testing sequentially again all those that have not been selected. If K > 1, then this is repeated.

In the end, the backward selection is performed to remove any falsely included variables. This backward phase has not been implemented yet.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Borboudakis G. and Tsamardinos I. (2019). Forward-backward selection with early dropping. Journal of Machine Learning Research, 20(8): 1-39.

See Also

cor.fsreg, ompr, correls, fs.reg

Examples

x <- matrnorm(100, 100)
y <- rnorm(100)
a <- cor.fbed(y, x)
a
x <- NULL

Fast and general represantation of a factor variable

Description

Fast and general represantation of a factor variable.

Usage

ufactor(x)
## S3 method for class 'ufactor'
x[i]
## S3 method for class 'ufactor'
print(x,...)

Arguments

Details

This is a general implementation of factor structure. For access the fields of a "ufactor" use the "$" operator.

Value

An object of class "ufactor". This object holds 2 fields:

levels: the levels of the variable in his initial type values: the values of the variable in his initial type

Author(s)

Manos Papadakis

R implementation and documentation: and Manos Papadakis <papadakm95@gmail.com>.

See Also

colVars, factor

Examples

x <- rnorm(10)
R.factor<- as.factor(x)
Rfast.factor <- ufactor(x)

identical(levels(R.factor),Rfast.factor$levels) # TRUE
identical(as.numeric(R.factor),Rfast.factor$values) # TRUE
x<-R.factor<-Rfast.factor<-NULL

Find element

Description

Search a value in an unordered vector.

Usage

is_element(x, key)

Arguments

Details

Find if the key exists in the vector and return returns TRUE/FALSE if the value is been found. If the vector is unordered it is fast but if the vector is ordered then use binary_search. The functions is written in C++ in order to be as fast as possible.

Value

TRUE/FALSE if the value is been found.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

binary_search (buit-in R function)

Examples

x <- rnorm(500)
key <- x[50]
b <- is_element(x, key) 

Find the given value in a hash table

Description

Find the given value in a hash table or list.

Usage

hash.find(x,key)

Arguments

Details

This function search the given key.

Value

If the given key exists return its value else returns 0.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>

See Also

hash.list

Examples

x <- hash.list(letters,c(1:26))
value <- hash.find(x,"a")
x[["a"]]==value

Fitted probabilities of the Terry-Bradley model

Description

Fitted probabilities of the Terry-Bradley model.

Usage

btmprobs(x, tol = 1e-09)

Arguments

Details

It fits a Bradley-Terry model to the given matrix and returns the fitted probabilities only.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Bradley R.A. and Terry M.E. (1952). Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons. Biometrika, 39(3/4):324-345.

Huang Tzu-Kuo, Ruby C. Weng and Chih-Jen Lin (2006). Generalized Bradley-Terry models and multi-class probability estimates. Journal of Machine Learning Research, 7:85-115.

Agresti A. (2002). Categorical Data Analysis (2nd ed). New York: Wiley.

See Also

g2tests, poisson.anova, anova, poisson_only, poisson.mle

Examples

x <- matrix( rpois(10 * 10, 10), ncol = 10) ## not the best example though
res<-btmprobs(x)

Fitting a Dirichlet distribution via Newton-Rapshon

Description

Fitting a Dirichlet distribution via Newton-Rapshon.

Usage

diri.nr2(x, type = 1, tol = 1e-07)

Arguments

Details

Maximum likelihood estimation of the parameters of a Dirichlet distribution is performed via Newton-Raphson. Initial values suggested by Minka (2012) are used.

Value

A list including:

Author(s)

Michail Tsagris and Manos Papadakis

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>

References

Minka Thomas (2012). Estimating a Dirichlet distribution. Technical report.

Ng Kai Wang, Guo-Liang Tian, and Man-Lai Tang (2011). Dirichlet and related distributions: Theory, methods and applications. John Wiley & Sons.

See Also

beta.mle

Examples

x <- matrix( rgamma(100 * 4, c(5, 6, 7, 8), 1), ncol = 4)
x <- x / rowsums(x)
res<-diri.nr2(x) 

Floyd-Warshall algorithm for shortest paths in a directed graph

Description

Floyd-Warshall algorithm for shortest paths in a directed graph.

Usage

floyd(x) 

Arguments

Details

The Floyd-Warshall algorithm is designed to find the shortest path (if it exists) between two nodes in a graph.

Value

A matrix, say z, with 0 and positive numbers. The elements denote the length of the shortest path between each pair of points. If z[i, j] is zero it means that there is no cost from i to j. If z[i, j] has a positive value it means that the length of going from i to j is equal to that value.

Author(s)

John Burkardt (C++ code)

Ported into R and documentation: Manos Papadakis <papadakm95@gmail.com>.

References

Floyd, Robert W. (1962). Algorithm 97: Shortest Path. Communications of the ACM. 5(6): 345.

Warshall, Stephen (1962). A theorem on Boolean matrices. Journal of the ACM. 9 (1): 11-12.

https://en.wikipedia.org/wiki/Floyd

See Also

colSort, rowSort

Examples

x <- matrix(NA, 10, 10)
x[sample(1:100, 10)] <- rpois(10, 3)
res<-floyd(x)

Variable selection in generalised linear regression models with forward selection

Description

Variable selection in generalised linear regression models with forward selection

Usage

fs.reg(y, ds, sig = 0.05, tol = 2, type = "logistic") 

Arguments

Details

The classical forward regression is implemented. The difference is that we have an extra step of check. Even if a variable is significant, the BIC of the model (with that variable) is calculated. If the decrease from the previous BIC (of the model without this variable) is less thatn a prespecified by the user value (default is 2) the variable wil enter. This way, we guard somehow against over-fitting.

Value

A matrix with for columns, the selected variables, the logarithm of their p-value, their test statistic and the BIC of the model with these variables included. If no variable is selected, the matrix is empty.

Author(s)

Marios Dimitriadis

Documentation: Marios Dimitriadis <kmdimitriadis@gmail.com>.

See Also

cor.fsreg, logistic_only, poisson_only, glm_logistic, glm_poisson

Examples

set.seed(123)

#simulate a dataset with continuous data
x <- matrnorm(100, 50)
y <- rpois(100, 10)
a <- fs.reg(y, x, sig = 0.05, tol = 2, type = "poisson") 
x <- NULL

G-square test of conditional indepdence

Description

G-square test of conditional indepdence with and without permutations.

Usage

g2Test(data, x, y, cs, dc)
g2Test_perm(data, x, y, cs, dc, nperm)
chi2Test(data, x, y, cs, dc)

Arguments

Details

The functions calculates the test statistic of the G^2 test of conditional independence between x and y conditional on a set of variable(s) cs.

Value

A list including:

Author(s)

Giorgos Borboudakis. The permutation version used a C++ code by John Burkardt.

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

References

Tsamardinos, I., & Borboudakis, G. (2010). Permutation testing improves Bayesian network learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 322-337). Springer Berlin Heidelberg

See Also

g2Test_univariate, g2Test_univariate_perm, correls, univglms

Examples

nvalues <- 3
nvars <- 10
nsamples <- 5000
data <- matrix( sample( 0:(nvalues - 1), nvars * nsamples, replace = TRUE ), nsamples, nvars )
dc <- rep(nvalues, nvars)

res<-g2Test( data, 1, 2, 3, c(3, 3, 3) )
res<-g2Test_perm( data, 1, 2, 3, c(3, 3, 3), 1000 )

dc<-data<-NULL

Gamma regression with a log-link

Description

Gamma regression with a log-link.

Usage

gammareg(y, x, tol = 1e-07, maxiters = 100)
gammacon(y, tol = 1e-08, maxiters =50)

Arguments

Details

The gamma.reg fits a Gamma regression with a log-link. The gamma.con fits a Gamma regression with a log link with the intercept only ( glm(y ~ 1, Gamma(log) ) ).

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris mtsagris@uoc.gr.

References

McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

See Also

gammaregs, normlog.reg, invgauss.reg

Examples

y <- abs( rnorm(100) )
x <- matrix( rnorm(100 * 2), ncol = 2)
mod <- glm(y ~ x, family = Gamma(log) )
res<-summary(mod)

res<-gammareg(y, x)

mod <- glm(y ~ 1, family = Gamma(log) )
res<-summary(mod)
res<-gammacon(y)

Gaussian regression with a log-link

Description

Gaussian regression with a log-link.

Usage

normlog.reg(y, x, tol = 1e-07, maxiters = 100)

Arguments

Details

A Gaussian regression with a log-link is fitted.

Value

A list including:

Author(s)

Stefanos Fafalios

R implementation and documentation: Stefanos Fafalios <stefanosfafalios@gmail.com>

See Also

normlog.regs, score.glms, prop.regs, allbetas

Examples

y <- abs( rnorm(100) )
x <- matrix( rnorm(100 * 2), ncol = 2)
a <- normlog.reg(y, x)
b <- glm(y ~ x, family = gaussian(log) )
summary(b)
a

Generates random values from a normal and puts them in a matrix

Description

Generates random values from a normal and puts them in a matrix.

Usage

matrnorm(n, p, seed = NULL)

Arguments

Details

How many times did you have to simulated data from a (standard) normal distribution in order to test something? For example, in order to see the speed of logistic_only, one needs to generate a matrix with predictor variables. The same is true for other similar functions. In sftests, one would like to examine the typer I error of this test under the null hypothesis.

By using the Ziggurat method of generating standard normal variates, this function is really fast when you want to generate big matrices.

Value

An n x p matrix with data simulated from a standard normal distribution.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

See Also

rvmf, Rnorm, rmvnorm, rvonmises

Examples

x <- matrnorm(100, 100)

Get specific columns/rows fo a matrix

Description

Get specific columns/rows of a matrix.

Usage

columns(x,indices)
rows(x,indices)

Arguments

Value

A matrix with the specific columns/rows of argumment indices.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

rowMins, rowFalse, nth, colrange, colMedians, colVars, colSort, rowSort, rowTrue

Examples

x <- matrix(runif(100*100),100,100)
indices = sample(1:100,50)
all.equal(x[,indices],columns(x,indices))
all.equal(x[indices,],rows(x,indices))

x<-indices<-NULL

Hash - Pair function

Description

Hash - Pair function.

Usage

hash.list(key,x)

Arguments

Details

This function pairs each item of of key and value make a unique hash table.

Value

Returns the hash-list table.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>

See Also

hash.find

Examples

x <- hash.list(letters,c(1:26))
x[["a"]]==1

Hash object

Description

Hash object.

Usage

Hash(keys=NULL,values=NULL)
Hash.key.multi(x,...,sep = " ")
## S3 replacement method for class 'Hash'
x[...,sep = " "] <- value
## S3 method for class 'Hash'
x[...,sep = " "]
## S3 method for class 'Hash'
print(x,...)
## S3 method for class 'Hash'
length(x)

Arguments

Details

If you want to delete a key just insert the global variable "Rfast:::delete".

Hash: Create Hash object where every key has a value. Specify the type from the beggining (for speed). Use the argument "type" with one of the values "new.env, logical, character, integer, numeric". Hash.key.multi: search if key exists. If the keys are multiple, then use the argument "substr" to search inside each multiple for the specific key.

Value

A Hash object.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

hash.list, hash.find

Examples

x <- Hash(rnorm(10),sample(1:10))

x[1,2,13] <- 0.1234 # insert value using multi key. the same as x["1 2 13"] <- 0.1234
x[1,2,3] <- 15 # insert value using multi key. the same as x["1 2 3"] <- 15

Hash.key.multi(x,"1")
x # print Hash object using S3 generic
#x[1,2,3] <- Rfast:::delete # delete multi key. the same as x["1 2 3"] <- NULL
length(x)

Hash object to a list object

Description

Hash object to a list object.

Usage

hash2list(x, sorting = FALSE)

Arguments

Details

For every key, there is a key value. This function creates a list and puts every pair of keys and value in a component of a list.

Value

A list whose length is equal to the size of the hash table.

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

hash.list, hash.find

Examples

x=list("1 2 4 3"=2.56,"2.34 1.05"=2)
res<-hash2list(x)
res<-hash2list(x,TRUE)

High dimensional MCD based detection of outliers

Description

High dimensional MCD based detection of outliers.

Usage

rmdp(y, alpha = 0.05, itertime = 100, parallel = FALSE)

Arguments

Details

High dimensional outliers (n<<p) are detected using a properly constructed MCD. The variances of the variables are used and the determinant is simply their product.

Value

A list including: runtime = runtime, dis = dis, wei = wei

Author(s)

Initial R code: Changliang Zou <nk.chlzou@gmail.com> R code modifications: Michail Tsagris <mtsagris@uoc.gr> C++ implementation: Manos Papadakis <papadakm95@gmail.com> Documentation: Michail Tsagris <mtsagris@uoc.gr> and Changliang Zhou <nk.chlzou@gmail.com>

References

Ro K., Zou C., Wang Z. and Yin G. (2015). Outlier detection for high-dimensional data. Biometrika, 102(3): 589–599.

Tsagris M., Papadakis M., Alenazi A. and Alzeley O. (2024). Computationally Efficient Outlier Detection for High-Dimensional Data Using the MDP Algorithm. Computation, 12(9): 185.

See Also

colmeans, colVars, colMedians

Examples

x <- matrix(rnorm(30 * 100), ncol = 100)
#a <- rmdp(x, itertime = 5)

x<-a<-NULL

Hypothesis test for the distance correlation

Description

Hypothesis test for the distance correlation.

Usage

dcor.ttest(x, y, logged = FALSE)

Arguments

Details

The bias corrected distance correlation is used. The hypothesis test is whether the two matrices are independent or not. Note, that this test is size correct as both the sample size and the dimensionality goes to infinity. It will not have the correct type I error for univariate data or for matrices with just a couple of variables.

Value

A vector with 4 elements, the bias corrected distance correlation, the degrees of freedom, the test statistic and its associated p-value.

Author(s)

Manos Papadakis

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

G.J. Szekely, M.L. Rizzo and N. K. Bakirov (2007). Measuring and Testing Independence by Correlation of Distances. Annals of Statistics, 35(6): 2769–2794.

Szekely G. J. and Rizzo M. L. (2023). The Energy of Data and Distance Correlation. Chapman and Hall/CRC.

See Also

dcov, edist

Examples

x <- as.matrix(iris[1:50, 1:4])
y <- as.matrix(iris[51:100, 1:4])
#res<-dcor.ttest(x, y)

Hypothesis test for two means of percentages

Description

Hypothesis test for two means of percentages.

Usage

percent.ttest(x, y, logged = FALSE)

Arguments

Details

This is the prop.reg but with a single categorical predictor which has two levels only. It is like a t-test for the means of two samples haivng percentages.

Value

A vector with three elements, the phi parameter, the test statistic and its associated p-value.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Papke L. E. & Wooldridge J. (1996). Econometric methods for fractional response variables with an application to 401(K) plan participation rates. Journal of Applied Econometrics, 11(6): 619-632.

McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

See Also

link{percent.ttests}, prop.reg, ttest2, ftest

Examples

x <- rbeta(100, 3, 1)
y <- rbeta(100, 7.5, 2.5)
res<-percent.ttest(x, y)

Hypothesis test for von Mises-Fisher distribution over Kent distribution

Description

The null hypothesis is whether a von Mises-Fisher distribution fits the data well, and the altenrative is that the Kent distribution is more suitable.

Usage

fish.kent(x, logged = FALSE)

Arguments

Details

Essentially it is a test of rotational symmetry, whether Kent's ovalness parameter (beta) is equal to zero. This works for spherical data only.

Value

A vector with two elements, the value of the test statistic and its associated p-value.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Rivest, L. P. (1986). Modified Kent's statistics for testing goodness of fit for the Fisher distribution in small concentrated samples. Statistics & probability letters, 4(1): 1-4.

See Also

vmf.mle, iag.mle

Examples

#x <- rvmf(100, rnorm(3), 15)
#res<-fish.kent(x)
x <- NULL

Hypothesis testing between two skewness or kurtosis coefficients

Description

Hypothesis testing between two skewness or kurtosis coefficients.

Usage

skew.test2(x, y)

kurt.test2(x, y) 

Arguments

Details

The skewness of kurtosis coefficients between two samples are being compared.

Value

A vector with the test statistic and its associated p-value.

Author(s)

Klio Lakiotaki

R implementation and documentation: Klio Lakiotaki <kliolak@gmail.com>.

References

https://en.wikipedia.org/wiki/Skewness

https://en.wikipedia.org/wiki/Kurtosis

See Also

skew, colskewness, colmeans, colVars, colMedians

Examples

x <- rgamma(150,1, 4)
y <- rgamma(100, 1, 4)
res<-skew.test2(x, y)
res<-kurt.test2(x, y)

Index of the columns of a data.frame which are a specific type

Description

Index of the columns of a data.frame which are a specific type.

Usage

which.is(x,method="factor")

Arguments

Details

The function is written in C++ and this is why it is very fast.

Value

A vector with the column indices which are factor variables. If there are no factor variables it will return an empty vector.

Author(s)

Manos Papadakis <papadakm95@gmail.com>

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

nth, Match

Examples

res<-which.is(iris)

Inverese Gaussian regression with a log-link

Description

Inverse Gaussian regression with a log-link.

Usage

invgauss.reg(y, x, tol = 1e-07, maxiters = 100)

Arguments

Details

An inverse Gaussian regression with a log-link is fitted.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Stefanos Fafalios <mtsagris@uoc.gr>

References

McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

Zakariya Yahya Algamal and Intisar Ibrahim Allyas (2017). Prediction of blood lead level in maternal and fetal using generalized linear model. International Journal of Advanced Statistics and Probability, 5(2): 65-69.

See Also

invgauss.regs, normlog.reg, score.glms

Examples

y <- abs( rnorm(100) )
x <- matrix( rnorm(100 * 2), ncol = 2)
a <- invgauss.reg(y, x)
a

Inverse of a symmetric positive definite matrix

Description

Inverse of a symmetric positive definite matrix.

Usage

spdinv(A)

Arguments

Details

After calculating the Cholesky decomposition of the matrix we use this upper triangular matrix to invert the original matrix.

Value

The inverse of the input matrix.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

http://econ.nsysu.edu.tw/ezfiles/124/1124/img/Chapter17_MaximumLikelihoodEstimation.pdf

See Also

cholesky, cova

Examples

s <- cova( as.matrix(iris[, 1:4]) )
res<-spdinv(s)
res<-solve(s)

Iterator

Description

A way to traverse a list, data.frame, matrix or vector.

Usage

iterator(x,method="ceil",type="vector",by=1)
## S3 method for class 'iterator'
print(x,...)
## S3 replacement method for class 'iterator'
Elem(x) <- value
Elem(x)
Elem(x) <- value
## S3 method for class 'iterator'
Elem(x)
## S3 method for class 'iterator'
x == y
## S3 method for class 'iterator'
x != y

Arguments

Details

iterator: is an object that helps a programmer to traverse the given object.

print.iterator: print an object of class iterator.

"Elem<-": access to element and change the value.

Elem: access to element.

Value

An object of class "iterator". This object holds 4 fields:

copy: deep copy of iterator. end: get iterator tha have access to points to the last element. equals: equality of iterators nextElem: move iterator to point to the next element using argument "by". prevElem: move iterator to point to the previous element using argument "by".

Author(s)

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

See Also

colShuffle, colVars, colmeans

Examples

y<-rnorm(100)
x<-iterator(y,method="ceil",type="vector",by=1)

s<-0
while(x != x$end()){
	s <- s + Elem(x)
	x$nextElem()
}

all.equal(s,sum(y))

James multivariate version of the t-test

Description

James test for testing the equality of two population mean vectors without assuming equality of the covariance matrices.

Usage

james(y1, y2, a = 0.05, R = 1)

Arguments

Details

Multivariate analysis of variance without assuming equality of the covariance matrices. The p-value can be calculated either asymptotically or via bootstrap. The James test (1954) or a modification proposed by Krishnamoorthy and Yanping (2006) is implemented. The James test uses a corected chi-square distribution, whereas the modified version uses an F distribution.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>.

References

G.S. James (1954). Tests of Linear Hypothese in Univariate and Multivariate Analysis when the Ratios of the Population Variances are Unknown. Biometrika, 41(1/2): 19-43

Krishnamoorthy K. and Yanping Xia. On Selecting Tests for Equality of Two Normal Mean Vectors (2006). Multivariate Behavioral Research 41(4): 533-548

See Also

mv.eeltest2

Examples

james( as.matrix(iris[1:25, 1:4]), as.matrix(iris[26:50, 1:4]), R = 1 )
james( as.matrix(iris[1:25, 1:4]), as.matrix(iris[26:50, 1:4]), R = 2 )

Limited number of eigenvalues and eigenvectors of a symmetric matrix

Description

Limited number of eigenvalues and eigenvectors of a symmetric matrix.

Usage

eigen.sym(A, k, vectors = TRUE)

Arguments

Details

The function calls the same function from the Armadillo library in C++. It is quite faster than R's built in function "eigen" if the number of eigenvalues and eigenvectors (argument k) is small.

The k largest, in magnitude, eigenvalues are returned. Hence, if the matrix is not positive definite you may get negative eigenvalues as well. So, it is advised to use it with positive definite matrices.

Value

A list including:

Author(s)

Armadillo library in C++ and Stefanos Fafalios and Manos Papadakis.

R implementation and documentation: Stefanos Fafalios <stefanosfafalios@gmail.com> and Manos Papadakis <papadakm95@gmail.com>.

See Also

hd.eigen

Examples

x <- matrnorm(500, 100 )
s <- Rfast::cova(x)
res<-eigen.sym(s, 5)
x <- s <- NULL

Linear models for large scale data

Description

Linear models for large scale data.

Usage

lmfit(x, y, w = NULL)

Arguments

Details

We have simply exploitted R's powerful function and managed to do better than .lm.fit which is a really powerful function as well. This is a bare bones function as it returns only two things, the coefficients and the residuals. .lm.fit returns more and lm.fit even more and finally lm returns too much. The motivatrion came form this site https://m-clark.github.io/docs/fastr.html . We changed the function a bit.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Draper, N.R. and Smith H. (1988). Applied regression analysis. New York, Wiley, 3rd edition.

See Also

regression, allbetas, correls, mvbetas, cor.fsreg

Examples

n <- 200  ;  p <- 5
X <- matrnorm(n, p)
y <- rnorm(n)
a1 <- .lm.fit(X, y) 
a2 <- lmfit(X, y) 
x <- NULL

Logistic and Poisson regression models

Description

Logistic and Poisson regression models.

Usage

glm_logistic(x, y, full = FALSE,tol = 1e-09, maxiters = 100)
glm_poisson(x, y, full = FALSE,tol = 1e-09)

Arguments

Details

The function is written in C++ and this is why it is very fast.

Value

When full is FALSE a list including:

When full is TRUE a list including:

Author(s)

Manos Papadakis <papadakm95@gmail.com>

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

See Also

poisson_only, logistic_only, univglms, regression

Examples

x <- matrix(rnorm(100 * 3), ncol = 3)
y <- rbinom(100, 1, 0.6)   ## binary logistic regression
a1 <- glm_logistic(x, y, full = TRUE) 
a2 <- glm(y ~ x, binomial)

x <- matrix(rnorm(100 * 3), ncol = 3)
y <- rpois(100, 10)   ## binary logistic regression
b1 <- glm_poisson(x, y, full = TRUE) 
b2 <- glm(y ~ x, poisson)

x<-y<-a1<-a2<-b1<-b2<-NULL

Logistic or Poisson regression with a single categorical predictor

Description

Logistic or Poisson regression with a single categorical predictor.

Usage

logistic.cat1(y, x, logged = FALSE)
poisson.cat1(y, x, logged = FALSE) 

Arguments

Details

There is a closed form solution for the logistic regression in the case of a single predictor variable. See the references for more information.

Value

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Stan Lipovetsky (2015). Analytical closed-form solution for binary logit regression by categorical predictors. Journal of Applied Statistics, 42(1): 37–49.

See Also

poisson.anova, poisson.anovas, anova, logistic_only, poisson_only

Examples

y <- rbinom(20000, 1, 0.6)
x <- as.factor( rbinom(20000, 3, 0.5) )
a1 <- logistic.cat1(y, x)
#a2 <- glm(y ~ x, binomial) 

y <- rpois(20000, 10)
x <- as.factor( rbinom(20000, 3, 0.5) )
a1 <- poisson.cat1(y, x)
#a2 <- glm(y ~ x, poisson) 

x<-y<-a1<-a2<-NULL

MLE for multivariate discrete data

Description

MLE for multivariate discrete data.

Usage

multinom.mle(x)
dirimultinom.mle(x, tol = 1e-07) 
colpoisson.mle(x)
colgeom.mle(x, type = 1)

Arguments

Details

For the Poisson and geometric distributions we simply fit independent Poisson and geometric distributions respectively.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Johnson Norman L., Kotz Samuel and Balakrishnan (1997). Discrete Multivariate Distributions. Wiley

Minka Thomas (2012). Estimating a Dirichlet distribution. Technical report.

See Also

poisson.mle, zip.mle, ztp.mle, negbin.mle, poisson.nb

Examples

x <- t( rmultinom(1000, 20, c(0.4, 0.5, 0.1) ) )
res<-multinom.mle(x)
res<-colpoisson.mle(x)
x <- NULL

MLE of (hyper-)spherical distributions

Description

MLE of (hyper-)spherical distributions.

Usage

vmf.mle(x, tol = 1e-07)
multivmf.mle(x, ina, tol = 1e-07, ell = FALSE)
acg.mle(x, tol = 1e-07)
iag.mle(x, tol = 1e-07)

Arguments

Details

For the von Mises-Fisher, the normalised mean is the mean direction. For the concentration parameter, a Newton-Raphson is implemented. For the angular central Gaussian distribution there is a constraint on the estimated covariance matrix; its trace is equal to the number of variables. An iterative algorithm takes place and convergence is guaranteed. Newton-Raphson for the projected normal distribution, on the sphere, is implemented as well. Finally, the von Mises-Fisher distribution for groups of data is also implemented.

Value

For the von Mises-Fisher a list including:

For the multi von Mises-Fisher a list including:

For the angular central Gaussian a list including:

For the spherical projected normal a list including:

Author(s)

Michail Tsagris R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>

References

Mardia, K. V. and Jupp, P. E. (2000). Directional statistics. Chicester: John Wiley & Sons.

Sra, S. (2012). A short note on parameter approximation for von Mises-Fisher distributions: and a fast implementation of Is(x). Computational Statistics, 27(1): 177–190.

Tyler D. E. (1987). Statistical analysis for the angular central Gaussian distribution on the sphere. Biometrika 74(3): 579-589.

Paine P.J., Preston S.P., Tsagris M and Wood A.T.A. (2017). An Elliptically Symmetric Angular Gaussian Distribution. Statistics and Computing (To appear).

See Also

racg, vm.mle, rvmf

Examples

m <- c(0, 0, 0, 0)
s <- cov(iris[, 1:4])
#x <- racg(100, s)
#mod <- acg.mle(x)
#res<-cov2cor(mod$cova)  ## estimated covariance matrix turned into a correlation matrix
#res<-cov2cor(s)  ## true covariance matrix turned into a correlation matrix
#res<-vmf.mle(x)
#x <- rbind( rvmf(100,rnorm(4), 10), rvmf(100,rnorm(4), 20) )
#a <- multivmf.mle(x, rep(1:2, each = 100) )

MLE of continuous univariate distributions defined on the positive line

Description

MLE of continuous univariate distributions defined on the positive line.

Usage

gammamle(x, tol = 1e-09) 
chisq.mle(x, tol = 1e-09)
weibull.mle(x, tol = 1e-09, maxiters = 100)
lomax.mle(x, tol = 1e-09)
foldnorm.mle(x, tol = 1e-09)
betaprime.mle(x, tol = 1e-09)
logcauchy.mle(x, tol = 1e-09)
loglogistic.mle(x, tol = 1e-09)
halfnorm.mle(x)
invgauss.mle(x)
lognorm.mle(x)
pareto.mle(x)
expmle(x)
exp2.mle(x)
maxboltz.mle(x)
rayleigh.mle(x)
normlog.mle(x)
lindley.mle(x)

Arguments

Details

Instead of maximising the log-likelihood via a numerical optimiser we have used a Newton-Raphson algorithm which is faster. See wikipedia for the equations to be solved. For the t distribution we need the degrees of freedom and estimate the location and scatter parameters. If you want to to fit an inverse gamma distribution simply do "gamma.mle(1/x)". The log-likelihood and the parameters are for the inverse gamma.

The "normlog.mle" is simply the normal distribution where all values are positive. Note, this is not log-normal. It is the normal with a log link. Similarly to the inverse gaussian distribution where the mean is an exponentiated. This comes from the GLM theory.

Value

Usually a list with three elements, but this is not for all cases.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Kalimuthu Krishnamoorthy, Meesook Lee and Wang Xiao (2015). Likelihood ratio tests for comparing several gamma distributions. Environmetrics, 26(8):571-583.

N.L. Johnson, S. Kotz and N. Balakrishnan (1994). Continuous Univariate Distributions, Volume 1 (2nd Edition).

N.L. Johnson, S. Kotz a nd N. Balakrishnan (1970). Distributions in statistics: continuous univariate distributions, Volume 2.

Tsagris M., Beneki C. and Hassani H. (2014). On the folded normal distribution. Mathematics, 2(1):12-28.

Sharma V. K., Singh S. K., Singh U. and Agiwal V. (2015). The inverse Lindley distribution: a stress-strength reliability model with application to head and neck cancer data. Journal of Industrial and Production Engineering, 32(3): 162-173.

You can also check the relevant wikipedia pages for these distributions.

See Also

zip.mle, normal.mle, beta.mle

Examples

x <- rgamma(100, 3, 4)
for (i in 1:20) gammamle(x)
## for (i in 1:20) fitdistr(x,"gamma")
#a <- glm(x ~ 1, gaussian(log) )
res<-normlog.mle(x)

MLE of continuous univariate distributions defined on the real line

Description

MLE of continuous univariate distributions defined on the real line.

Usage

normal.mle(x) 
gumbel.mle(x, tol = 1e-09)
cauchy.mle(x, tol = 1e-09)
logistic.mle(x, tol = 1e-07)
ct.mle(x, tol = 1e-09)
tmle(x, v = 5, tol = 1e-08)
wigner.mle(x, tol = 1e-09)
laplace.mle(x)

Arguments

Details

Instead of maximising the log-likelihood via a numerical optimiser we have used a Newton-Raphson algorithm which is faster. See wikipedia for the equation to be solved. For the t distribution we need the degrees of freedom and estimate the location and scatter parameters.

The Cauchy is the t distribution with 1 degree of freedom. If you want to fit such a distribution used the cauchy.mle and not the t.mle with 1 degree of freedom as it's faster. The Laplace distribution is also called double exponential distribution.

The wigner.mle refers to the wigner semicircle distribution.

Value

Usually a list with three elements, but this is not for all cases.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Johnson, Norman L. Kemp, Adrianne W. Kotz, Samuel (2005). Univariate Discrete Distributions (third edition). Hoboken, NJ: Wiley-Interscience.

https://en.wikipedia.org/wiki/Wigner_semicircle_distribution

See Also

zip.mle, gammamle, vm.mle

Examples

x <- rt(1000,10)
a <- ct.mle(x)
res<-tmle(x, v = a$nu)
res<-cauchy.mle(x)
res<-normal.mle(x)
res<-logistic.mle(x)
res<-gumbel.mle(x)

MLE of count data

Description

MLE of count data.

Usage

zip.mle(x, tol = 1e-09)
ztp.mle(x, tol = 1e-09)
negbin.mle(x, type = 1, tol = 1e-09) 
binom.mle(x, N = NULL, tol = 1e-07)
borel.mle(x)
geom.mle(x, type = 1)
logseries.mle(x, tol = 1e-09)
poisson.mle(x)
betageom.mle(x, tol = 1e-07)
betabinom.mle(x, N, tol = 1e-07)

Arguments

Details

Instead of maximising the log-likelihood via a numerical optimiser we used a Newton-Raphson algorithm which is faster.

See wikipedia for the equation to be solved in the case of the zero inflated distribution. https://en.wikipedia.org/wiki/Zero-inflated_model. In order to avoid negative values we have used link functions, log for the lambda and logit for the \pi as suggested by Lambert (1992). As for the zero truncated Poisson see https://en.wikipedia.org/wiki/Zero-truncated_Poisson_distribution.

zip.mle is for the zero inflated Poisson, whereas ztp.mle is for the zero truncated Poisson distribution.

Value

The following list is not inclusive of all cases. Different functions have different names. In general a list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Lambert Diane (1992). Zero-Inflated Poisson Regression, with an Application to Defects in Manufacturing. Technometrics. 34 (1): 1-14

Johnson Norman L., Kotz Samuel and Kemp Adrienne W. (1992). Univariate Discrete Distributions (2nd ed.). Wiley

See Also

poisson_only, colrange

Examples

x <- rpois(100, 2)
res<-zip.mle(x)
res<-poisson.mle(x)
## small difference in the two log-likelihoods as expected.

x <- rpois(100, 10)
x[x == 0 ] <- 1
res<-ztp.mle(x)
res<-poisson.mle(x)
## significant difference in the two log-likelihoods. 

x <- rnbinom(100, 10, 0.6)
res<-poisson.mle(x)
res<-negbin.mle(x)

MLE of distributions defined in the (0, 1) interval

Description

MLE of distributions defined in the (0, 1) interval.

Usage

beta.mle(x, tol = 1e-09)
ibeta.mle(x, tol = 1e-09)
logitnorm.mle(x)
hsecant01.mle(x, tol = 1e-09)

Arguments

Details

Maximum likelihood estimation of the parameters of the beta distribution is performed via Newton-Raphson. The distributions and hence the functions does not accept zeros. "logitnorm.mle" fits the logistic normal, hence no nwewton-Raphson is required and the "hypersecant01.mle" uses the golden ratio search as is it faster than the Newton-Raphson (less calculations)

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>

See Also

diri.nr2,

Examples

x <- rbeta(100, 1, 4)
res<-beta.mle(x)
res<-hsecant01.mle(x)
res<-logitnorm.mle(x)

x[sample(1:100, 5)] <- 0
res<-ibeta.mle(x)

MLE of some circular distributions

Description

MLE of some circular distributions.

Usage

vm.mle(x, tol = 1e-09)
spml.mle(x, tol = 1e-09, maxiters = 100)
wrapcauchy.mle(x, tol = 1e-09)

Arguments

Details

The parameters of the von Mises, the bivariate angular Gaussian and wrapped Cauchy distributions are estimated. For the Wrapped Cauchy, the iterative procedure described by Kent and Tyler (1988) is used. As for the von Mises distribution, we use a Newton-Raphson to estimate the concentration parameter. The angular Gaussian is described, in the regression setting in Presnell et al. (1998).

Value

A list including:

Author(s)

Michail Tsagris and Stefanos Fafalios

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Stefanos Fafalios <stefanosfafalios@gmail.com>

References

Mardia K. V. and Jupp P. E. (2000). Directional statistics. Chicester: John Wiley & Sons.

Sra S. (2012). A short note on parameter approximation for von Mises-Fisher distributions: and a fast implementation of Is(x). Computational Statistics, 27(1): 177-190.

Presnell Brett, Morrison Scott P. and Littell Ramon C. (1998). Projected multivariate linear models for directional data. Journal of the American Statistical Association, 93(443): 1068-1077.

Kent J. and Tyler D. (1988). Maximum likelihood estimation for the wrapped Cauchy distribution. Journal of Applied Statistics, 15(2): 247–254.

See Also

vmf.mle, rvonmises, rvmf

Examples

y <- rcauchy(100, 3, 1)
x <- y 
res<-vm.mle(x)
res<-spml.mle(x)
res<-wrapcauchy.mle(x)
x <- NULL

MLE of the inverted Dirichlet distribution

Description

MLE of the inverted Dirichlet distribution.

Usage

invdir.mle(x, tol = 1e-07)

Arguments

Details

Maximum likelihood estimation of the parameters of the inverted is performed via Newton-Raphson. We took the initial values suggested by Bdiri T. and Bouguila N. (2012) and modified them a bit.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>

References

Bdiri T. and Bouguila N. (2012). Positive vectors clustering using inverted Dirichlet finite mixture models. Expert Systems with Applications, 39(2): 1869-1882.

See Also

diri.nr2, multinom.mle

Examples

x <- as.matrix(iris[, 1:4])
for(i in 1:10) invdir.mle(x)
res<-invdir.mle(x)

MLE of the multivariate (log-) normal distribution

Description

MLE of the multivariate (log-) normal distribution.

Usage

mvnorm.mle(x)
mvlnorm.mle(x)

Arguments

Details

The mean vector, covariance matrix and the value of the log-likelihood of the multivariate normal or log-normal distribution is calculated. For the log-normal distribution we also provide the expected value and the covariance matrix.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Kotz, S., Balakrishnan, N., & Johnson, N. L. (2004). Continuous multivariate distributions, Volume 1: Models and applications (Vol. 1). John wiley & sons.

http://isi.cbs.nl/iamamember/CD2/pdf/329.PDF

https://en.wikipedia.org/wiki/Log-normal_distribution#Multivariate_log-normal

See Also

multinom.mle, dmvnorm, gaussian.nb

Examples

x <- matrnorm(100, 4)
res<-mvnorm.mle(x)
x <- NULL

MLE of the multivariate t distribution

Description

MLE of the multivariate t distribution.

Usage

mvt.mle(x, v = 5, tol = 1e-07)

Arguments

Details

The location vector, scatter matrix and the value of the log-likelihood is calculated.

Value

A list including:

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>.

References

Nadarajah S. and Kotz S. (2008). Estimation methods for the multivariate t distribution. Acta Applicandae Mathematicae, 102(1):99-118.

See Also

mvnorm.mle, dmvnorm, gaussian.nb

Examples

x <- matrnorm(100, 4)
#res<-mvnorm.mle(x)
#res<-mvt.mle(x, v = 5)
#res<-mvt.mle(x, v = 100)

MLE of the ordinal model without covariates

Description

MLE of the ordinal model without covariates.

Usage

ordinal.mle(y, link = "logit")

Arguments

Details

Maximum likelihood of the ordinal model (proportional odds) is implemented. See for example the "polr" command in R or the examples.

Value

A list including:

Author(s)

Manos Papadakis

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

References

Agresti, A. (2002) Categorical Data. Second edition. Wiley.

See Also

beta.mle, diri.nr2

Examples

y <- factor( rbinom(100,3,0.5), ordered = TRUE )
res<-ordinal.mle(y)
res<-ordinal.mle(y, link = "probit")

MLE of the tobit model

Description

MLE of the tobit model.

Usage

tobit.mle(y, tol = 1e-09) 

Arguments

Details

The tobin model is useful for (univariate) positive data with left censoring at zero. There is the assumption of a latent variable. Tthe values of that variable which are positive concide with the observed values. If some values are negative, they are left censored and the observed values are zero. Instead of maximising the log-likelihood via a numerical optimiser we have used a Newton-Raphson algorithm which is faster.

Value

A list with three elements including

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Tobin James (1958). Estimation of relationships for limited dependent variables. Econometrica. 26(1):24–36.

https://en.wikipedia.org/wiki/Tobit_model

See Also

gammamle, normal.mle

Examples

x <- rnorm(300, 3, 5)
x[ x < 0 ] <- 0   ## left censoring. Values below zero become zero
for (i in 1:50) tobit.mle(x)

Mahalanobis distance

Description

Mahalanobis distance.

Usage

mahala(x, mu, sigma, ischol = FALSE)

Arguments

Value

A vector with the Mahalanobis distances.

Author(s)

Matteo Fasiolo <matteo.fasiolo@gmail.com>,

C++ and R implementation and documentation: Matteo Fasiolo <matteo.fasiolo@gmail.com>.

See Also

dista, colmeans

Examples

x <- matrix( rnorm(100 * 50), ncol = 50 )
m <- colmeans(x)
s <- cov(x)
#a1 <- mahala(x, m, s) 

Many area under the curve values

Description

Many area under the curve values.

Usage

colaucs(group, preds)
auc(group, preds)

Arguments

Details

The AUCs are calculated column-wise or just an AUC if the vector function is used.

Value

A vector with length equal to the number of columns of the "preds" argument, with the AUC values for each column. If the "auc" function is used then a signle number is returned.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

ttests, ttest, ftests

Examples

## 200 variables, hence 200 AUCs will be calculated
x <- matrix( rnorm(100 * 200), ncol = 200 )
ina <- rbinom(100, 1, 0.6)
colaucs(ina, x)
a <- colaucs(ina, x) 
b <- auc(ina, x[, 1])
x <- NULL

Many 2 sample proportions tests

Description

It performs very many 2 sample proportions tests.

Usage

proptests(x1, x2, n1, n2) 

Arguments

Details

The 2-sample proportions test is performed for each pair of proportions of teh two groups.

Value

A matrix with the proportions of each group (two columns), the test statistic and the p-value of each test.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

B. L. Welch (1951). On the comparison of several mean values: an alternative approach. Biometrika, 38(3/4), 330-336.

See Also

ttests, ftests, colVars

Examples

## 10000 variables, hence 10000 t-tests will be performed
set.seed(12345)
x1 <- rpois(500, 5)
x2 <- rpois(500, 5)
n1 <- rpois(1000, 40)
n2 <- rpois(1000, 40)
a <- proptests(x1, x2, n1, n2)
mean(a[, 4]<0.05)

x1 <- rbinom(500, 500, 0.6)
x2 <- rbinom(500, 500, 0.6)
b <- proptests(x1, x2, 500, 500)
mean(b[, 4]<0.05)

Many 2 sample tests tests

Description

It performs very many 2 sample tests.

Usage

ttests(x, y = NULL, ina, alternative = "unequal", paired = FALSE, 
logged = FALSE, parallel = FALSE)
mcnemars(x, y = NULL, ina, logged = FALSE) 
var2tests(x, y = NULL, ina, alternative = "unequal", logged = FALSE) 

Arguments

Details

For the ttests, if the groups are independent, the Welch's t-test (without assuming equal variances) is performed. Otherwise many paired t-tests are performed. The McNemar's test requires a number of observations, at least 30 would be good in order for the test to have some power and be size corect.

Value

A matrix with the test statistic, the degrees of freedom (if the groups are independent) and the p-value (or their logarithm) of each test.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

B. L. Welch (1951). On the comparison of several mean values: an alternative approach. Biometrika, 38(3/4), 330-336.

McNemar Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 12(2):153-157.

See Also

ftests, anovas, ttest

Examples

## 1000 variables, hence 1000 t-tests will be performed
x = matrnorm(100, 100)
## 100 observations in total
ina = rbinom(100, 1, 0.6) + 1   ## independent samples t-test
ttests(x, ina = ina)
x1 = x[ina == 1, ]
x2 = x[ina == 2, ]
ttests(x1, x2)
x <- NULL

Many ANCOVAs

Description

Many ANCOVAs.

Usage

ancovas(y, ina, x, logged = FALSE)

Arguments

Details

Many Analysis of covariance tests are performed. No interaction between the factor and the covariate is tested. Only the main effects. The design need not be balanced. The values of ina need not have the same frequency. The sums of squares have been adjusted to accept balanced and unbalanced designs.

Value

A matrix with the test statistic and the p-value for the factor variable and the covariate.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

D.C. Montgomery (2001). Design and analysis of experiments (5th Edition). New York: John Wiley & Sons

See Also

ftests, ttests, anovas

Examples

## 100 variables, hence 100 F-tests will be performed
y <- matrix( rnorm(90 * 100), ncol = 100 )
ina <- rbinom(90, 2, 0.5) + 1
x <- rnorm(90)
a <- ancovas(y, ina, x)

m1 <- lm(y[, 15] ~ factor(ina) + x)
m2 <- lm(y[, 15] ~ x + factor(ina))
res<-anova(m1)
res<-anova(m2)
y <- NULL
a[15, ]  ## the same with the m2 model, but not the m1

Many ANOVAS for count data with Poisson or quasi Poisson models

Description

Many ANOVAS for count data with Poisson or quasi Poisson models.

Usage

colpoisson.anovas(y, x, logged = FALSE) 
colquasipoisson.anovas(y, x, logged = FALSE) 

Arguments

Details

Poisson or quassi Poisson ANOVA takes place at each column.

Value

A matrix with the test statistic and the (logged) p-value for each predictor variable. In the case of the quasi Poisson, the \phi is returned as well.

Author(s)

Michail Tsagris and Manos Papadakis

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>

See Also

poisson.anova boot.ttest2, ttest2, ftest

Examples

y <- rpois(200, 10)
x <- matrix(rbinom(200 * 10, 3, 0.5 ), ncol = 10)

Many F-tests with really huge matrices

Description

Many F-tests with really huge matrices.

Usage

list.ftests(x, logged = FALSE)

Arguments

Details

The Welch's F-test (without assuming equal variances) is performed just like in the "ftests" function. The difference is that you have a really huge matrix which you cannot load into R. In the "ftests" function, the argument "ina" denotes the different groups. Here, you "cut" the matrix into smaller ones, each of which denotes a different group and put them in a list.

Value

A matrix with the test statistic and the p-value of each test.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr>.

References

B.L. Welch (1951). On the comparison of several mean values: an alternative approach. Biometrika, 38(3/4), 330-336.

See Also

ftests, ttests

Examples

x <- matrnorm(300, 500)
ina <- rbinom(300, 2, 0.6) + 1
a <- list()
a[[ 1 ]] <- x[ina == 1, ]
a[[ 2 ]] <- x[ina == 2, ]
a[[ 3 ]] <- x[ina == 3, ]
mod <- list.ftests(a)
z <- NULL
a <- NULL

Many G-square tests of indepedence

Description

Many G-square tests of indepdence with and without permutations.

Usage

g2tests(data, x, y, dc)
g2tests_perm(data, x, y, dc, nperm) 
chi2tests(data, x, y, dc)

Arguments

Details

The function does all the pairwise G^2 test of independence and gives the position inside the matrix. The user must build the associations matrix now, similarly to the correlation matrix. See the examples of how to do that. The p-value is not returned, we leave this to the user. See the examples of how to obtain it.

Value

A list including:

Author(s)

Giorgos Borboudakis. The permutation version used a C++ code by John Burkardt.

R implementation and documentation: Manos Papadakis <papadakm95@gmail.com>.

References

Tsagris M. (2017). Conditional independence test for categorical data using Poisson log-linear model. Journal of Data Science, 15(2):347-356.

Tsamardinos, I., & Borboudakis, G. (2010). Permutation testing improves Bayesian network learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases (pp. 322-337). Springer Berlin Heidelberg.

See Also

g2Test, g2Test_perm, correls, univglms

Examples

nvalues <- 3
nvars <- 10
nsamples <- 2000
data <- matrix( sample( 0:(nvalues - 1), nvars * nsamples, replace = TRUE ), nsamples, nvars )
dc <- rep(nvalues, nvars)
a <- g2tests(data = data, x = 2:9, y = 1, dc = dc)
pval <- pchisq(a$statistic, a$df, lower.tail = FALSE)  ## p-value
b <- g2tests_perm(data = data, x = 2:9, y = 1, dc = dc, nperm = 1000)
a<-b<-data<-NULL

Many Shapiro-Francia normality tests

Description

Many Shapiro-Francia normality tests.

Usage

sftests(x, logged = FALSE)
sftest(x, logged = FALSE)

Arguments

Details

The Shapiro-Francia univariate normality test is performed for each column (variable) of the matrix x.

Value

A matrix with the squared correlation between the ordered values and the standard normal ordered statistics, the test statistic and the p-value of each test. If the "sftest" has been used, the output is a vector with these three elements.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Royston J. P. (1983). A simple method for evaluating the Shapiro-Francia W' test of non-normality. The Statistician, 32(3): 297-300.

Mbah A. K. & Paothong A. (2015). Shapiro-Francia test compared to other normality test using expected p-value. Journal of Statistical Computation and Simulation, 85(15): 3002-3016.

See Also

ttests, ttest, ftests

Examples

x <- matrnorm(200, 100)
sftests(x)
a <- sftests(x)
mean(a[, 3]<0.05) 
x <- rnorm(100)
res<-sftest(x)

Many Welch's F-tests

Description

Many Welch's F-tests.

Usage

colanovas(y, x, logged = FALSE)

Arguments

Details

For each categorical variable in the x matrix Welch's F test is performed. This is the opposie of ftests, where there are many dependent variables and one categorical variable.

Value

A matrix with the test statistic and the p-value for each predictor variable.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Draper, N.R. and Smith H. (1988). Applied regression analysis. New York, Wiley, 3rd edition.

McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

See Also

regression, ftests, allbetas, correls

Examples

y <-  rnorm(100)
x <- matrix( rbinom(100 * 50, 2, 0.5) + 1 , ncol = 50)  
a <- colanovas(y, x)
x <- NULL

Many analysis of variance tests with a discrete variable

Description

Many analysis of variance tests with a discrete variable.

Usage

poisson.anovas(y, ina, logged = FALSE) 
quasipoisson.anovas(y, ina, logged = FALSE)
geom.anovas(y, ina, type = 1, logged = FALSE) 

Arguments

Details

This is the analysis of variance with count data. What we do is many log-likelihood ratio tests. For the quasi Poisson case we scale the difference in the deviances.

Value

A matrix with two values, the difference in the deviances (test statistic) and the relevant p-value. For the case of quasi Poisson the estimated \phi parameter is also returned.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

g2tests, poisson.anova, anova, poisson_only, poisson.mle

Examples

ina <- rbinom(500, 3, 0.5) + 1 
## Poisson example
y <- matrix( rpois(500 * 100, 10), ncol= 100 )
a1 <- poisson.anovas(y, ina)
y <- NULL

Many exponential regressions

Description

Many exponential regressions.

Usage

expregs(y, x, di, tol = 1e-09, logged = FALSE)  

Arguments

Details

We have implemented the newton-Raphson in order to avoid unnecessary calculations.

Value

A matrix with three columns, the test statistic, its associated (logged) p-value and the BIC of each model.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

See Also

univglms, score.glms, logistic_only, poisson_only, regression

Examples

## 200 variables, hence 200 univariate regressions are to be fitted
x <- matrnorm(100, 100)
y <- rexp(100, 4)
expregs(y, x, di = rep(1, length(y)))
x <- NULL

Many hypothesis tests for two means of percentages

Description

Many hypothesis tests for two means of percentages.

Usage

percent.ttests(x, y, logged = FALSE)

Arguments

Details

This is the prop.reg but with a single categorical predictor which has two levels only. It is like a t-test for the means of two samples haivng percentages.

Value

A matrix with three columns, the phi parameter, the test statistic and its associated p-value.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Papke L. E. & Wooldridge J. (1996). Econometric methods for fractional response variables with an application to 401(K) plan participation rates. Journal of Applied Econometrics, 11(6): 619-632.

McCullagh, Peter, and John A. Nelder. Generalized linear models. CRC press, USA, 2nd edition, 1989.

See Also

link{percent.ttest}, prop.reg, ttest2, ftest

Examples

x <- matrix( rbeta(100 * 10, 3, 1), ncol = 10)
y <- matrix( rbeta(50 * 10, 7.5, 2.5), ncol = 10)
res<-percent.ttests(x, y)

Many moment and maximum likelihood estimations of variance components

Description

Many moment and maximum likelihood estimations of variance components.

Usage

colvarcomps.mom(x, id, parallel = FALSE) 
colvarcomps.mle(x, id, ranef = FALSE, tol= 1e-08, maxiters = 100, 
parallel = FALSE)

Arguments

Details

Note that the "colvarcomp.mom" works for balanced designs only, i.e. for each subject the same number of measurements have been taken. The "colvarcomps.mle" works for unbalanced as well.

The variance components, the variance of the between measurements and the variance of the within are estimated using moment estimators. The "colvarcomps.mom" is the moment analogue of a random effects model which uses likelihood estimation ("colvarcomps.mle"). It is much faster, but can give negative variance of the random effects, in which case it becomes zero.

The maximum likelihood version is a bit slower (try youselves to see the difference), but statistically speaking is to be preferred when small samples are available. The reason why it is only a little bit slower and not a lot slower as one would imagine is because we are using a closed formula to calculate the two variance components (Demidenko, 2013, pg. 67-69). Yes, there are closed formulas for linear mixed models.

Value

For the "colvarcomps.mom": A matrix with 5 columns, The MSE, the estimate of the between variance, the variance components ratio and a 95% confidence for the ratio.

For the "colvarcomps.mle": If ranef = FALSE a list with a single component called "info". That is a matrix with 3 columns, The MSE, the estimate of the between variance and the log-likelihood value. If ranef = TRUE a list including "info" and an extra component called "ranef" containing the random effects. It is a matrix with the same number of columns as the data. Each column contains the randome effects of each variable.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

D.C. Montgomery (2001). Design and analysis of experiments (5th Edition). New York: John Wiley & Sons.

Charles S. Davis (2002). Statistical methods for the analysis of repeated measures. New York: Springer-Verlag.

Demidenko E. (2013). Mixed Models: Thoery and Applications with R 2nd Edition). New Jersey: John Wiley & Sons (Excellent book).

See Also

varcomps.mle, colrint.regbx

Examples

## example taken from Montgomery, page 514-517.
y <- c(98, 97, 99, 96, 91, 90, 93, 92,
96, 95, 97, 95, 95, 96, 99, 98)
y <- matrix(y)
id <- rep(1:4, each = 4)

x <- rmvnorm(100, numeric(20), diag(rexp(20)) )
id <- rep(1:25, each = 4)
n <- 25  ;  d <- 4
a <- colvarcomps.mom(x, id) 
mean(a[, 4]<0 & a[, 5]>0)  
b <- colvarcomps.mle(x, id) 
x <- NULL

Many multi-sample tests

Description

Many multi-sample tests.

Usage

ftests(x, ina, logged = FALSE)
anovas(x, ina, logged = FALSE)
vartests(x, ina, type = "levene", logged = FALSE)
block.anovas(x, treat, block, logged = FALSE)

Arguments

Details

The Welch's F-test (without assuming equal variances) is performed with the "ftests" function. The "anovas" function perform the classical (Fisher's) one-way analysis of variance (ANOVA) which assumes equal variance across the groups.

The "vartests" perform hypothesis test for the equality of the variances in two ways, either via the Levene or via the Brown-Forshythe procedure. Levene's test employs the means, whereas the Brown-Forsythe procedure employs the medians and is therefore more robust to outliers. The "var2tests" implement the classical F test.

The "block.anova" is the ANOVA with blocking, randomised complete block design (RCBD). In this case, for every combination of the block and treatment values, there is only one observation. The mathematics are the same as in the case of two way ANOVA, but the assumptions different and the testing procedure also different. In addition, no interaction is present.

Value

A matrix with the test statistic and the p-value of each test.

Author(s)

Michail Tsagris.

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Welch B.L. (1951). On the comparison of several mean values: an alternative approach. Biometrika, 38(3/4), 330-336.

Montgomery D.C. (2001). Design and analysis of experiments (5th Edition). New York: John Wiley & Sons.

See Also

ttests

Examples

x <- matrix( rnorm(300 * 50), ncol = 50 )
## 300 observations in total
ina <- rbinom(300, 3, 0.6) + 1   
a1 <- ftests(x, ina) 
a2 <- anovas(x, ina) 
a3 <- vartests(x, ina) 
x <- NULL

Many multivariate simple linear regressions coefficients

Description

Many multivariate simple linear regressions coefficients.

Usage

mvbetas(y, x, pvalue = FALSE)

Arguments

Details

It is a function somehow opposite to the allbetas. Instead of having one y and many xs we have many ys and one x.

Value

A matrix with the constant (alpha) and the slope (beta) for each simple linear regression. If the p-value is set to TRUE, the correlation of each y with the x is calculated along with the relevant p-value.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

allbetas, correls, univglms

Examples

y <- matrnorm(100, 100)
x <- rnorm(100)
a <- mvbetas(y, x, pvalue = FALSE)
b <- matrix(nrow = 100, ncol = 2)
z <- cbind(1, x)

a <- mvbetas(y, x)
b[2, ] <- coef( lm.fit( z, y[, 1] ) )
b[2, ] <- coef( lm.fit( z, y[, 2] ) )
x <- NULL

Many multi-sample tests

Description

Many multi-sample tests.

Usage

kruskaltests(x, ina, logged = FALSE) 
cqtests(x, treat, block, logged = FALSE)

Arguments

Details

The "kruskaltests" performs the Kruskal-Wallis non parametric alternative to analysis of variance test. The "cqtests" performs the Cocrhan's Q test for the equality of more than two groups whose values are strictly binary (0 or 1). This is a generalisation of the McNemar's test in the multi-sample case.

Value

A matrix with the test statistic and the p-value of each test.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

block.anovas, ftests

Examples

x <- matrix( rexp(300 * 200), ncol = 200 )
ina <- rbinom(300, 3, 0.6) + 1   
kruskaltests(x, ina)
x <- matrix( rbinom(300 * 200, 1, 0.6), ncol = 200 )
treat <- rep(1:3, each = 100)
block <- rep(1:3, 100)  
cqtests(x, treat, block)
x <- NULL

Many odds ratio tests

Description

It performs very many odds ratio tests.

Usage

odds(x, y = NULL, ina, logged = FALSE)

Arguments

Details

Many odds ratio tests are performed.

Value

A matrix with the test statistic and the p-value (or their logarithm) of each test.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

References

Mosteller Frederick (1968). Association and Estimation in Contingency Tables. Journal of the American Statistical Association. 63(321):1-28.

Edwards A.W.F. (1963). The measure of association in a 2x2 table. Journal of the Royal Statistical Society, Series A. 126(1):109-114.

See Also

odds.ratio, g2Test_univariate

Examples

x <- matrix( rbinom(100 * 100, 1, 0.5), ncol = 100 )
ina <- rep(1:2, each = 50)
a <- odds(x, ina = ina)

Many one sample goodness of fit tests for categorical data

Description

Many one sample goodness of fit tests for categorical data.

Usage

cat.goftests(x, props, type = "gsquare", logged = FALSE)

Arguments

Details

Given a matrix of integers, where each column refers to a sample, the values of a categorical variable the function tests wether these values can be assumed to fit a specific distribution.

Value

A matrix with the test statistic and the p-value of each test.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

ttests, ttest, ftests

Examples

x <- matrix( rbinom(300 * 100, 4, 0.6), ncol = 100 ) + 1
props <- dbinom(0:4, 4, 0.6)
## can we assume that each column comes from a distribution  whose mass is given by props?
cat.goftests(x, props)
a1 <- cat.goftests(x, props)  ## G-square test
a2 <- cat.goftests(x, props, type = "chisq")  ## Chi-square test
cor(a1, a2)
mean( abs(a1 - a2) ) 
x <- NULL

Many one sample tests

Description

Many one sample tests.

Usage

proptest(x, n, p, alternative = "unequal", logged = FALSE) 
ttest(x, m, alternative = "unequal", logged = FALSE, conf = NULL)
vartest(x, sigma, alternative = "unequal", logged = FALSE, conf = NULL) 

Arguments

Details

Despite the functions having been written in R, they are very fast.

Value

For all tests except for the "sftests" a matrix with two colums, the test statistic and the p-value respectively.

Author(s)

Michail Tsagris

R implementation and documentation: Michail Tsagris <mtsagris@uoc.gr> and Manos Papadakis <papadakm95@gmail.com>.

See Also

ftests, ttests

Examples

R <- 100
## protest
x <- rbinom(R, 50, 0.6)
n <- rep(50, R)
p <- rep(0.6, R)
a1 <- proptest(x, n, p, "unequal", logged = FALSE)
res<-sum( a1[, 2] < 0.05 ) / R

## vartest
x <- matrnorm(100, 100)
a2 <- vartest(x, rep(1, R) )
res<-sum( a2[, 2] < 0.05 )

## ttest
a4 <- ttest(x, numeric(R) )
res<-sum(a4[, 2] < 0.05) / R
x <- NULL

Many random intercepts LMMs for balanced data with a single identical covariate

Description

Many random intercepts LMMs for balanced data with a single identical covariate.

Usage

colrint.regbx(y, x, id)  

Arguments