readMat() now preserves the dimension of cell arrays
and empty matrices produces by MATLAB. See below Bug Fixes for
details.readMat() for the upcoming Matrix (>=
1.4.2).readMat() read empty MATLAB matrices as NULL. Now it
preserves the original data type and dimension,
e.g. matrix(integer(), nrow = 0, ncol = 5). Thanks to
Gordon Turner for the troubleshooting, the bug fix, and the package test
solving this.
readMat() lost the dimension for MATLAB cell arrays.
Thanks to Brian James for the troubleshooting, the bug fix, and the
package test solving this.
Matlab$getOption() produced “Error in
base::getOption(…) : ‘x’ must be a character string” in R-devel (to
become R 3.6.0). Thank you Kurt Hornik for the troubleshooting and the
fix.readMat() detects when MAT v7.3 files are trying to
be read and gives an informative error explaining those are not
supported (Issue #23). Thanks to Thomas Beutlich (author of C library
matio) for (non-official) pointers on how to distinguish MAT v5 from MAT
v7.3 files.writeMat() gained argument
fixNames = TRUE for automatically replacing periods
(.) in variable and field names with underscores
(_). Thanks to Eugenio Piasini at University College London
for suggesting this feature.
The MatlabServer.m script now outputs verbose/debug
messages to standard error (or more precisely MATLAB file #2) (Issue
#27).
Internal writeCommand() now gives an informative
error if an unknown command is used. Previously it silently sent a
0L command to MATLAB.
writeMat() could not write logical elements. They are
now converted to integers as MATLAB does not have a native logical /
boolean data type. Previously, writeMat() silently ignored
writing the data section of these logical elements resulting in a MAT
file that could not be parsed by MATLAB (although readMat()
could). Thanks to Eugenio Piasini for reporting on this problem.utils::setInternet2()
in help("Matlab").readMat() parses (effectively skips) also objects
of array class mxOPAQUE_CLASS (=17). The official ‘MAT-File
Format R2015b’ (Sept 2015) documentation does not describe this type.
The parsing/skipping was done by reverse engineering and clever guesses.
Thanks to GitHub user ‘yakzan’ for reporting on this type of MAT
files.Now readMat() parses also functional objects of
array type (class) 16. Note that the official ‘MAT-File Format R2015b’
(Sept
ROBUSTNESS: Now writeMat(pathname, ...) writes the
MAT file atomically by first writing to a temporary file which is
renamed only if successful.
ROBUSTNESS: Now writeMat() asserts that a maximum of
2^31-1 bytes are written per variable, otherwise an informative error is
thrown. This byte limit is set by the MAT file format definition. Thanks
to GitHub user ‘intelinsight’ for reporting on this.
evaluate() gained argument capture for
capturing MATLAB output while evaluating MATLAB expressions. Thanks to
Rohan Shah (GitHub user ‘rohan-shah’) for contributing code for
this.
ROBUSTNESS: Now the temporary filenames generated by MATLAB also includes the port number in addition to a semi-random sequence string. This should lower the risk for two MATLAB server using the same temporary filenames.
readMat() gave an error when reading empty sparse
matrices. Thanks to Gregory Jefferis at University of Cambridge for the
bug report and bug fix.Matlab$startServer() always overwrites any existing
MatlabServer.m and
InputStreamByteWrapper.class to make sure the most recent
versions are always used. Previously, existing files would be kept,
requiring a manual removal in order to use more resent versions of these
files.readMat(..., sparseMatrixClass = "matrix") did not
return the correct results in all cases. Added more package
tests.
readMat(..., sparseMatrixClass = "SparseM") did not
handle all-zero sparse matrices. Added package test for this.
readMat(..., sparseMatrixClass = "Matrix") would
give “Error in validObject(.Object) : invalid class”dgCMatrix” object:
lengths of slots ‘i’ and ‘x’ must match”, whenever trying to read an
sparse matrix with all zeros. Added package test for this case. Thanks
to Abhinav Jauhri at Carnegie Mellon University for reporting on this
and donating a test MAT file.
Trying to retrieve a non-existing MATLAB variable using
getVariable() would crash the R-to-MATLAB communication if
remote = FALSE. Similarly, Trying to
evaluate() a MATLAB expression that is invalid or by other
means fails would also break the R-to-MATLAB bridge. The only solution
is both cases was to restart R and MATLAB. Thanks to GitHub user
‘badbye’ (yalei) for the report.
InputStreamByteWrapper.java is no longer installed
with this package. It is only the precompiled
InputStreamByteWrapper.class file that is installed.
Updated documentation accordingly.readMat() parsed an mxCELL_CLASS structure
incorrectly if one of its names was empty. Thanks Dieter Menne at Menne
Biomed Consulting in Germany for reporting on this.help("Matlab").Added argument workdir to
Matlab$startServer() for setting the working directory of
MATLAB. Thanks to Steven Jaffe at Morgan Stanley for the
suggestion.
ROBUSTNESS: Under certain circumstances the MATLAB server script
(MatlabServer.m) would try to access variables that were
not yet defined. Thanks to Steven Jaffe at Morgan Stanley for spotting
this.
uncompressZlib() of readMat()
opened several raw connections without closing them. Added system
package test. Thanks Thilo Klein at University of Cambridge, UK, for the
report.readMat() can now read all (zlib) compressed MAT files
where it previously gave an error message on: “INTERNAL ERROR: Failed to
decompress data […] internal error -3 in memDecompress(2)”.miUINT32),
readMat() would drop any characters that were not in the
ASCII set (0,32-127). This was overly conservative as it should at least
have kept 7-bit ASCII characters (in 0-127), e.g. newline characters
were dropped. This bug was introduced in R.matlab
v1.2.0 (2008-02-12) when adding support for UTF-* strings via
iconv(). Now we keep all 8-bit ASCII characters, which is
also what the MATLAB file format documentation suggests are used in
non-UTF-encoded text matrices. Thanks to Steven Pav (San Francisco, CA)
for reporting on this.mxCHAR_CLASS structures are now read
slighly faster.finalize() is called on any deleted
Matlab objects.setOption() for Matlab gave Error in
UseMethod(“setOption”) : no applicable method for ‘setOption’ applied to
an object of class “c(‘Options’, ‘Object’)”. Added a package system test
for this. Thanks to Griet Laenen at KU Leuven for reporting on
this.Added option R.matlab::readMat/v4/textMatrixCollapse
controlling whether MAT v4 text matrixes are collapsed into strings by
row ("byrow"; default), by column ("bycolumn")
or not at all ("none").
Whenever there is an uncompress error in readMat(),
it now tries to infer what type of compression the buffer has by
inspecting the first two bytes and include the type in the error
message.
readMat(..., drop = "singletonLists") would throw an
error if the singleton list dropped contained NULL and that NULL was
assigned to an element of an outer list, resulting in that element being
dropped.ROBUSTNESS: Now Matlab$startServer() asserts that
all MATLAB server files that are indeed successfully copied, iff
missing.
open() for Matlab no longer generates
warnings on “socketConnection(…) […] cannot be opened”, which occured
while waiting/polling for the MATLAB server to startup and
respond.
MatlabServer.m script would incorrectly consider
MATLAB v8 and above as MATLAB v6. Thanks to Frank Stephen at NREL for
reporting on this and providing a patch.Now the R.matlab Package object is also
available when the package is only loaded (but not attached).
ROBUSTNESS: Added package system test asserting that
readMat() can be called without attaching the
package.
Package now only imports R.oo (no longer attaches it). Also, the package now imports R.utils (used to suggest and attach it when needed). These updates make the package more “silent”.
Dropped obsolete autoload():s.
readMat() throws on
failed decompression also includes the first and last bytes of the data
block that it tried to decompress. This will further help
troubleshooting.R.methodsS3::appendVarArgs().CONSISTENCY: Added argument drop to
readMat() to control how singleton dimensions of for
instance nested lists are dropped or not. The defaults are now such that
R.matlab v2.0.4 is consistent with
R.matlab v1.7.1 (R.matlab v2.0.0-2.0.3
were not). Thanks to Claudia Beleites at IPHT Jena, Germany for
reporting on this.
Bumped package dependencies.
readMat() would produce an error when parsing an empty
mxCHAR_CLASS matrix with a 0x0 dimension. It is not clear
whether this is a valid MAT v5 structure or not, but I`ve added a
workaround. Thanks Claudia Beleites at IPHT Jena, Germany for reporting
on this.help("readMat").MATLAB
instead of Matlab.readMat() significant faster. More than
15x speedup(!) is observed on MAT files with many small structures. This
was achieved by (i) dramatically decreasing the amount of memory
allocation and garbage collection needed by the internal raw buffer
mechanism, by (ii) dropping an internal rm() calls that was
called very often, as well as by (iii) minimizing the number of times
local functions and objects are created.Now readMat() also accepts a raw vector as
input.
Now readMat() no longer generates warnings reporting
on “In readBin( con = rawBufferT, what = what, size = size, n = n,
signed = signed, : ‘signed = FALSE’ is only valid for integers of sizes
1 and 2.”
help("readMat"). Added a section on ‘Speed
performance’.\usage{} lines are at most 90 characters
long.readMat() now support reading 64-bit integers [array
type (class) mxINT64_CLASS and mxUINT64_CLASS]
on machines/versions of R with .Machine$sizeof.long >= 8
or .Machine$sizeof.longlong >= 8.
Package utilizes new startupMessage() of
R.oo.
Authors@R field to the DESCRIPTION.R CMD check no longer warns on global
assignments.
Bumped up package dependencies.
readMat() could not read sparse matrices containing
logical values, only numerics. This was because the logical
bit in the Array Flags was not utilized by readMat(), which
in turn was because the file format documentation is rather vague on how
to use that bit. This means that readMat() now represents
sparse logical matrices using logical values, except when
sparseMatrixClass = "SparseM", because
SparseM matrices can only hold numeric values. Thanks
to Irtisha Sinhg at Cornell University for reporting on this.It is now possible to specify the decompression method for
readMat().
Now readMat() decompression error messages are more
informative.
readMat(..., verbose = -111) gave an error on object
zraw not being found.SOFTWARE QUALITY:
readMat() and writeMat() to formal system
tests of the package, and made those examples simpler.memDecompress() of the base package
instead. However, just in case someone is still running older versions
of R, readMat() does indeed still look for
Rcompression as a fallback..Internal() calls.help(Matlab) on how to connect
through firewalls. Thanks Bas de Regt at University of Adelaide for the
suggestion.R CMD check NOTE that would show up in R
v2.15.0 devel.The version 1.5.0 that was submitted to CRAN did not include the adjustments for startup messages. Thus, resubmitting this one to CRAN.
Updated dependencies on R and packages.
GENERALIZATION: Now readMat() utilizes
base::memDecompress() to uncompress compressed data
structures, unless running R v2.9.x or before, in case
Rcompression::uncompress() is used, iff installed. R
v2.10.0 was released October 2009. Thanks Stephen Eglen at University of
Cambridge for reminding me about memDecompress().
CLEANUP: Now using packageStartupMessage() instead
of cat().
Added a namespace to the package, which will be more or less a requirement in the next major release of R.
CLEANUP: Now all references to the Rcompression
package is within the local uncompress() function of
readMat(). This makes the code more modular making it
easier to implement alternatives to
Rcompression::uncompress().
example(readMat).imaginary (not partial
imag) in calls to complex().Now writeMat() gives an error if duplicated object
names are used,
e.g. writeMat(..., A = 1, B = 2, A = 3).
Now writeMat() gives an error, not a warning, if
there are non-named objects.
writeMat() did not
work correctly. Thanks Claudia Beleites for the report.readMat() can read non-named
objects, in MATLAB it is only the load() function call that
can read it but not the plain ‘load’ call. Because of this,
writeMat() will now give a warning if it detects non-named
objects. For instance, in writeMat("foo.mat", x = a, y) the
object a is named (as x) whereas
y is not. To name the objects and avoid the warning, use
writeMat("foo.mat", x = a, y = y). Thanks Claudia Beleites
at University of Trieste for reporting on this.writeMat() that it can only
write ‘uncompressed’ MAT files.Clarified and corrected some sentences in
help(Matlab).
Added MATLAB 7.11.0.584 (R2010b) to the list of confirmed version that are compatible with R.matlab.
MatlabServer.m script incorrectly referred to the
InputStreamByteWrapper class as
java.io.InputStreamByteWrapper. This hopefully solves
previously reports on obtaining “??? Undefined variable”java” or class
“java.io.InputStreamByteWrapper”. Error in ==> MatlabServer at 262
reader = java.io.InputStreamByteWrapper(4096);“. This bug was introduced
in R.matlab v1.2.5 (2009-08-25). Thanks Kenvor Cothey
at GMO LCC for the report.R.utilsR.utils # Version 1.3.3 [2010-09-18]
readMat() would throw an exception on “Error in
dim(matrix) <- dimensionsArray$dim : dims [product 1] do not match
the length of object [0]” in rare cases related to empty strings. Not
sure if those cases are legal, but added an ad hoc workaround for them.
Thanks Claude Flene at University of Turku for reporting this.MatlabServer script reports it’s version when
started.Removed some minor ambiguities in help(Matlab) and
updated example(Matlab) with some more troubleshooting
statements.
Added MATLAB 7.4.0 (R2007a), 7.7.0.471 (R2008b), and 7.10.0.499 (R2010a) to the list of confirmed version that are compatible with R.matlab. Thanks Ajay Tripathy at UC Berkeley and Marco Colombo at University of Edinburgh for reporting this.
MatlabServer.m saves variables using the function
form, i.e. save(). This solves the problem of having single
quotation marks in the pathname. Thanks Michael Q. Fan at NC State
University for reporting this problem.fixNames of
help(readMat). Thanks Stephen Eglen (University of
Cambridge) for the suggestion.readMat() can read also nested
miMATRIX structures. Issue reported by Jonathan Chard at
Mango Solutions, UK.readMat() tries
smaller and smaller initial buffers, before giving in up. Also, it now
starts with a buffer 3 (not 10) times the compressed size. Thanks to
Michael Sumner for the report.MatlabServer.m) gave the
error “Undefined function or method ServerSocket for input
arguments of type double.”, when launching it via
Matlab$startServer(). It seems like
import java.net.*, etc. does not work. A workaround is to
specify the full path for all Java classes,
e.g. java.net.ServerSocket. Thanks Nicolas Stadler for the
report.onWrite == NULL, then there is no longer an outer
two-pass scan for figuring out the size of MAT structure. However,
instead each internal writeDataElement() has to do a
two-pass scan in order to figure out the number of bytes before writing
each element.Arrays with more than two dimension would be written as vectors. Thanks Minho Chae for the report.
From v1.2.2, writeMat() would generate corrupt MAT
files. Reverted writeMat() back to v1.2.1 and updated as
above.
example(readMat) so that it gives no error if
the optional Rcompression package is not
installed.writeMat() that counts the number
of bytes to be written is skipped if argument onWrite is
not given. This means that writing to file, which is the most common use
case, will no longer involve the counting. Thank to Adam Grossman at
Stanford University for suggesting the modification and providing
initial code. He reports a substantial speedup in writing large files
(38 MB file).Removed internal debugging/asserting code.
Renamed the HISTORY file to NEWS.
readMat()/writeMat() we use an internal
character vector to represent the 8-bit ASCII character set. However, in
R one cannot represent ASCII 0x00 - it resolves to an empty string, and
from R v2.8.0 we will get a warning if we try. For now, we have replaced
the ASCII for 0x00 with the empty string (unless before R v2.6.0). We
might want to move away from using this ASCII vector for
intToChar()/charToInt() and instead make use
of intToUtf8()/utf8ToInt() or alternatively
just rawToChar()/charToRaw().Added support for reading compressed MAT files. It requires that the Rcompression package (Omegahat.org) is available.
Added support for also returning sparse matrices according to the representations given by either the Matrix or the SparseM package.
Added simple UTF-* string support. Conversion should be accurate when the R installation supports iconv.
Added mat-files/StructWithSparseMatrix-v4.mat from
JR.
Credits for all of the above new feature goes to Jason Riedy at the Computer Science Dept, UC Berkeley. Thanks also to Duncan Temple-Lang at UC Davis for the Rcompression package.
readMat() does not support
compressed MAT files.Update so that package passes R CMD check on R
v2.5.0.
Replaced Sys.putenv() with new
Sys.setenv() (to be deprecated in R v2.5.0).
Now open() throws an error if connection to MATLAB
failed (before FALSE was returned).
Extended the accepted range of ports to [1023,66535].
Added more details on available options in
setOption() and how to avoid timeouts.
The example in help(Matlab) is now using default
values for host and port making the example
less confusing.
MATLABSERVER_PORT)
was out of range, a MATLAB syntax error occurred, i.e ‘… Line: 109
Column: 45 “)” expected, “identifier” found.’ Thanks Alexander Nervedi
for the report.port to
Matlab$startServer(..., port = NULL) to start the MATLAB
server such that it listens to another port than the default 9999.
Internally the server first tries to obtain the port number by
environment variable MATLABSERVER_PORT. This feature was
requested by Wang Yu at Iowa State University.?Matlab.Matlab class, the package requires the
R.utils where the classes Java and
Verbose was moved. The methods for dealing with user
options in the Matlab class are provided by the
Options class also in R.utils. Similarly,
if verbose != FALSE in readMat() or
writeMat(), then the R.utils package is
required.Forgot to “implement” miSINGLE, i.e. to set
knownTypes and knownWhats for this. Thanks to
Craig Aumann for the report.
readMat() did not read ‘complex’ matrices in MAT v5
correctly. Thanks to Chris Sims for the report.
readMat() did not read ‘text’ matrices in MAT v4
correctly. Thanks to Chris Sims, Princeton University, for the
patch.
Added support for user-defined options to Matlab
objects via methods setOption() and
getOption(). Current supported options are
"readResult/maxTries" and
"readResult/interval" to provide a “workaround” for the
problem with timeout errors in evaluate(). Thanks Thomas
Romary, France, for (re-)reporting on this problem.
Added class Options. This will probably move to
another package whenever it becomes stable and widely used.
misc/ with two new
directories mat-files/ and externals/.seq(0) and not seq(length = 0) was used.
Updated all occurrences of this problem.readMat() failed to read some MATLAB struct`s
because of a parsing error.
Internally in readMat() - in
readMat5DataStructure() it might be data
readTag() returns a tag referring to a data block of zero
length (nbrOfBytes == 0). Now list(NULL) is
returned in this case.
The MatlabServer script did (indeed) not work with
MATLAB v7; a wrong version was included. Thanks Patrick Drechsler,
University of Wuerzburg for reporting the above two problems.
SIGNFICANT CHANGES:
MatlabServer.m script to automatically set
the Java classpath for MATLAB v7 and higher. This will assure that
InputStreamByteWrapper.class is found as long as it is in
the same directory as the *.m script. Thanks Yichun Wei at
University of Southern California for suggesting this and also for
suggesting additional help instructions.Added a Wishlist and an Acknowledgements section to “about”.
Added more information on how to start the MATLAB server.
Matlab$startServer() and MATLAB v7.autoload("hasVarArgs") to
R/000.R.Package now passes R CMD check on R v2.1.0 devel
without warnings.
Added appendVarArgs = TRUE to
setMethodS3(), which specifies that ... should
be added, if missing.
Add argument ... to all methods to make it even more
consistent with any generic function. This is also done for a few
methods in R base packages.
readMat() would not read non-signed bytes
correctly.DOCMENTATION:
Updated description of package. Preparing for moving to CRAN.
Added Rdoc comments to internal Verbose classes.
Package passes R CMD check on R v2.1.0 devel
too.
Moved the Java class from R.io to
this package, since it is currently only used here. It is used for
reading and writing Java bytes, ints and (UTF) strings when
communicating with MATLAB.
Added a setVerbose() method to the
Matlab class, which allows the user to get details at
various levels of the MATLAB process.
Added support for reading sparse matrices. Sparse matrices are expanded to regular matrices, which means that some may be too large to be loaded.
WORKAROUND: It sometimes happens that the reply from MATLAB is
not transferred “quick enough” and even if the connection should block
we receive NULL giving an error. The workaround for now is to try to
read the answer again. The symptom was that an error in
readResult() complain about “if (answer == 0)” where answer
was of length 0.
The header of MAT v4 files was not parsed correctly, which made MAT v4 files unrecognized and assumed to be MAT v5 files which in turn could not be read.
Used "sparce" instead of "sparse" in
MOPT[4] tag (MAT v4), which most likely would generate an
error for such structure.
<- was used instead of
<<- in one new method. I do not like
<<-, but that is how it works.Created an readMat() that can read both MAT v4 and
MAT v5 files. The MAT v4 reader was kindly provided by Andy Jacobson at
the Princeton University.
writeMat() was created for consistency, but it does
only support MAT v5.
Note that I have now chosen to rename readMAT() and
writeMAT() to readMat() and
writeMat(), respectively. This is in line with the RCC,
cf. readHtml() instead of readHTML(). If you
really need readMAT() and writeMAT() you can
easily write your own wrappers.
require(R.io) is loaded with the package and not
when the first MATLAB client/server method is called. The former
approach was a bit annoying.Java class of the R.io
package. Correcting these bugs made the MATLAB part work again.\keyword{internal}.readMAT() would not work because getBits()
previously in package R.oo was removed.
getBits() is now added as a local function inside
readMAT().Initial methods are readMAT() and
writeMAT() for reading and writing MATLAB MAT file
structures.
Initial class is Matlab, which provides static methods for communicating with the running MATLAB server.
Package created.