\documentclass[nojss]{jss}

\usepackage[english]{babel}

%\documentclass[fleqn, a4paper]{article}
%\usepackage{a4wide}
%\usepackage[round,longnamesfirst]{natbib}
%\usepackage{graphicx,keyval,thumbpdf,url}
%\usepackage{hyperref}
%\usepackage{Sweave}
\SweaveOpts{strip.white=true}
\AtBeginDocument{\setkeys{Gin}{width=0.6\textwidth}}

\usepackage[utf8]{inputenc}

%% end of declarations %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\usepackage{amsmath}
\usepackage{amsfonts}


%\newcommand{\strong}[1]{{\normalfont\fontseries{b}\selectfont #1}}
\newcommand{\class}[1]{\mbox{\textsf{#1}}}
\newcommand{\func}[1]{\mbox{\texttt{#1()}}}
%\newcommand{\code}[1]{\mbox{\texttt{#1}}}
%\newcommand{\pkg}[1]{\strong{#1}}
\newcommand{\samp}[1]{`\mbox{\texttt{#1}}'}
%\newcommand{\proglang}[1]{\textsf{#1}}
\newcommand{\set}[1]{\mathcal{#1}}
\newcommand{\sQuote}[1]{`{#1}'}
\newcommand{\dQuote}[1]{``{#1}''}
\newcommand\R{{\mathbb{R}}}

\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}

%% almost as usual
\author{Michael Hahsler\\Southern Methodist University \And
Kurt Hornik\\Wirtschaftsuniversit\"at Wien \AND
Christian Buchta\\Wirtschaftsuniversit\"at Wien}
\title{Getting Things in Order:\\
An Introduction to the \proglang{R}~Package~\pkg{seriation}}

%% for pretty printing and a nice hypersummary also set:
\Plainauthor{Michael Hahsler, Kurt Hornik, Christian Buchta} %% comma-separated
\Plaintitle{Getting Things in Order: An Introduction to the R Package seriation} %% without formatting
\Shorttitle{Getting Things in Order} %% a short title (if necessary)

%% an abstract and keywords
\Abstract{Seriation, i.e., finding a suitable linear order for a set of
objects given data and a loss or merit function, is a basic problem in
data analysis.  Caused by the problem's combinatorial nature, it is
hard to solve for all but very small sets.  Nevertheless, both exact
solution methods and heuristics are available.  In this paper we
present the package~\pkg{seriation} which provides an infrastructure
for seriation with \proglang{R}.  The infrastructure comprises data
structures to represent linear orders as permutation vectors, a wide
array of seriation methods using a consistent interface, a method to
calculate the value of various loss and merit functions, and several
visualization techniques which build on seriation. To illustrate how
easily the package can be applied for a variety of applications, a
comprehensive collection of examples is presented.}

\Keywords{combinatorial data analysis, seriation, permutation, \proglang{R}}
\Plainkeywords{combinatorial data analysis, seriation, permutation, R} %% without formatting

\Address{
Michael Hahsler\\
    Engineering Management, Information, and Systems\\
    Lyle School of Engineering\\
    Southern Methodist University\\
    P.O. Box 750123 \\
    Dallas, TX 75275-0123\\
    E-mail: \email{mhahsler@lyle.smu.edu}\\
    URL: \url{http://lyle.smu.edu/~mhahsler}

Kurt Hornik\\
Department f\"ur Statistik \& Mathematik\\
Wirtschaftsuniversit\"at Wien\\
1090 Wien, Austria\\
E-mail: \email{kurt.hornik@wu.ac.at}\\
URL: \url{http://statmath.wu.ac.at/~hornik/}

Christian Buchta\\
Department f\"ur Welthandel\\
Wirtschaftsuniversit\"at Wien\\
1090 Wien, Austria\\
E-mail: \email{christian.buchta@wu.ac.at}\\
URL: \url{http://www.wu.ac.at/itf/institute/staff/buchta}
			    }

\hyphenation{Brusco}
\sloppy

%% \VignetteIndexEntry{An Introduction to the R package seriation}

\begin{document}

%\title{Getting Things in Order: An introduction to the
%R~package~\pkg{seriation}}
%\author{Michael Hahsler, Kurt Hornik and Christian Buchta}
\maketitle


%\abstract{Seriation, i.e., finding a suitable linear order for a set of
%  objects given data and a loss or merit function, is a basic problem in
%  data analysis.  Caused by the problem's combinatorial nature, it is
%  hard to solve for all but very small sets.  Nevertheless, both exact
%  solution methods and heuristics are available.  In this paper we
%  present the package~\pkg{seriation} which provides an infrastructure
%  for seriation with \proglang{R}.  The infrastructure comprises data
%  structures to represent linear orders as permutation vectors, a wide
%  array of seriation methods using a consistent interface, a method to
%  calculate the value of various loss and merit functions, and several
%  visualization techniques which build on seriation. To illustrate how
%  easily the package can be applied for a variety of applications, a
%  comprehensive collection of examples is presented.}
%


<<echo=FALSE>>=
options(scipen=3, digits=4)
### for sampling
set.seed(1234)
@

\section{Introduction}

A basic problem in data analysis, called \emph{seriation} or sometimes
\emph{sequencing}, is to arrange all objects in a set in a linear order
given available data and some loss or merit function in order to
reveal structural information.  Together with
cluster analysis and variable selection, seriation is an important
problem in the field of \emph{combinatorial data
  analysis}~\citep{seriation:Arabie:1996}.  Solving problems in
combinatorial data analysis requires the solution of discrete
optimization problems which, in the most general case, involves
evaluating all feasible solutions.  Due to the combinatorial nature, the
number of possible solutions grows with problem size (number of objects, $n$)
by the order~$O(n!)$.  This makes a brute-force enumerative approach
infeasible for all but very small problems.  To solve larger problems
(currently with up to 40 objects), partial enumeration methods can be used.  For
example, \cite{seriation:Hubert:2001} propose dynamic programming and
\cite{seriation:Brusco:2005} use a branch-and-bound strategy.  For even
larger problems only heuristics can be employed.

It has to be noted that seriation has a rich history in archaeology.
\cite{seriation:Petrie:1899} was the first to use seriation as a formal method.
He applied it to find a chronological order for graves discovered in the Nile
area given objects found there. He used a cross-tabulation of grave sites and
objects and rearranged the table using row and column permutations till all
large values were close to the diagonal. In the rearranged table graves with
similar objects are closer to each other. Together with the assumption that
different objects continuously come into and go out of fashion, the order of
graves in the rearranged table suggests a chronological order.
Initially, the rearrangement of rows and columns of this
contingency table was done manually and the adequacy was only judged
subjectively by the researcher.  Later, \cite{seriation:Robinson:1951},
\cite{seriation:Kendall:1971} and others proposed measures of agreement
between rows to quantify optimality of the resulting table.  A
comprehensive description of the development of seriation in archeology
is presented by \cite{seriation:Ihm:2005}.

Techniques related to seriation are also popular in several other
fields.  Especially in ecology scaling techniques are used under the
name \emph{ordination}.  For these applications several \proglang{R} packages
already exist (e.g.,
\pkg{ade4}~\citep{seriation:Chessel:2007,seriation:Dray:2007} and
\pkg{vegan}~\citep{seriation:Oksanen:2007}). This paper describes the new
package \pkg{seriation} which differs from existing packages in the
following ways:
\begin{itemize}
 \item \pkg{seriation} provides a flexible infrastructure for seriation;
 \item \pkg{seriation} focuses on seriation as a combinatorial
  optimization problem.
\end{itemize}

This paper starts with a formal introduction of the seriation problem as
a combinatorial optimization problem in Section~\ref{sec:seriation}.  In
Section~\ref{sec:methods} we give an overview of seriation methods.  In
Section~\ref{sec:infrastructure} we present the infrastructure provided
by the package~\pkg{seriation}.  Several examples and applications for
seriation are given in Section~\ref{sec:example}.
Section~\ref{sec:conclusion} concludes.

A previous version of this manuscript was published in the \emph{Journal
  of Statistical Software} \citep{seriation:Hahsler+Hornik:2008}.


\section{Seriation as a combinatorial optimization problem}
\label{sec:seriation}

To seriate a set of $n$ objects $\{O_1,\dots,O_n\}$ one typically starts
with an $n \times n$ symmetric dissimilarity matrix~$\mathbf{D} =
(d_{ij})$ where $d_{ij}$ for $1 \le i,j \le n$ represents the
dissimilarity between objects $O_i$ and $O_j$, and $d_{ii} = 0$ for
all~$i$.  We define a permutation function $\Psi$ as a function which
reorders the objects in $\mathbf{D}$ by simultaneously permuting rows
and columns.  The seriation problem is to find a permutation function
$\Psi^*$
%$\{1,\dots,n\} \rightarrow \{1,\dots,n\}$, i.e. a
%bijection that maps the set of indices of the objects (and equally of rows and
%columns of $\mathbf{D}$) onto itself,
which optimizes the value of a given loss function~$L$ or merit function~$M$.
This results in the optimization problems
\begin{equation}
    \Psi^* = \argmin_\Psi L(\Psi(\mathbf{D})) \quad \text{or} \quad
    \Psi^* = \argmax_\Psi M(\Psi(\mathbf{D})),
\end{equation}
respectively.
%This is clearly a hard discrete optimization problem since the number of
%possible permutations is $n!$ which makes an exhaustive
%search for sets with a medium to large number of objects infeasible.
%Partial enumeration methods and heuristics can be used. Such methods are
%presented in Section~\ref{sec:methods}.
%But first, we review commonly used loss functions in the following section.
%\marginpar{two-mode data missing}

A symmetric dissimilarity matrix is known as \emph{two-way one-mode}
data since it has columns and rows (two-way) but only represents one set
of objects (one-mode).  Seriation is also possible for two-way two-mode
data which are represented by a general nonnegative matrix.  In such data
columns and rows represent two sets of objects which are reordered
simultaneously.  For loss/merit functions for two-way two-mode data the
optimal order of columns can depend of the order of rows and vice
versa or it can be independent allowing for breaking the optimization
down into two separate problems, one for the columns and one for the
rows.  Another way to deal with the seriation for two-way two-mode data
is to calculate two dissimilarity matrices, one for each mode, and then
solve two seriation problems for two-way one-mode data.  Furthermore,
seriation can be generalized to $k$-way $k$-mode data in the form of a
$k$-dimensional array by defining suitable loss/merit functions for such data
or by breaking the problem down into several lower dimensional
independent problems.

To assess the complexity of seriation of $k$-way $k$-mode data, let us
assume the data is a $k$-dimensional array with the dimensions
containing $n_1, n_2, \ldots, n_k$ objects. If the loss/merit function allows
for separating the problem into $k$ independent problems, the problem
size is just the sum of the individual problems. By using complete
enumeration the size is $O(\sum_{i=1}^k{n_i!})$. If the problem is not
separable and the optimal seriation of each dimension depends on the
order of the objects of the other dimensions, the problem size is
$O((\sum_{i=1}^k{n_i})!)$. For example for $k=5$ and all dimensions
containing 5 objects, the search space for separable dimensions is only
600 while without separability it is larger than $10^{25}$ clearly too
big to be solvable in reasonable time. This shows that for data with
even only a few dimensions and a few objects each, finding the optimal
solution is infeasible and loss/merit functions which allow for separating the
problem are highly desirable.

In the following subsections, we review some commonly employed loss/merit
functions.  Most functions are used for two-way one-mode data but the
measure of effectiveness and stress can be also used for two-way
two-mode data. For the implementation of various loss or merit measures
see function~\func{criterion} in Section~\ref{sec:infrastructure}.



%\section{Loss functions}
%\label{sec:criteria}
%In the literature several loss functions are suggested.
%We review the most commonly used functions.

\subsection{Column/row gradient measures}

A symmetric dissimilarity matrix where the values in all rows and
columns only increase when moving away from the main diagonal is called
a perfect \emph{anti-Robinson matrix} after the statistician
\cite{seriation:Robinson:1951}. Formally, an $n \times n$ dissimilarity
matrix $\mathbf{D}$ is in anti-Robinson form if and only if the
following two gradient conditions hold~\citep{seriation:Hubert:2001}:
\begin{align}
    \text{within rows:} & \quad  d_{ik} \le d_{ij} \quad \text{for}
        \quad 1 \le i < k < j \le n; \\
    \text{within columns:} & \quad  d_{kj} \le d_{ij} \quad \text{for}
        \quad 1 \le i < k < j \le n.
\end{align}

In an anti-Robinson matrix the smallest dissimilarity values appear
close to the main diagonal, therefore, the closer objects are together
in the order of the matrix, the higher their similarity.  This provides
a natural objective for seriation.

It has to be noted that $\mathbf{D}$ can be brought into a perfect
anti-Robinson form by row and column permutation whenever $\mathbf{D}$  is an
ultrametric or $\mathbf{D}$ has an exact Euclidean representation in a single
dimension~\citep{seriation:Hubert:2001}. However, for most data only an
approximation to the anti-Robinson form is possible.

A suitable merit measure which quantifies the divergence of a matrix from the
anti-Robinson form was given by \cite{seriation:Hubert:2001} as
\begin{equation}
    M(\mathbf{D}) =
    \sum_{i<k<j}f(d_{ik}, d_{ij}) + \sum_{i<k<j}f(d_{kj}, d_{ij})
    \label{equ:gradient}
\end{equation}
where $f(\cdot,\cdot)$ is a function which defines how a violation or
satisfaction of a gradient condition for an object triple ($O_i$, $O_k$ and
$O_j$) is counted.  \cite{seriation:Hubert:2001} suggest two functions. The
first function is given by:
\begin{equation}
    f(z,y) = \mathrm{sign}(y-z) =
    \begin{cases}
        +1 \quad \text{if} \quad z < y; \\
        \phantom{+}0 \quad \text{if} \quad z = y; \\
        -1 \quad \text{if} \quad z > y.
    \end{cases}
\end{equation}
It results in the raw number of triples satisfying the gradient
constraints minus triples which violate the constraints.

The second function is defined as:
\begin{equation}
    f(z,y) = |y-z|\mathrm{sign}(y-z) = y-z
\end{equation}
It weighs each satisfaction or violation by its
magnitude given by the absolute difference between the values.

\subsection{Anti-Robinson events}
An even simpler loss function can be created in the same way as the gradient
measures above by concentrating on violations only.
\begin{equation}
    L(\mathbf{D}) =
    \sum_{i<k<j}f(d_{ik}, d_{ij}) + \sum_{i<k<j}f(d_{kj}, d_{ij})
\end{equation}

To only count the violations we use
\begin{equation}
    f(z, y) = I(z, y) =
    \begin{cases}
        1 \quad \text{if} \quad z > y \quad \text{and} \\
        0 \quad \text{otherwise.}
    \end{cases}
\end{equation}
$I(\cdot)$ is an indicator function returning $1$ only for violations.
\cite{seriation:Chen:2002} presented a formulation for an equivalent
loss function and called the violations \emph{anti-Robinson events}.
\cite{seriation:Chen:2002} also introduced a weighted versions of the loss
function resulting in
\begin{equation}
    f(z, y) = |y-z|I(z, y)
\end{equation}
using the absolute deviations as weights.

\subsection{Hamiltonian path length}
The dissimilarity matrix $\mathbf{D}$ can be represented as a finite weighted
graph $G = (\Omega,E)$ where the set of objects~$\Omega$ constitute the
vertices and each edge~$e_{ij} \in E$ between the objects $O_i, O_j \in \Omega$
has a weight~$w_{ij}$ associated which represents the dissimilarity~$d_{ij}$.

Such a graph can be used for seriation~\citep[see,
e.g.,][]{seriation:Hubert:1974,seriation:Caraux:2005}.  An order~$\Psi$
of the objects can be seen as a path through the graph where each node
is visited exactly once, i.e., a Hamiltonian path.  Minimizing the
Hamiltonian path length results in a seriation optimal with respect to
dissimilarities between neighboring objects.  The loss function based on
the Hamiltonian path length is:
\begin{equation}
    L(\mathbf{D}) = \sum_{i=1}^{n-1} d_{i,i+1}.
\end{equation}

Note that the length of the Hamiltonian path is equal to the value of the
\emph{minimal span loss function} \citep[as used by][]{seriation:Chen:2002},
and both notions are related to the \emph{traveling salesperson
problem}~\citep{seriation:Gutin:2002}.

\subsection{Inertia criterion}
Another way to look at the seriation problem is not to focus on placing small
dissimilarity values close to the diagonal, but to push large values away from
it. A function to quantify this is the moment of inertia of dissimilarity
values around the diagonal \citep{seriation:Caraux:2005} defined as
\begin{equation}
    M(\mathbf{D}) = \sum_{i=1}^n \sum_{j=1}^n d_{ij}|i-j|^2.
\end{equation}
$|i-j|^2$ is used as a measure for the distance to the diagonal and $d_{ij}$
gives the weight.  This is a merit function since the sum increases when higher
dissimilarity values are placed farther away from the diagonal.

\subsection{Least squares criterion}
Another natural loss function for seriation is to quantify the deviations
between the dissimilarities in $\mathbf{D}$ and the rank differences of the
objects.  Such deviations can be measured, e.g, by the sum of squares of
deviations \citep{seriation:Caraux:2005} defined by
\begin{equation}
    L(\mathbf{D}) = \sum_{i=1}^n \sum_{j=1}^n (d_{ij} - |i-j|)^2,
\end{equation}
where $|i-j|$ is the rank difference or gap between $O_i$ and $O_j$.

The least squares criterion defined here is related to uni-dimensional
scaling~\citep{seriation:Leeuw:2005}, where the objective is to place all
$n$ objects on a straight line using a position vector~$\mathbf{z} =
z_1,z_2,\ldots,z_n$ such that the dissimilarities
in $\mathbf{D}$ are
preserved by the relative positions in the best possible way. The optimization
problem of uni-dimensional scaling is to find the
position vector~$\mathbf{z^*}$ which minimizes $\sum_{i=1}^n \sum_{j=1}^n
(d_{ij} - |z_i-z_j|)^2$.
This is close to the seriation problem, but in
addition to the ranking of the objects also takes the distances between objects
on the resulting scale into account.

Note that if Euclidean distance is used to calculate $\mathbf{D}$ from a data
matrix~$\mathbf{X}$, using the order of the elements in $\mathbf{X}$ as they
occur projected on the first principal component of $\mathbf{X}$
minimizes the loss function of uni-dimensional scaling (using squared
distances). Using this order, also provides a good solution
for the least square seriation criterion.

\subsection{Linear Seriation Criterion}
The Linear Seriation Criterion (Hubert and Schultz 1976)
weights the distances with the absolute rank differences.

$$L(\mathbf{D}) \sum_{i=1}^n \sum_{j=1}^n d_{ij} (-|i-j|)$$

\subsection{2-Sum Problem}
The 2-Sum loss criterion \citep{seriation:Barnard:1993}
multiplies the similarity between objects
with the squared rank differences.

$$L(\mathbf{D}) \sum_{i,j=1}^p \frac{1}{1+d_{ij}} (i-j)^2,$$

where $s_{ij} = \frac{1}{1+d_{ij}}$ represents the similarity between
objects $i$ and $j$.


\subsection{Measure of effectiveness}
\label{sec:ME}

\cite{seriation:McCormick:1972} defined the
\emph{measure of effectiveness (ME)} for an $n \times m$ matrix~$\mathbf{X} =
(x_{ij})$ as
\begin{equation}
    M(\mathbf{X}) =
    \frac{1}{2}
    \sum_{i=1}^{n} \sum_{j=1}^{m} x_{ij}[x_{i,j+1}+x_{i,j-1}+
        x_{i+1,j}+x_{i-1,j}]
    \label{equ:ME}
\end{equation}
with, by convention $x_{0,j}=x_{n+1,j}=x_{i,0}=x_{i,m+1}=0$.  ME is maximized
if each element is as closely related numerically to its four neighboring
elements as possible.

ME was developed for two-way two-mode data, however, ME can also be used for a
symmetric  matrix (one-mode data) and gets maximal only if all large values are
grouped together around the main diagonal.

Note that the definition in equation~(\ref{equ:ME})
can be rewritten as
\begin{equation}
    M(\mathbf{X}) =
    \frac{1}{2}
    \sum_{i=1}^{n} \sum_{j=1}^{m} x_{ij}[x_{i,j+1}+x_{i,j-1}] +
    \sum_{i=1}^{n} \sum_{j=1}^{m} x_{ij}[x_{i+1,j}+x_{i-1,j}]
\end{equation}
showing that the contributions of column and row order to the merit function
are independent.

\subsection{Stress}
\label{sec:stress}

Stress measures the conciseness of the presentation of a matrix
(two-mode data) and can be seen as a purity function which compares the
values in a matrix with their neighbors. The stress measures used here
are computed as the sum of squared distances of each matrix entry from
its adjacent entries.  \cite{seriation:Niermann:2005} defined for an $n
\times m$ matrix~$\mathbf{X} = (x_{ij})$ two types of neighborhoods:

\begin{itemize}
    \item The Moore neighborhood comprises the (at most) eight adjacent entries.
        The local stress measure for element~$x_{ij}$ is defined as
        \begin{equation}
            \sigma_{ij} = \sum_{k=\max(1,i-1)}^{\min(n,i+1)}
                \sum_{l=\max(1,j-1)}^{\min(m,j+1)}
                (x_{ij} - x_{kl})^2
        \end{equation}

    \item The Neumann neighborhood comprises the (at most) four adjacent entries
        resulting in the local stress of $x_{ij}$ of
        \begin{equation}
            \sigma_{ij} =
            \sum_{k=\max(1,i-1)}^{\min(n,i+1)} (x_{ij} - x_{kj})^2 +
            \sum_{l=\max(1,j-1)}^{\min(m,j+1)} (x_{ij} - x_{il})^2
            %(x_{ij} - x(i-1,j))^2 + (x_{ij} - x(i+1,j))^2 +
            %(x_{ij} - x(i,j-1))^2 + (x_{ij} - x(i,j+1))^2
        \end{equation}
\end{itemize}

Both local stress measures can be used to construct a global measure for the
whole matrix by summing over all entries which can be used as a loss function:
\begin{equation}
    L(\mathbf{X}) =
    \sum_{i=1}^n \sum_{j=1}^m \sigma_{ij}
\end{equation}

The major difference between the Moore and the Neumann neighborhood is
that for the later the contributions of row and column order to
stress are independent.

Stress can be also used as a loss function for
symmetric proximity matrices (one-mode data).
%,
%since it can only be optimal, if large values are
%concentrated around the main diagonal.
Note also, that stress with Neumann
neighborhood is related to the measure of effectiveness defined above (in
Section~\ref{sec:ME}) since both measures are optimal if for each cell the cell
and its four neighbors are numerically as similar as possible.

\section{Seriation methods}
\label{sec:methods}

Solving the discrete optimization problem for seriation with most loss/merit
functions is clearly very hard. The number of possible permutations for $n$
objects is $n!$ which makes an exhaustive search for sets with a medium to
large number of objects infeasible.  In this section, we describe some methods
(partial enumeration, heuristics and other methods) which are typically used
for seriation. For each method we state for which type of loss/merit functions
it is suitable and whether it finds the optimum or is a heuristic.  For the
implementation of various seriation methods see function~\func{seriate} in
Section~\ref{sec:infrastructure}.


\subsection{Partial enumeration methods}

Partial enumeration methods search for the exact solution of a
combinatorial optimization problem.  Exploiting properties of the search
space, only a subset of the enormous number of possible combinations has
to be evaluated.  Popular partial enumeration methods which are used for
seriation are \emph{dynamic programming}~\citep{seriation:Hubert:2001}
and \emph{branch-and-bound}~\citep{seriation:Brusco:2005}.

Dynamic programming recursively searches for the optimal solution checking and
storing $2^n-1$ results. Although $2^n-1$ grows at a lower rate than $n!$ and
is for $n \gg 3$ considerably smaller, the storage requirements of $2^n-1$
results still grow fast, limiting the maximal problem size severely. For
example, for $n=30$ more than one billion results have to be calculated and
stored, clearly a number too large for the main memory capacity of most current
computers.

Branch-and-bound has only very moderate storage requirements. The
forward-branching procedure~\citep{seriation:Brusco:2005} starts to build
partial permutations from left (first position) to right. At each step, it is
checked if the permutation is valid and several fathoming tests are performed
to check if the algorithm should continue with the partial permutation.  The
most important fathoming test is the boundary test, which checks if the partial
permutation can possibly lead to a complete permutation with a better solution
than the currently best one.  In this way large parts of the search space can
be omitted. However, in contrast to the dynamic programming approach, the
reduction of search space is strongly data dependent and poorly structured
data can lead to very poor performance.  With branch-and-bound slightly larger
problems can be solved than with dynamic programming in reasonable time.
\cite{seriation:Brusco:2005} state that depending on the data, in some cases
proximity matrices with 40 or more objects can be handled with current
hardware.

Partial enumeration methods can be used to find the exact solution
independently of the loss/merit function. However, partial enumeration is
limited to only relatively small problems.

\subsection{Traveling salesperson problem solver}

Seriation by minimizing the length of a Hamiltonian path through a graph
is equal to solving a traveling salesperson problem.  The traveling
salesperson or salesman problem (TSP) is a well known and well
researched combinatorial optimization
problem~\citep[see, e.g.,][]{seriation:Gutin:2002}. The goal is to find the
shortest tour that, starting from a given city, visits each city in a
given list exactly once and then returns to the starting city.
In graph theory a TSP tour is called
a \emph{Hamiltonian cycle.} But for the seriation problem, we are
looking for a Hamiltonian path. \cite{seriation:Garfinkel:1985}
described a simple transformation of the TSP to find the shortest
Hamiltonian path.  An additional row and column of 0's is added
(sometimes this is referred to as a \emph{dummy city}) to the original
$n \times n$ dissimilarity matrix~$\mathbf{D}$.  The solution of this
$(n+1)$-city TSP, gives the shortest path where the city representing
the added row/column cuts the cycle into a linear path.

As the general seriation problem, solving the TSP is difficult.  In the
seriation case with $n+1$ cities, $n!$ tours have to be checked. However,
despite this vast searching space, small instances can be solved efficiently
using dynamic programming \citep{seriation:Held:1962} and larger instances of
several hundred objects can be solved using \emph{branch-and-cut}
algorithms~\citep{seriation:Padberg:1990}. For even larger instances or if
running time is critical, a wide array of heuristics are available, ranging
from simple nearest neighbor approaches to construct a
tour~\citep{seriation:Rosenkrantz:1977} to complex heuristics like the
Lin-Kernighan heuristic~\citep{seriation:Lin:1973}.  A comprehensive overview
of heuristics and exact methods can be found in \cite{seriation:Gutin:2002}.


\subsection{Bond energy algorithm}

The \emph{bond energy algorithm}~\citep[BEA;][]{seriation:McCormick:1972} is a
simple heuristic to rearrange columns and rows of a matrix
(two-way two-mode data) such that each entry
is as closely numerically related to its four neighbors as possible.  To
achieve this, BEA tries to maximize the measure of effectiveness (ME) defined
in Section~\ref{sec:ME}.  For optimizing the ME, columns and rows can be
treated separately since changing the order of rows does not influence the ME
contributions of the columns and vice versa. BEA consists of the
following three steps:
\begin{enumerate}
 \item Place one randomly chosen column.
 \item Try to place each remaining column at each possible position
  left, right and between the already placed columns and calculate every
  time the increase in ME.  Choose the column and position which gives
  the largest increase in ME and place the column.  Repeat till all
  columns are placed.
 \item Repeat procedure with rows.
\end{enumerate}

This greedy algorithm works fast and only depends on the choice of the
first column/row. This dependence can be reduced by repeating the
procedure several times with different choices and returning the solution
with the highest ME.

Although \cite{seriation:McCormick:1972} use BEA also for non-binary data,
\cite{seriation:Arabie:1990} argue that the measure of effectiveness only
serves its intended purpose of finding an arrangement which is
close to Robinson form for binary data and should therefore only be
used for binary data.

\cite{seriation:Lenstra:1974} notes that the optimization problem of BEA
can be stated as two independent traveling salesperson problems (TSPs).
For example, the row TSP for an $n \times m$ matrix~$\mathbf{X}$
consists of $n$ cities with an $n \times n$ distance matrix~$\mathbf{D}$
where the distances are
\begin{displaymath}
    d_{ij} = -\sum_{k=1}^m x_{ik}x_{jk}.
\end{displaymath}
BEA is in fact a simple suboptimal TSP heuristic using this distances
and instead of BEA any TSP solver can be used to obtain an order.
With an exact TSP solver, the optimal solution can be found.

\subsection{Hierarchical clustering}
\label{sec:hierarchical_clustering}

Hierarchical clustering produces a series of nested clusterings which
can be visualized by a dendrogram, a tree where each internal node
represents a split into subtrees and has a measure of
similarity/dissimilarity attached to it.  As a simple heuristic to find
a linear order of objects, the order of the leaf nodes in a dendrogram
structure can be used. This idea is used, e.g., by heat maps to reorder
rows and columns with the aim to place more similar objects and
variables closer together.

%For hierarchical clustering several methods are available (e.g.,
%single linkage, average linkage, complete linkage, ward method) resulting in
%different dendrograms.
%However,
The order of leaf nodes in a dendrogram is not unique.  A binary
(two-way splits only) dendrogram for $n$ objects has $2^{n-1}$ internal
nodes and at each internal node the left and right subtree (or leaves)
can be swapped resulting in $2^{n-1}$ distinct leaf orderings.  To find
a unique or optimal order, an additional criterion has to be defined.
\cite{seriation:Gruvaeus:1972} suggest to obtain a unique order by
requiring to order the leaf nodes such that at each level the objects at
the edge of each cluster are adjacent to that object outside the cluster
to which it is nearest.

\cite{seriation:Bar-Joseph:2001} suggest to rearrange the dendrogram such that
the Hamiltonian path connecting the leaves is minimized and called this the
optimal leaf order. The authors also present a fast algorithm with time
complexity $O(n^4)$ to solve this optimization problem. Note that this problem
is related to the TSP described above, however, the given dendrogram structure
significantly reduces the number of permissible permutations making the problem
easier.

Although hierarchical clustering solves an optimization problem different to
the seriation problem discussed in this paper, hierarchical clustering still
can produce useful orderings, e.g., for visualization.

\subsection{Rank-two ellipse seriation}

\cite{seriation:Chen:2002} proposes to
generate a sequence of correlation matrices
$R^1, R^2, \ldots$. $R^1$ is the correlation matrix
of the original distance matrix $\mathbf{D}$ and
\begin{equation}
R^{n+1} = \phi R^n,
\end{equation}
where $\phi(\cdot)$ calculates a correlation matrix.

\cite{seriation:Chen:2002} shows that the
rank of the matrix $R^n$ falls with increasing $n$ and that if the sequence
is continued till the first matrix in the sequence has a rank of 2,
projecting all points in this matrix on its first two eigenvectors,
all points will fall on an ellipse.
\cite{seriation:Chen:2002} suggests to use the order of the points on
this ellipse as a seriation where the ellipse can be cut at any of the
two interception points (top or bottom) with the vertical axis.

Although the rank-two ellipse seriation procedure does not try to solve a
combinatorial optimization problem, it still provides for some cases a useful
ordering.

\subsection{Spectral Seriation}
Spectral seriation uses a relaxation to minimize the 2-Sum Problem
\citep{seriation:Barnard:1993}.
Rewriting the minimization problem using a permutation vector $\pi$,
its inverse, rescaling to $\mathrm{q}$ and using a Lagrangian
multiplier for the constraint on the permutation yields \citep{seriation:Ding:2004} the following equivalent optimization problem:

$$\mathrm{min}_\mathbf{q} \frac{\mathbf{q}^T L_\mathbf{S}\mathbf{q}}{\mathbf{q}^T\mathbf{q}}$$

where $L_\mathbf{S}$ is the Laplacian of $\mathbf{S}$.

The optimal order can be recovered by the sorting order of
the Fiedler vector (i.e., the second smallest eigenvector of the Laplacian
of the similarity matrix).

\subsection{Quadratic Assignment Problem}
Both, the linear seriation criterion and the 2-Sum problem formulation
can be written as a Quadratic Assignment Problem (QAP). However,
the QAP is in general NP-hard. Methods include
QIP, linearization, branch and bound and cutting planes as well as
heuristics including Tabu search, simulated annealing, genetic algorithms, and
ant systems \citep{seriation:Burkard:1998}.

\section{The package infrastructure}
\label{sec:infrastructure}

The \pkg{seriation} package provides the data structures and some algorithms
to efficiently handle seriation with \proglang{R}. As the
input data for seriation
\proglang{R}
already provides

\begin{itemize}
    \item for two-way one-mode data the class \code{dist},
    \item for two-way two-mode data the class \code{matrix}, and
    \item for $k$-way $k$-mode data the class \code{array}.
\end{itemize}


\begin{figure}[tp]
    \centerline{
    %\includegraphics[width=12cm]{infrastructure}}
    \includegraphics[width=10cm]{classes}}

    \caption{UML class diagram of the data structures for permutations provided by \pkg{seriation}}
    \label{fig:infrastructure}
\end{figure}

However, \proglang{R} provides no classes for representing permutation vectors.
\pkg{seriation} adds the necessary data structure (using the S3 class
system) as depicted in the UML class diagram \citep{seriation:Fowler:2004} in
Figure~\ref{fig:infrastructure}. In this diagram classes are represented by
rectangles and different symbols are used to state the type of
relationship between the classes. The class
\code{ser\_permutation}
in Figure~\ref{fig:infrastructure}
represents the permutation information for $k$-mode
data (including the cases of $k=1$ and $k=2$). It consists of $k$ permutation
vectors (class \code{ser\_permutation\_vector}). This relationship is
represented by the solid diamond and the star above the connection between the
two classes. Class \code{ser\_permutation\_vector} is defined \emph{abstract}
and only its concrete implementations (classes connected with the triangle
symbol) are used to store a permutation vector.
This design with an abstract class was chosen to allow to use
different representations for the permutation vectors.
Currently, the permutation vector can be stored as a simple
integer vector or as an object of class \code{hclust} (defined in
package \pkg{stats}). \code{hclust} describes a hierarchical clustering
tree (dendrogram) including an ordering for the tree's node leaves which
provides a permutation for all objects (see
Section~\ref{sec:hierarchical_clustering}).

Class \code{ser\_permutation\_vector} has a constructor
\func{ser\_permutation\_vector} which converts data into the correct concrete
subclass of \code{ser\_permutation\_vector} and checks if it contains a proper
permutation vector.  For \code{ser\_permutation\_vector} the methods
\func{print}, \func{length} for the length of the permutation vector,
\func{get\_method} to get the method used to generate the permutation, and
\func{get\_order} to access the raw (integer) permutation vector are available.
To use an additional class to represent permutations as a concrete subclass of
\code{ser\_permutation\_vector} only an appropriate accessor method
\func{get\_order} has to be implemented for the new class.

For \code{ser\_permutation} a constructor is provided which can bind $k$
\code{ser\_permutation\_vector} objects together into an object for $k$-mode
data. \code{ser\_permutation} is implemented as a list of length~$k$ and each
element contains a \code{ser\_permutation\_vector} object.  Methods like
\func{length}, accessing elements with \code{[[},
% ]]
\code{[[<-},
% ]]
subsetting with \code{[}, and combining with \func{c} work as expected.
Also a \func{print} method is provided. Finally, direct access to the
raw permutation vectors is available using \func{get\_order}.  Here a
second argument (which defaults to $1$) specifies the dimension (mode)
for which the order vector is requested.

All seriation algorithms are available via the function \func{seriate}
defined as:
\begin{quotation}
\code{seriate(x,  method = NULL, control = NULL, ...)}
\end{quotation}
where \code{x} is the input data, \code{method} is a string defining the
seriation method to be used and \code{control} can contain a list with
additional information for the algorithm.  \func{seriate} returns an object
of class \code{ser\_permutation} with a length conforming to the number of
dimensions of~\code{x}.
Typical input data are a dissimilarity matrix (class~\code{dist};
see package \pkg{stats} for more information) for one-mode two-way data,
\code{matrix} for two-mode two-way data and \code{array} for $k$-mode $k$-way
data.
For \code{matrix} and \code{array} the additional argument
\code{margin} can be used to restrict the dimensions which should be seriated
(e.g., with \code{margin = 1} only the first dimension,
i.e., the columns of a matrix, are seriated).

%\begin{landscape}
\begin{table}[tp]
\centering
    \begin{tabular}{p{5cm}p{3cm}p{4cm}l}
        \hline
Algorithm & \code{method} & Optimizes & Input data \\
        \hline
Simulated annealing & \code{"ARSA"} & Linear seriation crit.&\code{dist} \\
Branch-and-bound & \code{"BBURCG"} & Gradient measure &\code{dist} \\
Branch-and-bound & \code{"BBWRCG"} & Gradient measure (weighted)& \code{dist} \\
TSP solver & \code{"TSP"} & Hamiltonian path length& \code{dist} \\
Optimal leaf ordering  & \code{"OLO"}
\code{"OLO_single"}
\code{"OLO_average"}
\code{"OLO_complete"}
& Hamiltonian path length (restricted)& \code{dist} \\
Gruvaeus and Wainer & \code{"GW"}
\code{"GW_single"}
\code{"GW_average"}
\code{"GW_complete"}
& Hamiltonian path length (restricted) & \code{dist} \\
MDS & \code{"MDS"}
\code{"MDS_metric"}
\code{"MDS_nonmetric"}
\code{"MDS_angle"}
& Least square crit.& \code{dist} \\

Spectral seriation & \code{"Spectral"}
\code{"Spectral_norm"}
& 2-Sum crit. & \code{dist} \\

QAP & \code{"QAP_2SUM"}
& 2-Sum crit. & \code{dist} \\

 & \code{"QAP_LS"}
& Linear seriation crit. & \code{dist} \\

 & \code{"QAP_BAR"}
& Banded AR form  & \code{dist} \\

 & \code{"QAP_Inertia"}
& Inertia crit. & \code{dist} \\

	Genetic Algorithm & \code{"GA"}*
& various & \code{dist} \\

	DendSer & \code{"DendSer"}*
& various & \code{dist} \\

Hierarchical clustering & \code{"HC"}
\code{"HC_single"}
\code{"HC_average"}
\code{"HC_complete"}
& Other& \code{dist} \\
Rank-two ellipse seriation  & \code{"R2E"} & Other& \code{dist} \\
Sorting Points Into Neighborhoods & \code{"SPIN_NH"}
\code{"SPIN_STS"} & Other& \code{dist} \\
Visual Assessment of (Clustering) Tendency & \code{"VAT"}& Other& \code{dist} \\
        \hline
Bond Energy Algorithm & \code{"BEA"} &
    Measure of effectiveness & \code{matrix} \\
TSP to optimize ME & \code{"BEA\_TSP"} &
    Measure of effectiveness& \code{matrix} \\
Principal component analysis& \code{"PCA"}
\code{"PCA_angle"}&
    Least square crit.& \code{matrix} \\
        \hline
    \end{tabular}
    \caption{Currently implemented methods for \func{seriation} (* methods need to be registered).}
\label{tab:methods}
\end{table}
%\end{landscape}

Various seriation methods were already introduced in this paper in
Section~\ref{sec:methods}. In Table~\ref{tab:methods} we summarize the
methods currently available in the package for seriation.  The code for
the simulated annealing heuristic~\citep{seriation:Brusco:2007} and the
two branch-and-bound implementations~\citep{seriation:Brusco:2005} was
obtained from the authors.  The TSP solvers (exact solvers and a variety
of heuristics) is provided by package
\pkg{TSP}~\citep{seriation:Hahsler:2007, seriation:Hahsler:2007b}.  For
optimal leaf ordering we implemented the algorithm
by~\cite{seriation:Bar-Joseph:2001}. The BEA code was kindly provided by
Fionn Murtagh.  For the Gruvaeus and Wainer algorithm, the
implementation in package \pkg{gclus}~\citep{seriation:Hurley:2007} is
used. For the rank-two ellipse seriation we implemented the algorithm
by~\cite{seriation:Chen:2002}.
Spectral seriation is described by~\cite{seriation:Ding:2004}.
Note that some methods implemented
(e.g., the rank-two ellipse seriation) do not fall within the
combinatorial optimization framework of this paper and thus are not
dealt with here in detail.  They are included in the package since they
can be useful for various applications.
A detailed empirical comparison of seriation methods and criteria can be found
in the study by \cite{hahsler:Hahsler2016d}.
%Over time more methods will be
%added to the package.

To calculate the value of a loss/merit function for data and
a certain permutation, the function
\begin{quotation}
\code{criterion(x, order = NULL, method = NULL, ...)}
\end{quotation}
is provided. \code{x} is the data object, \code{order} contains a suitable
object of class \code{ser\_permutation} (if omitted no permutation is
performed) and \code{method} specifies the type of loss/merit function. A
vector of several methods can be used resulting in a named vector
with the values of the requested functions.  If \code{method} is omitted
(\code{method = NULL}), the values for all applicable loss/merit functions are
calculated and returned.  We already defined different loss/merit functions for
seriation in Section~\ref{sec:seriation}. In Table~\ref{tab:criteria} we
indicate the loss/merit functions currently available in the package.

\begin{table}[t]
\centering
\begin{tabular}{llll}
        \hline
        Name & \code{method} & merit/loss & Input data \\
        \hline
        Anti-Robinson events& \code{"AR\_events"} &
loss & \code{dist} \\
        Anti-Robinson deviations& \code{"AR\_deviations"} &
loss & \code{dist} \\
        Banded Anti-Robinson& \code{"BAR"} &
loss & \code{dist} \\
        Gradient measure& \code{"Gradient\_raw"} &
merit & \code{dist} \\
        Gradient measure (weighted)& \code{"Gradient\_weighted"} &
merit & \code{dist} \\
        Hamiltonian path length & \code{"Path\_length"} & loss & \code{dist} \\
        Inertia criterion& \code{"Inertia"} & merit & \code{dist} \\
        Least squares criterion& \code{"Least\_squares"} & loss & \code{dist} \\

        Linear Seriation criterion& \code{"LS"} & loss & \code{dist} \\
        2-Sum criterion& \code{"2SUM"} & loss & \code{dist} \\

        \hline
        Measure of effectiveness& \code{"ME"} &
merit & \code{matrix} \\
        Stress (Moore neighborhood)& \code{"Moore\_stress"} &
loss & \code{matrix} \\
        Stress (Neumann neighborhood)& \code{"Neumann\_stress"} &
loss & \code{matrix} \\
        \hline
    \end{tabular}
    \caption{Implemented loss/merit functions in function \func{criterion}.}
\label{tab:criteria}
\end{table}

All methods for \func{seriate} and \func{criterion} are managed by a
registry mechanism which makes the seriation framework easily extensible
for users.  For example, a new seriation method can be registered using
\func{set\_seriation\_method} and then used in the same way as the
built-in methods with \func{seriate}. All available methods in the
registry can be viewed using \func{list\_seriation\_methods} and
\func{show\_seriation\_methods}.  For criterion methods, the same
interface is available by just substituting `seriation' by `criterion'
in the function names. An example for how to add new methods can be
found in section~\ref{sec:registering} of this paper.

In addition the package offers the (generic) function
\begin{quotation}
\code{permute(x, order)}
\end{quotation}
where \code{x} is the data (a \code{dist} object, a matrix, an
array, a list or a numeric vector) to be reordered and \code{order} is a
\code{ser\_permutation} object of suitable length.
%The permutation for
%\code{dist} objects uses package \pkg{proxy}~\citep{seriation:Meyer:2007}.

For visualization, the package offers several options:
\begin{itemize}
\item Matrix shading with \func{pimage}. In contrast to the
        standard \func{image} in package~\pkg{graphics}, \func{pimage}
        displays the matrix as is with the first element in the top
        left-hand corner and using a gamma-corrected gray scale.

\item Different heat maps (e.g., with optimally reordered
    dendrograms) with \func{hmap}.

\item Visualization of data matrices in the spirit
of~\cite{seriation:Bertin:1981} with \func{bertinplot}.

\item \emph{Dissimilarity plot}, a new visualization to judge the
    quality of a clustering using matrix shading
    and seriation with \func{dissplot}.
\end{itemize}

We will introduce the package usage and the visualization options
in the examples in the next section.

\section{Examples and applications}
\label{sec:example}


We start this section with a simple first session to demonstrate the basic
usage of the package. Then we present and discuss several seriation
applications.

\subsection{A first session using seriation}
In the following example, we use the well known iris data set
(from \proglang{R}'s \pkg{datasets} package) which gives the
measurements in centimeters of the variables sepal length and width and petal
length and width, respectively, for 50 flowers from each of 3 species of the
iris family (Iris setosa, versicolor and virginica).

First, we load the package \pkg{seriation} and the iris data set.  We
remove the species classification and reorder the objects randomly since
they are already sorted by species in the data set. Then we calculate
the Euclidean distances between objects.

<<echo=FALSE>>=
set.seed(1234)
@

<<>>=
library("seriation")

data("iris")
x <- as.matrix(iris[-5])
x <- x[sample(seq_len(nrow(x))),]
d <- dist(x)
@

To seriate the objects given the dissimilarities, we just call
\func{seriate} with the default settings.

<<>>=
o <- seriate(d)
o
@

The result is an object of class \code{ser\_permutation} for
one-mode data. The permutation vector length is $150$ for the
$150$ objects in the iris data set and the used seriation method is
\code{"ARSA"}, a simulated annealing heuristic
(see~Table~\ref{tab:methods}). The actual order can be accessed
using \func{get\_order}. In the following we show the first
15 elements in the permutation vector.

<<>>=
head(get_order(o), 15)
@


To visually inspect the effect of seriation on the distance matrix, we use
matrix shading with \func{pimage} (the result is shown in
Figure~\ref{fig:pimage1}).

<<label=pimage1, fig=TRUE, include=FALSE, width=7.5>>=
pimage(d, main = "Random")
@
<<label=pimage1-2, fig=TRUE, include=FALSE, width=7.5>>=
pimage(d, o, main = "Reordered")
@
\begin{figure}
    \centering
    \includegraphics[width=7.5cm]{seriation-pimage1}
    \includegraphics[width=7.5cm]{seriation-pimage1-2}
    \caption{Matrix shading of the  distance matrix for the iris data.}
    \label{fig:pimage1}
\end{figure}

We can also compare the improvement for different loss/merit functions
using \func{criterion}.

<<>>=
cbind(random = criterion(d), reordered = criterion(d, o))
@

Naturally, the reordered dissimilarity matrix achieves better values for all
criteria. Note that the gradient measures, inertia and the measure of
effectiveness are merit functions and for these measures larger values are
better (use \code{show\_criterion\_methods("dist")} to find out which measures
are loss and merit functions).

To visually compare the original data matrix and the
result of seriation, we can also use \func{pimage}.
We standardize the data using scale such that the visualized value
is the number of standard deviations an object differs from the
variable mean. For matrices containing negative values, \code{pimage}
uses automatically a divergent palette.
After using \func{pimage} for the original random data matrix,
we create a suitable \code{ser\_permutation} object for the original
two-mode data. Since the seriation above only produced an order for the rows
of the data, we add an identity permutation vector
for the columns (represented by \code{NA})
to the permutations object using the combine
function \func{c}. This new permutation object for $2$-mode data is used
for displaying the reordered scaled data. The two plots are shown in
Figure~\ref{fig:pimage2}.

<<label=pimage2, fig=TRUE, include=FALSE, width=7.5>>=
pimage(scale(x), main = "Random", prop = FALSE)
@

<<label=pimage2-2, fig=TRUE, include=FALSE, width=7.5>>=
o_2mode <- c(o, NA)
pimage(scale(x), o_2mode, main = "Reordered", prop = FALSE)
@

\begin{figure}
    \centering
    \includegraphics[width=7.5cm]{seriation-pimage2}
    \includegraphics[width=7.5cm]{seriation-pimage2-2}
    \caption{Matrix shading of the iris data matrix.}
    \label{fig:pimage2}
\end{figure}

\subsection{Comparing different seriation methods}

To compare different seriation methods we use again the randomized iris data
set and the distance matrix \code{d} from the previous example.  We include in
the comparison several seriation methods for dissimilarity matrices described
in Section~\ref{sec:methods}.

<<>>=
methods <- c("TSP","R2E", "ARSA", "HC", "GW", "OLO")
o <- sapply(methods, FUN = function(m) seriate(d, m))
@

<<echo=FALSE>>=
timing <- sapply(methods, FUN = function(m) system.time(seriate(d, m)),
    simplify = FALSE)
@


\begin{table}
    \centering
     \begin{tabular}{lcccccc}
     \hline
     Seriation Method &
     \Sexpr{methods[1]}&
     \Sexpr{methods[2]}&
     \Sexpr{methods[3]}&
     \Sexpr{methods[4]}&
     \Sexpr{methods[5]}&
     \Sexpr{methods[6]} \\
     \hline
     Execution time [sec] &
     \Sexpr{round(timing[[methods[1]]][1],4)}&
     \Sexpr{round(timing[[methods[2]]][1],4)}&
     \Sexpr{round(timing[[methods[3]]][1],4)}&
     \Sexpr{round(timing[[methods[4]]][1],4)}&
     \Sexpr{round(timing[[methods[5]]][1],4)}&
     \Sexpr{round(timing[[methods[6]]][1],4)}\\
     \hline
     \end{tabular}
%%% fix me: for the vignette we need something else
\caption{Execution time of seriation of the iris data set for different
    methods.}
    \label{tab:timings}
\end{table}

Table~\ref{tab:timings} contains the execution times for running
seriation with the different methods. Except for the simulated annealing
method (ARSA) the seriation only takes a fraction of a second.
The direction of the resulting orderings is first normalized (aligned)
and then the orderings are displayed using matrix shading
(see Figure~\ref{fig:pimage3}).


<<label=pimage3-pre, eval=FALSE>>=
o <- ser_align(o)
for(s in o) pimage(d, s, main = get_method(s), key = FALSE)
@

<<label=pimage3, echo=FALSE, fig=TRUE, include=FALSE>>=
o <- ser_align(o)
for(i in 1:length(o)) {
    pdf(file=paste("seriation-pimage_comp_", i , ".pdf", sep=""))
    pimage(d, o[[i]], main = get_method(o[[i]]), key = FALSE)
    dev.off()
}
@

\begin{figure}
    \centering
    \includegraphics[width=.3\linewidth]{seriation-pimage_comp_1.pdf}
    \includegraphics[width=.3\linewidth]{seriation-pimage_comp_2.pdf}
    \includegraphics[width=.3\linewidth]{seriation-pimage_comp_3.pdf}\\
    \includegraphics[width=.3\linewidth]{seriation-pimage_comp_4.pdf}
    \includegraphics[width=.3\linewidth]{seriation-pimage_comp_5.pdf}
    \includegraphics[width=.3\linewidth]{seriation-pimage_comp_6.pdf}
    \caption{Image plot of the distance matrix for the iris data
    using rearrangement by different seriation methods.}
    \label{fig:pimage3}
\end{figure}


The first row of matrices in Figure~\ref{fig:pimage3} contains the
orders obtained by a TSP solver the rank-two ellipse seriation by Chen
and using the simulated annealing method (ARSA). The results of Chen and
ARSA are very similar (except that the order is reversed). The TSP
solver produces a smoother image with some lighter lines visible.  The
reason for these lines is that the TSP only optimizes distances locally
between two neighboring objects. Therefore it is possible that in a
quite homogeneous block several objects are enclosed gradually getting
more different and then getting more similar again (see, e.g., the light
line close to the upper left corner of the TSP image in
Figure~\ref{fig:pimage3}).

The second row of Figure~\ref{fig:pimage3} contains three images based on
hierarchical clustering. The visual impression gets better from left (just
hierarchical clustering) to right (first using the Gruvaeus Wainer heuristic
and then optimal leaf ordering to rearrange the branches of the dendrogram
obtained by hierarchical clustering). The most striking feature in the
image for hierarchical clustering (HC in Figure~\ref{fig:pimage3}) is the
distinct cross going right through the center of the plot. This indicates that
several relatively dissimilar objects are caught in an otherwise homogeneous
block. This effect vanishes after rearranging the dendrogram branches
(see GW and OLO in Figure~\ref{fig:pimage3}).

%' To investigate this effect,
%' we plot the dendrogram obtained by hierarchical clustering which is used
%' to order the objects and compare it to the dendrogram rearranged
%' using the Gruvaeus Wainer heuristic.
%'
%' <<label=pimage3_dend, eval=FALSE>>=
%' plot(o[["HC"]], labels = FALSE, main = "Dendrogram HC")
%' plot(o[["GW"]], labels = FALSE, main = "Dendrogram GW")
%' @
%' <<echo=FALSE, fig=FALSE, include=FALSE, width=9>>=
%' def.par <- par(no.readonly = TRUE)
%' pdf(file="seriation-pimage3_dendrogram.pdf", width=9, height=4)
%' layout(t(1:2))
%' plot(o[["HC"]], labels = FALSE, main = "Dendrogram HC")
%' symbols(74.7,.5, rect = matrix(c(4, 3), ncol=2), add= TRUE,
%'     inches = FALSE, lwd =2)
%'
%' plot(o[["GW"]], labels = FALSE, main = "Dendrogram GW")
%' symbols(98.7,.5, rect = matrix(c(4, 3), ncol=2), add= TRUE,
%'     inches = FALSE, lwd =2)
%' par(def.par)
%' tmp <- dev.off()
%' @
%'
%' \begin{figure}
%'     \centering
%'     \includegraphics[width=\linewidth, trim=0 80 0 0, clip=TRUE]{seriation-pimage3_dendrogram}
%'     \caption{Dendrograms for the seriation with HC and GW.}
%'     \label{fig:pimage3_dendrogram}
%' \end{figure}
%'
%' Comparing the two dendrograms in Figure~\ref{fig:pimage3_dendrogram}, we see
%' that the branch left from the top is almost unchanged. The branch which is
%' responsible for the light cross in the shaded image is highlighted by a box.
%' The Gruvaeus Wainer heuristic rotates the highlighted branch towards the right
%' since the objects in it are more similar to the objects in there.

Finally, we compare the values of the loss/merit functions
for the different seriation methods.
<<>>=
crit <- sapply(o, FUN = function(x) criterion(d, x))
t(crit)
@

<<echo=FALSE, fig=TRUE, include=FALSE, label=crit1, width=6, height=8>>=
def.par <- par(no.readonly = TRUE)
m <- c("Path_length", "AR_events", "Moore_stress")
layout(matrix(seq_along(m), ncol=1))
#tmp <- apply(crit[m,], 1, dotchart, sub = m)
tmp <- lapply(m, FUN = function(i) dotchart(crit[i,], sub = i))
par(def.par)
@

\begin{figure}
    \centering
    \includegraphics[width=14cm]{seriation-crit1}
    \caption{Comparison of different methods and seriation criteria}
    \label{fig:crit1}
\end{figure}

For easier comparison, Figure~\ref{fig:crit1} contains a plot of the criteria
Hamiltonian path length, anti-Robinson events (\code{AR\_events}) and stress
using the Moore neighborhood. Clearly, the methods which directly try to
minimize the Hamiltonian path length (hierarchical clustering with optimal leaf
ordering (\code{OLO}) and the TSP heuristic) provide the best results
concerning the path length. For the number of anti-Robinson events, using the
simulated annealing heuristic (\code{ARSA}) provides the best result.  Regarding stress, the
simulated annealing heuristic also provides the best result although, it does
not directly minimize this loss function.

\subsection{Registering new methods}
\label{sec:registering}
New methods to calculate criterion values and to compute a seriation can
be easily added by the user via the method registry mechanism provided
in \pkg{seriation}. Here we give a simple example of how to implement and
register a new seriation method.

In the registry we distinguish between methods for different types of
input data.  With the following two commands we produce a list
of the available seriation methods for input data of class \code{dist}
and \code{matrix}.

<<>>=
list_seriation_methods("dist")
list_seriation_methods("matrix")
@

To get detailed information on a seriation method use the following.
<<>>=
get_seriation_method("dist", name = "ARSA")
@

To add a new seriation method, we first have to implement the seriation code as
a function with the two formal arguments \code{x} and \code{control}, and for
arrays also an additional argument \code{margin}.
\code{x} is the data
object and \code{control} contains a list with additional information for the
method passed on from \func{seriate}. The function has to return a list
of objects which can be coerced into \code{ser\_permutation\_vector}
objects (e.g., a list of integer vectors). The elements in the list
have to be in order corresponding to the dimensions of \code{x}.

In this example we just create a method to return a permutation
which reverses the original order of the objects, i.e., which returns
the reverse identity order.

<<>>=
seriation_method_reverse <- function(x, control = NULL,
                                     margin = seq_along(dim(x))) {
          lapply(seq_along(dim(x)), function(i)
            if (i %in% margin) rev(seq(dim(x)[i]))
            else NA)
}
@

The function produces integer sequences of the correct lengths,
one for each dimension of \code{x} (\code{control} is not used).
Since the function works for \code{matrix} and \code{array} we can register
it for both data types under the short name `Reverse'.

<<>>=
set_seriation_method("matrix", "New_Reverse", seriation_method_reverse,
    "Reverse identity order")

set_seriation_method("array", "New_Reverse", seriation_method_reverse,
    "Reverse identity order")
@

Now the new seriation method is registered and can be found by the user
and applied to data.
<<>>=
list_seriation_methods("matrix")

o <- seriate(matrix(1, ncol = 3, nrow = 4), "New_Reverse")
o

get_order(o, 1)
get_order(o, 2)
@


Criterion methods can be added in the same way. We refer the interested reader
to the documentation accompanying the package for detailed information and
an example.

If you have implemented a new criterion or seriation method, please consider
submitting the code to one of the maintainers of \pkg{seriation} for
inclusion in a future release of the package.

\subsection{Heat maps}

A heat map is a shaded/color coded data matrix with a dendrogram added to one
side and to the top to indicate the order of rows and columns. Typically,
reordering is done according to row or column means within the restrictions
imposed by the dendrogram. Heat maps recently became popular for visualizing
large scale genome expression data obtained via DNA microarray technology
\citep[see, e.g.,][]{seriation:Eisen:1998}.

From Section~\ref{sec:hierarchical_clustering} we know that it is possible to
find the optimal ordering of the leaf nodes of a dendrogram which minimizes
the distances between adjacent objects in reasonable time.  Such an order might
provide an improvement over using simple reordering such as the row or column
means with respect to presentation. In \pkg{seriation} we provide
the function \func{hmap} which uses optimal ordering and can also use
seriation directly on distance matrices without using hierarchical
clustering to produce dendrograms first.

For the following example, we use again the randomly reordered iris data set
\code{x} from the examples above. To make the variables (columns) comparable,
we use standard scaling.

<<>>=
x <- scale(x, center = FALSE)
@

To produce a heat map with optimally reordered dendrograms (using
by default Optimal Leaf Ordering), the function
\func{hmap} can be used with its default settings.

<<eval=FALSE>>=
hmap(x, margin = c(7, 4), cexCol = 1, row_labels = FALSE)
@

With these settings, the
Euclidean distances between rows and between columns are calculated (with
\func{dist}), hierarchical clustering (\func{hclust}) is performed, the
resulting dendrograms are optimally reordered, and \func{heatmap.2} in package
\pkg{gplots} is used for plotting
(see Figure~\ref{fig:heatmap}(a) for the resulting plot).

<<eval=FALSE>>=
hmap(x, method = "MDS")
@

If a seriation method is used that does not depend on dendrograms, instead of hierarchical clustering,
seriation on the dissimilarity matrices for rows and columns is
performed and the reordered matrix
with the reordered dissimilarity matrices to the left and on top is
displayed (see Figure~\ref{fig:heatmap}(b)). A \code{method} argument can be used to choose different
seriation methods.

<<echo=FALSE, fig=FALSE, include=FALSE>>=
#bitmap(file = "seriation-heatmap1.png", type = "pnggray",
#    height = 6, width = 6, res = 300, pointsize=14)
pdf(file = "seriation-heatmap1.pdf")
hmap(x, margin = c(7, 4), row_labels = FALSE, cexCol = 1)
tmp <- dev.off()
@
<<echo=FALSE, fig=FALSE, include=FALSE>>=
pdf(file = "seriation-heatmap2.pdf")
hmap(x, method="MDS")
tmp <- dev.off()
@

\begin{figure}
    \begin{minipage}[b]{.48\linewidth}
        \centering
        \includegraphics[width=\linewidth]{seriation-heatmap1} \\
            (a)
    \end{minipage}
    \begin{minipage}[b]{.48\linewidth}
        \centering
        \includegraphics[width=\linewidth]{seriation-heatmap2} \\
            (b)
    \end{minipage}
    \caption{Two presentations of the rearranged iris data matrix. (a) as an
optimally reordered heat map and (b) as a seriated data matrix with reordered
dissimilarity matrices to the left and on top.}
    \label{fig:heatmap}
\end{figure}


\subsection{Bertin's permutation matrix}

\cite{seriation:Bertin:1981,seriation:Bertin:1999}
introduced permutation matrices to analyze
multivariate data with medium to low sample size.  The idea is to reveal a more
homogeneous structure in a data matrix~$\mathbf{X}$ by simultaneously
rearranging rows and columns. The rearranged matrix is displayed and cases and
variables can be grouped manually to gain a better understanding of the data.

%To quantify homogeneity, a purity function
%\begin{displaymath}
%  \phi = \Phi(\mathbf{X})
%\end{displaymath}
%is defined. Let $\Pi$ be the set of all permutation functions
%$\pi$ for matrix $\mathbf{X}$.
%Note that function $\pi$ performs row and column permutations on a matrix.
%The optimal permutation with respect to
%purity can be found by
%\begin{displaymath}
%  \pi^* = \argmax\nolimits_{\pi \in \Pi} \Phi(\pi(\mathbf{X})).
%\end{displaymath}
%Since, depending on the purity function, finding the optimal
%solution can be hard, often a near optimal solution is also acceptable
%for visualization.
%
%A possible purity function $\Phi$ is:
%Given distances between rows and columns of the data matrix, define purity as
%the sum of distances of adjacent rows/columns.  Using this purity function,
%finding the optimal permutation $\pi^*$ means solving two (independent) TSPs,
%one for the columns and one for the rows.

To find a rearrangement of columns and rows which reveals structure a
purity function is used. A possible purity function is: Given distances between rows and columns of the data matrix, define purity as
the sum of distances of adjacent rows/columns. Using this purity function,
finding the optimal permutation means solving two (independent) TSPs,
one for the columns and one for the rows which can be done very conveniently
using the infrastructure provided by \pkg{seriation}.

As an example, we use the results of $8$ constitutional referenda for $41$
Irish communities~\citep{seriation:Falguerolles:1997}\footnote{The Irish data
set is included in this package. The original data and the text of the
referenda can be obtained from~\url{http://www.electionsireland.org/}}.  To
make values comparable across columns (variables), the ranks of the values for
each variable are used instead of the original values.

<<>>=
data("Irish")
orig_matrix <- apply(Irish[,-6], 2, rank)
@

For seriation, we calculate distances between rows and between columns using
the sum of absolute rank differences (this is equal to the Minkowski distance
with power $1$). Then we apply seriation (using a TSP heuristic) to both
distance matrices and combine the two resulting \code{ser\_permutation} objects
into one object for two-mode data. The original and the reordered matrix are
plotted using \func{bertinplot}.

<<>>=
o <- c(
    seriate(dist(orig_matrix, "minkowski", p = 1), method = "TSP"),
    seriate(dist(t(orig_matrix), "minkowski", p = 1), method = "TSP")
)
o
@

In a newer version of the package this can be also done with the new heatmap seriation method for matrices.
<<>>=
get_seriation_method("matrix", name = "heatmap")

o <- seriate(orig_matrix, method = "heatmap", dist_fun = function(d) dist(d, "minkowski", p = 1),
             seriation_method = "TSP")
o
@



<<eval=FALSE>>=
bertinplot(orig_matrix)
bertinplot(orig_matrix, o)
@
<<echo=FALSE, fig=TRUE, include=FALSE, label=bertin1, width=10>>=
bertinplot(orig_matrix)
@
<<echo=FALSE, fig=TRUE, include=FALSE, label=bertin2, width=10>>=
bertinplot(orig_matrix, o)
@
\begin{figure}
    \centering
    \includegraphics[width=15cm, trim=60 60 0 0]{seriation-bertin1} \\
    (a)

    \includegraphics[width=15cm, trim=60 60 0 0]{seriation-bertin2} \\
    (b)
    \caption{Bertin plot for the (a) original arrangement and the (b)
    reordered Irish data set.}
    \label{fig:bertin}
\end{figure}



The original matrix and the rearranged matrix are shown in
Figure~\ref{fig:bertin} as a matrix of bars where high values are highlighted
(filled blocks).  Note that following Bertin, the cases (communities) are
displayed as the columns and the variables (referenda) as rows.  Depending on
the number of cases and variables, columns and rows can be exchanged to obtain
a better visualization.

Although the columns are already ordered (communities in the same city
appear consecutively) in the original data matrix in
Figure~\ref{fig:bertin}(a), it takes some effort to find structure in
the data.  For example, it seems that the variables `Marriage',
`Divorce', `Right to Travel' and `Right to Information' are correlated
since the values are all high in the block made up by the columns of the
communities in Dublin.  The reordered matrix confirms this but makes the
structure much more apparent. Especially the contribution of low values
(which are not highlighted) to the overall structure becomes only
visible after rearrangement.

\subsection{Binary data matrices}

Binary or $0$-$1$ data matrices are quite common.  Often such matrices are
called \emph{incidence matrices} since a $1$ in a cell indicates the incidence
of an event. In archeology such an event could be that a special type of
artifact was found at a certain archaeological site.
This can be seen as a simplification of a so-called \emph{abundance matrix}
which codes in each cell the (relative) frequency or quantity of an artifact
type at a site.  See \cite{seriation:Ihm:2005} for a comparison of
incidence and abundance matrices in archeology.

Here we are interested in binary data.
For the example we use an artificial data set from~\cite{seriation:Bertin:1981}
called \emph{Townships}.  The data set contains $9$ binary characteristics
(e.g., has a veterinary or has a high school) for $16$ townships. The idea of
the data set is that townships evolve from a rural to an urban environment over
time.

After loading the data set (which comes with the package), we use
\func{bertinplot} to visualize the data (\func{pimage} could also be used
but \func{bertinplot} allows for a nicer visualization).
Bars, the standard visualization of \func{bertinplot}, do not
make much sense for binary data. We therefore use the
panel function \func{panel.squares} without spacing
to plot black squares.

<<fig=TRUE, include=FALSE, label=binary1, width=9>>=
data("Townships")

bertinplot(Townships, panel = panel.tiles)
@

The original data in Figure~\ref{fig:binary}(a) does not reveal structure in
the data. To improve the display, we run the  bond energy algorithm (BEA) for
columns and rows $10$ times with random starting points and report the best
solution.

<<echo=FALSE>>=
## to get consistent results
set.seed(10)
@

<<fig=TRUE, include=FALSE, label=binary2, width=9>>=
o <- seriate_rep(Townships, method = "BEA", criterion = "ME", rep = 10)
bertinplot(Townships, o, panel = panel.tiles)
@

The reordered matrix is displayed in Figure~\ref{fig:binary}(b).  A
clear structure is visible.  The variables (rows in a Bertin plot) can be
split into the three categories describing different evolution states of
townships:
\begin{enumerate}
    \item Rural: No doctor, one-room school and possibly also
        no water supply
    \item Intermediate: Land reallocation, veterinary and agricultural
        cooperative
    \item Urban: Railway station, high school and police station
\end{enumerate}

The townships also clearly fall into these three groups which
tentatively can be called villages (first~$7$), towns (next~5) and
cities (final~2).  The townships B and C are on the transition to the
next higher group.

\begin{figure}
    \centering
    \includegraphics[width=12cm, trim=0 40 0 30]{seriation-binary1} \\
    (a)

    \includegraphics[width=12cm, trim=0 40 0 30]{seriation-binary2} \\
    (b)

    \caption{The townships data set in original order (a) and
    reordered using BEA (b).}
    \label{fig:binary}
\end{figure}


<<>>=
rbind(
  original = criterion(Townships),
  reordered = criterion(Townships, o)
)
@

BEA tries to maximize the measure of effectiveness which is
much higher in the reordered matrix (in fact, 65 is the maximum for
the data set). Also the two types of stress are improved
significantly.

\subsection{Dissimilarity plot}

Assessing the quality of an obtained cluster solution has been a
research topic since the invention of cluster analysis. This is
especially important since all popular cluster algorithms produce a
clustering even for data without a ``cluster'' structure.

%A method to judge the quality of a cluster solution is by inspecting a
%visualization. For hierarchical clustering
%dendrogramms~\cite{seriation:Hartigan:1967} are available which show the
%hierarchical structure of the clustering as a binary tree and cluster quality
%can be judged by looking at the dissimilarities between objects in a cluster
%and objects in other clusters. However, such a visualization is
%only possible for heirarchical/nested clusterings.
%
%\marginpar{Cite Pison et al 1999 and Kaufmann and Rousseeuw}
%For the an arbitrary partitional clustering, the original objects can
%be displayed in a 2 dimensional scatter plot
%after using dimensionality reduction (e.g., PCA, MDS).
%Objects belonging to the same cluster can be marked and thus, if the
%dimensionality reduction preserves a large proportion of the
%variavility in the original data, the separation between clusters can be
%visually judged.
%
%Silhouettes

Matrix shading is an old technique to visualize clusterings by
displaying the rearranged matrices~\citep[see,
e.g.,][]{seriation:Sneath:1973,seriation:Ling:1973,seriation:Gale:1984}.
Initially matrix shading was used in connection with hierarchical
clustering, where the order of the dendrogram leaf nodes was used to
arrange the matrix.  However, with some extensions, matrix shading can
also be used with any partitional clustering method.

\cite{seriation:Strehl:2003} suggest a matrix shading visualization called
\emph{CLUSION} where the dissimilarity matrix is arranged such that all objects
pertaining to a single cluster appear in consecutive order in the matrix. The
authors call this \emph{coarse seriation}. The result of a ``good'' clustering
should be a matrix with low dissimilarity values forming blocks around the main
diagonal. However, using coarse seriation, the order of the clusters has to be
predefined and the objects within each cluster are unordered.

The dissimilarity plots implemented by the function \func{dissplot} in
\pkg{seriation} improve \emph{CLUSION} using seriation methods.  It aims
at visualizing global structure (similarity between different clusters
is reflected by their position relative to each other) as well as the
micro structure within each cluster (position of objects).

To position the clusters in the dissimilarity plot, an inter-cluster
dissimilarity matrix is calculated using the average between cluster
dissimilarities. \func{seriate} is used on this inter-cluster dissimilarity
matrix to arrange the clusters relative to each other resulting in on average
more similar clusters to appear closer together in the plot.  Within each
cluster, \func{seriate} is used again on the sub-matrix of the dissimilarity
matrix concerning only the objects in the cluster.

For the example, we use again Euclidean distance between the objects in the
iris data set.

<<>>=
data("iris")
iris <- iris[sample(seq_len(nrow(iris))), ]
x_iris <- iris[, -5]
d_iris <- dist(x_iris, method = "euclidean")
@

First, we use \func{dissplot} without a clustering. We set \code{method} to
\code{NA} to prevent reordering and display the original matrix (see
Figure~\ref{fig:dissplot1}(a)). Then we omit the method argument which results
in using the default seriation technique from \func{seriate}. Since we
did not provide a clustering, the whole matrix is reordered
in one piece.  From the result shown in Figure~\ref{fig:dissplot1}(b) it seems
that there is a clear structure in the data which suggests a two cluster
solution.

<<eval=FALSE, label=dissplot1>>=
## plot original matrix
dissplot(d_iris, method = NA)
@

<<eval=FALSE, label=dissplot2>>=
## plot reordered matrix
dissplot(d_iris, main = "Dissimilarity plot with seriation")
@

<<echo=FALSE, fig=FALSE, include=FALSE>>=
pdf(file = "seriation-dissplot1.pdf")
<<dissplot1>>
tmp <- dev.off()
pdf(file = "seriation-dissplot2.pdf")
<<dissplot2>>
tmp <- dev.off()
@


\begin{figure}
    \begin{minipage}[b]{.48\linewidth}
    \centering
    \includegraphics[width=\linewidth]{seriation-dissplot1} \\
    (a)
    \end{minipage}
    \begin{minipage}[b]{.48\linewidth}
    \centering
    \includegraphics[width=\linewidth]{seriation-dissplot2} \\
    (b)
    \end{minipage}
    \caption{Two dissimilarity plots.
    (a) the original dissimilarity matrix and
    (b) the seriated dissimilarity matrix.}
    \label{fig:dissplot1}
\end{figure}

Next, we create a cluster solution using the $k$-means algorithm.
Although we know
that the data set should contain $3$ groups representing the three species
of iris, we let $k$-means produce a $10$ cluster solution to study how such a
misspecification can be spotted using \func{dissplot}.

<<echo=FALSE>>=
set.seed(1234)
@
<<>>=
l <- kmeans(x_iris, 10)$cluster
#$
@

We create a standard dissimilarity plot by providing the cluster
solution as a vector of labels. The function rearranges the matrix and
plots the result. Since rearrangement can be a time consuming procedure for
large matrices, the rearranged matrix and all
information needed for plotting is returned as the result.

<<eval=FALSE, label=dissplot3>>=
res <- dissplot(d_iris, labels = l,
    main = "Dissimilarity plot - standard")
@
<<echo=FALSE, fig=FALSE, include=FALSE>>=
pdf(file = "seriation-dissplot3.pdf")

## visualize the clustering
<<dissplot3>>
tmp <- dev.off()


pdf(file = "seriation-dissplot4.pdf")
## threshold
plot(res, main = "Dissimilarity plot - threshold",
    threshold = 3)

tmp <- dev.off()
@

\begin{figure}
    \centering
    \includegraphics[width=10cm]{seriation-dissplot3}\\
    (a)

    \includegraphics[width=10cm]{seriation-dissplot4}\\
    (b)
    \caption{Dissimilarity plot for $k$-means solution with 10 clusters.
    (a) standard plot and (b) plot with threshold.}
    \label{fig:dissplot3}
\end{figure}

<<>>=
res
@

The resulting plot is shown in Figure~\ref{fig:dissplot3}(a).  The
inter-cluster dissimilarities are shown as solid
gray blocks and the average object
dissimilarity within each cluster as gray triangles below the main diagonal of
the matrix. Since the clusters are arranged such that more similar clusters
are closer together, it is easy to see in Figure~\ref{fig:dissplot3}(a)
that clusters 6, 3 and 1 as well as clusters 10, 9, 5, 7, 8, 4 and 2
are very similar and form two blocks.
This suggests again that a two cluster solution would
be reasonable.

Since slight variations of gray values are hard to distinguish,
we plot the matrix again (using \func{plot} on the result above) and
use a threshold on the dissimilarity to suppress high dissimilarity
values in the plot.

<<eval=FALSE>>=
plot(res, options = list(main = "Seriation - threshold",
    threshold = 3))
@

In the resulting plot in Figure~\ref{fig:dissplot3}(b), we see that the
block containing 10, 9, 5, 7, 8, 4 and 2 is very well defined and
cleanly separated from the other block. This suggests that these clusters
should form together a cluster in a solution with less clusters.
The other block is less well defined. There is considerable overlap between
clusters 6 and 3, but also cluster 3 and 1 share similar objects.

Using the information stored in the result of \func{dissplot} and
the class information available for the iris data set, we can analyze
the cluster solution and the interpretations of the dissimilarity plot.

<<>>=
#names(res)
table(iris[res$order, 5], res$label)[,res$cluster_order]
#$
@

As the plot in Figure~\ref{fig:dissplot3} indicated, the clusters 10, 9, 5, 7,
8, 4 and 2 should be a single cluster containing only flowers of the species
Iris setosa. The clusters 6, 3 and 1 are more problematic since they contain a
mixture of Iris versicolor and virginica.

To illustrate the results of the dissimilarity plot in case a clustering
with a $k$ smaller than the actual number of groups in the data is used,
we use the Ruspini data set which consists of 75 points in four groups and
is also often used to illustrate clustering techniques. We load the data set,
calculate distances, perform $k$-means clustering with $k=3$ (although the
real number of groups is 4) and produce
a dissimilarity plot.

<<label=ruspini, fig=TRUE, include=FALSE>>=
data("ruspini", package = "cluster")
d <- dist(ruspini)
l <- kmeans(ruspini, 3)$cluster
dissplot(d, labels = l)
@

\begin{figure}
    \centering
    \includegraphics[width=10cm]{seriation-ruspini}\\
    \caption{Dissimilarity plot for $k$-means solution with 3 clusters
    for the Ruspini data set with 4 groups.}
    \label{fig:ruspini}
\end{figure}

The dissimilarity plot in Figure~\ref{fig:ruspini} shows that
cluster 3 actually should be two separate clusters represented
by the two clearly visible darker triangles next to the main diagonal.

The dissimilarity plot using seriation is a useful tool to inspect the result
of clustering. It is especially useful to spot
misspecifications of the number of clusters employed.
A more detailed treatment of dissimilarity plots as a tool for exploring
partitional clustering can be found in
\cite{seriation:Hahsler+Kornik:2011}.

\section{Conclusion}
\label{sec:conclusion}

In this paper we presented the infrastructure provided by the
package~\pkg{seriation}.  The infrastructure contains the necessary data
structures to store the linear order for one-, two- and $k$-mode data.
It also provides a wide array of seriation methods for different input
data, e.g., dissimilarities, binary and general data matrices focusing
on combinatorial optimization.  New seriation methods can be easily
incorporated into the \pkg{seriation} framework by the user with the
method registry mechanism provided.

Based on seriation, \pkg{seriation} features several visualization
techniques.  In particular, the optimally reordered heat map, the
Bertin plot and the dissimilarity plot present clear improvements over
standard plots.

A natural extension to \pkg{seriation} is the synthesis of ensembles of
seriations into a ``consensus'' one.  Such ensembles do not only arise
when using different seriation methods, but also when varying data or
control parameters to obtain more robust solutions (see
e.g.~\cite{seriation:Jurman:2008} for a recent application of such ideas
in a molecular profiling context).  The \proglang{R}~extension package
\pkg{relations}~\citep{seriation:Hornik+Meyer:2008} contains a variety
of methods for obtaining consensus \emph{relations}, covering consensus
seriation (where the relations are linear orders on the objects) as a
special case.

Future work on \pkg{seriation} will focus on adding further seriation
methods, such as for example methods for higher dimensional arrays and
methods for block seriation which aim at finding simultaneous partitions
of rows and columns in a data matrix~\citep[see,
e.g.,][]{seriation:Marcotorchino:1987}.

\section*{Acknowledgments}

The authors would like to thank Michael Brusco, Hans-Friedrich K{\"o}hn
and Stephanie Stahl for their seriation code, Fionn Murtagh for his BEA
implementation and the anonymous reviewers for their valuable comments
and suggestions.

%
%\bibliographystyle{abbrvnat}
\bibliography{seriation}
%
\end{document}

