---
title: "Training neural networks"
author: "Maximilian Pichler"
date: "2023-09-29"
abstract: "This vignette helps to address certain problems that occur when training neural networks (NN) and gives hints on how to increase the likelihood of their convergence."
output:
 rmarkdown::html_vignette:
    toc: true
    toc_depth: 4
    html_document:
      toc: true
      theme: cerulean
vignette: >
  %\VignetteIndexEntry{Training neural networks}
  %\VignetteEncoding{UTF-8}
  %\VignetteEngine{knitr::rmarkdown}
editor_options:
  chunk_output_type: console
---




## Possible issues

-   Convergence issues, (often because of the learning rate), **training loss above baseline loss**:

    <div class="figure" style="text-align: center">
    <img src="B/B-unnamed-chunk-2-1.png" alt="plot of chunk unnamed-chunk-2"  />
    <p class="caption">plot of chunk unnamed-chunk-2</p>
    </div>

    If it looks like that, go to the [adjusting the learning rate section](#lr)

-   Overfitting, difference between training and testing/holdout/new data error is too high, or validation loss starts to increase again at some point during the training

    <div class="figure" style="text-align: center">
    <img src="B/B-unnamed-chunk-3-1.png" alt="plot of chunk unnamed-chunk-3"  />
    <p class="caption">plot of chunk unnamed-chunk-3</p>
    </div>

    if it loos like that, go to the [overfitting section](#overfitting)

## Convergence issues {#lr}

Ensuring convergence can be tricky when training neural networks. Their training is sensitive to a combination of the learning rate (how much the weights are updated in each optimization step), the batch size (a random subset of the data is used in each optimization step), and the number of epochs (number of optimization steps).

### Epochs

Give the neural network enough time to learn. The epochs should be high enough so that the training loss "stabilizes":


```r
m = dnn(Species~., data = iris, epochs = 10L, loss = "softmax", verbose=FALSE)
```

<div class="figure" style="text-align: center">
<img src="B/B-unnamed-chunk-4-1.png" alt="plot of chunk unnamed-chunk-4"  />
<p class="caption">plot of chunk unnamed-chunk-4</p>
</div>

After 10 epochs the loss was still decreasing, let's run the model for more epochs:


```r
m = dnn(Species~., data = iris, epochs = 200L, loss = "softmax", verbose=FALSE)
```

<div class="figure" style="text-align: center">
<img src="B/B-unnamed-chunk-5-1.png" alt="plot of chunk unnamed-chunk-5"  />
<p class="caption">plot of chunk unnamed-chunk-5</p>
</div>

It takes around 190-200 epochs until the loss doesn't decrease anymore. The "speed" of the learning depends also on the learning rate. Higher rates means larger steps into direction of the minima of the loss function:


```r
m = dnn(Species~., data = iris, epochs = 200L, loss = "softmax", lr = 0.05, verbose=FALSE)
```

<div class="figure" style="text-align: center">
<img src="B/B-unnamed-chunk-6-1.png" alt="plot of chunk unnamed-chunk-6"  />
<p class="caption">plot of chunk unnamed-chunk-6</p>
</div>

Now it only takes about 100 epochs, but we also see that the training loss becomes wobbly. Larger learning rates increase the probability that local minima are skipped and the optimizer has problems to hit a minima.

### Learning rate

Typically, the learning rate should be decreased with the size of the neural networks (depth of the network and width of the hidden layers). We provide a baseline loss (intercept only model) that can give hints about an appropriate learning rate.


```r
nn.fit_good<- dnn(Species~., data = datasets::iris, lr = 0.09, epochs = 20L, loss = "softmax", verbose = FALSE, plot = FALSE)
nn.fit_high<- dnn(Species~., data = datasets::iris, lr = 2.09, epochs = 20L, loss = "softmax", verbose = FALSE, plot = FALSE)
nn.fit_low<- dnn(Species~., data = datasets::iris, lr = 0.00000001, epochs = 20L, loss = "softmax", verbose = FALSE, plot = FALSE)

par(mfrow = c(1, 3), mar = c(4, 3, 2, 2))
cito:::visualize.training(nn.fit_good$losses, epoch = 20, new = TRUE, baseline = nn.fit_good$base_loss)
cito:::visualize.training(nn.fit_high$losses, epoch = 20, new = TRUE, baseline = nn.fit_good$base_loss)
cito:::visualize.training(nn.fit_low$losses, epoch = 20, new = TRUE, baseline = nn.fit_good$base_loss)
```

<div class="figure" style="text-align: center">
<img src="B/B-unnamed-chunk-7-1.png" alt="plot of chunk unnamed-chunk-7"  />
<p class="caption">plot of chunk unnamed-chunk-7</p>
</div>

If the training loss of the model doesn't fall below the baseline loss, the learning rate is either too high or too low. If this happens, try higher and lower learning rates.

A common strategy is to try (manually) a few different learning rates to see if the learning rate is on the right scale.

### Solution: learning rate scheduler

A common strategy to deal with the learning rate problem is to start with a high learning rate, and if the loss does not decrease, the learning rate is reduced according to a specific plan.

I favor the "reduce learning rate on plateau" scheduler. If a loss plateau isn't resolved for a certain number of epochs (patience), the learning rate will be reduced ($lr_{new} = factor * lr_{old}$):


```r
nn.fit_high<- dnn(Species~., data = datasets::iris,
                  lr = 0.2,
                  epochs = 60L,
                  loss = "softmax",
                  lr_scheduler = config_lr_scheduler("reduce_on_plateau", patience = 5, factor = 0.5),
                  verbose = TRUE,
                  plot = TRUE)
#> Loss at epoch 1: 1.172509, lr: 0.20000
```

<div class="figure" style="text-align: center">
<img src="B/B-unnamed-chunk-8-1.png" alt="plot of chunk unnamed-chunk-8"  />
<p class="caption">plot of chunk unnamed-chunk-8</p>
</div>

```
#> Loss at epoch 2: 0.781517, lr: 0.20000
#> Loss at epoch 3: 1.024170, lr: 0.20000
#> Loss at epoch 4: 0.768242, lr: 0.20000
#> Loss at epoch 5: 0.514967, lr: 0.20000
#> Loss at epoch 6: 0.528964, lr: 0.20000
#> Loss at epoch 7: 0.430172, lr: 0.20000
#> Loss at epoch 8: 0.579465, lr: 0.20000
#> Loss at epoch 9: 0.483767, lr: 0.20000
#> Loss at epoch 10: 0.372494, lr: 0.20000
#> Loss at epoch 11: 0.726314, lr: 0.20000
#> Loss at epoch 12: 0.385350, lr: 0.20000
#> Loss at epoch 13: 0.586158, lr: 0.20000
#> Loss at epoch 14: 0.580966, lr: 0.20000
#> Loss at epoch 15: 0.359812, lr: 0.20000
#> Loss at epoch 16: 0.512657, lr: 0.20000
#> Loss at epoch 17: 0.350033, lr: 0.20000
#> Loss at epoch 18: 0.610410, lr: 0.20000
#> Loss at epoch 19: 0.328625, lr: 0.20000
#> Loss at epoch 20: 0.487544, lr: 0.20000
#> Loss at epoch 21: 0.374787, lr: 0.20000
#> Loss at epoch 22: 0.660259, lr: 0.20000
#> Loss at epoch 23: 0.353881, lr: 0.20000
#> Loss at epoch 24: 0.244274, lr: 0.20000
#> Loss at epoch 25: 0.475878, lr: 0.20000
#> Loss at epoch 26: 0.325557, lr: 0.20000
#> Loss at epoch 27: 0.510104, lr: 0.20000
#> Loss at epoch 28: 0.255266, lr: 0.20000
#> Loss at epoch 29: 0.581988, lr: 0.20000
#> Loss at epoch 30: 0.340278, lr: 0.10000
#> Loss at epoch 31: 0.306466, lr: 0.10000
#> Loss at epoch 32: 0.285408, lr: 0.10000
#> Loss at epoch 33: 0.278103, lr: 0.10000
#> Loss at epoch 34: 0.273333, lr: 0.10000
#> Loss at epoch 35: 0.249927, lr: 0.10000
#> Loss at epoch 36: 0.215362, lr: 0.10000
#> Loss at epoch 37: 0.198717, lr: 0.10000
#> Loss at epoch 38: 0.187999, lr: 0.10000
#> Loss at epoch 39: 0.221773, lr: 0.10000
#> Loss at epoch 40: 0.372954, lr: 0.10000
#> Loss at epoch 41: 0.358555, lr: 0.10000
#> Loss at epoch 42: 0.205245, lr: 0.10000
#> Loss at epoch 43: 0.199998, lr: 0.10000
#> Loss at epoch 44: 0.163184, lr: 0.10000
#> Loss at epoch 45: 0.259405, lr: 0.10000
#> Loss at epoch 46: 0.298692, lr: 0.10000
#> Loss at epoch 47: 0.319238, lr: 0.10000
#> Loss at epoch 48: 0.219220, lr: 0.10000
#> Loss at epoch 49: 0.189692, lr: 0.10000
#> Loss at epoch 50: 0.126413, lr: 0.10000
#> Loss at epoch 51: 0.195314, lr: 0.10000
#> Loss at epoch 52: 0.113725, lr: 0.10000
#> Loss at epoch 53: 0.113329, lr: 0.10000
#> Loss at epoch 54: 0.208422, lr: 0.10000
#> Loss at epoch 55: 0.161362, lr: 0.10000
#> Loss at epoch 56: 0.222583, lr: 0.10000
#> Loss at epoch 57: 0.577362, lr: 0.10000
#> Loss at epoch 58: 0.115498, lr: 0.10000
#> Loss at epoch 59: 0.137783, lr: 0.05000
#> Loss at epoch 60: 0.113263, lr: 0.05000
```

At the end of the training, the learning rate is 0.025

Note: The learning rate scheduler is a powerful approach to improve the likeliness of convergence, BUT it cannot help with much too high learning rates!


```r
nn.fit_high<- dnn(Species~., data = datasets::iris,
                  lr = 2,
                  epochs = 60L,
                  loss = "softmax",
                  lr_scheduler = config_lr_scheduler("reduce_on_plateau", patience = 5, factor = 0.5),
                  verbose = TRUE,
                  plot = TRUE)
#> Loss at epoch 1: 355.402261, lr: 2.00000
```

<div class="figure" style="text-align: center">
<img src="B/B-unnamed-chunk-9-1.png" alt="plot of chunk unnamed-chunk-9"  />
<p class="caption">plot of chunk unnamed-chunk-9</p>
</div>

```
#> Loss at epoch 2: 237.208636, lr: 2.00000
#> Loss at epoch 3: 1.134408, lr: 2.00000
#> Loss at epoch 4: 1.109911, lr: 2.00000
#> Loss at epoch 5: 1.126410, lr: 2.00000
#> Loss at epoch 6: 1.113606, lr: 2.00000
#> Loss at epoch 7: 1.111348, lr: 2.00000
#> Loss at epoch 8: 1.149916, lr: 2.00000
#> Loss at epoch 9: 1.142611, lr: 2.00000
#> Loss at epoch 10: 1.140599, lr: 1.00000
#> Loss at epoch 11: 1.125465, lr: 1.00000
#> Loss at epoch 12: 1.112052, lr: 1.00000
#> Loss at epoch 13: 1.105067, lr: 1.00000
#> Loss at epoch 14: 1.105355, lr: 1.00000
#> Loss at epoch 15: 1.114075, lr: 1.00000
#> Loss at epoch 16: 1.105306, lr: 1.00000
#> Loss at epoch 17: 1.113121, lr: 1.00000
#> Loss at epoch 18: 1.103184, lr: 1.00000
#> Loss at epoch 19: 1.101075, lr: 1.00000
#> Loss at epoch 20: 1.103349, lr: 1.00000
#> Loss at epoch 21: 1.113014, lr: 1.00000
#> Loss at epoch 22: 1.108363, lr: 1.00000
#> Loss at epoch 23: 1.111946, lr: 1.00000
#> Loss at epoch 24: 1.104911, lr: 1.00000
#> Loss at epoch 25: 1.114638, lr: 0.50000
#> Loss at epoch 26: 1.102755, lr: 0.50000
#> Loss at epoch 27: 1.102358, lr: 0.50000
#> Loss at epoch 28: 1.103404, lr: 0.50000
#> Loss at epoch 29: 1.100128, lr: 0.50000
#> Loss at epoch 30: 1.102469, lr: 0.50000
#> Loss at epoch 31: 1.104513, lr: 0.50000
#> Loss at epoch 32: 1.103203, lr: 0.50000
#> Loss at epoch 33: 1.103002, lr: 0.50000
#> Loss at epoch 34: 1.099864, lr: 0.50000
#> Loss at epoch 35: 1.103847, lr: 0.50000
#> Loss at epoch 36: 1.104482, lr: 0.50000
#> Loss at epoch 37: 1.101944, lr: 0.50000
#> Loss at epoch 38: 1.104321, lr: 0.50000
#> Loss at epoch 39: 1.101755, lr: 0.50000
#> Loss at epoch 40: 1.103744, lr: 0.25000
#> Loss at epoch 41: 1.100082, lr: 0.25000
#> Loss at epoch 42: 1.102116, lr: 0.25000
#> Loss at epoch 43: 1.104146, lr: 0.25000
#> Loss at epoch 44: 1.103228, lr: 0.25000
#> Loss at epoch 45: 1.100558, lr: 0.25000
#> Loss at epoch 46: 1.101082, lr: 0.12500
#> Loss at epoch 47: 1.099932, lr: 0.12500
#> Loss at epoch 48: 1.099542, lr: 0.12500
#> Loss at epoch 49: 1.099541, lr: 0.12500
#> Loss at epoch 50: 1.099415, lr: 0.12500
#> Loss at epoch 51: 1.099269, lr: 0.12500
#> Loss at epoch 52: 1.099922, lr: 0.12500
#> Loss at epoch 53: 1.099074, lr: 0.12500
#> Loss at epoch 54: 1.100799, lr: 0.12500
#> Loss at epoch 55: 1.099098, lr: 0.12500
#> Loss at epoch 56: 1.101013, lr: 0.12500
#> Loss at epoch 57: 1.099656, lr: 0.12500
#> Loss at epoch 58: 1.099287, lr: 0.12500
#> Loss at epoch 59: 1.099364, lr: 0.06250
#> Loss at epoch 60: 1.099803, lr: 0.06250
```

Although the learning rate ended up being 0.01562, the loss never outperformed the baseline loss. The optimizer jumped right at the beginning into a completely unrealistic solution space for the parameters of the NN, from which we could not recover.

## Overfitting {#overfitting}

Overfitting means that the model fits the training data well, but generalizes poorly to new observations. We can use the validation argument to detect overfitting. If the validation loss starts to increase again at a certain point, it often means that the models are starting to overfit your training data:


```r
library(EcoData) # can be install from github using devtools::install_github(repo = "TheoreticalEcology/EcoData", dependencies = FALSE, build_vignettes = FALSE)
df = elephant$occurenceData
m = dnn(Presence~., data = df, lr = 0.03, epochs = 600L, loss = "binomial", validation = 0.2,  hidden = c(350L, 350L, 350L), activation = "relu", batchsize = 150L, verbose = FALSE, plot = TRUE)
```

<div class="figure" style="text-align: center">
<img src="B/B-unnamed-chunk-10-1.png" alt="plot of chunk unnamed-chunk-10"  />
<p class="caption">plot of chunk unnamed-chunk-10</p>
</div>

**Solutions**:

-   Re-train with epochs = point where model started to overfit

-   Early stopping, stop training when model starts to overfit, can be specified using the `⁠early_stopping=…⁠` argument

-   Use regularization (dropout or elastic-net, see next section)

### Early stopping and regularization

Early stopping = stop training when validation loss cannot be improved for x epochs (if there is no validation split, the training loss is used).

lambda = 0.001 is the regularization strength and alpha = 0.2 means that 20% L1 and 80% L2 weighting.


```r
m = dnn(Presence~., data = df, lr = 0.03, epochs = 600L, loss = "binomial", validation = 0.2,  hidden = c(350L, 350L, 350L), activation = "relu", batchsize = 150L, verbose = FALSE, plot = TRUE, early_stopping = 10, lambda = 0.001, alpha = 0.2)
```

<div class="figure" style="text-align: center">
<img src="B/B-unnamed-chunk-11-1.png" alt="plot of chunk unnamed-chunk-11"  />
<p class="caption">plot of chunk unnamed-chunk-11</p>
</div>

The training is aborted!
