21 - Concentration and covariance matrix in an autoregressive model and in a dynamic linear model

0.1 Autoregressive model (\(AR(1)\))

Consider this model: \[ x_i = a x_{i-1} + e_i, \quad i=1, \dots, 3 \] and \(x_0=e_0\). All terms \(e_0, \dots, e_3\) are independent and \(N(0,v^2)\) distributed. Let \(e=(e_0, \dots, e_3)\) and \(x=(x_0, \dots x_3)\). Hence \(e \sim N(0, v^2 I)\). Isolating error terms gives \[ e= \left[\begin{matrix}e_{0}\\e_{1}\\e_{2}\\e_{3}\end{matrix}\right] = \left[\begin{matrix}1 & 0 & 0 & 0\\- a & 1 & 0 & 0\\0 & - a & 1 & 0\\0 & 0 & - a & 1\end{matrix}\right] \left[\begin{matrix}x_{0}\\x_{1}\\x_{2}\\x_{3}\end{matrix}\right] = L_1 x \]

N <- 3
L1 <- diag(4)
L1[cbind(1 + (1:N), 1:N)] <- "-a"
L1 <- as_sym(L1)

Since \(\mathbf{Var}(e)=v^2 I\) we have \(\mathbf{Var}(e)=v^2 I=L \mathbf{Var}(x) L'\) so the covariance matrix of \(x\) is \(V_1=\mathbf{Var}(x) = v^2 L^- (L^-)'\) while the concentration matrix (the inverse covariances matrix) is \(K=v^{-2}L' L\).

def_sym(v2)
L1inv <- inv(L1)
V1 <- v2 * L1inv %*% t(L1inv)
K1 <- (t(L1) %*% L1) / v2

\[\begin{align} K_1 &= \left[\begin{matrix}\frac{a^{2} + 1}{v_{2}} & - \frac{a}{v_{2}} & 0 & 0\\- \frac{a}{v_{2}} & \frac{a^{2} + 1}{v_{2}} & - \frac{a}{v_{2}} & 0\\0 & - \frac{a}{v_{2}} & \frac{a^{2} + 1}{v_{2}} & - \frac{a}{v_{2}}\\0 & 0 & - \frac{a}{v_{2}} & \frac{1}{v_{2}}\end{matrix}\right] \\ V_1 &= \left[\begin{matrix}v_{2} & a v_{2} & a^{2} v_{2} & a^{3} v_{2}\\a v_{2} & v_{2} \left(a^{2} + 1\right) & v_{2} \left(a^{3} + a\right) & v_{2} \left(a^{4} + a^{2}\right)\\a^{2} v_{2} & v_{2} \left(a^{3} + a\right) & v_{2} \left(a^{4} + a^{2} + 1\right) & v_{2} \left(a^{5} + a^{3} + a\right)\\a^{3} v_{2} & v_{2} \left(a^{4} + a^{2}\right) & v_{2} \left(a^{5} + a^{3} + a\right) & v_{2} \left(a^{6} + a^{4} + a^{2} + 1\right)\end{matrix}\right] \end{align}\]

0.2 Dynamic linear model

Augment the \(AR(1)\) process above with \(y_i=b x_i + u_i\) for \(i=1,2,3\). Suppose \(u_i\sim N(0, w^2)\) and all \(u_i\) are independent and inpendent of \(e\). Then \((e,u)\) can be expressed in terms of \((x,y)\) as \[ (e,u) = \left[\begin{matrix}e_{0}\\e_{1}\\e_{2}\\e_{3}\\u_{1}\\u_{2}\\u_{3}\end{matrix}\right] = \left[\begin{matrix}1 & 0 & 0 & 0 & 0 & 0 & 0\\- a & 1 & 0 & 0 & 0 & 0 & 0\\0 & - a & 1 & 0 & 0 & 0 & 0\\0 & 0 & - a & 1 & 0 & 0 & 0\\0 & - b & 0 & 0 & 1 & 0 & 0\\0 & 0 & - b & 0 & 0 & 1 & 0\\0 & 0 & 0 & - b & 0 & 0 & 1\end{matrix}\right] \left[\begin{matrix}x_{0}\\x_{1}\\x_{2}\\x_{3}\\y_{1}\\y_{2}\\y_{3}\end{matrix}\right] = L_2 (x,y) \] where

N <- 3
L2 <- diag("1", 1 + 2*N)
L2[cbind(1 + (1:N), 1:N)] <- "-a"
L2[cbind(1 + N + (1:N), 1 + 1:N)] <- "-b"
L2 <- as_sym(L2)

Veu <- diag(1, 7)
diag(Veu)[1:4] <- "v2"
diag(Veu)[5:7] <- "w2"
Veu
#>      [,1] [,2] [,3] [,4] [,5] [,6] [,7]
#> [1,] "v2" "0"  "0"  "0"  "0"  "0"  "0" 
#> [2,] "0"  "v2" "0"  "0"  "0"  "0"  "0" 
#> [3,] "0"  "0"  "v2" "0"  "0"  "0"  "0" 
#> [4,] "0"  "0"  "0"  "v2" "0"  "0"  "0" 
#> [5,] "0"  "0"  "0"  "0"  "w2" "0"  "0" 
#> [6,] "0"  "0"  "0"  "0"  "0"  "w2" "0" 
#> [7,] "0"  "0"  "0"  "0"  "0"  "0"  "w2"
Veu <- as_sym(Veu)
Veu
#> c: ⎡v₂  0   0   0   0   0   0 ⎤
#>    ⎢                          ⎥
#>    ⎢0   v₂  0   0   0   0   0 ⎥
#>    ⎢                          ⎥
#>    ⎢0   0   v₂  0   0   0   0 ⎥
#>    ⎢                          ⎥
#>    ⎢0   0   0   v₂  0   0   0 ⎥
#>    ⎢                          ⎥
#>    ⎢0   0   0   0   w₂  0   0 ⎥
#>    ⎢                          ⎥
#>    ⎢0   0   0   0   0   w₂  0 ⎥
#>    ⎢                          ⎥
#>    ⎣0   0   0   0   0   0   w₂⎦
L2inv <- inv(L2) 
V2 <- L2inv %*% Veu %*% t(L2inv) 
K2 <- t(L2) %*% inv(Veu) %*% L2

\[\begin{align} K_2 &= \left[\begin{matrix}\frac{a^{2}}{v_{2}} + \frac{1}{v_{2}} & - \frac{a}{v_{2}} & 0 & 0 & 0 & 0 & 0\\- \frac{a}{v_{2}} & \frac{a^{2}}{v_{2}} + \frac{b^{2}}{w_{2}} + \frac{1}{v_{2}} & - \frac{a}{v_{2}} & 0 & - \frac{b}{w_{2}} & 0 & 0\\0 & - \frac{a}{v_{2}} & \frac{a^{2}}{v_{2}} + \frac{b^{2}}{w_{2}} + \frac{1}{v_{2}} & - \frac{a}{v_{2}} & 0 & - \frac{b}{w_{2}} & 0\\0 & 0 & - \frac{a}{v_{2}} & \frac{b^{2}}{w_{2}} + \frac{1}{v_{2}} & 0 & 0 & - \frac{b}{w_{2}}\\0 & - \frac{b}{w_{2}} & 0 & 0 & \frac{1}{w_{2}} & 0 & 0\\0 & 0 & - \frac{b}{w_{2}} & 0 & 0 & \frac{1}{w_{2}} & 0\\0 & 0 & 0 & - \frac{b}{w_{2}} & 0 & 0 & \frac{1}{w_{2}}\end{matrix}\right] \\ V_2 &= \left[\begin{matrix}v_{2} & a v_{2} & a^{2} v_{2} & a^{3} v_{2} & a b v_{2} & a^{2} b v_{2} & a^{3} b v_{2}\\a v_{2} & a^{2} v_{2} + v_{2} & a^{3} v_{2} + a v_{2} & a^{4} v_{2} + a^{2} v_{2} & a^{2} b v_{2} + b v_{2} & a^{3} b v_{2} + a b v_{2} & a^{4} b v_{2} + a^{2} b v_{2}\\a^{2} v_{2} & a^{3} v_{2} + a v_{2} & a^{4} v_{2} + a^{2} v_{2} + v_{2} & a^{5} v_{2} + a^{3} v_{2} + a v_{2} & a^{3} b v_{2} + a b v_{2} & a^{4} b v_{2} + a^{2} b v_{2} + b v_{2} & a^{5} b v_{2} + a^{3} b v_{2} + a b v_{2}\\a^{3} v_{2} & a^{4} v_{2} + a^{2} v_{2} & a^{5} v_{2} + a^{3} v_{2} + a v_{2} & a^{6} v_{2} + a^{4} v_{2} + a^{2} v_{2} + v_{2} & a^{4} b v_{2} + a^{2} b v_{2} & a^{5} b v_{2} + a^{3} b v_{2} + a b v_{2} & a^{6} b v_{2} + a^{4} b v_{2} + a^{2} b v_{2} + b v_{2}\\a b v_{2} & a^{2} b v_{2} + b v_{2} & a^{3} b v_{2} + a b v_{2} & a^{4} b v_{2} + a^{2} b v_{2} & a^{2} b^{2} v_{2} + b^{2} v_{2} + w_{2} & a^{3} b^{2} v_{2} + a b^{2} v_{2} & a^{4} b^{2} v_{2} + a^{2} b^{2} v_{2}\\a^{2} b v_{2} & a^{3} b v_{2} + a b v_{2} & a^{4} b v_{2} + a^{2} b v_{2} + b v_{2} & a^{5} b v_{2} + a^{3} b v_{2} + a b v_{2} & a^{3} b^{2} v_{2} + a b^{2} v_{2} & a^{4} b^{2} v_{2} + a^{2} b^{2} v_{2} + b^{2} v_{2} + w_{2} & a^{5} b^{2} v_{2} + a^{3} b^{2} v_{2} + a b^{2} v_{2}\\a^{3} b v_{2} & a^{4} b v_{2} + a^{2} b v_{2} & a^{5} b v_{2} + a^{3} b v_{2} + a b v_{2} & a^{6} b v_{2} + a^{4} b v_{2} + a^{2} b v_{2} + b v_{2} & a^{4} b^{2} v_{2} + a^{2} b^{2} v_{2} & a^{5} b^{2} v_{2} + a^{3} b^{2} v_{2} + a b^{2} v_{2} & a^{6} b^{2} v_{2} + a^{4} b^{2} v_{2} + a^{2} b^{2} v_{2} + b^{2} v_{2} + w_{2}\end{matrix}\right] \end{align}\]