From 7ef953b6d269358693e351f687b03d4df47c0fa7 Mon Sep 17 00:00:00 2001
From: Morten Hjorth-Jensen Representing the wave fun
To find the marginal distribution of \( \boldsymbol{x} \) we set:
$$ - F_{rbm}(\mathbf{x}) =\frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}. + P_{rbm}(\mathbf{x}) =\frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}. $$Now this is what we use to represent the wave function, calling it a neural-network quantum state (NQS)
$$ - \Psi (\mathbf{X}) = F_{rbm}(\mathbf{x}), + \vert\Psi (\mathbf{X})\vert^2 = P_{rbm}(\mathbf{x}). $$ -or we could square the wave function.
or in terms of its column vectors \( \boldsymbol{a}_i \) as
$$ - \mathbf{A} = + \boldsymbol{A} = \begin{bmatrix}\boldsymbol{a}_{0} & \boldsymbol{a}_{1} & \boldsymbol{a}_{2} & \dots & \dots & \boldsymbol{a}_{n-2} & \boldsymbol{a}_{n-1}\end{bmatrix}. $$ +We can think of a matrix as a diagram of in general \( n \) rowns and \( m \) columns. In the example here we have a square matrix.
The inverse of a matrix (if it exists) is defined by
+The inverse of a square matrix (if it exists) is defined by
$$ -\mathbf{A}^{-1} \cdot \mathbf{A} = I, +\boldsymbol{A}^{-1} \cdot \boldsymbol{A} = I, $$where \( \boldsymbol{I} \) is the unit matrix.
@@ -1300,15 +1324,15 @@For an \( N\times N \) matrix \( \mathbf{A} \) the following properties are all equivalent
+For an \( n\times n \) matrix \( \boldsymbol{A} \) the following properties are all equivalent
The basic matrix operations that we will deal with are addition and subtraction
$$ -\mathbf{A}= \mathbf{B}\pm\mathbf{C} \Longrightarrow a_{ij} = b_{ij}\pm c_{ij}, +\boldsymbol{A}= \boldsymbol{B}\pm\boldsymbol{C} \Longrightarrow a_{ij} = b_{ij}\pm c_{ij}, $$and scalar-matrix multiplication
$$ -\mathbf{A}= \gamma\mathbf{B} \Longrightarrow a_{ij} = \gamma b_{ij}. +\boldsymbol{A}= \gamma\boldsymbol{B} \Longrightarrow a_{ij} = \gamma b_{ij}. $$ @@ -1335,19 +1359,19 @@We have also vector-matrix multiplications
$$ -\mathbf{y}=\mathbf{Ax} \Longrightarrow y_{i} = \sum_{j=1}^{n} a_{ij}x_j, +\boldsymbol{y}=\boldsymbol{Ax} \Longrightarrow y_{i} = \sum_{j=1}^{n} a_{ij}x_j, $$and matrix-matrix multiplications
$$ -\mathbf{A}=\mathbf{BC} \Longrightarrow a_{ij} = \sum_{k=1}^{n} b_{ik}c_{kj}, +\boldsymbol{A}=\boldsymbol{BC} \Longrightarrow a_{ij} = \sum_{k=1}^{n} b_{ik}c_{kj}, $$and transpositions of a matrix
$$ -\mathbf{A}=\mathbf{B}^T \Longrightarrow a_{ij} = b_{ji}. +\boldsymbol{A}=\boldsymbol{B}^T \Longrightarrow a_{ij} = b_{ji}. $$ @@ -1357,13 +1381,13 @@Similarly, important vector operations that we will deal with are addition and subtraction
$$ -\mathbf{x}= \mathbf{y}\pm\mathbf{z} \Longrightarrow x_{i} = y_{i}\pm z_{i}, +\boldsymbol{x}= \boldsymbol{y}\pm\boldsymbol{z} \Longrightarrow x_{i} = y_{i}\pm z_{i}, $$scalar-vector multiplication
$$ -\mathbf{x}= \gamma\mathbf{y} \Longrightarrow x_{i} = \gamma y_{i}, +\boldsymbol{x}= \gamma\boldsymbol{y} \Longrightarrow x_{i} = \gamma y_{i}, $$ @@ -1371,32 +1395,127 @@and vector-vector multiplication (called Hadamard multiplication)
$$ -\mathbf{x}=\mathbf{yz} \Longrightarrow x_{i} = y_{i}z_i. +\boldsymbol{x}=\boldsymbol{yz} \Longrightarrow x_{i} = y_{i}z_i. $$Finally, as already metnioned, the inner or so-called dot product resulting in a constant
$$ -x=\mathbf{y}^T\mathbf{z} \Longrightarrow x = \sum_{j=1}^{n} y_{j}z_{j}, +x=\boldsymbol{y}^T\boldsymbol{z} \Longrightarrow x = \sum_{j=1}^{n} y_{j}z_{j}, $$and the outer product, which yields a matrix,
$$ -\mathbf{A}= \mathbf{yz}^T \Longrightarrow a_{ij} = y_{i}z_{j}, +\boldsymbol{A}= \boldsymbol{y}\boldsymbol{z}^T \Longrightarrow a_{ij} = y_{i}z_{j}, $$ + +Neural networks, its so-called feed-forward form, where each +
Neural networks, in its so-called feed-forward form, where each iterations contains a feed-forward stage and a back-propgagation stage, consist of series of affine matrix-matrix and matrix-vector multiplications. The unknown parameters (the so-called biases and -weights which deternine the architecture of a neural network), are uptaded iteratively +weights which deternine the architecture of a neural network), are +uptaded iteratively using the so-called back-propagation algorithm. +This algorithm corresponds to the so-called reverse mode of the +automatic differentation algorithm. These algorithms will be discussed +in more detail below.
+We start however first with the definitions of the various variables which make up a neural network.
+ + +The architecture of a neural network defines our model. This model +aims at describing some function \( f(\boldsymbol{x} \) which aims at describing +some final result (outputs or tagrget values) given a specific inpput +\( \boldsymbol{x} \). Note that here \( \boldsymbol{y} \) and \( \boldsymbol{x} \) are not limited to be +vectors. +
+ +The architecture consists of
+The cost function is a function of the unknown parameters +\( \boldsymbol{\Theta} \) where the latter is a container for all possible +parameters needed to define a neural network +
+ +If we are dealing with a regression task a typical cost/loss function +is the mean squared error +
+$$ +C(\boldsymbol{\Theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}. +$$ + +This function represents one of many possible ways to define +the so-called cost function. +
+ +For neural networks the parameters +\( \boldsymbol{\Theta} \) are given by the so-called weights and biases (to be +defined below). +
+ +The weights are given by matrix elements \( w_{ij}^{(l)} \) where the +superscript indicates the layer number. The biases are typically given +by vector elements representing each single node of a given layer, +that is \( b_j^{(l)} \). +
+ + +Having defined the architecture of a neural network, the optimization +of the cost function with respect to the parameters \( \boldsymbol{\Theta} \), +involves the calculations of gradients and their optimization. The +gradients represent the derivatives of a multidimensional object and +are often approximated by various gradient methods, including +
+In addition to the above, there are often additional hyperparamaters +which are included in the setup of a neural network. These will be +discussed below. +
+ + +
$$
- F_{rbm}(\mathbf{x},\mathbf{h}) = \frac{1}{Z} e^{-\frac{1}{T_0}E(\mathbf{x},\mathbf{h})}.
+ P_{rbm}(\mathbf{x},\mathbf{h}) = \frac{1}{Z} e^{-\frac{1}{T_0}E(\mathbf{x},\mathbf{h})}.
$$
@@ -965,18 +965,16 @@
$$
- F_{rbm}(\mathbf{x}) =\frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}.
+ P_{rbm}(\mathbf{x}) =\frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}.
$$
Now this is what we use to represent the wave function, calling it a neural-network quantum state (NQS)
$$
- \Psi (\mathbf{X}) = F_{rbm}(\mathbf{x}),
+ \vert\Psi (\mathbf{X})\vert^2 = P_{rbm}(\mathbf{x}).
$$
-
-
or we could square the wave function.
or in terms of its column vectors \( \boldsymbol{a}_i \) as
$$
- \mathbf{A} =
+ \boldsymbol{A} =
\begin{bmatrix}\boldsymbol{a}_{0} & \boldsymbol{a}_{1} & \boldsymbol{a}_{2} & \dots & \dots & \boldsymbol{a}_{n-2} & \boldsymbol{a}_{n-1}\end{bmatrix}.
$$
+
+
We can think of a matrix as a diagram of in general \( n \) rowns and \( m \) columns. In the example here we have a square matrix.
-
The inverse of a matrix (if it exists) is defined by
+The inverse of a square matrix (if it exists) is defined by
$$
-\mathbf{A}^{-1} \cdot \mathbf{A} = I,
+\boldsymbol{A}^{-1} \cdot \boldsymbol{A} = I,
$$
@@ -1238,21 +1238,21 @@
-
For an \( N\times N \) matrix \( \mathbf{A} \) the following properties are all equivalent
+For an \( n\times n \) matrix \( \boldsymbol{A} \) the following properties are all equivalent
@@ -1272,7 +1272,7 @@
@@ -1283,7 +1283,7 @@
We have also vector-matrix multiplications
$$
-\mathbf{y}=\mathbf{Ax} \Longrightarrow y_{i} = \sum_{j=1}^{n} a_{ij}x_j,
+\boldsymbol{y}=\boldsymbol{Ax} \Longrightarrow y_{i} = \sum_{j=1}^{n} a_{ij}x_j,
$$
@@ -1291,7 +1291,7 @@
$$
-\mathbf{A}=\mathbf{BC} \Longrightarrow a_{ij} = \sum_{k=1}^{n} b_{ik}c_{kj},
+\boldsymbol{A}=\boldsymbol{BC} \Longrightarrow a_{ij} = \sum_{k=1}^{n} b_{ik}c_{kj},
$$
@@ -1299,7 +1299,7 @@
$$
-\mathbf{A}=\mathbf{B}^T \Longrightarrow a_{ij} = b_{ji}.
+\boldsymbol{A}=\boldsymbol{B}^T \Longrightarrow a_{ij} = b_{ji}.
$$
@@ -1311,7 +1311,7 @@
@@ -1319,7 +1319,7 @@
@@ -1329,7 +1329,7 @@
and vector-vector multiplication (called Hadamard multiplication)
$$
-\mathbf{x}=\mathbf{yz} \Longrightarrow x_{i} = y_{i}z_i.
+\boldsymbol{x}=\boldsymbol{yz} \Longrightarrow x_{i} = y_{i}z_i.
$$
@@ -1337,7 +1337,7 @@
$$
-x=\mathbf{y}^T\mathbf{z} \Longrightarrow x = \sum_{j=1}^{n} y_{j}z_{j},
+x=\boldsymbol{y}^T\boldsymbol{z} \Longrightarrow x = \sum_{j=1}^{n} y_{j}z_{j},
$$
@@ -1345,22 +1345,132 @@
$$
-\mathbf{A}= \mathbf{yz}^T \Longrightarrow a_{ij} = y_{i}z_{j},
+\boldsymbol{A}= \boldsymbol{y}\boldsymbol{z}^T \Longrightarrow a_{ij} = y_{i}z_{j},
$$
Neural networks, its so-called feed-forward form, where each
+ Neural networks, in its so-called feed-forward form, where each
iterations contains a feed-forward stage and a back-propgagation
stage, consist of series of affine matrix-matrix and matrix-vector
multiplications. The unknown parameters (the so-called biases and
-weights which deternine the architecture of a neural network), are uptaded iteratively
+weights which deternine the architecture of a neural network), are
+uptaded iteratively using the so-called back-propagation algorithm.
+This algorithm corresponds to the so-called reverse mode of the
+automatic differentation algorithm. These algorithms will be discussed
+in more detail below.
+ We start however first with the definitions of the various variables which make up a neural network. The architecture of a neural network defines our model. This model
+aims at describing some function \( f(\boldsymbol{x} \) which aims at describing
+some final result (outputs or tagrget values) given a specific inpput
+\( \boldsymbol{x} \). Note that here \( \boldsymbol{y} \) and \( \boldsymbol{x} \) are not limited to be
+vectors.
+ The architecture consists of The cost function is a function of the unknown parameters
+\( \boldsymbol{\Theta} \) where the latter is a container for all possible
+parameters needed to define a neural network
+ If we are dealing with a regression task a typical cost/loss function
+is the mean squared error
+ This function represents one of many possible ways to define
+the so-called cost function.
+ For neural networks the parameters
+\( \boldsymbol{\Theta} \) are given by the so-called weights and biases (to be
+defined below).
+ The weights are given by matrix elements \( w_{ij}^{(l)} \) where the
+superscript indicates the layer number. The biases are typically given
+by vector elements representing each single node of a given layer,
+that is \( b_j^{(l)} \).
+ Having defined the architecture of a neural network, the optimization
+of the cost function with respect to the parameters \( \boldsymbol{\Theta} \),
+involves the calculations of gradients and their optimization. The
+gradients represent the derivatives of a multidimensional object and
+are often approximated by various gradient methods, including
+
+ In addition to the above, there are often additional hyperparamaters
+which are included in the setup of a neural network. These will be
+discussed below.
+Further mathematical notations
+
+
+Setting up the basic equations for neural networks
-Overarching view of a neural network
+
+
+
+The optimzation problem
+
+
+$$
+C(\boldsymbol{\Theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
+$$
+
+
+Other ingredients of a neural network
+
+
+
+
+
+
+Other parameters
+
+Setting up the equations for a neural network
+
To find the marginal distribution of \( \boldsymbol{x} \) we set:
$$ - F_{rbm}(\mathbf{x}) =\frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}. + P_{rbm}(\mathbf{x}) =\frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}. $$Now this is what we use to represent the wave function, calling it a neural-network quantum state (NQS)
$$ - \Psi (\mathbf{X}) = F_{rbm}(\mathbf{x}), + \vert\Psi (\mathbf{X})\vert^2 = P_{rbm}(\mathbf{x}). $$ -or we could square the wave function.
or in terms of its column vectors \( \boldsymbol{a}_i \) as
$$ - \mathbf{A} = + \boldsymbol{A} = \begin{bmatrix}\boldsymbol{a}_{0} & \boldsymbol{a}_{1} & \boldsymbol{a}_{2} & \dots & \dots & \boldsymbol{a}_{n-2} & \boldsymbol{a}_{n-1}\end{bmatrix}. $$ +We can think of a matrix as a diagram of in general \( n \) rowns and \( m \) columns. In the example here we have a square matrix.
-
The inverse of a matrix (if it exists) is defined by
+The inverse of a square matrix (if it exists) is defined by
$$ -\mathbf{A}^{-1} \cdot \mathbf{A} = I, +\boldsymbol{A}^{-1} \cdot \boldsymbol{A} = I, $$where \( \boldsymbol{I} \) is the unit matrix.
@@ -1215,15 +1233,15 @@-
For an \( N\times N \) matrix \( \mathbf{A} \) the following properties are all equivalent
+For an \( n\times n \) matrix \( \boldsymbol{A} \) the following properties are all equivalent
and scalar-matrix multiplication
$$ -\mathbf{A}= \gamma\mathbf{B} \Longrightarrow a_{ij} = \gamma b_{ij}. +\boldsymbol{A}= \gamma\boldsymbol{B} \Longrightarrow a_{ij} = \gamma b_{ij}. $$ @@ -1249,19 +1267,19 @@We have also vector-matrix multiplications
$$ -\mathbf{y}=\mathbf{Ax} \Longrightarrow y_{i} = \sum_{j=1}^{n} a_{ij}x_j, +\boldsymbol{y}=\boldsymbol{Ax} \Longrightarrow y_{i} = \sum_{j=1}^{n} a_{ij}x_j, $$and matrix-matrix multiplications
$$ -\mathbf{A}=\mathbf{BC} \Longrightarrow a_{ij} = \sum_{k=1}^{n} b_{ik}c_{kj}, +\boldsymbol{A}=\boldsymbol{BC} \Longrightarrow a_{ij} = \sum_{k=1}^{n} b_{ik}c_{kj}, $$and transpositions of a matrix
$$ -\mathbf{A}=\mathbf{B}^T \Longrightarrow a_{ij} = b_{ji}. +\boldsymbol{A}=\boldsymbol{B}^T \Longrightarrow a_{ij} = b_{ji}. $$ @@ -1271,13 +1289,13 @@scalar-vector multiplication
$$ -\mathbf{x}= \gamma\mathbf{y} \Longrightarrow x_{i} = \gamma y_{i}, +\boldsymbol{x}= \gamma\boldsymbol{y} \Longrightarrow x_{i} = \gamma y_{i}, $$ @@ -1285,32 +1303,127 @@and vector-vector multiplication (called Hadamard multiplication)
$$ -\mathbf{x}=\mathbf{yz} \Longrightarrow x_{i} = y_{i}z_i. +\boldsymbol{x}=\boldsymbol{yz} \Longrightarrow x_{i} = y_{i}z_i. $$Finally, as already metnioned, the inner or so-called dot product resulting in a constant
$$ -x=\mathbf{y}^T\mathbf{z} \Longrightarrow x = \sum_{j=1}^{n} y_{j}z_{j}, +x=\boldsymbol{y}^T\boldsymbol{z} \Longrightarrow x = \sum_{j=1}^{n} y_{j}z_{j}, $$and the outer product, which yields a matrix,
$$ -\mathbf{A}= \mathbf{yz}^T \Longrightarrow a_{ij} = y_{i}z_{j}, +\boldsymbol{A}= \boldsymbol{y}\boldsymbol{z}^T \Longrightarrow a_{ij} = y_{i}z_{j}, $$ +Neural networks, its so-called feed-forward form, where each +
Neural networks, in its so-called feed-forward form, where each iterations contains a feed-forward stage and a back-propgagation stage, consist of series of affine matrix-matrix and matrix-vector multiplications. The unknown parameters (the so-called biases and -weights which deternine the architecture of a neural network), are uptaded iteratively +weights which deternine the architecture of a neural network), are +uptaded iteratively using the so-called back-propagation algorithm. +This algorithm corresponds to the so-called reverse mode of the +automatic differentation algorithm. These algorithms will be discussed +in more detail below.
+We start however first with the definitions of the various variables which make up a neural network.
+ +The architecture of a neural network defines our model. This model +aims at describing some function \( f(\boldsymbol{x} \) which aims at describing +some final result (outputs or tagrget values) given a specific inpput +\( \boldsymbol{x} \). Note that here \( \boldsymbol{y} \) and \( \boldsymbol{x} \) are not limited to be +vectors. +
+ +The architecture consists of
+The cost function is a function of the unknown parameters +\( \boldsymbol{\Theta} \) where the latter is a container for all possible +parameters needed to define a neural network +
+ +If we are dealing with a regression task a typical cost/loss function +is the mean squared error +
+$$ +C(\boldsymbol{\Theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}. +$$ + +This function represents one of many possible ways to define +the so-called cost function. +
+ +For neural networks the parameters +\( \boldsymbol{\Theta} \) are given by the so-called weights and biases (to be +defined below). +
+ +The weights are given by matrix elements \( w_{ij}^{(l)} \) where the +superscript indicates the layer number. The biases are typically given +by vector elements representing each single node of a given layer, +that is \( b_j^{(l)} \). +
+ +Having defined the architecture of a neural network, the optimization +of the cost function with respect to the parameters \( \boldsymbol{\Theta} \), +involves the calculations of gradients and their optimization. The +gradients represent the derivatives of a multidimensional object and +are often approximated by various gradient methods, including +
+In addition to the above, there are often additional hyperparamaters +which are included in the setup of a neural network. These will be +discussed below. +
+ +To find the marginal distribution of \( \boldsymbol{x} \) we set:
$$ - F_{rbm}(\mathbf{x}) =\frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}. + P_{rbm}(\mathbf{x}) =\frac{1}{Z}\sum_\mathbf{h} e^{-E(\mathbf{x}, \mathbf{h})}. $$Now this is what we use to represent the wave function, calling it a neural-network quantum state (NQS)
$$ - \Psi (\mathbf{X}) = F_{rbm}(\mathbf{x}), + \vert\Psi (\mathbf{X})\vert^2 = P_{rbm}(\mathbf{x}). $$ -or we could square the wave function.
or in terms of its column vectors \( \boldsymbol{a}_i \) as
$$ - \mathbf{A} = + \boldsymbol{A} = \begin{bmatrix}\boldsymbol{a}_{0} & \boldsymbol{a}_{1} & \boldsymbol{a}_{2} & \dots & \dots & \boldsymbol{a}_{n-2} & \boldsymbol{a}_{n-1}\end{bmatrix}. $$ +We can think of a matrix as a diagram of in general \( n \) rowns and \( m \) columns. In the example here we have a square matrix.
-
The inverse of a matrix (if it exists) is defined by
+The inverse of a square matrix (if it exists) is defined by
$$ -\mathbf{A}^{-1} \cdot \mathbf{A} = I, +\boldsymbol{A}^{-1} \cdot \boldsymbol{A} = I, $$where \( \boldsymbol{I} \) is the unit matrix.
@@ -1292,15 +1310,15 @@-
For an \( N\times N \) matrix \( \mathbf{A} \) the following properties are all equivalent
+For an \( n\times n \) matrix \( \boldsymbol{A} \) the following properties are all equivalent
and scalar-matrix multiplication
$$ -\mathbf{A}= \gamma\mathbf{B} \Longrightarrow a_{ij} = \gamma b_{ij}. +\boldsymbol{A}= \gamma\boldsymbol{B} \Longrightarrow a_{ij} = \gamma b_{ij}. $$ @@ -1326,19 +1344,19 @@We have also vector-matrix multiplications
$$ -\mathbf{y}=\mathbf{Ax} \Longrightarrow y_{i} = \sum_{j=1}^{n} a_{ij}x_j, +\boldsymbol{y}=\boldsymbol{Ax} \Longrightarrow y_{i} = \sum_{j=1}^{n} a_{ij}x_j, $$and matrix-matrix multiplications
$$ -\mathbf{A}=\mathbf{BC} \Longrightarrow a_{ij} = \sum_{k=1}^{n} b_{ik}c_{kj}, +\boldsymbol{A}=\boldsymbol{BC} \Longrightarrow a_{ij} = \sum_{k=1}^{n} b_{ik}c_{kj}, $$and transpositions of a matrix
$$ -\mathbf{A}=\mathbf{B}^T \Longrightarrow a_{ij} = b_{ji}. +\boldsymbol{A}=\boldsymbol{B}^T \Longrightarrow a_{ij} = b_{ji}. $$ @@ -1348,13 +1366,13 @@scalar-vector multiplication
$$ -\mathbf{x}= \gamma\mathbf{y} \Longrightarrow x_{i} = \gamma y_{i}, +\boldsymbol{x}= \gamma\boldsymbol{y} \Longrightarrow x_{i} = \gamma y_{i}, $$ @@ -1362,32 +1380,127 @@and vector-vector multiplication (called Hadamard multiplication)
$$ -\mathbf{x}=\mathbf{yz} \Longrightarrow x_{i} = y_{i}z_i. +\boldsymbol{x}=\boldsymbol{yz} \Longrightarrow x_{i} = y_{i}z_i. $$Finally, as already metnioned, the inner or so-called dot product resulting in a constant
$$ -x=\mathbf{y}^T\mathbf{z} \Longrightarrow x = \sum_{j=1}^{n} y_{j}z_{j}, +x=\boldsymbol{y}^T\boldsymbol{z} \Longrightarrow x = \sum_{j=1}^{n} y_{j}z_{j}, $$and the outer product, which yields a matrix,
$$ -\mathbf{A}= \mathbf{yz}^T \Longrightarrow a_{ij} = y_{i}z_{j}, +\boldsymbol{A}= \boldsymbol{y}\boldsymbol{z}^T \Longrightarrow a_{ij} = y_{i}z_{j}, $$ +Neural networks, its so-called feed-forward form, where each +
Neural networks, in its so-called feed-forward form, where each iterations contains a feed-forward stage and a back-propgagation stage, consist of series of affine matrix-matrix and matrix-vector multiplications. The unknown parameters (the so-called biases and -weights which deternine the architecture of a neural network), are uptaded iteratively +weights which deternine the architecture of a neural network), are +uptaded iteratively using the so-called back-propagation algorithm. +This algorithm corresponds to the so-called reverse mode of the +automatic differentation algorithm. These algorithms will be discussed +in more detail below.
+We start however first with the definitions of the various variables which make up a neural network.
+ +The architecture of a neural network defines our model. This model +aims at describing some function \( f(\boldsymbol{x} \) which aims at describing +some final result (outputs or tagrget values) given a specific inpput +\( \boldsymbol{x} \). Note that here \( \boldsymbol{y} \) and \( \boldsymbol{x} \) are not limited to be +vectors. +
+ +The architecture consists of
+The cost function is a function of the unknown parameters +\( \boldsymbol{\Theta} \) where the latter is a container for all possible +parameters needed to define a neural network +
+ +If we are dealing with a regression task a typical cost/loss function +is the mean squared error +
+$$ +C(\boldsymbol{\Theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}. +$$ + +This function represents one of many possible ways to define +the so-called cost function. +
+ +For neural networks the parameters +\( \boldsymbol{\Theta} \) are given by the so-called weights and biases (to be +defined below). +
+ +The weights are given by matrix elements \( w_{ij}^{(l)} \) where the +superscript indicates the layer number. The biases are typically given +by vector elements representing each single node of a given layer, +that is \( b_j^{(l)} \). +
+ +Having defined the architecture of a neural network, the optimization +of the cost function with respect to the parameters \( \boldsymbol{\Theta} \), +involves the calculations of gradients and their optimization. The +gradients represent the derivatives of a multidimensional object and +are often approximated by various gradient methods, including +
+In addition to the above, there are often additional hyperparamaters +which are included in the setup of a neural network. These will be +discussed below. +
+ +5C35fxf;v{P zmWogO9@Ce814KxMp#9c*)K15Q=7r&|3#^6-6B(2(C|ZOB2t2G3O);-P0kk^ZHFUPz zGQd(OI&n5`l4j>)?)Ex|OBUHLp5VSI||zFvyJ{Y8Wz&IHx69=6n{{V$9)t zY{QD7O1xuZEGuAz;DS{9O^>l1=}&=ChH`5~4{0sdYTa|SdYM1BN`!BREE&|1m=xw?_E-ye|-{tZLgaL80 zNg63WJx1D$Iz48a|1>QTS(;WLJyu)&4XpxY+H%H!DH;Zy2|?OTCp{Jt7Yhp t>2S_2CmD*z{%W06%(W-ehdM{SXW1Yrn7e1?VH111R#O}wpH z#yd1gP40J4F%Ss1IK%x*;FkCH $Vu3l$?WWWbQ(dqhP-3@~8BPzoVs z9;xSo-7n5f9TbEjBBCL-9}EElfdUac{6jUEhz3Gg&p!>7KZ5K7;~iWGvw?X4!%Q(f zOBB0vdMgb=4TL8lVfqewY2z2!#f<`r_A`MnvTPDv#O$Dgd4Zh`k+RTiUiOK($ZZrY zz`&Bs4-JJ9U&S0okY8K01AIV I~j4 z4}f6$8iL^HALPBOpim$LOZ_V{QR=gU_=XF0Y-)^-CEMP>y$F#){hm~Y4F`(V2fToN z^BPEFw?{ |YhgmU6(tf(*2N73M&0uYWx#Ea7h=OI8H3?u{y z7YhuY31UlV2g)}(A-uTbkDmD)G3*0GdT{+cI!I7p!1seiux2{q_Q=E~@%t?~RHXpN zg4*)woBBN<1`f_eIS4r~fh0&-1p)#E1U4F!$b|Uwn}3!Zz;dNb^mDK2-4F&I2>Fr1 z+^PyVz514h$OcdW;qUeanh$iz%vXm$uFz(P#=AindSzPFBllu!B!KaEqq`*17T zd3nED-M=MYf7S %!9V?f2|e4mvzydAi^`?qlwA6^V0v@bS) zb@3Yn@J$pj2smI#hMPvWdw>ZKU;m-Z>T?+~20w*x4;Bh3GU= XgzMG;p1%!+S7g0;PI2856mg0P^$K9da-33*Y1-J$l!RUj@u zv~_}t1Y+~xU4r$mUP0G~k;4KAIkbvJ>d^;@KH&l1vA=(V2dx))eM6)AfPjcC{OG}h z26^oR*d-tv#_RA=x2F48MI9`t>zi6;`fNxT9;-TkKn6ibz+xnZ`jxm?^tS+RC_{%j zGyRU=;HI9468R=Sz%i<@kc}^X#cFgdBLLc09h2yGf{X(`U(9*};aiPvQsxwN*Lwn;8m@P;oJa6OyX zauJ-({AoUIQX5jcr1T#jD{OQcGb7~p_4IKB`|CdJ qOB(|Xt=TXc8bMmKc@4-(8Ntn5jyaUt>pd&Ts{TOKl6 eMIjCi&iR1U?yPe~i+%AI>neVS0vk8Wi9jhz({8w;4TGeI zE3%gw4me_aO}P%ntlTwt1KF{a&8}zqE`2O-UUWXt zDw%_jby$Oe^Vy#Pf`XV%HB&q1)2Ut-ULgedg}cm*Gdx29? 4_>#<^bK! z` &cbE{wLfKuff@C^lqIsI}5OM-?aLEcKHsQtNSGLw<%%R6td*!a=oO%KSRlO0));>3vylm H*Jxk+Waon6KHwkvTQp5OL=ffWij}m43kM_M;la-`FRg0` zIbGh>+|ebf#oZ|H23igIWl-N02(uFnUZm94&D`p;1hNVIt#_M68Gj>#2Z;4tk9 kHP`aBF&_o7(y!5kWX*{T^?41WDQtn0OMxQ5?goxin|`BR_X zqTgtsYdpYsdqu;7cg&pq3=qN}uG-`59lOs#@-7i#|GFmg@dp3afh<8E!^}>Y>vbe= z-#S7q*-_)FrE(-{1S6HCvEe zRqojJNr!S2pXL!kg5_VK;z90?yN01vzdC=!7jxrv5_&2UF}RO)1Z*k^fd =IAD^-K(;12i+%P$}+jo0qT_Z {^f(;he{O_V|O?XB-4NAA7nfc zZoPe7%UPvJ^SkZRZzx>p-wcmKV?{jYcZyr3zbXi!V;`{pIsoEgl_DOlQa@%Luq-ML zr~G$Bd8;_$OlAhk9-F9Xp>By`U;SZ h^A9rk#z`ay=1LJ 7CC_{xy8xL(>}>RKqb$}5M7!P>5|cGd!fqSCi#6E>uFQT-TskjtW9My(h3IfG zI&e+;vxM@8Vo%!gjF_GWuNYG+_wmB`X$cONvgX5-LUw;sOpZaLix;$i8KWmv@q#V& z3J{VIyXeRzgM_^0H@%3mFHedr)1w}b6NLuSISfLFTmS``qy|2ff->1$*k4|w+id`s zkN2}4HpKDKOiI#&-s!h4cktJ84 UEz%wptL!jIA0&s8TMbBAz0pziao+ zNntmqp~3@c)mK!Bga6Y1PUJH@_Ec4pFJA5WdwPbF6%>@;fp%S^A_@ M; z!x5w4=n8mwH!%GHe#B;udZg67eJ7v0BTw@Es;LnmHM;4%f$>Ha{!Rk>Czd&R p&E?;jbYf6wE3V88YPzdBGBGJ(Y9N)bCmZ|~prdr(E{`fr&^b_G zHi|2o?nmT?=BGx(FNPmG9Qm5NxarVgOcWw5WK|XFEbs&B-NDnfehtClje&g)6T^I3 zjsoCq|8lB+Ex!4SA!>rT7gAcW@E7gUd5}uiN}QyP6xt5=7 HDv>%kI8gwb?&7`lg<%;zA C7Vmst0pO-&?UG|T2 zp$z>B7m9r|g-MJ4^vm5$&ja&r66n}yMraCQjMKXXoWLGB4jfy#9LHie4*EZ~)w?Go zoGpE&-G4b1(tnTnY1SQpII?Me%J4HDcnU&~&W7w&PIX lL${R;=JZJ26GQl7f>K z6De;zp3Z}4a-|A|@l;GKz~BD(S_Vnc{!q85q?ZrN8hAGe%8})OUkgQs M<;o`SkcryiX8(2b9xmT(#2s&so}sB8;yJ+5QSO5A&iRUI37N#3p%83 zel-~E-xu_NrA*dr&E>%jc*$&hez+BE|D(7XMn&Bo**Hk~4{Jue>Eqc%cejVsxBHhh zAx|MEqTAyBQ_H1LG#+IDO@<+J^i7P^EAmb9tpIhWC0lf2ABubGORmssrt46tq5l1@ zT&3`J&BE)U*Ra^7>3xnApX7>I5=hK*#MLDB4t#GC2x2#Jp+(L;yfk`%mKOar5}F8^ z0bcePCo8CST6oyic3C09iMrntoV|Q{#{UOiK%u`lse#SaM^S8u>kefBoqxR>9$n}V z9$4`_tUUi=$wXGe`2?WNu%Ksf2eoe6oaqqamrH2=7OkeZiMKs)!kh{uR9adEv -W;P#Dl8R-A`WIPi=BJR-`w<2*j+% HtLh(trXmyT4(;i3u zaHv>Tp)#0l{Xopi2Hp;$*ZW*~in2|p&C8!McMgC!3c#y))}G>r4S%gI+z5E1ycuS_ z9m%U$?z-2-5qe=lxuu?$-aC LNXVh;!9 z4N0#l$H`SRC-83zK3PQ86i8%qGO5XwLf2;=4b9a|NLCz8^?iC7?B{HpvBCGVPHRbi z#2aOWvi6J3ikiOYLN&@!Sz=JFK-}c4WtH%bU{alU<+rbsoqj~l+ncLeSzkK@GmGV? zHL!8c@Af}26n~ wl-jc;czc#alLre!v!k$la3o ztY%PR#FrVlkh@9s{wqoZ 7k_0!GS@O@H4W|&g{1_ zMcTRB6J(3G+8wtRyKNRd->BOwOZNp8g%?=FTYn${2JxMK#LYj{pPhYA7o4LjXC*EO zyODai*<(gfnN0XG9^!W4)nup_=(u=27W+n0H}IU{^>AQEXs~aq$)kxJqqZK-4e$BU zO=lnXwijL6XzuKxpJ@;>9HrE%A^LbCOG&ij#5`Ym=B6+cdr?vSO_k!4?Nqwk$8+DS z>3^?Xm)FVWEDuG!v#nz1oIOK~D1BDDC&+O#g%1%yq_OQ^`P_PWU+Av1oARFSMnW<^ z=a!O`WQ{{*_dvtx!nFG8i6DxKDYt&%Oup7GR24I L0aU~EO}1zZA&_+Rljuow z$}S1j9dfHj{`yxe(i(GDvdSlG8Vu>_95p<9&-0MxY=Q)AW}l*fo|6cncATv*riL#| zU4w4capPL$AOWwA lzzV8b$^|tgprBm!h%~5y}$n9s`+Kqtwa}3L;E)S812D`oYHY zqKAD}JMR)qxb!@?=i@hGuXlSAh>Y^j*cY?nH!G>pZDvn{`f(w3{%@M79~RtR4sF~H zERb-$LEYPL@1-$Ud#NHRWqOtUA%EoDzsq;`b;AO|1Ucot0T9`7QOHuUE#Gau+1i&^ zd$!_K%I@U74@v|3qrPI49w}-IYna#1?XJlxv;Nd`89Pe6x4GoVI+wj4FQ=b#>idw0 z>uO)k{074 Ghka+ z*E(GCio{71qZS5gG5J R5px`1hUKo`lCnNY%*bxKM!1|! z8_vTlDNUSyL-q61r`}>@5A=};j?ikh=Z*&}!*a{`b%iOBV6A}gw@EZqtOM0h-?Nl` z>xwVTsYJKYnqgw2RiNa7aDU(T Q;-SWq(~TFxVH|i)!QJ zf0${2xF^&vFUPUwCD- 1;H)z*(x9mb7ea7y0ViC2~X>SVh{wA)sR5r)=@F zRp{kHkv=;khgPFkbe^eVegc8@$#P?1G!KEi#~qD4!lO0-V_94VL7uYc3X_vDGEC~0 z@a3`AfF-?!L3pbF{S@~KrZZVxFG7dKkPlh;O9@Zk5T*F?RewS>$!-Mi*Baa)&;H@< z-%n1Cn{maUc}gGpV4|Q*mFIEiV75Y3)j~wargS4dH^)$*!GqUg{!hbVSyRRqdpn#G z--@x^sb{I~X=}89c-Bz(CWkSX(;@oGdX<}6+7}(gzzZ3}fGB!EP7T|YwZkYWqh=4B z(msK)grk=%+ 4k@{GQf;%%H~4_u)JY5CMeQq~1T2n4#iVScO tgF`IY?Vi(`{Ux}4HMJ24GaD$5_Bn!ACM@O|$> #9yU(91g+M2s@b#-c z6AzJL;D5crC11zXVooR>8<2MX;p=u)qQ*rGbKk+6EJl#l0i4wyF>0{E-o3kejZZI! zQ>K7> MXTldz<+hR?)>F|cCSL3cxPSaOWVifc6VYiqvw-Vz{Sv2Lq*s8{ zrZm4P?z=|!7KNE6p;x0TL9Ie(JzJODeY)@g(T%&tTZ@Dvv8T#+>U-6jwndM5&(sax zu!Timj(DX{hyB1@Q%gd?n%1t2Aof`nv`!qK7hbA#%}F%9CDX~@U}No%z`v^`QH&Ew zj(-!v&V`Fd1*Tn2*Yd<%^!MGf)J#5=WqfNx!F?(bHVN7mw0Pjj`K?(PK~(-otd*OQ z|AxB=Qx 0`4%j=~;}u+FQsiE~%}^2mW>rjEG}1z|MZBL!Q1P6 INy!Wsh0IWzs_|I z)NUB9u6db`-d`(E@2tDy))6X;z-tK=eB^z)V+PR|S*>FV@jU+OqYXP2OVTZJ)-eo* zD+M}DvVHmR4*t0HRlid%-+!$HI+gEq&P ^g3O^g)Hi6g5ATEb}UB*5>Y8vc>0vms}=O$*RQcoESOomsBvNzL#J} z-y-=?WB=}9_lmuG?kgZMf`kSldBU{h5SSZd!BwVqs@b35Kco%vaS( zd3CA0Wegl99NE-m0Dnw4dfhLHH(Tw9F$twK-KyEHA$fgar_ThDZtYWUh{D s`lOqXx;rkwfdO0^jMfv&Ri0W^XEymgJ9Fz^WJPl%+{eP}ulq&L1lVhI4@6VNY z(Cw4&zm&8x?VSuDFFp(*Wo8Kdd8C+PN~5e^N-V|;Kj`Pw@!-#JED5F1HXTkr%@SF? zj1oa(63LfK-MKcV^s`yZ&|jRF0mRLepEIHIEz2!b=CgFY%QyS!p{B#*tamA1w8A}P z1Z%v#Z-4*c6@RV}(%GY)L+_5Laj~d3@lKSI!>lW C0F@Q?$!xXm& zQL8x_#jFt?RN4_7GoKK>rNUER9CUdb8;h0Pad3o1 TLSF-Teni1 zY4B jBy86jChD=D?12GX?RE4D^UXYTn2Qm+utunsJkqb?b5f#iiwQ62` z5MTQc8j<;!b0_vQ+#R6iG6YBQO73s$N?pxDzkg*6u RrT9Rijev?3e>O= z(`kiXJn4pKkl?n|84kQIgeca0sus9urh_>=Ay{_X*H+=oeLRW2?0GTaHLiuRIOKmH zcYmh~l9YGjZo%C(Az2+<7kNJM<9OWqu2f%KqRP-s-;Ys6JATs`e@4nIGHL%> =VxOL_8Fs> 7z0s2D@V;Gr(HEV omJ5YpAS$(vV_x6LBWrrEk>F0xS1gE9E{RlzsXVL0AtdiKuu zJNQ*^UmYD(gcGbC+kP0eO(`{GmVcQWiUI>REI-E!IP=E5Y!9(}#`BJn7%KA#hlgjp zi2wD_w4%b`YV=XhG2rmST<*oGA5@!{)>uKS;&P`n_g=X0S;cA=bB|8k>O^Ob1t)t( z+83=vj3<2#?yL6=?xiXT9@GgU#*2@dE_X0tb5U}$jjVN}Z^V`bf%d-rYd1 Ur0p`lV%?T6%$f#;-Kj#JlfIwbOAdm-(kx>WgY6tmu z0gF)&;_L!-fQkI!eX-CpyXx;;N}Hz3y5$FiU5HC9w1QoKaLL0 zA^=&CJJbT8$_Y?%fI(ca7-bwBy_}&|)~@i4{__Z60Y3$B3kwUf{}m39vWGZB!5|nw z7369Sv4`&n2H63$9l%hCtJi-?u!vc^x;lz*ad~)naDwbze>feSt)4w)2Y5hTtpQpP z7l^Yv!~*cEXn-2X9`c{sII$Q3I@VB^zgM&!EL}Z7&JY0HUyxU=SGWVDAWmc|l=T086MH1fVXj z!s+Vi$_@a*e=L4Eg6v!z;PD`L5Y!H24tMw~cMw2cN)rHrm-tVCE?{S IrxOgm)k}0Ql?Y-zQ^ulUX>x?7V)v|Mix+)T9+;wPe};Q}e&8($Wr|0B;UqK>!C2 zKM=qTe-so12=WU9eE(ZW0|fmikw0Re!z>*D!hh!rKdArY?EX(1u>3P5PXYg}rRD(d zUI>8YceB3)@&m!}U)=xCE&o@@|6iy7Qu)6(`~RjS?`CKBmjlZ`9{(Q)kUiAS>mL_* z`?|TpM?uvAJ`k|~b=8ObJ;16E3#gm@|E@lFe+9uuLJDRDZ)*;2p1+My7kQ{B#6kn= z3by{cYyP(D{+c*DC=8 ^U=zT#&H-%S+kfZ-rnp&jLxeGz#N#x_Nm=YFNUD}Lz zKFnV_W)eM^h>GL|mEJU|wS288N>?GF64u*a)pD&j@73AhKP6+%TD6aTy%zN{JVh;y zPFbww3M;{*{PA*6fA2hg=aaU`e?TSopp7Gj5LbLT(OxQ9KJ;?E`CQ!FtU~v_Gy-pP z1wzR^i>{Ixg_zsI=J#7Aj@-!LCHz=Uz0<~euLeT}V4LWH&w1ZZ)z<}vjL`;D7-L%R z6m9YD4Pt?mP6xo3^(W|)VK5^JQff4O>0ys{>)`#*5gT23;xq3xG)6;Re?L3wK~yWw zn!rlaPc!R$TOMrLrP1)FhG)Ni4>5o#XHQ#dsO9p}rL1p;MTE0yer#B})Sp{xyA`5O z)ze*N*GX4qZl0~$3N~S(N=l>IF{U#^G^xA Cv9wAP!K0kUwDyqUr8m7M)*P@hJWW^U-}Dh&rx!f4nf zd;t$ h$#Kj({nt)5y6~= zXmsr!krwAF*4G?be gB ziX3sGPm5Ox-}RPZehKLraQSS@lEK>Ph!HKU@#Vk_gUOA@W<{beb-BIM f(K*2>34UMh3Z^8eh~$U1**f4$F4_|;w9?R^0%u&$=u zC%A5$(14h;|3k_WDM K>FgMX45Vv>9e0c`h2yuaXHYizo%P zJqP1n @nq8& z`0%aaSSz{Bf2a)y!_zo8i%9 ktPoXL^SA(fA1}7(paYIG$$Oc>i7IT$*a*8ES~q^Sbfc`J+A= z3&XhGs8JDr>uz5iebuE9TqWNIkF5pqz?uL}oNx5;p4p`zVzZH-N_Ciy5Nnt6rtg Ke+Fr-7v0 ?57uA#^A;jS$J13@)7oKj{rX)FeRazRY^<^B zb(x&l(fDl^w6LNO90XsvVY%FQFDa#Ye_b703)`IPht+mCqe=vMUv @ay F~-e?ISL zwE`8-hhI 2ulUDuqi@uMlt5pC)-M+YmWhod#6LnUlr5D@4O#~17PMF(SpGV zuEaoOUJJ>i1eRt@r6`^g3W8crBu3V6NZB?vYbV3%OI_;b3Qba{%%agoI+1BN#6lfA z-(17*?6HX6h3jKHE~p{F+Yq>Qe{Sh+UAm^q8k0xu(`LB~U53a2hs#OYYHJOrMJ1>T zLF8$B_QFFR3dh>JVp9yHh6=Zs25t0=%SiR@Op^!Z0tSscET35;zes?%0eiX?$)fh0 za|$0C$iE2DgJRW^s!50}?Pgv0fbqL#EBk=zdBaOh;?!XT77_KR=7t;@fAnRQ{tR#N z0ROzV#K?tsy0D*jP 4M z;`?dck4#N#kL N=L$h{?U1tb<1gi`%Gyy~gg^QKu(#(aR;^WIq--0p-m}qC zQq)Ja3Iaaz0`9q6S<%(+f3(p$hjRtSBblemGz^$cUpan8cp-t zTN}ZzM54oRbe(I6TXy{4Qp+) zxjawBP?MVI9Oyww#y+aesWG1wlGdnyWR)T+l!bab9NF+fSy!oa(~FqG?&B@lp4F!N zeg3Q7v%)z3 XO%kU+h( z$N0C+0a!B4-k7Muvlp%3e@^K2%GSt>e&*(W@Mw)@m9m{08%o5*GO7WlBGo^94j^9n zTt{iE1FB8GGjG|F(>IVg0kok9; ek& zA6K#ckZ1O&sinC^KLPtFJ?_}Px@_8a%LuYD^n nq1o0f431&FJA^Ro3f#D5|1Bp0qPeR3#=Ej^<{a>9c})7+Yr IR{tnx< zWkW5lz_mgUSxhQ-JosQKQ|Dn^U!vVa3yF<)$rOJVm%JA?c1ctFg*#AT+k0tLh5Zeo ztP @F31JSS$Tc }@eoX_ObW#~`VjN|lOqu~ znMOL9wujE2gs4aa5ea+j@;xla1eO_HjUApF;iOl)RdYzmP>Hb6P4F^Wi>omw(Cocn z(oYs#KKz(qm^;KMS;a!NKLXg&bS~JHR!+Wi)00OBQdmpzym(7ZAnzNsU( OSfNFPm&h+-3oWIPhE+`MHZ35AKZfzFU1u|F!sc>oaS*l z2C_18%2DT@P2w5SH*!@ehiqU>bebVX4D{$;-}PNTXTlAmH^e|O!l^1cGR0>D0g5eW z)%%{<+w5oAG3kO(x?^mDOO&m2Wabu@JNU)Kf8?>CM%H|JwobWta2xJaQVJ}fy({j3 z8jVI(q5Y*H7sM9vf=};Ezx@ZYh8`F1;-z%mjVren?f2`*njZvCec;$qyqfp*N>$Hv zuj|gSPI;@%?US#IpM2IF52OGJlc*D~^D6uJAbnUaL|lb`7Ilb!D5jNuC5f0)@<;`H ze{=Fh=))b&WA(m;z);g!ceNi|2gn@3mTRJBUM2KqsOlwtLR$v>d1UV>rik%B4m*Fb z)3!?qevaf{gK+Wv6rnjj&kw2#1XMaV2SvOE<=_r}c@Q!fVwcOdU7|iU66TXBwd}j$ z8Qsg *yNK- zQUW_GbGQTEDnC(6`c5kEdXA5Czx `UH zxCf{CUc-G$C4qPgU+rM?1{Rijx^D8-OgvbwIp2cg8N?nNQ7@gnh3=tHe_30NIystm zIbS?lWE%e}neNkU(raBL-XgglUUt}5EfI;&Or?B9+P<^8%~6K$#9iB<$hWaYr>^mS zWRF;xuW++Yn5_3aE*b98BRHP7tHG$I!nBbuS+pH@^H1`8H=yhX85}*k-#NtkQBzJV zPG5(i=Ad{=fw{m*i9OSBf2x%CwAB06C &3zL{!Swquga1H8s1@gG4$j6e>VDjj94al2clU9&j{vO&+c_W~9}98e}6 zk91#&+&vhX7>J?rwhg|%#c-CdEK8GIEs!=D_p@Gwe#kH*`0m6}e|CS_ !`b3V(sa4xz8h&!hq>K%r!kacmBgjY^VX8M?2@-mq>BxsqL_Kqbsm{`~U7S*F< zqx{5m7|LFxzuT2JMu84Nhde6tIBA;1b$XOR`n}o!O=2fJ1x1X9SV@Xq^~Wh&m**#k z6;8LWHeF&Lt(l8Lf1-}BTdpEg5E#`3(T6x>c+l`G_tlA&PQJft{i$ST2vqKSYZG*P zxY-|j&~3yO6&lnv +)n*2QkYh*38-+QLTs~z}RU#5?D;`psia(8J z9q#_RZv@-%N5{{(teLqY2E8fw m)ErN{dI#@Iod@*t5zDzGCP1TR0A^0 zN`pdP;0klhe@9qZT_{P5AZdI9@JQ&k%oDI*L%k(|CpC6VW*A3b9*?2}gACDh0UAf< zK*I9oip6f@p*$}%b>bChQQ*4H#u=j(rPaiQq$CXe)185VB48TJFq=qwqYo>M!OENi z ;%mlo_p{Y=j&7S2+IP*T)h{%6uQ^vVUYifo z3S-!j05rtF^>Ma@-N^e`hpbG~ iQC?mV%qPxBx%aa8r`HR> z;+ICLe+rs~odA^^CcQ98XYYgK*Nc1?+e_(u4Z6mkSnx#CBpuE;4_6?FK?oPe&E|2> zPF+T}z8Y?dR3~eTf3?uATjmg7;=p` Qhue=B^RpGhV|A9ij8pEg7XXO1FCt5Rm) zt1?C=)CG42H~Lxm2|E$4zc8SeVt?qa_)M2u>EzY7bHKlxcrw=T1Gm Y z)+*)OIzFA}S|ehDp&L!2ya;CldgG8Mdwq>x^=T*l`2e>*1+=zjVacW=Yz9Dd%ZmR~ |nk}I>zyF+nZMfB*a>NQLzSnza$|77&GcTNX{+6WjBtZKuJ?@V+ zBiOxsC#QBg_Q|@TSvTE#MvIwfhk=t!NavXUr^kHi8lnd>N4hImdRSH%hCJBgvf3D$ zi30NHISJ7C0ra+yI7BUg$X3sxe~fp7dtIxvKZ2ix>h29*ZP%Hf#oa!UHh5J=V$aZ< zKC|#1`{W00td0U%sJ@9}cpTYsY)E^*#a3qbjG#TC2+y0bITYwytnqMRtS|S^d|t=C zUV73`_F#dDVf~mQu%S;_Ml%ZSnS5apR##}*iFDrmR2&B7Js!n% z^0c@_xtsSoBj@k+AKjRl@wsZh(xbU0#QoOD^D2khx$dzVk%zTJFSM8c`#ZFfgJS2A zzFU8pG)+B+f&-Jq=+1YrosQX~ZmBRN$D{ZciaA{*sId(2?@x0t)F&1MzR;&qPAe2` zhoz2aNqUSfnnI^n3h;Pee|_8tDUM>i;!7P5v_HO0U^o}Vz6*b$kAVmj#F?wnSf@x{ z#@by^-1tsP47_tq6`vzhN~~2^=O~^KW8EIC={yp5uBz&hn{jM(3ma@MR{&2$iF^ zqtlOWjM&bQ-^h2QE~vvvjs&;9wX`R~0NJvF6p^m&7!5K$u*hb#e+_sxqkMx6lyhrM zy%#oIFC>qvr_TtY;H#bav qn~>_Xg$E*;c^a?7gzXpV&^ zePR @@wc2Ias=fC;@$I%#6YkTUe;X2F6)sxwi2md+?~=#i zuo71I-1LXiPkgW0M05cE#t`r`Uh_VRC!a(Ua%E5zW1bivPJiR4Pih?Q8PJWn&GM}Y z^MMR+E4FlKZoZSAULc8LZfU_pUCErP6%DzH(vzMn))yx8`9ai*0bClp-OFL(@&{r5 z#T^6+IbXZ0fA0CxS805!6DqH4z}PNy-F+%bYFhZMOZu#xN9o0RJ_jc;6jztFn>m*B z+Re^qM_q}@?wss-S$)^9{b|$yXKSimOzSJ&p-9R#)qu(evpC13Dw8?1&(Y>yHXpWa zFiAUqT-C#RP5nWGx(^8%(CV4720T0z#}Yd$*6Jipe-n>ZBq@l@5(9tZMBDVCcXM$t zjZnJnV$AKi%0-Rtffuo`5q-O}PYs*@>Nfv*piS#`Tyf)%yqBQ^5CSnbuAE|}Por9# zSxjjykZq f2W%jM+^4a@Xiua-+D)!(jk@p!;us!uV`!T%DUy*LK0+q+EDGMHYrOeo{nsz z$aK6({XY3ySi(G~XLI2S--o5f@JC5Zohm?B>sk3|mw{Z-xrWnL)4+E!H7aURHZ{o+ zON}hyR)k`!gi}Tz?C|B^>_l8B#a72hW6hvZe Zgy^TuP4#1leUCXpzu ze@X{Hp4zyaw9tXw%4w-TN(i5yg$#o0d9^laK$g|jm%CXTM7L48sdq7)*+dTv?d&-u zGi)0p_yi9p+GtJk`W0In2t(^T+sbtzPP|1P@9UWlVOyXBnKGMa)1Q9U&+Zv(`22|5 zU&td~Y!sa#JQCYJorLff<1J(jx})y{e_QuR0?Vm@hlAQi5&0?tA3Oxc^UOi7a|{{C zGq7LjDkNo1x%G*@tYN$IW?TDNJhS~6vD`~!fSB3ajNazuCch@_QXMWD=KB@pJiF*G zilba->pbW*;jcOGc7=Nu@Qc&0W!;qf@%TLXmp1U$+=bGcFxj(s+i1(%9wn$pe OCMCYD#H7AY`ZTnc#lJg&Oeb z?RmBXRdx;dRo+&gSE&lpM?Oov%z%n@Oy-%N>RK!D+DW9z9kE!U>XeS1m5LNg%R@Z! zv?Ege0UoR^VkK^Eo>>}oUnu5=f8DB#v{;H%Vp<;u<7@4ZZ_tFJV2`d_O3#|8nndq7 zf68>UOX`?uIy|pz24(P=P=LZ658er+x{c8jiKQ8#-cINW=$nh8E9(@EFza7bV;9s4 zFgNcB8nD#9C`6yyUn`^{VhfW>T$#LUkFY6UYtAhwzfJeu03#9o=>Lpxf0fOQhK1Nk zRFt!JVm?4nu>I(p`2dnZxX*Huv?m40Cs16vh3fL)fMI*52aj MJr9E57q__vagJU$e@e%KZq?y^Z3&gcwu&U-c0tvnu13)8@@IWb`kx2FuRD0k zDdbp#?U`%~gcR>cA{-x~6aqyhNBiRO$pi>>-+#M}MCNb+89PzdoiC{(s|Q|=y)4EL zQM>^oTpBT%TXafv=&`l_R1-Pu<0Q9uPPj_OZI{n$3uFjIkAK6kf4C*hO|#y@#FSoG zCu44n?p^hE!Hjr2eXcVdQHuT3KoTyKY2ep+njUi74Yzs$hU=L+UZu?8ds0kjQt;r+ zn`hg5nD-PhzQV5W>8LD!rfYH{Q%bii{ZM57K7awsku`VjgUQNz_LG8S7k###eo^SC zG>a(8P3lBeDvx3$f2z%H^Y3|D@XB|!*5c(+&+{vK<;>~?zhReJ%Y|rrR%wcKEuH=} zf0rJGSVo|YiS-5Pqzr8t-lh!A5d|)%N&jY~>E8BoV(%md%CJ;ZlWq>;jT<-fPL4%X z5sG?tL&Kv2emz-K{^B>pDV!!^p{-L|^8$KLzSFOZM4){le>S**ik(3Q#&r#+%N4*0 z_4=$uljZE4V a@!0j7Nrl_t~FC6QI0+YT@Tk&j7Pdvk7|C&~|rC{OB z^K;>F$!9IsfAbX&l4h2D+b(omB#J%0XeO?K&1CdjI&YX2x*392@*?H-_-A6xgO_(L z?T6)}{@AXb6zVc{s3!~C0Fy5c(UnYJWtkGvsuns$R-h#-Ia*DdFKgLrMFmW8ft`gs zcT{}36S1*{9+WgBrZH@i54okHat+C 1zTRbM8Gg6 zC+WI&zIS!0B-%;W&~qA%wlh7e0@G`ESWaN_Z&D5J y0k|WYiaXk8 C ze~O<(LwwNE8&vx@AMe{vg!l*Y`Vn11_HZ&~-ns56y6QCN@%nNIWoS=z_?=qE?v *xkb$Hp`NL@KV ztFBI~iq^R}x%&)*iG@a8Sc`0NY8RsR0Vj{AWwqYR?P-_Ko>UBpwk>elcn#kX@m;%K z_dvrK&HI*xN3(91DYqk{3_|S2Vk87ncGrP8x#R9a!)>4PE666O+X!D#2QZ=Ue@>oA z*}59rmGXj^)GB_gQJh;YGMlpI9X5`s`XRVk@ls0Z=D+X7 ;1mpsP`ce1JSWz|zmyE`!HV6^pHMDQRQn%q$EPXOb;kL`S zb=H*%EQY%RL!2(VQm4Y W2_I!M$nZvtxJ$gKM(>3?zUQN z%rA2IsVhLaazO_(?gWPERHA#n5=JgjL-}CZl1(#W(X1Q9(3S-CG|lHNuH9wL$y}{Y zX;vxA!?~XgB8s-jC8ZN_2DFEPcU8k4mW3 qRvi>5!iSV zYjeWpmgVhnTe(fWH_N{kJqE*V>kHf}6xMlQ!7TY6Hvgj&nuVK$-%$C{T BAnE27Sr|e;k|l1^MUi`1Ta5iBRSY2ewaGUE=8$!P~@kK%Q7B%( i$q4!?jJ}IUCg05E8%*eXmp%G(!SVV4>>xs?e^+bO-0eZM@~kNC zI={t+m)n&D$1^41V~x*&m8P2l8?x+n9}Quk!TRyIf35d0-u-6(5!X)bYB zy7?o*8c}P%F=N?x?Lq3S*Pj^tzJEc8wHGNSzS6sLF&*Tj-6EfqHs=oENI&A|WT{P$ zxnlG()3tzZm-SYhe@imG^j@kKRv!-sV0hP#VVh(K9*?%PqGa`JGYh#0jNJ_oQyeq; z>taNnDDb4cX4b{+n&>u}U}Rkm+O61XI3>3(jp7@L@Cc_$YNDb}UhElCK(7nC<+Nhq zK&It~^&WW iBC<$E$q>o} z_AC;7(%Cvx^eFP?G5MGj <^yD1ki`#a+uJ{>^pEreyg^qZaQaxYU>CKxDP)Zrz nFIJr{3}+P= zrL#n&^9$;Br$OW!*&5hK(KZElaeR4)`AH9$t-V!aeQ!zV3uB$FT+0l5Tv;_o9NS=i zR_90nXm3MU8;_NY=qnK&@V+d4vr zsQo|4BQsdizcP-IE!(9mt(c_-N!@hRhQJD4bx#V^Dp8;~ -9`>(TS?0?*C1+ATc|VdYD9Erz#tD6^PT2pjQKO(ajw{6kgFS0D+gWIErZe;c ziCJm^f7|E=3H>;PI^HG@Kj^;?8Fu@)dVXOd-HyOIbsRSL>iI#lmfZ2}5|W3I-ySRu zVa;+HVz9`7Zky97je2@nesfHbG3xUah%ZCK_mSzOY*D)?kvWVqIdC>eD?I$e>#m;u z_j!K1<*LDZ&XQHR@99}FT{aMMzE2&kMrH54f0!BT>r;z0X^MMFoh$u8X2rR?GTycB zjrX=Oh9-#hwcq37PR9nM_nGsF+zzVU${! <&66pw OM6$mwNg*QAc8|41evk%Vv$H7Oy+gOqw^>*z^bOj7ux_ zCce!xtr!WrslvOcI>EA{@4bHV0cqP?LpkXkBHa0m)Bgu HWJd^pg2;R zjqV2NkdkI>14e8xMu#AcNS8E7OG` ZtU;n@FobNk3XWMh{bMJfa zeV_ZXGwT`fDL|37FjXW1%?IKKN&=L$4Gh%90YIP-KM*KLz|L+6M?1s*vJ KPfn+9nff3NdW;bFE4(ui#tCOWiQLc3-E%Ye;oi1VeT-LCkzVs)iOW_ z>;n6 P+DEr6~o z4Dkn9>kk4i;LqUzK>VP;)BPF!D-j&=I~WXsAYEL+2p>4Y9$*J|h5>X{wfNEAXkGvq z0sTb;JG&!q;=!I^xPLR)_J;7gbTB|wK_39V>G03~+#x8qE83mk9q# U4*4RH|ob%3FdE9|!u^vitHLw|suE7BEUchd$e0B#4n z`62Lg2YbQ*Xp{#m!0+FV{~`hq2mpmc&;VPQJsd&sPxc!#% !T z`0MlcKg%11L6Hb&pMU7TuUJ4|TgUXF2G5@@|Ls&%M0x}K_&_%r +akT=qW0XY9rx&=@K z2)X$J{eSEIcgX)u=D)K1Kb8K!D^m4vcK+?>{Dbg+_`xo4XP-YAH`4V$->iW)@@5$j z|C?$8`(t;tVNke-%m4POqro@ppn$Nyku)Dj@DC&0U4Iqs4TI{z(GZ6}H1mhu=-0M6 z!x1n&q&xiA`vUlVPyhAZtQf@U<_&Sb5$10f>}G@hd#DNmf`tBBH9-+k02qY=`w-k* z^bHaL_4NL&~oVE8wZxbfNkjX*#VfB^ho@FuI{zu-+;r+>klSm%Gin}S^afgm72 z0P!y<2oP}n7rdzg^&9?YC=d@6>gIj;E!~@m{e^!&MlhH+3_>t9j)X{sI@X1@oz^HY zdhx9eN#p!TKYb*~H&keahwfGRV!f1TkexNHS%0oN^%kUR<=CmC=ysYjKXKr<^6Fl_ zkMJ?yJ=G2Sd$#f8*T=U;ll)HZu#A&7W8|BpDl$oypn5U%?mi{!!k2^E_ZqO1`rog6 zGnwSYK~<_MrV-Yr)6rE^nx#c^N5XjP>qGQw+kV4Ukwd!sxnEu4LY8AJqSACSSTv 5?{RCdKpH6YbJmQ6if=GzYc|ne-*N-Y4QvxijEh;%^aMBfA7RDM^ z&Zr+fpg75%h!H_#di?Y&q@%2MWe&M3zkh2(VAd2s#2hiVoub+5)66h)rrk3(zh$ew z+#EkYtIlm#hF>n5EJkKuXSjfntvDAvB4pXz(C-|JG*7Fodi?dEiK+SN6v~!jI)jFB z zZKDr3BH_{os3~Rf>#SlZ&a0_n#(%uxZbd8FvaC5(kodUCXbZ2LlN=@--wxNDS)J?+ z)0w*zY^jR6m?5?iGtYIU5tI@-cd@vZR6Vn@HriCM>oLxin$j#(``tPFg8WLDMS?Tq zzUeT^FJ0Yklb+Dis`p7clVG0Mn$jOWBUTa$x3Fk)#};vdP~ko1DtG-G9>Gu?TOX z>qqJ8=Wwovpj0e&H#WvZo7*AIuSoY0bDIcdYxRNJ T8^*zyfADF~x@kkk?J zp-022%sq^em>-l~VKb)zu7CMC-sZ%0Jr$HL{fF=J%_!2IGg_)xKuG&AF{`f5bgybk z1cSDL-xNJ3`$yd9l*1{-?|%8vDa^cYv1eNAy?9^Lt5H!LYquX1F`=x1;R6iV0%WNV z)Rn0z0I`$zzgRGSn1qso(^uqB{jnW~pzgck<#+mi>@)=}Ky3`bynpy9bKB1YPWP@P z3J;$Tw)wOZJJM^sqc%Bs8YHPbrV*sb6S%#B#f{mA=M=j|$JH+7eMhe9t~J5YSU&Wf zcJiP>sQEqS&;#9TUCjGs7u&^kd^z!bpJ@{*xv(rP>lYBkr{wBCzCEO)LRwAx6K 5pW(+(~l)Om@%Gydkt zQLdvCw)~xZ+9&xlSA_=8SH&H8C ccG!fP^QcVsWPfln6v$$!Uyy>MHIkbU|q;?V#- zx!t)kj-=ZjX^hhiLde#$b 054cGi!X8nsdKiNF=%8#vx>UCnQ3QGjEhhb%<@Byj0dP*92Bxx0>FI!n0 zrlMKutjgaW`F}&ZMu||x@V83Q90qb!p>K@c9v@%BGj$hFy&m&P8QVwTmrnxmA0b_< zdaXtp0x4|7CEnSBt7fi}z%yJ6d3CBKs 4KIR-*)G1#1_?=hC$hX>-NDJ(? z8LdAzC9FXX*x8G6(PnoR{WMu4+ZzNFGOlqkxL(bp@_&xY+Rvt#ImU;n{Kw$ExJNs^ zNVr}IGQw+Wnu$p_i!>pv!|qXzbWjo*ct0POPvg<#1YgQ^7xz+TnEmM|h1)ZBrQ588 z*DUYH*p1k`?-BFD%q^_=*;7Q&EB|!@ueb*in0y;a*zDZ*UJGlIk@oU_pCCc9KOL)N zgocK5_-CR#>WYN}I&r$3AIDY{JgA)gZJ`hcy` zg5!PGa#U}9`ruF8gFV}wMX=jca%9WE)4_&_K7S3U ;Jz?G+A- zNi| *~^QqPsQC<$z2^ zihsl@jNFez-^1VYyT;i-{$d)2e7KkxC@##!7&x5u-~~w*=l0hz!^P5=Ybs$|tj`?o zSj4Y|l4+ |0=j8JKsHJrE#-lpT9b5R zsU bFQkXYEN6+v+I$rG*2CBy)z2j;7eg_l#$r nCHbai)xAo+4$R96NWL6SQ;fk1JDF`&(ULn<9j!jtaUyG=` zB^$c@VdN`DlqmDuV~gh@6~Jy*e}&4o&Lh3qb8p`hdd`t*;4oS}3$9i cGx8*w1uDjOJc|?u=
gau2OdC!`)v|ZIz*mTORmA#h>8s7}xURRY$b+o?;Z%B^PrGzy5VM=YOYJJP8!U zgWcsmX@$r|pjA7Rk$7rF1gbd+RbPeks2AytB;#4q?ToBwy=HICO%vwtIiAycnrzYF ztNgT`NaZv2hTrJSVaM4qYdJ@F?EpQj<7n8Od-&eNfJx-@hI)XUIxg
m?u4PdY7V zzlT#@O^|A96Gg-IpA)fI^?x`f?iS^;eFXZW-W(;0aM6{s6R8+wAZhoW)rK z(TLd-rDqGdR^?NQP58K$^=+?A#^ Mh+lEyyF zFhK=VK=naWE^RVwA|IMC@YAunqrv2yP{>imMvp;5{Frb>IBou`;Trp}Xq~VRsb;r- zQ)>H!!a~TRJ$vFE`PGK^5WM?`xteD7MA-tI_t_KQezn9zXQc ~J-G2fY2W*3O<_U|S4i`gf_p5@EiaZB~o#o8jWX-2IG0$<^ z4k+@2_Weo2j)@3JotWuyVp8ccf-ur3$%&Z2;sy)c`1aSwUG%3uDqthq{ld$UpSsN; zY-D2=Y 6fnb=M;FHV7+>`6`ote_MOgF_dYnTdcOK{PSlr=SvHlx z&3#2pmV?AlF#M2RhZfT(b0Ga4xApi~2@^ekAbDmBF@y6f?YnQf+tp+wl$R5+V)jk9 zBUhbX041e<3V_v8HcM&i-MHfM4}zCRb&`=*Oj|fEaesNb82PtSb(ixjZ9&)}edM#m zA)Q7<7R)4Wp%rBeOH J*awJWfD zw6oruas{?7rQhn>5I=o6BuApzhVemV`>na(!!@r$T2y_S=D~xKG&yu%+Lc7HL8iyz zLIYhzuz&9uQ>n@47iuNUOD;3j7`3{ICq!#^SWh*q@o}Y6tbC~0g<7a|YI;3b7%Osq zyf|+ZkIYi*dT2D9`&2i_b)Sy6WoobA-N_k?h|H;B^~#_*ez47KV^%gf+U0sXa x{mICPMWZ2sKZ1=5r}Xmt<_GL|NhapjR2IYhEH`dxDq6p-1-l=Q8p6 zXw!latZdzt;X1m<;_F)p%9cFQ@4$L)Cc)N?hu^JzldSoC!lGxgc*XVoCY-K%(uE^? zwtr52TS>JUbvMIK&0}q8leYKN`nxQQF6y#EdnJ#tPCTriWTjC%OYU3x1d<6|rvbt5 zVr=jmK`z)~%QRfz?6J2#Pi3;3U1Y{>>XZD_ul6>zH&6RrZ2-2^_vud}y=s!H+3u4G zZghXYBJ*9$hv1!543WCYQ4{JAhX@?|>VJh$2v!8Z$sk`w&?Fa+Hw~TEDr+iaG*3ke zslfQQ3f!vdFE4`hPTGDxAUl)y|6CN*K}YbR*VtaBjU^`+2r@(rb_Gy5$F92w@{HsI zWf9|t#M=$fgd>A~B%_+q4AIX%!h(dK1h+y?1$QHL;>~Y;noK#BcpuumlWB&?IDe&K z3p?GP>rRxB#2QJ0nf0UZz5RyKTwE<@N>!3U7af1}W{Ehan;ToE+wA1A1po$&8f>(| zRql#{kaFrf3qB5+3Bg16S)46OJQ2@1D_bexjsz_=Ne&~p@%ck)`k39X+Wkm4<1*hF z7Hw0#r`+C>@|@FVTehiRD1tSCxPMSzC!LChC)>X^=~M=qg*DVd-MIN6tXj~6d!vMb z#emAOOR=HED~<`C5-s0EpbnBzP_C3_AWd{6FCHk$shLw<`9@J6nT7V+A4N*w?IGA} z#@oVd=Z!z#t-V-T{*I~>CCK)$>};j=CqE0caEBC{<=-BO%rc?vb2?4X$A5iG6nt7O z1mC_Bp)Oqa^z)l~xw=E-M~a~UO8YjS!(&q6ndN;dB<)xHlHv)8TPu_Y4?Imz_q5Ch z*%ZIYhpyo@QoP;36x<#C2JaTMtn1vks}P^8xdHtmns%+BqqEW1|Fyz*^F7O|?!Eyp zH0i_ztMW?#)P~?E{JWM}-hb%HaoMQ@qr~i@uOfe);qg;ey&usB;fU4m&Ros|A54}` z8dlFht`PyggP*6!L51h(ORK3!xg=KnXOQW|&N=r_Ol@1|8SrgnkD;%Ayh(Tv%O~l3 z3fbje$5hv;G|jco3!NmZJYx=9TH8P#5~9h5x2w4COhUsShv2U_@qeq}D!bNCi=R0H z0!AdZ{9hMDP{^A+^o7nPFbq{17{BR2X8JsYu4_|RYKT!yVeUC7ZYtF7?t1<_pQfzm z435}Q|KUJs;$+XG0E%txpKmZM7>$#n?GuhkdtJsTEIL=qCarWP*!R-up(NaEq|J7c z?CJ-emr8Y5l>M!2UVo;}4Qo>Sc^erQ3CV=68kOJ(wsb62!FjCP8ZX1b#uZ~MWLj50 znrRAe9w6qLwsOqF*Z1G!zzl4*I)K8Yq~5Xu3~WAj=U@5^I3-J!0*@#d-8g8BltPq} z!1N{z4z04Ou?CJwRdzKenNRXwCHgL5khI*+YNyEel?(kIB7f dT0%OFkx9$9M&(nOQdi$?t0^c@ z(N9@i6jS@~Dv6jep7ICA=J-}a16Vwcfx-{Z?wO{>n15(K@j)9N#S6wf+tuvtmqHgS zqVrKZuJ4RGz~zT6lJo&2wU_U <&f zjeDneHCf>Iugz^nL19~w>Cqt%w@DCQQyCZydNbxzTAeob>w3)6?aQA?uNaWKS^AQ8 z)wradoPWE!Cq1o}MFUTElCdhWpR-~P^0qS&Wqs^3_(+3T_&)dn#3E8YZ#K+MqXmfv z4)Yu`FOX!`7qmoXI}Uy@AFaayE6N(hm|}84B9)@*AqJdVra@!`;SiElN9XpZEk^qS zw~NzWlVRj|lXbexKj&O{{%!W7n+TyL(h)2Y2Y*?|*(juZl$R^GCmTP8&W+`1$RQl$ z*h{jbj2fB@9|XGU5^Vy0F!GQ}y086Yt0PUr+ >)NLOWgC4n6iXki@R#QR06InV6GBJtIaatp<-HytwXOYdggDS)-hN(ImDW) z5Pxi_Kw0@>*9CPXJ^OiI BT9QM*OgG+~2QIxs zQ3H>aUxbI?RpeIN`b#;rGW&zxiP3$840TF?O#r29!7E01LXk!C6meF8((76*o@(ry z%5^p67XCwTXkL=Q(iI=GJX3wdc*^%Woqt8cB5BvFaMSqvS01F~lF=+Hn;Q#_Y~0G` ziudbgAKv$v=ATdf%k8nDhpRsD0~&JrZ=I>^6A$@Z@$)-@&AM%0lndnt0<&h{2+f3u z89y)7@IEx zkF=3o!#xdWEH0oP)0K3%M?kVUakr47{B!b=Onn5#d8Qg(w3fM56XG5PgDVqDXeT`l zVD8W}WqK; fh+Jvj}HcPu|C+)mAXbjzC1{o|P1q0?OQOXl~aG_p YU*g>;KGEL-{t8T$K5jC_pk?E1+kX|E??Sh=K~mP1S9Xx_>l}=4_V>yPFyE) z;`A`IM*|KMGO;W3z#p4 m) zE2!f1$Z^!w@=41EJX&n$o4Oie{E~`9%2<41FEXvb)@RVm&8NG^d|x9LJi}Ae`!06w zu@L`||MsMsTQH`usVR0|eG(+{TcktO??l)0UbreTyMA>*T*SIGrW0ccV7{Ed{>=R$M9sLM&?RIrp7@$a zKRN0B7NO^9!t9+k4$3|+g6mYt*%ztbIfk+EI<(>}8`BqqFRhfA++^eyAM^=)yukcn zyx-A3jiyZeA M|ObYzEQC>rDQV%I4TE!Er6|T>AA?neB{GXIEm4 zBxkNa2vJ%YVoRUM&yslIx`{+ao~FfPQ6tGdgk>qXOC8}m?20*=fKo!`^=3zFpTHcd zkOZ$bCRul7>VFgpPao{!aJy_j>&$-9L<#|rUd#KX*Pt?JWt8qowOU7R3yEVZ38=XK z;573c!yN<5n6&OybkE8fRd~XcRWBDXOT3&kzPqYQwh_0=`dfYUZr_oP@K}59dW+@b z{)1^7W9JG&NBSv6R(n6y#3R;YRd39_ujkaSgGEGjv40{AH&iuA$_RDBmS}Vbe>}O} z<-B Gbeg+>v^lw>H#R1`D(i z`cS)TjR<}xw!f7kOPhur;Wc--I$`7Vv~VG`H8o+;J9l6mb3_X(G*AX#n!datGPS@u z(k~P|lz)$Rs;nbOR9{RJ)gg9ul1R*6!A^xqudFv|U8*k}l3;oKe5bY1=76_6+Mh}C zInRcPQJJuj_n;6#Y*q0}$Rv}?Pi*A$ShVU1dhZK2d+W3#2a&`R(YsE&I{k>luz`pa znRVC8UOO3vRz_T8dwH8EwXtMWIUo6a`SKYPt$)PWS|Sv%s!c}FoXdtQT_xK?=l$Vx znc?ToD__z&{|O;^Bh4}O+uT_S(jm`Umc)qh{TT4JH6MkP3Ql%yoe+NAMCZ+ghOr06 z+jv~Igs7hskxGzbEbZi1J0E%Wk{ySa_Pu#~nhBy%g4K&5AnfHI3G5yj+)h-X?~UuE zJ%6=x&QoNX+8hr2R{FwYyC&0ET *$#I~48PTmIbfJ-~U)80R^$6oEi zERXWZ-sz`Y$iZp7V)F@UWI(61Hmwz(z52mzI<=#EAY|JlTtr%Nojx*~6#O&tnLleh zIBFk?iqxC8cY93q8g)tUeHPD3W)~Sl6Mrp2)kq&ju=^~AH#4HS+tx!9UtGm0+?U~y z@4k@jG{t`9*+t8g>V%5u2|t`L;={D>rLG^a; *AOM(8^=LY#HuNro(aOkn^mTi9Eu5+ff z2M3s|UzXKIvl`v&O1*t8!(;AOO#mD{>hpNF*+uDn7@fQM&T3YZkc@CKkA5OOM&csK z_CpX`=Xyccy8tQToh>m_E6u>^?0;M`m$3N)M3?} n5W%xQp%=CphZ~bg&>rb$-%VUZz z9z)DWayDayzOFu7(ns9W 7jff2bfp4k)xt)d7VD>cF6Yhn^6CuqZ%S;=ZuteIX%$h>(!<-#{4R zK0wvM3+x2Y5db`dK|x4Db`==h7Xfy0MWN&T{RrR!asz~=r6u`(n*)?QKnO6<0SeG@ zK)HfE&=G+S5P%U32nL~i|K);9))j?<-xn10_VyNV@IVT{e-JM6+ ghxX%0YGK=?oI{#gHV1cv@Lb^rol9&iV!FBs|qa0Wv_06leW0hAAl z58wcG`UP}=e;{FKdj~HEFvP(T4fvh913+EL0N{YG@K1e6AOZ|WAq9|N$gdg&f4M=o zSq0`u?yK~YG;U-_wm5g;JC@4kY6hU*T6c|-mGJ~@M-PR_r|aPou;nn1yiJVBbO ze=ukf;Xg7L5DFkJBrPE!E(`!Y0)Tvgu7bY;82iFOf4`N##poIW{oyb;z!_ZzC=l!n zLVpnYBOSaz02IO#6zKooihrL7g@pl5U?2+M2yy{K3IFkq7K5DsI-}>00Q&%}gwXU8 z1_=Fn{pZOB%`hhz6yp02{P&0jRm{{()s1=ol>8s1vNFsE;Lk524d9oS5CRAb3rheb zCB*@OfB*KQ?*RUj$A8&sLY-j%>A%uN_v!DHz5Z~3>(7F41ODwx7lvjo2*C9Z(XEBV zg@EWc;s0m4|2FymWBD&H|5u{_Z$|2#5Xf&m*I$7DkKVxp4DtPgKr`19h28=k7 nLPvQW7Vf0oyfqZ_G86YSCg`v l&CHj35MrSSP0CjOh ze>ec$LFl{D8TDT}@&DHSHQv9KBENKSI0EJadgO_o<==8KVYD29fO$JPLjG+bEF=LC zbaX)cM}V$S5Cj4JTJ3*8= PUzK()FKU06{pKf0r;P^qq!g0h(QZYtWg(A)d&8J4d%$&=U&& zwGw}F_}5^7o(Kfm&+iRFPwnsX?<)}m@&N$}7v^9 H5b6Zb9D9A$GSH+WHntQABCYdf4HT5 z!e21*^{m$g#N-5{cbSmPh}ohzPBLI5v$w*w;!H7{yN;0T6Ww|FvFJxPa>r77AK(Ef zxgcjrXF$a(<#q34KCxh7+dz$4Sd%}!jj+n}G}bhk_<{KZ{Pa?}Gmh@7@0QimtkIQ~ z*SSN38KS3!TKzQ=)sd8RRDKUxfAYqXhv}%3av!!JF*uaPFI39Y5?SerJt8?)vFs&R zzt0p7btDaodN#S}s)P{fJ$}>d*#HxtlgsN{J-utUon!n}kmOEla*QiqZ3de%dyNyu z#c1YnjVL6)ADe~=fZNc$bfYr^D=oXU(8gSBj>CH#CTja}^3Iige0Gwse~3+*97Jk( z{e`lGK7 |Qq-Uqp*Tm9gaog}5yT>Q+<;5Vl(wsR3dcNEGx^uKI>IBAi z#7I3Il4#zQy+5#*)S@YV@s7fjM@<4%r}ZAzW5=D4moO7#n4gEO-_iX*#!^vwev&o8 zRFtg1nBu*$_6|m-E`i1Ue+Z@E8FEre_oH&gmy1;rZ=TDAw#ubR1q(f1nmoCsYH*N1 zUbjVL(DxO`HqjArA}E4ytCinv>1to9%lQx`mI!X%PrTk$wwMk%k(^u2SN>@dWJg){ z*gD@i{`=D}JLgTu`Q}Ou(Uy>y)Sl^}ExQ+Q7`rTop5D!THF?nAf5n-$ OFRhRnir7ya?E@-YN@!D4j)>C)%cz?&n z>=A=%s7@Phw1wDO=~-keMWIE)?3+S!eYF$qA0k_JB5g#~ZRIeHG=D;2TAyh;s*yd; zo=#7?fDRbR)x|e6U1mnZY_-(WlEbXFgc;29?;NRHnGk06e;XR>AZ-dvLs#LKOzL@# zpG&GI0(qk#Bq~=nfRf^+J}&(5yI#(-urOZdi95ME`-jX8ftL`4yFi}RXx5`@+aSZL zk2Vz^k=x|&z@_0@6?!AnPgyjH;_7GWzwg}nVo34R4mD#YlhyO)bLOz8BG $`G2j(Vqmnhul*;<5H)R@F7EM UVq|Fy& z=NhQG64H^*DBAJm4bzO}l6AS?ripiShU_S5Ug_Ine^xYPc`syBmN;pSX!yBv0WU?v ze7+|##g2p=7>%0fYc@FhYrQQWN76?SAB|hobQT;69(eU8Ojs4PeV=D))n?Z#P~6Yb z2#ZX)ExW4E`Sjt7v85odq*=ZpwnkQ0lTsap^M-wejTAV~T)3nc>3dqaC4FIJ;MN1O z6vZOle`Imn6VbF7;|~o|&h3!^dS8Q*sIWFY?(jyEDj~_4kJId=5d`56h>N}@8~VMV zJUes# pHy8Nr)zCs7QpA~4<;-_=v(|3-<;+V(8z7TZ zN*$D5@)2hk-N##6(br<`dUCXfKPwgIOaRGcf6EZQ6@8f6lU3S}?wIegct|DjA7NQ7 zvfv(EertI j<@d&(^{b_u=f+hC&LGmayG|Y~u(I?-fkk+_dG+W- t8{lTONvW?8=V( zTJh;54}j73enYp2CUpBlmFeR4+u^Vthw^Nj^Ie5mK4Q$nQ%6(*Bqh_8Tq5a4FWRfm zt`yhs)0+nAL?eJLmi{4&0sNAnH<978e+?mV+1V>o++dl$+cUK7HP0mNPV7tZl%`1| z*%vNDuR8L63y|VQcPN}YdH=*P+@@+J$I aFaaQ$z=waY*YbJgzf+5H-2w?yGG%#wRhaql$nU>gNe=MTL z@|u32P@7b0;??;=JaFu+X#z&IfLRrrqpeZ=u3c;S{H!^TJM# ze{aq3kZW;v!d7ISom>c0>H5ckM|cND+*`6wkN}S` z*v#vLaxprC5AD>|;RU|WC<9Uw7QL8nAM-v=EXrpZNL6bBK7CiefR(H>wEteP`ht;W zA2lC=b*%x)j$F`JY5e4+d$hy)Kq8R1 )yp60V**{b_ZP$8>&z%MHy7e$(3h zmf- EzHSJq9m1Pq=`D@EwVo*rZ$wq$M>o9Qpqp=1+D>ij?LtAi>)vYo zD6qmX4)H<0)hT3*(2wD1e|!!Cl_!)qmvm!zyDvuFVY<=~=9a`SO8~RRPxc>&vI#NH z{|srY>^D^a_uRf6<9D_rH8dE|Q!MXv f`utGXdFN>tr|hIf*BUN}|3FyqiZm`? zG9->iP5fSlQ`F}o*wb`>8