tutorials/custom_acquisition.ipynb

{
  "metadata": {
    "dataExplorerConfig": {},
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "captumWidgetMessage": {},
    "outputWidgetContext": {}
  },
  "nbformat": 4,
  "nbformat_minor": 2,
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "originalKey": "4c6694a4-a6f8-4fc6-a9a7-4a29617406cb",
        "showInput": false,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        ""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "originalKey": "c901c723-b2f3-4f75-96c0-6555954209f5",
        "showInput": false,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "### Upper Confidence Bound (UCB)\n",
        "\n",
        "The Upper Confidence Bound (UCB) acquisition function balances exploration and exploitation by assigning a score of $\\mu + \\sqrt{\\beta} \\cdot \\sigma$ if the posterior distribution is normal with mean $\\mu$ and variance $\\sigma^2$. This \"analytic\" version is implemented in the `UpperConfidenceBound` class. The Monte Carlo version of UCB is implemented in the `qUpperConfidenceBound` class, which also allows for q-batches of size greater than one. (The derivation of q-UCB is given in Appendix A of [Wilson et. al., 2017](https://arxiv.org/pdf/1712.00424.pdf)).\n",
        "\n",
        "### A scalarized version of q-UCB\n",
        "\n",
        "Suppose now that we are in a multi-output setting, where, e.g., we model the effects of a design on multiple metrics. We first show a simple extension of the q-UCB acquisition function that accepts a multi-output model and performs q-UCB on a scalarized version of the multiple outputs, achieved via a vector of weights. Implementing a new acquisition function in botorch is easy; one simply needs to implement the constructor and a `forward` method."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "collapsed": false,
        "pycharm": {
          "name": "#%%\n"
        },
        "originalKey": "bdd02e32-c993-4bc7-966a-dfa520e243d3",
        "requestMsgId": "2364e148-21c4-4037-998e-e4352a4f4929",
        "customOutput": null,
        "executionStartTime": 1668651266713,
        "executionStopTime": 1668651266719
      },
      "source": [
        "import plotly.io as pio\n",
        "# Ax uses Plotly to produce interactive plots. These are great for viewing and analysis,\n",
        "# though they also lead to large file sizes, which is not ideal for files living in GH.\n",
        "# Changing the default to `png` strips the interactive components to get around this.\n",
        "pio.renderers.default = \"png\""
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "75503e22-bf3b-49d6-87a8-cb9c741e941e",
        "code_folding": [],
        "hidden_ranges": [],
        "collapsed": false,
        "requestMsgId": "28d53a4f-be29-4ec6-8529-e1bdc80b5adf",
        "executionStartTime": 1668651267039,
        "executionStopTime": 1668651273452,
        "customOutput": null
      },
      "source": [
        "import math\n",
        "from typing import Optional\n",
        "\n",
        "from botorch.acquisition.monte_carlo import MCAcquisitionFunction\n",
        "from botorch.models.model import Model\n",
        "from botorch.sampling.base import MCSampler\n",
        "from botorch.sampling.normal import SobolQMCNormalSampler\n",
        "from botorch.utils import t_batch_mode_transform\n",
        "from torch import Tensor\n",
        "\n",
        "\n",
        "class qScalarizedUpperConfidenceBound(MCAcquisitionFunction):\n",
        "    def __init__(\n",
        "        self,\n",
        "        model: Model,\n",
        "        beta: Tensor,\n",
        "        weights: Tensor,\n",
        "        sampler: Optional[MCSampler] = None,\n",
        "    ) -> None:\n",
        "        # we use the AcquisitionFunction constructor, since that of\n",
        "        # MCAcquisitionFunction performs some validity checks that we don't want here\n",
        "        super(MCAcquisitionFunction, self).__init__(model=model)\n",
        "        if sampler is None:\n",
        "            sampler = SobolQMCNormalSampler(sample_shape=torch.Size([512]))\n",
        "        self.sampler = sampler\n",
        "        self.register_buffer(\"beta\", torch.as_tensor(beta))\n",
        "        self.register_buffer(\"weights\", torch.as_tensor(weights))\n",
        "\n",
        "    @t_batch_mode_transform()\n",
        "    def forward(self, X: Tensor) -> Tensor:\n",
        "        \"\"\"Evaluate scalarized qUCB on the candidate set `X`.\n",
        "\n",
        "        Args:\n",
        "            X: A `(b) x q x d`-dim Tensor of `(b)` t-batches with `q` `d`-dim\n",
        "                design points each.\n",
        "\n",
        "        Returns:\n",
        "            Tensor: A `(b)`-dim Tensor of Upper Confidence Bound values at the\n",
        "                given design points `X`.\n",
        "        \"\"\"\n",
        "        posterior = self.model.posterior(X)\n",
        "        samples = self.get_posterior_samples(posterior)  # n x b x q x o\n",
        "        scalarized_samples = samples.matmul(self.weights)  # n x b x q\n",
        "        mean = posterior.mean  # b x q x o\n",
        "        scalarized_mean = mean.matmul(self.weights)  # b x q\n",
        "        ucb_samples = (\n",
        "            scalarized_mean\n",
        "            + math.sqrt(self.beta * math.pi / 2)\n",
        "            * (scalarized_samples - scalarized_mean).abs()\n",
        "        )\n",
        "        return ucb_samples.max(dim=-1)[0].mean(dim=0)"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "I1116 181426.999 _utils_internal.py:179] NCCL_DEBUG env var is set to None\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "I1116 181427.000 _utils_internal.py:188] NCCL_DEBUG is INFO from /etc/nccl.conf\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/data/sandcastle/boxes/fbsource/buck-out/v2/gen/fbcode/f3b9a99e517e0a13/bento/kernels/__bento_kernel_axoptics__/bento_kernel_axoptics#link-tree/mpmath/ctx_mp_python.py:892: SyntaxWarning:\n\n\"is\" with a literal. Did you mean \"==\"?\n\n/data/sandcastle/boxes/fbsource/buck-out/v2/gen/fbcode/f3b9a99e517e0a13/bento/kernels/__bento_kernel_axoptics__/bento_kernel_axoptics#link-tree/mpmath/ctx_mp_python.py:986: SyntaxWarning:\n\n\"is\" with a literal. Did you mean \"==\"?\n\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/data/sandcastle/boxes/fbsource/buck-out/v2/gen/fbcode/f3b9a99e517e0a13/bento/kernels/__bento_kernel_axoptics__/bento_kernel_axoptics#link-tree/sympy/solvers/diophantine.py:3188: SyntaxWarning:\n\n\"is\" with a literal. Did you mean \"==\"?\n\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/data/sandcastle/boxes/fbsource/buck-out/v2/gen/fbcode/f3b9a99e517e0a13/bento/kernels/__bento_kernel_axoptics__/bento_kernel_axoptics#link-tree/sympy/plotting/plot.py:520: SyntaxWarning:\n\n\"is\" with a literal. Did you mean \"==\"?\n\n/data/sandcastle/boxes/fbsource/buck-out/v2/gen/fbcode/f3b9a99e517e0a13/bento/kernels/__bento_kernel_axoptics__/bento_kernel_axoptics#link-tree/sympy/plotting/plot.py:540: SyntaxWarning:\n\n\"is\" with a literal. Did you mean \"==\"?\n\n/data/sandcastle/boxes/fbsource/buck-out/v2/gen/fbcode/f3b9a99e517e0a13/bento/kernels/__bento_kernel_axoptics__/bento_kernel_axoptics#link-tree/sympy/plotting/plot.py:553: SyntaxWarning:\n\n\"is\" with a literal. Did you mean \"==\"?\n\n/data/sandcastle/boxes/fbsource/buck-out/v2/gen/fbcode/f3b9a99e517e0a13/bento/kernels/__bento_kernel_axoptics__/bento_kernel_axoptics#link-tree/sympy/plotting/plot.py:560: SyntaxWarning:\n\n\"is\" with a literal. Did you mean \"==\"?\n\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "b1d4ee8e-69e1-4914-b113-473add84f322",
        "showInput": false,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "Note that `qScalarizedUpperConfidenceBound` is very similar to `qUpperConfidenceBound` and only requires a few lines of new code to accomodate scalarization of multiple outputs. The `@t_batch_mode_transform` decorator ensures that the input `X` has an explicit t-batch dimension (code comments are added with shapes for clarity).\n",
        "\n",
        "See the end of this tutorial for a quick and easy way of achieving the same scalarization effect using `ScalarizedPosteriorTransform`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "7122ce31-f5ee-4fdc-8962-73f692eaaac0",
        "showInput": false
      },
      "source": [
        "#### Ad-hoc testing q-Scalarized-UCB\n",
        "\n",
        "Before hooking the newly defined acquisition function into a Bayesian Optimization loop, we should test it. For this we'll just make sure that it properly evaluates on a compatible multi-output model. Here we just define a basic multi-output `SingleTaskGP` model trained on synthetic data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "4958d0f5-cce4-4fc8-8fa2-8b9d7fe2bd5d",
        "collapsed": false,
        "requestMsgId": "534c2e0c-f4c0-4bb0-bdec-e44f86c0ed2b",
        "executionStartTime": 1668651273753,
        "executionStopTime": 1668651274217,
        "code_folding": [],
        "hidden_ranges": [],
        "customOutput": null
      },
      "source": [
        "import torch\n",
        "\n",
        "from botorch.fit import fit_gpytorch_mll\n",
        "from botorch.models import SingleTaskGP\n",
        "from botorch.utils import standardize\n",
        "from gpytorch.mlls import ExactMarginalLogLikelihood\n",
        "\n",
        "\n",
        "# generate synthetic data\n",
        "X = torch.rand(20, 2)\n",
        "Y = torch.stack([torch.sin(X[:, 0]), torch.cos(X[:, 1])], -1)\n",
        "Y = standardize(Y)  # standardize to zero mean unit variance\n",
        "\n",
        "# construct and fit the multi-output model\n",
        "gp = SingleTaskGP(X, Y)\n",
        "mll = ExactMarginalLogLikelihood(gp.likelihood, gp)\n",
        "fit_gpytorch_mll(mll)\n",
        "\n",
        "# construct the acquisition function\n",
        "qSUCB = qScalarizedUpperConfidenceBound(gp, beta=0.1, weights=torch.tensor([0.1, 0.5]))"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "5070d96c-5673-4bbf-ab72-781d1eabd1bf",
        "collapsed": false,
        "requestMsgId": "1a5d87ac-e176-46dd-b7b9-fb29bfd14971",
        "executionStartTime": 1668651274500,
        "executionStopTime": 1668651274569,
        "customOutput": null
      },
      "source": [
        "# evaluate on single q-batch with q=3\n",
        "qSUCB(torch.rand(3, 2))"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": "tensor([0.4412], grad_fn=<MeanBackward1>)"
          },
          "metadata": {
            "bento_obj_id": "140146377780496"
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "originalKey": "b3e0a586-b362-4ab4-b4c1-7fc0c99a1be4",
        "collapsed": false,
        "requestMsgId": "4f73099f-0dd6-4494-9706-71f50e5dbccf",
        "executionStartTime": 1668651274799,
        "executionStopTime": 1668651274876,
        "customOutput": null
      },
      "source": [
        "# batch-evaluate on two q-batches with q=3\n",
        "qSUCB(torch.rand(2, 3, 2))"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": "tensor([0.5129, 0.5216], grad_fn=<MeanBackward1>)"
          },
          "metadata": {
            "bento_obj_id": "140146572704240"
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "originalKey": "aa45db76-36cb-42e9-9386-649d9ea5e741",
        "showInput": false
      },
      "source": [
        "### A scalarized version of analytic UCB (`q=1` only)\n",
        "\n",
        "We can also write an *analytic* version of UCB for a multi-output model, assuming a multivariate normal posterior and `q=1`. The new class `ScalarizedUpperConfidenceBound` subclasses `AnalyticAcquisitionFunction` instead of `MCAcquisitionFunction`. In contrast to the MC version, instead of using the weights on the MC samples, we directly scalarize the mean vector $\\mu$ and covariance matrix $\\Sigma$ and apply standard UCB on the univariate normal distribution, which has mean $w^T \\mu$ and variance $w^T \\Sigma w$. In addition to the `@t_batch_transform` decorator, here we are also using `expected_q=1` to ensure the input `X` has a `q=1`.\n",
        "\n",
        "*Note:* BoTorch also provides a `ScalarizedObjective` abstraction that can be used with any existing analytic acqusition functions and automatically performs the scalarization we implement manually below. See the end of this tutorial for a usage example."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "5528fc1c-5cf1-4cab-9598-9a41e652d6ea",
        "collapsed": false,
        "requestMsgId": "22877611-7d10-4e11-8da5-17ae46e362b8",
        "executionStartTime": 1668651275106,
        "executionStopTime": 1668651275190,
        "customOutput": null
      },
      "source": [
        "from botorch.acquisition import AnalyticAcquisitionFunction\n",
        "\n",
        "\n",
        "class ScalarizedUpperConfidenceBound(AnalyticAcquisitionFunction):\n",
        "    def __init__(\n",
        "        self,\n",
        "        model: Model,\n",
        "        beta: Tensor,\n",
        "        weights: Tensor,\n",
        "        maximize: bool = True,\n",
        "    ) -> None:\n",
        "        # we use the AcquisitionFunction constructor, since that of \n",
        "        # AnalyticAcquisitionFunction performs some validity checks that we don't want here\n",
        "        super(AnalyticAcquisitionFunction, self).__init__(model)\n",
        "        self.maximize = maximize\n",
        "        self.register_buffer(\"beta\", torch.as_tensor(beta))\n",
        "        self.register_buffer(\"weights\", torch.as_tensor(weights))\n",
        "\n",
        "    @t_batch_mode_transform(expected_q=1)\n",
        "    def forward(self, X: Tensor) -> Tensor:\n",
        "        \"\"\"Evaluate the Upper Confidence Bound on the candidate set X using scalarization\n",
        "\n",
        "        Args:\n",
        "            X: A `(b) x d`-dim Tensor of `(b)` t-batches of `d`-dim design\n",
        "                points each.\n",
        "\n",
        "        Returns:\n",
        "            A `(b)`-dim Tensor of Upper Confidence Bound values at the given\n",
        "                design points `X`.\n",
        "        \"\"\"\n",
        "        self.beta = self.beta.to(X)\n",
        "        batch_shape = X.shape[:-2]\n",
        "        posterior = self.model.posterior(X)\n",
        "        means = posterior.mean.squeeze(dim=-2)  # b x o\n",
        "        scalarized_mean = means.matmul(self.weights)  # b\n",
        "        covs = posterior.mvn.covariance_matrix  # b x o x o\n",
        "        weights = self.weights.view(1, -1, 1)  # 1 x o x 1 (assume single batch dimension)\n",
        "        weights = weights.expand(batch_shape + weights.shape[1:])  # b x o x 1\n",
        "        weights_transpose = weights.permute(0, 2, 1)  # b x 1 x o\n",
        "        scalarized_variance = torch.bmm(\n",
        "            weights_transpose, torch.bmm(covs, weights)\n",
        "        ).view(batch_shape)  # b\n",
        "        delta = (self.beta.expand_as(scalarized_mean) * scalarized_variance).sqrt()\n",
        "        if self.maximize:\n",
        "            return scalarized_mean + delta\n",
        "        else:\n",
        "            return scalarized_mean - delta"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "54811aec-bcad-4d24-8346-fa69c9e1d4c1",
        "showInput": false
      },
      "source": [
        "#### Ad-hoc testing Scalarized-UCB\n",
        "\n",
        "Notice that we pass in an explicit q-batch dimension for consistency, even though `q=1`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "f7a98f13-7d87-4e87-afb8-a8ff087faeef",
        "collapsed": false,
        "requestMsgId": "99dedb82-13c3-45ea-8126-a2daa706a14a",
        "executionStartTime": 1668651275410,
        "executionStopTime": 1668651275490,
        "customOutput": null
      },
      "source": [
        "# construct the acquisition function\n",
        "SUCB = ScalarizedUpperConfidenceBound(gp, beta=0.1, weights=torch.tensor([0.1, 0.5]))"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "78de901c-d537-4b20-8ab2-f1161d12cca3",
        "collapsed": false,
        "requestMsgId": "af017973-596f-4bf5-9efe-54062c3c2715",
        "executionStartTime": 1668651275716,
        "executionStopTime": 1668651275791,
        "customOutput": null
      },
      "source": [
        "# evaluate on single point\n",
        "SUCB(torch.rand(1, 2))"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": "tensor([0.5031], grad_fn=<AddBackward0>)"
          },
          "metadata": {
            "bento_obj_id": "140146378011088"
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "3ffbd8b7-11e6-48d1-8d05-eddd3fc8221c",
        "collapsed": false,
        "requestMsgId": "79891745-83b5-459c-bfc7-d1683396b748",
        "executionStartTime": 1668651276011,
        "executionStopTime": 1668651276084,
        "customOutput": null
      },
      "source": [
        "# batch-evaluate on 3 points\n",
        "SUCB(torch.rand(3, 1, 2))"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": "tensor([-0.6162, -0.8318, -0.1927], grad_fn=<AddBackward0>)"
          },
          "metadata": {
            "bento_obj_id": "140146378289936"
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "23509c96-6cab-4b3c-a8f6-ca87e607f642",
        "showInput": false,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "## Using the custom acquisition function with Ax's Service API"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "636b7375-e46f-4bfb-bd38-724bcaae1b71",
        "showInput": false,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "### Registering the new acquisition function\n",
        "\n",
        "In order to use an acquisition function, Ax needs to know how to generate inputs to construct the acquisition function."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "cd81abdf-050f-470c-be2a-432a52f911b5",
        "showInput": true,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": [],
        "collapsed": false,
        "requestMsgId": "94f7428a-6053-4fca-b902-3d2729c34559",
        "executionStartTime": 1668651276301,
        "executionStopTime": 1668651276386,
        "customOutput": null
      },
      "source": [
        "from typing import List\n",
        "from typing import Any, Dict\n",
        "\n",
        "from botorch.acquisition.input_constructors import acqf_input_constructor\n",
        "\n",
        "\n",
        "@acqf_input_constructor(ScalarizedUpperConfidenceBound)\n",
        "def construct_inputs_scalarized_ucb(\n",
        "    model: Model,\n",
        "    beta: float,\n",
        "    weights: List[float],\n",
        "    **kwargs: Any,\n",
        ") -> Dict[str, Any]:\n",
        "    return {\n",
        "        \"model\": model,\n",
        "        \"beta\": torch.as_tensor(beta, dtype=torch.double),\n",
        "        \"weights\": torch.as_tensor(weights, dtype=torch.double),\n",
        "    }"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "71d0ade7-1341-4dda-b59e-b4f362d9991d",
        "showInput": false,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "### Setting up a `GenerationStrategy` using `BOTORCH_MODULAR` with our custom acquistion function.\n",
        "\n",
        "`BOTORCH_MODULAR` is a convenient wrapper implemented in Ax that facilitates the use of custom BoTorch models and acquisition functions in Ax experiments. In order to customize the way the candidates are generated, we need to construct a new `GenerationStrategy` and pass it into the `AxClient`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "80c4b078-9787-46da-ac78-a574cc73a17d",
        "showInput": true,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": [],
        "collapsed": false,
        "requestMsgId": "a4881c76-cd0f-48f0-9b5f-72f08ff15a21",
        "executionStartTime": 1668651276695,
        "executionStopTime": 1668651283047,
        "customOutput": null
      },
      "source": [
        "from ax.modelbridge.generation_strategy import GenerationStep, GenerationStrategy\n",
        "from ax.modelbridge.registry import Models\n",
        "\n",
        "\n",
        "gs = GenerationStrategy(\n",
        "    steps=[\n",
        "        # Quasi-random initialization step\n",
        "        GenerationStep(\n",
        "            model=Models.SOBOL,\n",
        "            num_trials=5,  # How many trials should be produced from this generation step\n",
        "            model_kwargs={\"seed\": 999},  # Any kwargs you want passed into the model\n",
        "        ),\n",
        "        # Bayesian optimization step using the custom acquisition function\n",
        "        GenerationStep(\n",
        "            model=Models.BOTORCH_MODULAR,\n",
        "            num_trials=-1,  # No limitation on how many trials should be produced from this step\n",
        "            # For `BOTORCH_MODULAR`, we pass in kwargs to specify what surrogate or acquisition function to use.\n",
        "            # `acquisition_options` specifies the set of additional arguments to pass into the input constructor.\n",
        "            model_kwargs={\n",
        "                \"botorch_acqf_class\": ScalarizedUpperConfidenceBound,\n",
        "                \"acquisition_options\": {\"beta\": 0.1, \"weights\": [1.0, 1.0]},\n",
        "            },\n",
        "        ),\n",
        "    ]\n",
        ")"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "b7e98402-9d53-4d33-bfbf-230833edc121",
        "showInput": false,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "### Setting up the experiment\n",
        "\n",
        "We will set up a simple experiment to optimize a simple scalarization of the BraninCurrin function (per the weights above). A detailed tutorial on Service API can be found [here](https://ax.dev/tutorials/gpei_hartmann_service.html).\n",
        "\n",
        "In order to use the `GenerationStrategy` we just created, we will pass it into the `AxClient`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "e1e4f622-4678-4ae1-98a5-02c770c51f87",
        "showInput": true,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": [],
        "collapsed": false,
        "requestMsgId": "1a068448-2ed8-4090-8d3b-a2fa10ee42a5",
        "executionStartTime": 1668651283379,
        "executionStopTime": 1668651283560,
        "customOutput": null
      },
      "source": [
        "from ax.service.ax_client import AxClient\n",
        "from ax.service.utils.instantiation import ObjectiveProperties\n",
        "from botorch.test_functions import BraninCurrin\n",
        "\n",
        "\n",
        "# Initialize the client - AxClient offers a convenient API to control the experiment\n",
        "ax_client = AxClient(generation_strategy=gs)\n",
        "# Setup the experiment\n",
        "ax_client.create_experiment(\n",
        "    name=\"branincurrin_test_experiment\",\n",
        "    parameters=[\n",
        "        {\n",
        "            \"name\": f\"x{i+1}\",\n",
        "            \"type\": \"range\",\n",
        "            # It is crucial to use floats for the bounds, i.e., 0.0 rather than 0.\n",
        "            # Otherwise, the parameter would \n",
        "            \"bounds\": [0.0, 1.0],\n",
        "        }\n",
        "        for i in range(2)\n",
        "    ],\n",
        "    objectives={\n",
        "        \"branin\": ObjectiveProperties(minimize=True),\n",
        "        \"currin\": ObjectiveProperties(minimize=True),\n",
        "    },\n",
        ")\n",
        "# Setup a function to evaluate the trials\n",
        "branincurrin = BraninCurrin()\n",
        "\n",
        "\n",
        "def evaluate(parameters):\n",
        "    x = torch.tensor([[parameters.get(f\"x{i+1}\") for i in range(2)]])\n",
        "    bc_eval = branincurrin(x).squeeze().tolist()\n",
        "    # In our case, standard error is 0, since we are computing a synthetic function.\n",
        "    return {\"branin\": (bc_eval[0], 0.0), \"currin\": (bc_eval[1], 0.0)}"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 6 decimal points.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.utils.instantiation: Due to non-specification, we will use the heuristic for selecting objective thresholds.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x1. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x2. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.utils.instantiation: Created search space: SearchSpace(parameters=[RangeParameter(name='x1', parameter_type=FLOAT, range=[0.0, 1.0]), RangeParameter(name='x2', parameter_type=FLOAT, range=[0.0, 1.0])], parameter_constraints=[]).\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "ae279099-6f99-4d29-858b-d174d388e2d3",
        "showInput": false,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "### Running the BO loop\n",
        "\n",
        "Ax makes this part super simple!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "11c29a68-56a5-4d48-888b-04dd69cfb0af",
        "showInput": true,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": [],
        "collapsed": false,
        "requestMsgId": "050fef34-1c80-46c1-83f6-c245385b7f42",
        "executionStartTime": 1668651283818,
        "executionStopTime": 1668651290050,
        "customOutput": null
      },
      "source": [
        "for i in range(10):\n",
        "    parameters, trial_index = ax_client.get_next_trial()\n",
        "    # Local evaluation here can be replaced with deployment to external system.\n",
        "    ax_client.complete_trial(trial_index=trial_index, raw_data=evaluate(parameters))"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Generated new trial 0 with parameters {'x1': 0.62873, 'x2': 0.51481}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Completed trial 0 with data: {'branin': (46.244598, 0.0), 'currin': (6.842319, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Generated new trial 1 with parameters {'x1': 0.434883, 'x2': 0.396266}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Completed trial 1 with data: {'branin': (14.735401, 0.0), 'currin': (8.740173, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Generated new trial 2 with parameters {'x1': 0.075645, 'x2': 0.934926}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Completed trial 2 with data: {'branin': (2.808084, 0.0), 'currin': (4.10731, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Generated new trial 3 with parameters {'x1': 0.863245, 'x2': 0.038764}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Completed trial 3 with data: {'branin': (9.956846, 0.0), 'currin': (10.342199, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Generated new trial 4 with parameters {'x1': 0.953918, 'x2': 0.808236}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:43] ax.service.ax_client: Completed trial 4 with data: {'branin': (95.420815, 0.0), 'currin': (4.715139, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:44] ax.service.ax_client: Generated new trial 5 with parameters {'x1': 1.0, 'x2': 0.343958}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:45] ax.service.ax_client: Completed trial 5 with data: {'branin': (6.593266, 0.0), 'currin': (7.800417, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:45] ax.service.ax_client: Generated new trial 6 with parameters {'x1': 0.545885, 'x2': 0.0}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:45] ax.service.ax_client: Completed trial 6 with data: {'branin': (5.420934, 0.0), 'currin': (11.428976, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:47] ax.service.ax_client: Generated new trial 7 with parameters {'x1': 0.123588, 'x2': 0.0}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:47] ax.service.ax_client: Completed trial 7 with data: {'branin': (151.344849, 0.0), 'currin': (12.426076, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:48] ax.service.ax_client: Generated new trial 8 with parameters {'x1': 0.045172, 'x2': 0.0}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:48] ax.service.ax_client: Completed trial 8 with data: {'branin': (240.222977, 0.0), 'currin': (7.477064, 0.0)}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:49] ax.service.ax_client: Generated new trial 9 with parameters {'x1': 0.120649, 'x2': 0.032781}.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:49] ax.service.ax_client: Completed trial 9 with data: {'branin': (142.032639, 0.0), 'currin': (12.317077, 0.0)}.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "0e35aa1a-766a-4c5e-924f-852db76c3139",
        "showInput": false,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "### Viewing trials and plotting the Pareto frontier\n",
        "\n",
        "View the trials attached to the experiment."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "e3299fb3-d9f7-45e3-ba34-8b5b0c29dea1",
        "showInput": true,
        "customInput": null,
        "collapsed": false,
        "requestMsgId": "54a67064-79e2-454a-b3d9-18d0a4f23ac8",
        "executionStopTime": 1668651290364,
        "executionStartTime": 1668651290295,
        "customOutput": null
      },
      "source": [
        "ax_client.generation_strategy.trials_as_df"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[INFO 11-16 18:14:50] ax.modelbridge.generation_strategy: Note that parameter values in dataframe are rounded to 2 decimal points; the values in the dataframe are thus not the exact ones suggested by Ax in trials.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": "   Generation Step  ...              Arm Parameterizations\n0                0  ...  {'0_0': {'x1': 0.63, 'x2': 0.51}}\n1                0  ...   {'1_0': {'x1': 0.43, 'x2': 0.4}}\n2                0  ...  {'2_0': {'x1': 0.08, 'x2': 0.93}}\n3                0  ...  {'3_0': {'x1': 0.86, 'x2': 0.04}}\n4                0  ...  {'4_0': {'x1': 0.95, 'x2': 0.81}}\n5                1  ...   {'5_0': {'x1': 1.0, 'x2': 0.34}}\n6                1  ...   {'6_0': {'x1': 0.55, 'x2': 0.0}}\n7                1  ...   {'7_0': {'x1': 0.12, 'x2': 0.0}}\n8                1  ...   {'8_0': {'x1': 0.05, 'x2': 0.0}}\n9                1  ...  {'9_0': {'x1': 0.12, 'x2': 0.03}}\n\n[10 rows x 5 columns]",
            "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Generation Step</th>\n      <th>Generation Model</th>\n      <th>Trial Index</th>\n      <th>Trial Status</th>\n      <th>Arm Parameterizations</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>Sobol</td>\n      <td>0</td>\n      <td>COMPLETED</td>\n      <td>{'0_0': {'x1': 0.63, 'x2': 0.51}}</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0</td>\n      <td>Sobol</td>\n      <td>1</td>\n      <td>COMPLETED</td>\n      <td>{'1_0': {'x1': 0.43, 'x2': 0.4}}</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0</td>\n      <td>Sobol</td>\n      <td>2</td>\n      <td>COMPLETED</td>\n      <td>{'2_0': {'x1': 0.08, 'x2': 0.93}}</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0</td>\n      <td>Sobol</td>\n      <td>3</td>\n      <td>COMPLETED</td>\n      <td>{'3_0': {'x1': 0.86, 'x2': 0.04}}</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0</td>\n      <td>Sobol</td>\n      <td>4</td>\n      <td>COMPLETED</td>\n      <td>{'4_0': {'x1': 0.95, 'x2': 0.81}}</td>\n    </tr>\n    <tr>\n      <th>5</th>\n      <td>1</td>\n      <td>BoTorch</td>\n      <td>5</td>\n      <td>COMPLETED</td>\n      <td>{'5_0': {'x1': 1.0, 'x2': 0.34}}</td>\n    </tr>\n    <tr>\n      <th>6</th>\n      <td>1</td>\n      <td>BoTorch</td>\n      <td>6</td>\n      <td>COMPLETED</td>\n      <td>{'6_0': {'x1': 0.55, 'x2': 0.0}}</td>\n    </tr>\n    <tr>\n      <th>7</th>\n      <td>1</td>\n      <td>BoTorch</td>\n      <td>7</td>\n      <td>COMPLETED</td>\n      <td>{'7_0': {'x1': 0.12, 'x2': 0.0}}</td>\n    </tr>\n    <tr>\n      <th>8</th>\n      <td>1</td>\n      <td>BoTorch</td>\n      <td>8</td>\n      <td>COMPLETED</td>\n      <td>{'8_0': {'x1': 0.05, 'x2': 0.0}}</td>\n    </tr>\n    <tr>\n      <th>9</th>\n      <td>1</td>\n      <td>BoTorch</td>\n      <td>9</td>\n      <td>COMPLETED</td>\n      <td>{'9_0': {'x1': 0.12, 'x2': 0.03}}</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
            "application/vnd.dataresource+json": {
              "schema": {
                "fields": [
                  {
                    "name": "index",
                    "type": "integer"
                  },
                  {
                    "name": "Generation Step",
                    "type": "integer"
                  },
                  {
                    "name": "Generation Model",
                    "type": "string"
                  },
                  {
                    "name": "Trial Index",
                    "type": "integer"
                  },
                  {
                    "name": "Trial Status",
                    "type": "string"
                  },
                  {
                    "name": "Arm Parameterizations",
                    "type": "string"
                  }
                ],
                "primaryKey": [
                  "index"
                ],
                "pandas_version": "0.20.0"
              },
              "data": [
                {
                  "index": 0,
                  "Generation Step": 0,
                  "Generation Model": "Sobol",
                  "Trial Index": 0,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "0_0": {
                      "x1": 0.63,
                      "x2": 0.51
                    }
                  }
                },
                {
                  "index": 1,
                  "Generation Step": 0,
                  "Generation Model": "Sobol",
                  "Trial Index": 1,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "1_0": {
                      "x1": 0.43,
                      "x2": 0.4
                    }
                  }
                },
                {
                  "index": 2,
                  "Generation Step": 0,
                  "Generation Model": "Sobol",
                  "Trial Index": 2,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "2_0": {
                      "x1": 0.08,
                      "x2": 0.93
                    }
                  }
                },
                {
                  "index": 3,
                  "Generation Step": 0,
                  "Generation Model": "Sobol",
                  "Trial Index": 3,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "3_0": {
                      "x1": 0.86,
                      "x2": 0.04
                    }
                  }
                },
                {
                  "index": 4,
                  "Generation Step": 0,
                  "Generation Model": "Sobol",
                  "Trial Index": 4,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "4_0": {
                      "x1": 0.95,
                      "x2": 0.81
                    }
                  }
                },
                {
                  "index": 5,
                  "Generation Step": 1,
                  "Generation Model": "BoTorch",
                  "Trial Index": 5,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "5_0": {
                      "x1": 1,
                      "x2": 0.34
                    }
                  }
                },
                {
                  "index": 6,
                  "Generation Step": 1,
                  "Generation Model": "BoTorch",
                  "Trial Index": 6,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "6_0": {
                      "x1": 0.55,
                      "x2": 0
                    }
                  }
                },
                {
                  "index": 7,
                  "Generation Step": 1,
                  "Generation Model": "BoTorch",
                  "Trial Index": 7,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "7_0": {
                      "x1": 0.12,
                      "x2": 0
                    }
                  }
                },
                {
                  "index": 8,
                  "Generation Step": 1,
                  "Generation Model": "BoTorch",
                  "Trial Index": 8,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "8_0": {
                      "x1": 0.05,
                      "x2": 0
                    }
                  }
                },
                {
                  "index": 9,
                  "Generation Step": 1,
                  "Generation Model": "BoTorch",
                  "Trial Index": 9,
                  "Trial Status": "COMPLETED",
                  "Arm Parameterizations": {
                    "9_0": {
                      "x1": 0.12,
                      "x2": 0.03
                    }
                  }
                }
              ]
            }
          },
          "metadata": {
            "bento_obj_id": "140146254574736"
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "ca47b562-3c01-4557-9c76-a7143a978dae",
        "showInput": false,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "Plot the Pareto frontier.\n",
        "\n",
        "Note that we do not expect a good coverage of the Pareto frontier since we use very small number of evaluations and our acquisition function naively optimizes the sum of the two objectives."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "5bbd1ba3-1f88-45d3-b4a9-c660a4ff9449",
        "showInput": true,
        "customInput": null,
        "code_folding": [],
        "hidden_ranges": [],
        "collapsed": false,
        "requestMsgId": "6357abd4-a4b5-4bcb-bf11-9d0532db6072",
        "executionStopTime": 1668651301173,
        "executionStartTime": 1668651290670,
        "customOutput": null
      },
      "source": [
        "from ax.plot.pareto_frontier import plot_pareto_frontier\n",
        "from ax.plot.pareto_utils import compute_posterior_pareto_frontier\n",
        "from ax.utils.notebook.plotting import render\n",
        "\n",
        "\n",
        "objectives = ax_client.experiment.optimization_config.objective.objectives\n",
        "frontier = compute_posterior_pareto_frontier(\n",
        "    experiment=ax_client.experiment,\n",
        "    data=ax_client.experiment.fetch_data(),\n",
        "    primary_objective=objectives[1].metric,\n",
        "    secondary_objective=objectives[0].metric,\n",
        "    absolute_metrics=[\"branin\", \"currin\"],\n",
        ")\n",
        "render(plot_pareto_frontier(frontier, CI_level=0.90))"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAu4AAAH0CAYAAABiuKiqAAAgAElEQVR4XuzdB5hcVdnA8Xd2dmZ7SzY9lEACiAYlIlWKhKJUAxiQIiKGYuigCAiGiKDSgmAQsBBQQRSMiAiICvohCiogoAldQnrd3Wyd9j33bnZJtiTznnN2d87Mf57ne/SDM+/c+zsX/O/kzmwkk8lkhAcCCCCAAAIIIIAAAgjktECEcM/p/eHgEEAAAQQQQAABBBAIBQh3LgQEEEAAAQQQQAABBDwQINw92CQOEQEEEEAAAQQQQAABwp1rAAEEEEAAAQQQQAABDwQIdw82iUNEAAEEEEAAAQQQQIBw5xpAAAEEEEAAAQQQQMADAcLdg03iEBFAAAEEEEAAAQQQINy5BhBAAAEEEEAAAQQQ8ECAcPdgkzhEBBBAAAEEEEAAAQQId64BBBBAAAEEEEAAAQQ8ECDcPdgkDhEBBBBAAAEEEEAAAcKdawABBBBAAAEEEEAAAQ8ECHcPNolDRAABBBBAAAEEEECAcOcaQAABBBBAAAEEEEDAAwHC3YNN4hARQAABBBBAAAEEECDcuQYQQAABBBBAAAEEEPBAgHD3YJM4RAQQQAABBBBAAAEECHeuAQQQQAABBBBAAAEEPBAg3D3YJA4RAQQQQAABBBBAAAHCnWsAAQQQQAABBBBAAAEPBAh3DzaJQ0QAAQQQQAABBBBAgHDnGkAAAQQQQAABBBBAwAMBwt2DTeIQEUAAAQQQQAABBBAg3LkGEEAAAQQQQAABBBDwQIBw92CTOEQEEEAAAQQQQAABBHI23JetXCNTP3NRrx0aXlctk7YbL2eefJTsvutOXu7gb574q3z12jv7Pfay0rj847H+//5An/QnT/yKTJk8Sa69bMZAvxTzEUAAAQQQQAABBLIUyPlwn/apfeWoQ/YJTyedScvylWvlF795Sl545XWZM/scOXi/3bI8Vftl077wNTn/i8fJAXt/xGpYV7h/+ewTZPttx/WaVRwtkr12+6DVa2T75IamZtn7yJny/O++L+VlpeHTfvuHv0n9sBrZY9cPZDuGdQgggAACCCCAAAIDLJDz4X7uF46Rsz531CYMbe0dctjJl8rI4bVy//e/bkyUSCQlFivO6vktre2yx+Fnya3XnO8s3O+99XKZMnmHrF4/WJRKpSUSiUhRUSTr52xp4V/+/rKcdemNm4T7lp6Tzd/X2GYzjzUIIIAAAggggEChC3gZ7sGmfemym+Xl/74lf5l/a7iH7y5eLjff+Qv5579fk8b1LWHUHzZ1T5n5+U93x/nXvv1D+c9r78jpJx4u1373J3LAXh+Rb371i+Hz75v/B7nvV38I55SXl8q+u+8il5x9vIwYXivPvbBATrvwW93Xysa3svzpry/IHff+Rl57c1EY1ZMmjJMZJx0pU/ed0u+11fWO+5bCfeny1XLQ8RfLty4/Q+Y/9n/y/EsL5Hc//Y6MG10vW3rdlavXyQHHXiDXX3m2vPjq6/LYn56TltY22XH7reXKCz8nO03cWr7341/J3Hm/7j7O/fb8sNz+rQul560ywQ8t3/3hg/LE08/L6jWNMmJ4jRxx8N4y87RpEiuOhs/fnG2h/0PG+SOAAAIIIIAAAi4EvA33o069PAzyB38wW9LpjHzyxC9LTXWlXHH+yRLcB7/gjXflsmvvlFOOO1TO/+KxodXVN94tT//tJdlq7Eg54+QjZfyYEbLN+FFheAdhGry7H8T+ilVrZfbN90gmnQ7nR4qK5J8vLZTTL/6OfPtrZ8rHPzZZamsqpevd6s8ccYCcdOxBEpGI3PvgE/LLR56WO75zsXx898l97lG24b5qTYPsf8z5suP2W8lB++0me+/2QfnApG3CHySCd8k397prG5rk40efK2NH18tZpxwlRxy8l6xvbpUzvnxD+APGL++6WlrbOuTHP/9dGPC/v/8Gqa6qkMqKsl7hHpz3qwvfkasuPFU+/MHt5aVX35Srb7o7vE3pmktP36yti4uUGQgggAACCCCAAAIiOR/uwbu6Z5x8RLhXmYzIylVrZd4vHpefPPh7uerCz8nxRx8Yhvt7S1eE92gH92Z3Pc678ruyZNnqMFKDxzVz7u18Z33ulbLLztuHf629IxEG7r57TJabZs3sfm7wbv4JZ88O37E+bOoe8srCt+X4M6+W7117QfetMqeef114z/3vfvrtMIaDR3A7yyEnXCLbbztW7rz+ks2G+103XCIf+eDEXmuKioqktCQuXfEd/AAQ/CDQ9cjmdbuee+gBu8tNs77U/dy7fvqIzLnrl/Li738Q/uDz4/t/Jzd8/+eb3Cqz8Tvu/3r5dTnl3G/K1y44RT776andc35436Phn3D84YGbZNSIuj5t+QcMAQQQQAABBBBAwJ1Azod7X6daU1UR3qZx0jEHdf/t/77+P/nBz34rwX82t7RJJpORpubW8N33J39+Y3e4//zhP8pLT/6o+z7xlxe8LSecdbXMuuTz4TvYGz/2OPxsOeKgvcJbS/oK948eeoZ88hO7d99u0/XcC79+m/zjpYXdt/H0PIctfavMbh/eUebdcll3uAfn+qVTj+4ek83rdoX7xWdNly+ccFj3c3/+6z+Gf5rw9EO3hD/kbCncf3T/o3Lj9x+QR3/y7fBPJ7oegfNxM74e/lAQ/HAQ/FDU09bdZcokBBBAAAEEEEAAgZwP9+Ad9WMO2zfcqeBWlLqaShkzanj3O9zBXw/e9f70aVfIVuNGymXnnhT+/eJoNIzJILg3DvdH//g3+evD3+ve+Wf/8ap88ZLrw/WRHh/6DD5gedC+H5VbvnFur3BPplLy4amny6mfOVS+MvOzm1xJX7/hx/LwE3+VF564q88rrCvcgx8IdthufK81lRXl4V/viu/Lzzu5+4eUbF+3r+cGL6QN91t+8KDc+ZPf9Pkh3sDn8vNOkpOOOTi07mnLP14IIIAAAggggAAC7gRyPtz7+laZnqcf3P4ShOPj910f3rfe9QiC/J1FyzYb7sG929PPnCXBVzPut9eHe8mWl5XI6BHD+nzHfbdPniGH7P+xXt93fv6Vt8qLr74Rvqvd1yPbe9z7i+9sXtdVuN/988fk+tvvl3u+e7nU1Vb1Op3htdVSU11BuLv7Z5JJCCCAAAIIIIBAnwJ5Ee7BO8LBO8N//+3t4Ycrg8eiJSvkiFMuC78B5ckHbgr/Wl/vCnd0JGSfo8+VaZ/6uATvbG/8ePN/S2S7rceE7+533Spz27Xnyyf23jVcFnxo893FK+Txn13ffetN8I548IujJu+0nQRrByLcs3ldbbg/9+j3paK883vcN77HPfgB5KSZ1/T6zvzgG2oaGpvDP93oz5Z/5hBAAAEEEEAAAQTcCeRFuHd9XWPwTTEnTpsqb7y9OPwA5oRtxoRfgzj/R9fIuDEj5Nu3/azP2zmCb5W5fd58uejM6bL/Xh+RjkQi/CVP9//6j/LAHbPCr04MfhAIgja4LSS4dWf7bcaGXz0ZvKsf/P+nTv+kJJOp8FtafvvkszLvluA72icNSLj/7Z//2eLrZhvuDz36Z7nyOz8Kv3Iy+MaaiRPG9fpWmeAc33xncfiDzc47bBt+685tP/qV/G/xcnnknuvCD9Jyq4y7fyiZhAACCCCAAAII9CWQF+EenFjwbSk/fehJaWxqDuPy0nNOlOA2lzO/fIO0tnfIPbdcJvfN/2O/92Fv/D3uJSVx+dCOE+TsU4+W4IOiXY8rvvUD+d0f/x7eGvKrH14TfiXk08++FEb/wrfek6JIJHztmad9WvacsnO/V5ztrTLB4C29brbhHvzm1MAoOP7gB40f3viVPr/H/dYfPSRPPPW8rFyzTmqrK8Pzu/CMz/COO/9eQQABBBBAAAEEBkkgZ8N9kM6fl0EAAQQQQAABBBBAwAsBwt2LbeIgEUAAAQQQQAABBApdgHAv9CuA80cAAQQQQAABBBDwQoBw92KbOEgEEEAAAQQQQACBQhcg3Av9CuD8EUAAAQQQQAABBLwQINy92CYOEgEEEEAAAQQQQKDQBQj3Qr8COH8EEEAAAQQQQAABLwQIdy+2iYNEAAEEEEAAAQQQKHQBwr3QrwDOHwEEEEAAAQQQQMALAcLdi23iIBFAAAEEEEAAAQQKXYBwL/QrgPNHAAEEEEAAAQQQ8EKAcPdimzhIBBBAAAEEEEAAgUIXINwL/Qrg/BFAAAEEEEAAAQS8ECDcvdgmDhIBBBBAAAEEEECg0AUI90K/Ajh/BBBAAAEEEEAAAS8ECHcvtomDRAABBBBAAAEEECh0AcK90K8Azh8BBBBAAAEEEEDACwHC3Ytt4iARQAABBBBAAAEECl2AcC/0K4DzRwABBBBAAAEEEPBCgHD3Yps4SAQQQAABBBBAAIFCFyDcC/0K4PwRQAABBBBAAAEEvBAg3L3YJg4SAQQQQAABBBBAoNAFCPdCvwI4fwQQQAABBBBAAAEvBAh3L7aJg0QAAQQQQAABBBAodAHCvdCvAM4fAQQQQAABBBBAwAsBwt2LbeIgEUAAAQQQQAABBApdgHAv9CuA80cAAQQQQAABBBDwQoBw92KbOEgEEEAAAQQQQACBQhcg3Av9CuD8EUAAAQQQQAABBLwQINy92CYOEgEEEEAAAQQQQKDQBQj3Qr8COH8EEEAAAQQQQAABLwQIdy+2iYNEAAEEEEAAAQQQKHQBwr3QrwDOHwEEEEAAAQQQQMALAcLdi23iIPNJYMnq1nw6Hc4FAQQQQAABY4Gxw8uMn1uITyTcC3HXOechFSDch5SfF0cAAQQQyCEBwl23GYS7zovVCFgLEO7WhAxAAAEEEMgTAcJdt5GEu86L1QhYCxDu1oQMQAABBBDIEwHCXbeRhLvOi9UIWAsQ7taE3g8YM6xMIhGRpWtaJZPx/nQ4AY8FYtGI1FbGZWVDu8dnwaH7LEC463aPcNd5sRoBawHC3ZrQ+wGEu/dbmDcnQLjnzVZ6eyKEu27rCHedF6sRsBYg3K0JvR9AuHu/hXlzAoR73myltydCuOu2jnDXebEaAWsBwt2a0PsBhLv3W5g3J0C4581WensihLtu6wh3nRerEbAWINytCb0fQLh7v4V5cwKEe95spbcnQrjrto5w13mxGgFrAcLdmtD7AYS791uYNydAuOfNVnp7IoS7busId50XqxGwFiDcrQm9H0C4e7+FeXMChHvebKW3J0K467aOcNd5sRoBawHC3ZrQ+wGEu/dbmDcnQLjnzVZ6eyKEu27rCHedF6sRsBYg3K0JvR9AuHu/hXlzAoR73myltydCuOu2jnDXebEaAWsBwt2a0PsBhLv3W5g3J0C4581WensihLtu6wh3nRerEbAWINytCb0fQLh7v4V5cwKEe95spbcnQrjrto5w13mxGgFrAcLdmtD7AYS791uYNydAuOfNVnp7IoS7busId50XqxGwFiDcrQm9H0C4e7+FeXMChHvebKW3J0K467aOcNd5sRoBawHC3ZrQ+wGEu/dbmDcnQLjnzVZ6eyKEu27rCHedF6sRsBYg3K0JvR9AuHu/hXlzAoR73myltydCuOu2jnDXebEaAWsBwt2a0PsBhLv3W5g3J0C4581WensihLtu6wh3nRerEbAWINytCb0fQLjrtnBVc7tkMv0/Z0RliW4gq7sFCHcuhqEWINx1O0C467xYjYC1AOFuTej9AMJdt4ULVzRJZjPlvsPIKimKRHRDWR0KEO5cCEMtQLjrdoBw13mxGgFrAcLdmtD7AYS7bgs3fsd9dXN7+OThFXER6Yx13nHXeW68mnA3t+OZbgQId50j4a7zYjUC1gKEuzWh9wMId/MtXLi8UYK7ZniX3dyQcHdjxxQ3AoS7zpFw13mxGgFrAcLdmtD7AYS7+RYS7uZ2fT2Td9zdejJNL0C468wId50XqxGwFiDcrQm9H0C4m28h4W5uR7i7tWOaGwHCXedIuOu8WI2AtQDhbk3o/QDC3XwLCXdzO8LdrR3T3AgQ7jpHwl3nxWoErAUId2tC7wcQ7uZbSLib2xHubu2Y5kaAcNc5Eu46L1YjYC1AuFsTej+AcDffQsLd3I5wd2vHNDcChLvOkXDXebEaAWsBwt2a0PsBhLtuC1c2t0trRyp80nvrWsL/HFtT1v3d7fUVJVIej+qGsjoU4MOpXAhDLUC463aAcNd5sRoBawHC3ZrQ+wGEu24LF69rlab2RPikZY1t4X+OqiqRyIZfujS2tkyqS2K6oawm3LkGckKAcNdtA+Gu82I1AtYChLs1ofcDCHfdFrYn05La8JtTX1y0VjpSafnA6GqpKu2M9ZLiIonym1N1qBtW8467ERtPcihAuOswCXedF6sRsBYg3K0JvR9AuJtv4d/eXiVtybR8ZHyt1JYFvz2Vh40A4W6jx3NdCBDuOkXCXefFagRCgXUN6+WwUy6V808/Vo4/+sBeKiecPVsWvP4/kQ3vAlZXlsuff/XdcB3hzkVEuJtfA4S7uV1fzyTc3XoyTS9AuOvMCHedF6sRCAUuv+4uee7FBTLjxMP7DPfDT/mq3DL7XJk4YVwvMcKdi4hwN78GCHdzO8LdrR3T3AgQ7jpHwl3nxWoE5LkXFsjcefNl4rbjZNKEcX2G+/7HnC8/v+PrMnrEMMKda6aXAOFuflEQ7uZ2hLtbO6a5ESDcdY6Eu86L1QUukEgkZfqZs+TGWTPlZw892W+473rIDNlvj13kXy+/JsPqquWiM6bL/nt9ONTjHfcCv4hEhHA3vwYId3M7wt2tHdPcCBDuOkfCXefF6gIXmHv3fMlkMjLztGlyzZx7+wz3dDojV37nh3LQfh+Vj+++i/zfc/+Wr3zj+/LwvOtkzMhhvcK9rpIP2BXaZVVW0vmd463tnd9NziN7gT8tXC4tHSnZa/vhMqy8JPsnsrJPgaKISKy4SNoTaYQQGBSBtes7Nnkdwl3HTrjrvFhdwALvLFomF189V+6be6XE47F+w70voi9c+G055rD95IiD9+oV7l0RV8C0BXfqtZVxiQQfcl7fIZmCO3u7E/7L6yukNZmWj21TJ3WEux2miESLIlJRWiyNLZ3fk88DgYEW6PmGBeGuEyfcdV6sLmCBux94TO6452GJxYpDheaWNolGi+TEaQfJBTOO65ZpaW2X195aJB/54MTuv/a5866Vk445WA494GPcKlPA11DXqXOrjPlFwK0y5nZ9PZNvlXHryTS9AOGuMyPcdV6sRqBboOetMo88+azsOWXn8N34qZ+5UG6++hz5+O6T5f+ee1m+PPt2eeTeb8nwumrCnWuIe9wtrgHC3QKvj6cS7m49maYXINx1ZoS7zovVCPQb7vtNO0/mzD5HpkzeQf7y95fl+tvvl+Ur18j4MSPkKzM/K3vs+oHwuXw4lYuId9zNrwHC3dyOd9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRsBYg3K0JvR9AuJtvIeFubke4u7VjmhsBwl3nSLjrvFiNgLUA4W5N6P0Awt18Cwl3czvC3a0d09wIEO46R8Jd58VqBKwFCHdrQu8HEO7mW0i4m9sR7m7tmOZGgHDXORLuOi9WI2AtQLhbE3o/gHA330LC3dyOcHdrxzQ3AoS7zpFw13mxGgFrAcLdmtD7AYS7+RYS7uZ2hLtbO6a5ESDcdY6Eu86L1QhYCxDu1oTeDyDczbeQcDe3I9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRsBYg3K0JvR9AuJtvIeFubke4u7VjmhsBwl3nSLjrvFiNgLUA4W5N6P0Awt18Cwl3czvC3a0d09wIEO46R8Jd58VqBKwFCHdrQu8HEO7mW0i4m9sR7m7tmOZGgHDXORLuOi9WI2AtQLhbE3o/gHA330LC3dyOcHdrxzQ3AoS7zpFw13mxGgFrAcLdmtD7AYS7+RYS7uZ2hLtbO6a5ESDcdY6Eu86L1QhYCxDu1oTeDyDczbeQcDe3I9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRsBYg3K0JvR9AuJtvIeFubke4u7VjmhsBwl3nSLjrvFiNgLUA4W5N6P0Awt18Cwl3czvC3a0d09wIEO46R8Jd58VqBKwFCHdrQu8HEO7mW0i4m9sR7m7tmOZGgHDXORLuOi9WI2AtQLhbE3o/gHA330LC3dyOcHdrxzQ3AoS7zpFw13mxGgFrAcLdmtD7AYS7+RYS7uZ2hLtbO6a5ESDcdY6Eu86L1QhYCxDu1oTeDyDczbeQcDe3I9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRsBYg3K0JvR9AuJtvIeFubke4u7VjmhsBwl3nSLjrvFiNgLUA4W5N6P0Awt18Cwl3czvC3a0d09wIEO46R8Jd58VqBKwFCHdrQu8HEO7mW0i4m9sR7m7tmOZGgHDXORLuOi9WI2AtQLhbE3o/gHA330LC3dyOcHdrxzQ3AoS7zpFw13mxGgFrAcLdmtD7AYS7+RYS7uZ2hLtbO6a5ESDcdY6Eu86L1QhYCxDu1oTeDyDczbeQcDe3I9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRCAXWNayXw065VM4//Vg5/ugD+1Xpax3hzkVEuJtfA4S7uR3h7taOaW4ECHedI+Gu82I1AqHA5dfdJc+9uEBmnHj4ZsO9r3WEOxcR4W5+DRDu5naEu1s7prkRINx1joS7zovVCMhzLyyQufPmy8Rtx8mkCeP6Dff+1hHuXESEu/k1QLib2xHubu2Y5kaAcNc5Eu46L1YXuEAikZTpZ86SG2fNlJ899GS/4b65dYR7gV9EIkK4m18DhLu5HeHu1o5pbgQId50j4a7zYnWBC8y9e75kMhmZedo0uWbOvf2G++bW9Qz3eKyowFUL7/SHV5dIRERWN7ZLpvBO3+qMn3ljpbQmU/LRrYdJXXncahZPFikuikhlWUzWNXfAgcCgCHQk0pu8DuGuYyfcdV6sLmCBdxYtk4uvniv3zb1S4vFYv+G+pXU9w72+uqSAVQvz1Lt+WOv5P2CFqaE76z/8d5m0JFKy9/b1MryCf3Z0er1XRyIixdGIJJL8CGlryfOzE1jV2E64Z0fV5yrC3QKPpxaWwN0PPCZ33POwxGLF4Yk3t7RJNFokJ047SC6YcVw3xpbWcatMYV03fZ0tt8qYXwPcKmNu19czY9GI1FbGZWXDpjHl9lWYhkD/Arzjrrs6CHedF6sR6BboeavMI08+K3tO2Vnqh9VsotRzHeHORUS4m18DhLu5HeHu1o5pbgQId50j4a7zYjUC/Yb7ftPOkzmzz5Epk3cg3LlONitAuJtfIIS7uR3h7taOaW4ECHedI+Gu82I1AtYCvONuTej9AMLdfAsJd3M7wt2tHdPcCBDuOkfCXefFagSsBQh3a0LvBxDu5ltIuJvbEe5u7ZjmRoBw1zkS7jovViNgLUC4WxN6P4BwN99Cwt3cjnB3a8c0NwKEu86RcNd5sRoBawHC3ZrQ+wGEu/kWEu7mdoS7WzumuREg3HWOhLvOi9UIWAsQ7taE3g8g3M23kHA3tyPc3doxzY0A4a5zJNx1XqxGwFqAcLcm9H4A4W6+hYS7uR3h7taOaW4ECHedI+Gu82I1AtYChLs1ofcDCHfzLSTcze0Id7d2THMjQLjrHAl3nRerEbAWINytCb0fQLibbyHhbm5HuLu1Y5obAcJd50i467xYjYC1AOFuTej9AMLdfAsJd3M7wt2tHdPcCBDuOkfCXefFagSsBQh3a0LvBxDu5ltIuJvbEe5u7ZjmRoBw1zkS7jovViNgLUC4WxN6P4BwN99Cwt3cjnB3a8c0NwKEu86RcNd5sRoBawHC3ZrQ+wGEu/kWEu7mdoS7WzumuREg3HWOhLvOi9UIWAsQ7taE3g8g3M23kHA3tyPc3doxzY0A4a5zJNx1XqxGwFqAcLcm9H4A4W6+hYS7uR3h7taOaW4ECHedI+Gu82I1AtYChLs1ofcDCHfzLSTcze0Id7d2THMjQLjrHAl3nRerEbAWINytCb0fQLibbyHhbm5HuLu1Y5obAcJd50i467xYjYC1AOFuTej9AMLdfAsJd3M7wt2tHdPcCBDuOkfCXefFagSsBQh3a0LvBxDu5ltIuJvbEe5u7ZjmRoBw1zkS7jovViNgLUC4WxN6P4BwN99Cwt3cjnB3a8c0NwKEu86RcNd5sRoBawHC3ZrQ+wGEu/kWEu7mdoS7WzumuREg3HWOhLvOi9UIWAsQ7taE3g8g3M23kHA3tyPc3doxzY0A4a5zJNx1XqxGwFqAcLcm9H4A4W6+hYS7uR3h7taOaW4ECHedI+Gu82I1AtYChLs1ofcDCHfzLSTcze0Id7d2THMjQLjrHAl3nRerEbAWINytCb0fQLibbyHhbm5HuLu1Y5obAcJd50i467xYjYC1AOFuTej9AMLdfAsJd3M7wt2tHdPcCBDuOkfCXefFagSsBQh3a0LvBxDu5ltIuJvbEe5u7ZjmRoBw1zkS7jovViNgLUC4WxN6P4BwN9/CrnCfMLxCyuPF/Q4aXh6XoqKI+QsVyDNj0YjUVsZlZUN7gZwxp5lrAoS7bkcId50XqxGwFiDcrQm9H0C4m29hV7jXlcekpDja76CJI1TcF6wAACAASURBVKqkWBHuq5vbpbkj1e+8YeVxqSzp/wcF8zMa2mcS7kPrz6uLEO66q4Bw13mxGgFrAcLdmtD7AYS7+Rb29Y77mpYOyWQyEsR1JNL5Lrv2Hfdlja2yrjXR74GNri6T2rKY+YHn6DMJ9xzdmAI6LMJdt9mEu86L1QhYCxDu1oTeDyDczbewr3vcX1/ZJKl0RibWV0pxtMhoeEcqI8l0OnzumuZ2Wd+elOEVJVKx4V32eLRI9Q6+0UEMwZMI9yFA5yU3ESDcdRcE4a7zYjUC1gKEuzWh9wMId/MtHKhw3/iIlja0SkNbQkZXl0ptWdz8YD14JuHuwSbl+SES7roNJtx1XqxGwFqAcLcm9H4A4W6+hQMR7i0dKVnV/P6HM4N33FsSqTDag/vay2JRGVFZYn7QOfxMwj2HN6dADo1w12004a7zYjUC1gKEuzWh9wMId/MtHIhwX93SIYvXtXQfVGNbQtoSKakqDaK9WGrKY7JNbYX5QefwMwn3HN6cAjk0wl230YS7zovVCFgLEO7WhN4PINzNt3Agwn1pY5usXN8WHlR7Mi2rmtokuNu9vjIehvvIytLwtpl8fBDu+birfp0T4a7bL8Jd58VqBKwFCHdrQu8HEO7mWzgQ4b62pUOa2pPhQTW0dMh761rCb6fZelhFeKtMbXlMqkvy7xtlgvMl3M2vRZ7pRoBw1zkS7jovViMQCqxrWC+HnXKpnH/6sXL80Qf2Unl5wdtyzc33yFvvLpXRI+rk4rOOlwP2/ki4jnDnIiLcza+BgQj3jY/mrVXrZeHyRimNR2XX8XV8ONV8q3gmAlkJEO5ZMXUvItx1XqxGIBS4/Lq75LkXF8iMEw/vFe7B90lPnX6RXDjjM3LEwXvJU8++KF+efbs88/D3pCQeI9y5hoRwN78ICHdzu76eyTvubj2Zphcg3HVmhLvOi9UIyHMvLJC58+bLxG3HyaQJ43qFe1t7hzz+1PNy9KH7dGtNOWSGPDzvWhk/ZgThzjVEuFtcA4S7BV4fTyXc3XoyTS9AuOvMCHedF6sLXCCRSMr0M2fJjbNmys8eerLPcN+YKFj/0KN/lvvm/1Ee/MFsiUaLCPcCv4aC0+cdd/OLgHA3t+Mdd7d2THMjQLjrHAl3nRerC1xg7t3zw1+tPvO0aXLNnHs3G+5/+usLcu4V35VR9XUy5xvnyuSdJoR6Pe9xH1GTn98PXeCXymZPP1bc+ds9E8nO39TJI3uBJ/+7LPyO9X22rw9/s2nweGVxgyTSafnQ2BqJKX9zanN7UjIbvfzry5vk1aUNUh6Pym7bDJeth5Vnf3AeroxERKJFEUmmNlbw8EQ4ZG8EVja8/zsTgoMm3HVbR7jrvFhdwALvLFomF189V+6be6XE47EthntAlUyl5PkXFsil37xD7r/9Khk7ur5XuHdFXAHTFtyp19eUSEREVjW0bxKNBQdhcMLPvLky/I71KdsMk7oNv9U0+DBpKp2RSSOr1OH+2oomSabe/wFqRVObBL85NV4cDaN9162GSXFRsFv5+QjOrao8JmvXd+TnCXJWOSfQ8w0Lwl23RYS7zovVBSxw9wOPyR33PCyxWHGo0NzSFt76cuK0g+SCGcd1y6xe2yjP/uPV8IOpXY/PX/AtmX7kJ+SwqXtwq0wBX0Ndp86tMuYXgetbZRava5VUpvPd5paOpAThvrK5TUqiUdmuvkp2HFUl0eBt6Tx9cI97nm6sR6dFuOs2i3DXebEagW6BnrfKPPLks7LnlJ3DsD9o+kVy06yZsu8eu8jCNxfJ5867Vn5y2xUyacJ4wp1riHvcLa4Bl+GeSqdlTUui+2gWrW2RVevbwltxSouj8oHR1bLNsPz8jaldJ024W1yMPNWJAOGuYyTcdV6sRqDfcN9v2nkyZ/Y5MmXyDvKXv/9bbrrjAVmyfLXUVlfKGScfKccevl/4XL7HnYuId9zNrwGX4d6Ryshbq5q6DyZ4t31dayK8FackViQThlfKLmNrzQ/Wg2cS7h5sUp4fIuGu22DCXefFagSsBQh3a0LvBxDu5lvoMtw3fsc9nUnLwuVN0tCWCO+X5x138z3imQhoBAh3jZYI4a7zYjUC1gKEuzWh9wMId/MtdBnuGx9FRyotwezGtqREIhkpjxfzm1PNt4lnIpC1AOGeNVW4kHDXebEaAWsBwt2a0PsBhLv5FhLu5nZ9PZNbZdx6Mk0vQLjrzAh3nRerEbAWINytCb0fQLibbyHhbm5HuLu1Y5obAcJd50i467xYjYC1AOFuTej9AMLdfAsJd3M7wt2tHdPcCBDuOkfCXefFagSsBQh3a0LvBxDu5ltIuJvbEe5u7ZjmRoBw1zkS7jovViNgLUC4WxN6P4BwN99Cwt3cjnB3a8c0NwKEu86RcNd5sRoBawHC3ZrQ+wGEu/kWEu7mdoS7WzumuREg3HWOhLvOi9UIWAsQ7taE3g8g3M23kHA3tyPc3doxzY0A4a5zJNx1XqxGwFqAcLcm9H4A4W6+hYS7uR3h7taOaW4ECHedI+Gu82I1AtYChLs1ofcDCHfzLSTcze0Id7d2THMjQLjrHAl3nRerEbAWINytCb0fQLibbyHhbm5HuLu1Y5obAcJd50i467xYjYC1AOFuTej9AMLdfAsJd3M7wt2tHdPcCBDuOkfCXefFagSsBQh3a0LvBxDu5ltIuJvbEe5u7ZjmRoBw1zkS7jovViNgLUC4WxN6P4BwN99Cwt3cjnB3a8c0NwKEu86RcNd5sRoBawHC3ZrQ+wGEu/kWEu7mdoS7WzumuREg3HWOhLvOi9UIWAsQ7taE3g8g3M23kHA3tyPc3doxzY0A4a5zJNx1XqxGwFqAcLcm9H4A4W6+hYS7uR3h7taOaW4ECHedI+Gu82I1AtYChLs1ofcDCHfzLbQN99XN7ZLO9H79RCotryxtkI5kWiKRjJTHi2XX8XVSWxY3P1gPnhmLRqS2Mi4rG9o9OFoOMR8FCHfdrhLuOi9WI2AtQLhbE3o/gHA330LbcH9tRZOkM73LPZnOyFur1ktRRCQSEcLdfIt4JgIqAcJdxSWEu86L1QhYCxDu1oTeDyDczbfQNtw3fsc9+O/BY3h5XBLpDO+4m28Lz0TAWIBw19ER7jovViNgLUC4WxN6P4BwN99C23Df+JVfW9EY3jYzaUSVpDIZCWY3tiW5VcZ8e3gmAmoBwl1HRrjrvFiNgLUA4W5N6P0Awt18C3uG+5qWdnllSUMY4CMq4xItKtpkeG1ZTEZXl/X5goS7CPe4m1+LPNONAOGucyTcdV6sRsBagHC3JvR+AOFuvoWEu7ldX88k3N16Mk0vQLjrzAh3nRerEbAWINytCb0fQLibb+HmbpUZVhGTNc0J2dy77Bu/Mu+48467+ZXIM10JEO46ScJd58VqBKwFCHdrQu8HEO7mW0i4m9vxjrtbO6a5ESDcdY6Eu86L1QhYCxDu1oTeDyDczbeQcDe3I9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRsBYg3K0JvR9AuJtvIeFubke4u7VjmhsBwl3nSLjrvFiNgLUA4W5N6P0Awt18Cwl3czvC3a0d09wIEO46R8Jd58VqBKwFCHdrQu8HEO7mW0i4m9sR7m7tmOZGgHDXORLuOi9WI2AtQLhbE3o/gHA330LC3dyOcHdrxzQ3AoS7zpFw13mxeggF1je3ykOP/lneXrRM2ts7eh3JtZfNGMKjy/6lCffsrfJ1JeFuvrOEu7kd4e7WjmluBAh3nSPhrvNi9RAKnP3Vm+XNdxbLlF12kFhxca8j+cZXvjBoR7euYb0cdsqlcv7px8rxRx/Y63WD45x14zxZ+Oa7Uj+sRi45+wQ5cJ9dw3WE+6BtU86+EOFuvjWEu7kd4e7WjmluBAh3nSPhrvNi9RAK7H/M+fK7n35HystKhvAoOl/68uvukudeXCAzTjy8z3A/+rQr5LjD95eTjjlYnnn+Fblo1m3y51/dKmWlccJ9yHdv6A+AcDffA8Ld3I5wd2vHNDcChLvOkXDXebF6CAU+fdrX5Fc/+oZEIpEhPAqR515YIHPnzZeJ246TSRPG9Qr3ZColv/rdX2Tap/aV4mg0PNY9Dj9bfnHn1bL1uJGE+5DuXm68OOFuvg+Eu7kd4e7WjmluBAh3nSPhrvNi9RAKPPzEM7LwjUUy46QjpLamckiOJJFIyvQzZ8mNs2bKzx56ss9w73lgL//3LTn/qlvlyZ/fJEVFEcJ9SHYut16UcDffD8Ld3I5wd2vHNDcChLvOkXDXebF6CAUO/eyXZcXqddLRkZDSkrj0fOP9H4/dOeBHN/fu+ZLJZGTmadPkmjn3bjHc31u6Us748g1y5QWfk712+2B4fD3vcS8v6X2//oCfCC8wpAI1lTEJ/txo3frEkB6Hjy/+59dXSGsyJbtvM0zqyjtvm1uwrFGS6bTUV5bIqvXtUlcel3G15Vs8vf8sbZB0JiMfGF0jyXRG/vLGCmlo7Qj3pqIkJh/bZpgMqxj6W/O2eCIWC6JFIuWlxdLUkrSYwlMRyF6gpX3Ta41wz94uWEm467xYPYQCTz/7khQVFfV7BPvuMXlAj+6dRcvk4qvnyn1zr5R4PLbFcF/45iI5/8pb5avnnCgH7P2R7mPrGe61lbEBPW6G555A1w9rPf8HLPeONPeO6E8LV0hLR1L22n64DNsQ7q8uaQjDe2RVXFY0dcjwiriMr9tyuL+yZJ2k0iIfGtsZ7n9auFzWtSTCNwUqS4plz+2Gy/A8D/eiSETixUXSlkjl3mZzRHkp0PMNC8Jdt82Eu86L1QUscPcDj8kd9zwssVjnO+TNLW0SjRbJidMOkgtmHLeJzKIlK2TGJTdI8BWVUyZP2uTv8a0yBXwRbTh1bpUxvwa4Vcbcrq9nxqIRqa2My8qGdreDmYZAlgKEe5ZQG5YR7jovVg+ywAVX3SbnnDZNJk4YJ8F/39xjzuxzBvXoet4q88iTz8qeU3YOv/7x8xd8S44/6hPyqQP36HVMhPugblNOvhjhbr4thLu5HeHu1o5pbgQId50j4a7zYvUgC1x360/lpGMOkq3HjZLgv2/ucdm5Jw3q0fUM9/2mnSfBDw8j6+skuB+/6535roO64aqz5aB9P8qHUwd1l3LzxQh3830h3M3tCHe3dkxzI0C46xwJd50Xq4dIIPiKxeBbXI47Yn8pLysdoqNw87K84+7G0ecphLv57hHu5naEu1s7prkRINx1joS7zovVQyiwz9HnyE9v+5psu9XoITwK+5cm3O0NfZ9AuJvvIOFubke4u7VjmhsBwl3nSLjrvFg9hAK/fvwZefQPf5PDpu4pW40dKfH4pl+j+KEdJwzh0WX/0oR79lb5upJwN99Zwt3cjnB3a8c0NwKEu86RcNd5sXoIBT54wOc3++qvPnX3EB5d9i89GOH+7ppmaUmkZKu6cqno8QNO9kfKyoESINzNZQl3czvC3a2dy2ltiaS8s6Yl/GrO7YYPzS8YdHk+mlmEu0aL73HXabF6SAXWN7dKcXFUIj1/89KGoyqJ+/F96LkS7mtaOiSVzsjw8nj4G115DJ4A4a63bk+mJYj2N1eul1QmI+NqymTqTqOlpLhIXl/ZFF7LwypisqY5IbVlMRldXbbFF3ltRaOkMyKTRlSFM4P5jW1JiUQyUh4vll3H10ltWXyLc3xewNdBDs7uBW+kNLcnpTwe7fPNFMJ9cPYhH16Fd9zzYRcL4ByCD6de+PXvyTWXni41VRVen3GuhPsbK5vCXzqz/fBKiRX3/4utvMbO0YMn3PUb89TrK+T1FU2bPHFsdZkcPnks4a7n7H4G4W6Bp3jqqub27t/qO6qq9xcsEO4KzAJfSrgX+AXg0+kf+bnL5MoLT5Xdd93Jp8PudayDGe6bg1rR1Ba+2ziiskSivOM+qNdUTUXnnw41tCREMoP60t6+2HPvrJbgXfeej30njpCua7kiHpXmjpSUxYqkJot3ypc3tUkmIzKyqiT8Z+GtVeu7f4NocMvC1nXl4Tvv+fwI/tkvK4nK+tZNfw19Pp/zUJzb+vakBP8XvONeXdr/nw5zq8xQ7I5fr0m4+7VfBX20P77/d/LL3z4tu35oUueHUzf8BtMulNNO+JQXPrkT7u2SzgS3BPR/+5EXoB4eZGVpsQR3fAWxRLdnt4EvL14X3g7T87HjqGppTaQkIxkpjRVJWyId3j5TWbLlW+fWNLeH/sPK42G4L2lolfZESiQiEiuKyKjqMimNRbM7QE9XRSMiJfGotLSnPD0DPw47kUpLRzItwQ+XVYT7JpvGPe66a5hw13mxeggFjv3iVRIrjkpYPH087r/9qiE8uuxfejDDfXMfTu26VSb7I2elKwHecddL9veO++SxtdKRSoXhzTvuelfecdeb2Tyjrjwu3CqzqSDhrruiCHedF6sRsBbInXBfL8l0WurK+HCq9aYqB9RXl4Q/f65qbA9v1eCxZYFXlzb0uMc9I5KJyMcnjgjfcQ8+XFpTGpWGtpRUlURleMWWf1Fb8O1Lwc03W9eWh89/ZWlD+AHCYG9KiqOy48iqzb47uuWjzv0VxUURqSqPydr1Hbl/sB4fYVuy88OpwZ/ujOQe9012knDXXdiEu86L1UMo8Ie//KvfVw8+vHroAR8bwqPL/qVzJ9z5cGr2u+Z2JR9O1Xs2tSXloRcXSUfq/fvcx9SUyREf4sOpes33n8GHU230sn8uH07t34pwz/46ClYS7jovVg+hwN5Hzdzk1TPpjDSub5HSkriMHTVcfnPPdUN4dNm/9GCEezZHw7fKZKM0MGsIdzPXIN7/+Nryzm9Dqq+Qj4yvCwfxdZBmnsGzCHdzO80ztxTumln5tpZw1+0o4a7zYnWOCbS0tsnt8x6WCVuPlmMO2y/Hjq7vw8mVcOd73IfuciHcze35BUzmdn09k3B369nftC19j/vgHEVuvgrhrtsXwl3nxeocFfj0aV+T+T++JkePbtPDypVw9wIrTw+ScDffWMLd3I5wd2vHNDcChLvOkXDXebE6BwVaWtvlUyd9RZ5+6JYcPLreh0S4e7FNA3qQhLs5L+Fubke4u7VjmhsBwl3nSLjrvFg9hAKXXnNHr1dPJJPy8oK3ZaeJW8ut15w3hEeX/UsT7tlb5etKwt18Zwl3czvC3a0d09wIEO46R8Jd58XqIRS4+sa7e716SUlcJmw1Wo48ZB8pLysZwqPL/qUJ9+yt8nUl4W6+s4S7uR3h7taOaW4ECHedI+Gu82J1DgikUmmJRovCI0kkkhLr8RtUc+AQN3sIhHuu79DAHx/hbm5MuJvbEe5u7ZjmRoBw1zkS7jovVg+hwHtLV8olV8+Vk487RI44aK/wSO649zfyp2f+JTfOminjRtcP4dFl/9KEe/ZW+bqScDffWcLd3I5wd2vHNDcChLvOkXDXebF6CAW+eMn1MmJYrVx81nSpH1YTHsnqtY1y852/kBWr1sqd118yhEeX/UsT7tlb5etKwt18Zwl3czvC3a0d09wIEO46R8Jd58XqIRT46KFnyB9/ebPUVFVschRr1jXJISdcLP947M4hPLrsX5pwz94qX1cS7uY7S7ib2xHubu2Y5kaAcNc5Eu46L1YPocD+x5wvt117gUzeacImR/HsP16Vy667S556cM4QHl32L024Z2+VrysJd/OdJdzN7Qh3t3ZMcyNAuOscCXedF6uHUCC4n/2nD/1ejjp0Hxk/ZoSk0xl5639L5OEnnpEZJx0R/p8PD8Ldh13qPMb17QlZ05KQinhUhle4+9Yiwt38GiDcze2GMtyb2hOytiUhVSXFUlced3sSTPNagHDXbR/hrvNi9RALPP7U8/LQo3+WRUtWhEey9biRcuzh+8vB++02xEeW/csT7tlbDfXKda0dsqyxTWpKYzKmpszZ4RDu5pQ9w31JQ6u8urRBEqm0jKoqlYqSYqkti8no6i3v12srGiWdEZk0okpSmYwEsxvbkhKJZKQ8Xiy7jq+T2rL8jsxYNCK1lXFZ2dBuvilZPHNNS7usaGrPem+yGMmSPBEg3HUbSbjrvFiNgLUA4W5NOGgDCPdBo876hTYO9xVNbfL06ys3ee7Y2jL56FZ1hHuWooR7llAsGzABwl1HS7jrvFiNgLUA4W5NOGgD1rUmZFljqxQXFUm8uPN3B7h4DKvqfBd37foOyWRcTCycGQuXN0pHKiPb1VfIq0saZW1L73eKd992eFa3Yyxe1yIB/9iasvCd9wXLG6W1IyUSyUhJcVS2r6+UinhxXuMWF0WkorRYGloSA3qeiXRaEsm01JbHZHTVlv80ZEAPhuE5JUC467aDcNd5sRoBawHC3Zpw0AZ0vePu+gVrKmLhyDCWBjjck+mMtCZSEgRaWSzq+lQGfd5bq9ZLIp2RrerKZMHSJmnuSPY6hl3G14a3N23psbypLfzBaWRVSRjuwey2ZEoiIuEPalvVlUt5jv6Ct/ZkWjpSaSkpLpL4hl9It6Xz7evvR4ProiQq61t7O5rM29Jzsr2NaUtz+Pv5I0C46/aScNd5sRoBawHC3Zpw0AZ0veNeWVIswxx+OHVkbalEIiIr1nWG40A+gg/YBvfpl5cUy9gs7vseyGNxMfuf766R9lRaPjimRl5ctEbeW9u6ydjA9aNbD5MdR1Vv8eXeXNUU+k8YXhne4/6vd9dIU3sqvMe9LB6VD43J7geALb7QACxYub5dGlo7ZFh53OraLI5GpKY8JqubOgbgKN8f2dTWEX44lXfcB5TZy+GEu27bCHedF6sRsBYg3K0JB21APtzj3tSWkMUNrRL88DG+tnzQ7AbqhTa+xz2VysgjrywJ33l+/xH8JBQJvwXo8A+NDd+R7u/h84dTgz8tWNvSIfUVcamvLDXm5h53Yzqe6EiAcNdBEu46L1YjYC1AuFsTDtqAvAj39oQsXtca3vpRVbLl20cGDdfwhV5Zsk6C20R2HFUllSUxSaQy8tJ7a+W9tS2dE4P7XDY8Jo+rDe9T7+/x7ppmCZJ/69ry8B33V5Y2SHPwrTJFEt7jvuPIKqnK4pYbw1OxelpLRzK8BWpkVWn4rrvpg3A3leN5rgQId50k4a7zYjUC1gKEuzXhoA1IpoP7iDOdH06NblSElkcwmF8H2fWOu+Uh58zTN77Hvev+8wXLm2RlU1uvY9x6WIVsM6z/P2Xo8x73RCqcE/ygs3Vwj3uOfzjVl3fcg6/rDD6bUByJOP2gd85cmByIsQDhrqMj3HVerEbAWoBwtyb0fsCghnuev+MeXAz/Wdogr61o6nVd7DS6WnbazL3uvOMuMljvuHv/Dy0nMGAChLuOlnDXebEaAWsBwt2a0PsBgxrueXyPe9cvR3p9ZZM8/86aTb5hJvimlRN224Z73LfwTwvh7v2/Trw/AcJdt4WEu86L1QhYCxDu1oTeDyDczbew529ODSYF4d6eSEtTR0IaW5NSGiuSPbet32y0B8/jw6m8425+JfJMVwKEu06ScNd5sRoBawHC3ZrQ+wGDGe4dybQ0tCXCiK3O0Q9aaja0v3BPpTMyrCIma5oTku13hfsc7sH317d0pKSipFjKLb6fn3fcNVcfawdCgHDXqRLuOi9WIxAKrGtYL4edcqmcf/qxcvzRB/ap8siTz8rVN94t11z6RTn0gI91ryHcuYgGM9zzTZtwd7ujhLtbT6bpBQh3nRnhrvNiNQKhwOXX3SXPvbhAZpx4eJ/hfvcDj8k/X1ooK1evk9NOOIxw57rZRIBwN78gCHdzu76eSbi79WSaXoBw15kR7jovViMgz72wQObOmy8Ttx0nkyaM6zPcF7zxruy4/VbyxYuvl+lHfYJw57oh3B1dA4S7I8gNYwh3t55M0wsQ7jozwl3nxeoCF0gkkjL9zFly46yZ8rOHnuw33LuYTr/oO4R7gV8zfZ0+77ibXxSEu7kd77i7tWOaGwHCXedIuOu8WF3gAnPvni+ZTEZmnjZNrplzr5NwH11n/uvKC3w7sjr94LdqptIiHx5fK9Eid79EKasX72dR0YbjSKczNmMK8rm//88yae5Iyb6T6mV4RUlo8PLideFvUB1VXSLLG9ulvjIuwS9f2tJj42sjmc7IE/9ZKuuaE+FvX60qLZaPTxwh9ZWdr5G3j4hIUSQiXIub7nDXtbHLuBopjhbl7fYPxYktW7vpL0sj3HW7QLjrvFhdwALvLFomF189V+6be6XE4zFn4d4VcblE++7aFslkpNdvndz4rwe/CXFJQ5sUR0XG1Wz62ymDX8f+vzUtUhYrkm2H9/8r5zc+5yCcXl/RJMEvKN1hM780R+u0cHmjBH28w8iqnAn3UbWlEomILF/XFjrzyF7g2bdWSWsyJVO2qpOu73EPfvlS8K0ywytjsnp9QmrLYzKmuqzfoe3JtDz71kp5c+X6cM342nLZe/sR8s9Fa6SpNQj3jFTEY7LrVnVSVx7P/uA8XBkrikh1RUxWN3V4ePQDd8hd/96YNKJyUMK9NZGUd1a3SElxRLarr8r6xJY1tcna5g4ZWVXS/YNsX0/u+e/0FU1t0ppIy4iqkl7fStSWSEvwW4WDr1UdVeX+jaWePyQS7llvd7iQcNd5sbqABYIPnN5xz8MSixWHCs0tbRKNFsmJ0w6SC2Yc16eMr7fKLFjWIEFZ7jiySiJBYW54BP9jFnRmEMFBaAe/fj74H/7tR2z6PzRBuAf/Q1EWi8o2WbzzGYwP5r2xskmCN6N3GFnt7Erb+Cv/cuUdd26VMd9eF7fKPPX6ivCHxO5HRiT4ATod/BQVXOCRsN2lriIuh3xgTPjue74+uMe9753t+vfGxPrBCfe2INzXtEi8uEi2y/LNjuDIg8Be29IhIyo3H+49/52+aG1z+CdXwQ+tlSWbXt/BpilvVQAAIABJREFUV40uWtsSBn02f3Jl+88G4a4TJNx1XqxGoFug560ywdc/7jllZ6kfVtO9hnDPnXDPpUu3piIWHk5DS6IzFHlkLRD8sJhIZ2Sr2jIpj3cGR/DuYfCnKhXxaBgjwZ/01JT1/075c++skfZkqvdrboj2jf9GNBKR3bcdLsXBHwXl4SP4YbasJCrrW5N5eHb2p0S4b/mWM1tlwl0nSLjrvFiNQL/hvt+082TO7HNkyuQd5LgZX5c33lksyWRKokVFEimKyLevOEMOPWB38eF73BcsbwzPM3jHfON33Bevawk7M3hXKAil4J2eiER6RU0qlZHmhC4Egjc7+4wpy2suuKUn1x7BPcXBI3yHl4dKIJnKhNdgcNtxl2Nwm0zno7O8A96uv9fX8PaOlOrnpeB6z8Vb2lRwm1kc/DMefHaHR2+B4uDf34P5M1tEpDS4/1D5KI0WSXwzv4ir69+D9Rs+FxJ8JiSdSUvw/5f2eF4qlZb2VDr8wXjruk1vg1QeVlbLCfesmLoXEe46L1YjYC3gRbhvuFWm58kua3z/Q0WpTEbWt/cd50GQBvcRax8DEdldUdf5+bLB/F/g/s+ecNdeGe+vT6bSYXQXFwWB3rmfXXsckSDqtxzuiWS6+znZHEm+h3twexqfk970Skilg39/RcLP3AxuuWckFtWHe/DPQyyLD9FWlxaHb7Z0PYLPcAS/VbmvB7fKZPNvh8FfQ7gPvjmvWOACXoT7hnfct6otD/+0oOvxxqqm8E3NsTVlksiILF7bEn7gc0SPDzC1JVOysqld4sURGVXV/4cEN74Ugg8sLW5oCe9xH1fr7o9n3wv+lCATzCzb7Luwg3lZDquKhy2wpqmDD6cq4f+7rEE60mnZvr5KKjfcKhP8SVDww2JwL3pTW0oqSopkWHn/H6oL3m3879J14fUb/jgXkQ370Me9MiKyz8SRUl3WeXtTvj2CRqwqjXV+mw6PboGuf2+MrSkN/9R0oB8dyVR4v3rwDv+Ymuz+nRkc09rWDlnflgyvz5rS/q/RZQ2t4SlMGN7579ZVze3SnkiFr1Wx4Z+jrnNsS6TC2894x32gd91sPuFu5sazEDAW8CLc+XCq8f5m80Q+nJqNUt9rXHw4tWty1wcQh5WXyO8XLJWWjuC+903jfc8J9TJ57PufWzE/8tx8Jh9O7Xtf+HAqH07NzX9i+VaZXN0XjiuPBQj3vjd3oL5VJhcvJcLdfFcGItwnjaiS4NavYHZjWzIM+MqSqOy9XX33V06aH3FuP5Nwz4394VtlcmMffDgK3nH3YZc4xrwS8CHcg69zDO4fCO5x3PjRmuj8UF/w14N7YoNbYoIbaYIPsfLIXoBwz96q58rBCPdIJBPeJrDr+Pe/K978iHP7mYR7bu+Pq6Pr+e/09kRagn+bB/e3B9+ctPEjuG2xLZWWIomE3+U+0A8+nKoTJtx1XqxGwFrAh3C3PkkGbFaAcDe/QAh3c7u+nkm4u/Vkml6AcNeZEe46L1YjYC1AuFsTej+AcDffQsLd3I5wd2vHNDcChLvOkXDXebEaAWsBwt2a0PsBhLv5FhLu5naEu1s7prkRINx1joS7zovVCFgLEO7WhN4PINzNt5BwN7cj3N3aMc2NAOGucyTcdV6sRsBagHC3JvR+AOFuvoWEu7kd4e7WjmluBAh3nSPhrvNiNQLWAoS7NaH3Awh38y0k3M3tCHe3dkxzI0C46xwJd50XqxGwFiDcrQm9H0C4m28h4W5uR7i7tWOaGwHCXedIuOu8WI2AtQDhbk3o/QDC3XwLCXdzO8LdrR3T3AgQ7jpHwl3nxWoErAUId2tC7wcQ7uZbSLib2xHubu2Y5kaAcNc5Eu46L1YjYC1AuFsTej+AcDffQsLd3I5wd2vHNDcChLvOkXDXebEaAWsBwt2a0PsBhLv5FhLu5naEu1s7prkRINx1joS7zovVCFgLEO7WhN4PINzNt5BwN7cj3N3aMc2NAOGucyTcdV6sRsBagHC3JvR+AOFuvoWEu7kd4e7WjmluBAh3nSPhrvNiNQLWAoS7NaH3Awh38y0k3M3tCHe3dkxzI0C46xwJd50XqxGwFiDcrQm9H0C4m28h4W5uR7i7tWOaGwHCXedIuOu8WI2AtQDhbk3o/QDC3XwLCXdzO8LdrR3T3AgQ7jpHwl3nxWoErAUId2tC7wcQ7uZbSLib2xHubu2Y5kaAcNc5Eu46L1YjYC1AuFsTej+AcDffQsLd3I5wd2vHNDcChLvOkXDXebEaAWsBwt2a0PsBhLv5FhLu5naEu1s7prkRINx1joS7zovVCFgLEO7WhN4PINzNt5BwN7cj3N3aMc2NAOGucyTcdV6sRsBagHC3JvR+AOFuvoWEu7kd4e7WjmluBAh3nSPhrvNiNQLWAoS7NaH3Awh38y0k3M3tCHe3dkxzI0C46xwJd50XqxGwFiDcrQm9H0C4m28h4W5uR7i7tWOaGwHCXedIuOu8WI2AtQDhbk3o/QDC3XwLCXdzO8LdrR3T3AgQ7jpHwl3nxWoErAUId2tC7wcQ7uZbSLib2xHubu2Y5kaAcNc5Eu46L1YjYC1AuFsTej+AcDffQsLd3I5wd2vHNDcChLvOkXDXebEaAWsBwt2a0PsBhLv5FhLu5naEu1s7prkRINx1joS7zovVCFgLEO7WhN4PINzNt5BwN7cj3N3aMc2NAOGucyTcdV6sRsBagHC3JvR+AOFuvoWEu7kd4e7WjmluBAh3nSPhrvNiNQIqgXUN6+WwUy6V808/Vo4/+sDwuYS7ijAvFxPu5ttKuJvbEe5u7ZjmRoBw1zkS7jovViOgErj8urvkuRcXyIwTDyfcVXL5vZhwN99fwt3cjnB3a8c0NwKEu86RcNd5sRqBrAWee2GBzJ03XyZuO04mTRhHuGctl/8LCXfzPe4Z7k3tCXl1aaOkMxmpr4xLOi1SXFQk8eKi8EWqSoqlrjze5wu+tiJ4nsikEVWSymQkmN3YlpRIJCPl8WLZdXyd1Jb1/VzzM8itZ8aiEamtjMvKhvbcOjCOpmAECHfdVhPuOi9WI5CVQCKRlOlnzpIbZ82Unz30JOGelVrhLCLczfe6Z7ivbemQl5esCwN8RGVcokWdwd71qC2PyeiqMsK9H3LC3fxa5JluBAh3nSPhrvNiNQJZCcy9e75kMhmZedo0uWbOvZsN96ryWFYzWZQ/AlVlxeHJNLUm8+ekBulMnn5thbQmkrL7hOEyrLxEEsm0vLK0QZLpjEwaWSnF0U3DPVYUkZLiaJ9H9+qG4N95TE34/KdfWy4NrQmJREQq4sWyR/AaFSWDdGZD8zLRiEhpPCrN7amhOQBeteAEmloSm5wz4a67BAh3nRerEdiiwDuLlsnFV8+V++ZeKfF4bMvhviHitjiYBXkj0PXDWs//AcubExzAE+kM95TsPmFYGO7B479LGyWZTstOo6sl1iPcN3cory5tkHQ6I++H+wppaO2QSFEQ7jHZY9theR/uRUWRMNxb2vghcgAvW0ZvJNDzDQvCXXd5EO46L1YjsEWBux94TO6452GJxTrfVW1uaZNotEhOnHaQXDDjOL5VZouC+b+AW2XM93hzH06dWN/7HffNvRL3uItwq4z5tcgz3QgQ7jpHwl3nxWoE1AJbulVGPZAneC9AuJtvIeFubtfXMwl3t55M0wsQ7jozwl3nxWoE1AKEu5os759AuJtvMeFubke4u7VjmhsBwl3nSLjrvFiNgLUAv4DJmtD7AYS7+RYS7uZ2hLtbO6a5ESDcdY6Eu86L1QhYCxDu1oTeDyDczbeQcDe3I9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRsBYg3K0JvR9AuJtvIeFubke4u7VjmhsBwl3nSLjrvFiNgLUA4W5N6P0Awt18Cwl3czvC3a0d09wIEO46R8Jd58VqBKwFCHdrQu8HEO7mW0i4m9sR7m7tmOZGgHDXORLuOi9WI2AtQLhbE3o/gHA330LC3dyOcHdrxzQ3AoS7zpFw13mxGgFrAcLdmtD7AYS7+RYS7uZ2hLtbO6a5ESDcdY6Eu86L1QhYCxDu1oTeDyDczbeQcDe3I9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRsBYg3K0JvR9AuJtvIeFubke4u7VjmhsBwl3nSLjrvFiNgLUA4W5N6P0Awt18Cwl3czvC3a0d09wIEO46R8Jd58VqBKwFCHdrQu8HEO7mW0i4m9sR7m7tmOZGgHDXORLuOi9WI2AtQLhbE3o/gHA330LC3dyOcHdrxzQ3AoS7zpFw13mxGgFrAcLdmtD7AYS7+RYS7uZ2hLtbO6a5ESDcdY6Eu86L1QhYCxDu1oTeDyDczbeQcDe3I9zd2jHNjQDhrnMk3HVerEbAWoBwtyb0fgDhbr6FhLu5HeHu1o5pbgQId50j4a7zYjUC1gKEuzWh9wMId/MtJNzN7Qh3t3ZMcyNAuOscCXedF6sRsBYg3K0JvR9AuJtvIeFubke4u7VjmhsBwl3nSLjrvFiNgLUA4W5N6P0Awt18C23DfXVzu6Qzna8f/PfgMbw8Lol0Rl5Z2iAdybREIhkpjxfLruPrpLYsbn6wHjwzFo1IbWVcVjZ0WvBAYLAFCHedOOGu82I1AtYChLs1ofcDCHfzLbQN99dWNEk6s6HcNzqMZDojb61aL0URkUhECHfzLeKZCKgECHcVlxDuOi9WI2AtQLhbE3o/gHA330LbcN/4HfeNjyKRSvOOu/m28EwEjAUIdx0d4a7zYjUC1gKEuzWh9wMId/Mt7Ar3MTWlUhYrDge1dCTD/yyNRaUoeLtcRMbVlkl0w3/P5tU6UmkJZje2JblVJhsw1iDgSIBw10ES7jovViNgLUC4WxN6P4BwN9/CrnCvK49LSXFRv4MmjqiS4uC+lywfhDv3uGd5qbDMsQDhrgMl3HVerEbAWoBwtyb0fgDhbr6FXeG+0+gqqS7t/4Oj5bGo6kUId8JddcGw2JkA4a6jJNx1XqxGwFqAcLcm9H4A4W6+hX3d424+7f1nEu6Eu4vriBl6AcJdZ0a467xYjYC1AOFuTej9AMLdfAsJd3O7vp7J10G69WSaXoBw15kR7jovViNgLUC4WxN6P4BwN99Cwt3cjnB3a8c0NwKEu86RcNd5sRoBawHC3ZrQ+wGEu/kWEu7mdoS7WzumuREg3HWOhLvOi9UIWAsQ7taE3g8g3M23kHA3tyPc3doxzY0A4a5zJNx1XqxGwFqAcLcm9H4A4W6+hYS7uR3h7taOaW4ECHedI+Gu82I1AtYChLs1ofcDCHfzLSTcze0Id7d2THMjQLjrHAl3nRerEbAWINytCb0fQLibbyHhbm5HuLu1Y5obAcJd50i467xYXeACLy94W665+R55692lMnpEnVx81vFywN4f6aWy8M1FcvWNd8sb7yyWrcaOlOsuP0N22G58uI5wL/CLSEQId/NrgHA3tyPc3doxzY0A4a5zJNx1XqwuYIFMJiNTp18kF874jBxx8F7y1LMvypdn3y7PPPw9KYnHumXS6YwcfspX5eRjD5bPfnqq/PKRp+QnD/5eHp53LeFewNfPxqdOuJtfCIS7uR3h7taOaW4ECHedI+Gu82J1AQu0tXfI4089L0cfuk+3wpRDZoRBPn7MiO6/tmTZKjn6tCvkuUe/L5FIJPzr+x9zvvzgxi/LpAnjece9gK+hrlMn3M0vAsLd3I5wd2vHNDcChLvOkXDXebEagVAgkUjKQ4/+We6b/0d58AezJRot6pZZumKNHPm5r8rzv7ujO9wPOeESuXTmiTJ13ym9wn1D2yNbQAKj68ok2Pdla1slkymgE3dwqs++vUrakmnZdXyt1JbFHUzsHNGRTEswu7EtKZFIRsrjxbLr+DqpK3f3Gs4O1uGgWFFEairjsqqx3eFURiHQv0DPf+cR7rqrhXDXebEaAfnTX1+Qc6/4royqr5M53zhXJu80YROV4JaaI0+9PLxN5oSjD5THnnpOrrjuLrn2sjPksKl79Ar34N1XHoUl0PXDGtGu3/cnXl0qLYmU7DtphAyvKNEP6OcZ7cm0PP7qUlnX0hH+UFVZEgtfo77S3Ws4O1iXgyIiwZ8Lci26RGXW5gSWrmnd5G8T7rrrhXDXebEagVAgmUrJ8y8skEu/eYfcf/tVMnZ0/SYywYdTv3HzPbJs5Ro5aN+Pykv/eVO+dOqnZd89JnOrDNcQH061uAa4VcYCr4+nxqIRqa2My8oG3nF3K8u0bAUI92ylOtcR7jovVhewwOq1jfLsP14NP5ja9fj8Bd+S6Ud+Inwnvb9HcFtNcI/7/B9/U0bW1xLuBXwNdZ0697ibXwSEu7ldX88k3N16Mk0vQLjrzAh3nRerC1igoalZDpp+kdw0a6bsu8cuEryr/rnzrpWf3HZF+KHTR558VvacsrPUD6uRz37pG/KlU4+WvXf7kHz/nl+H77jfef0loR5fB1nAF9GGUyfcza8Bwt3cjnB3a8c0NwKEu86RcNd5sbrABf7y93/LTXc8IEuWr5ba6ko54+Qj5djD9wtV9pt2nsyZfY5MmbyD/O2f/5HZN8+T4F36yR/YTq67bIaMGF5LuBf49cM77vYXgMtwT6bTsqShLTyodDot765tkYbWhHSk0lISK5IdRlTJDiOr7Q86hyfwjnsOb06BHBrhrttowl3nxWoErAV4x92a0PsBvONuvoUuw70jlZG3VjV1H8yKpjZZ15qQtkQqDPcJwytkl7F15gfrwTMJdw82Kc8PkXDXbTDhrvNiNQLWAoS7NaH3Awh38y10Ge7pTCb8asmux1ur18uKxjZZ35GUslhUJo+pkdHV+f2tT4S7+bXIM90IEO46R8Jd58VqBKwFCHdrQu8HEO7mW+gy3IOjWNPSIal055fpB/99eWNr+J8lxVHZcWSVbDusQoqKOn+RWj4+CPd83FW/zolw1+0X4a7zYjUC1gKEuzWh9wMId/MtdB3ub6xskuSGcA+OKrhdZnlTm8SiRbJ1XbnsMq5Oigl38w3jmQhsQYBw110ihLvOi9UIWAsQ7taE3g8g3M230HW4b/yOe3BU761rkf+taeYdd/Mt4pkIqAQIdxUX3+Ou42I1AvYChLu9oe8TCHfzHXQd7j2P5K1V62Xh8kYpjUdl1/F1UlsWNz9YD57JrTIebFKeHyLhrttg3nHXebEaAWsBwt2a0PsBhLv5FhLu5nZ9PZNwd+vJNL0A4a4zI9x1XqxGwFqAcLcm9H4A4W6+hYS7uR3h7taOaW4ECHedI+Gu82I1AtYChLs1ofcDCHfzLSTcze0Id7d2THMjQLjrHAl3nRerEbAWINytCb0fQLibb+FAhHvwi5iC36IaPBavaZE3VjVJvLhIdh5TIzVlcQluJ4kVFZkfdA4/k1tlcnhzCuTQCHfdRhPuOi9WI2AtQLhbE3o/gHA338KBCPfg6x/XtnSEB9XakZJlja0SZPyY6lIpjxfLyKpSGVaenx9SJdzNr0We6UaAcNc5Eu46L1YjYC1AuFsTej+AcDffwoEI9yUNbbK0sTU8qOCXMTW0dkginZHq0lj4G1SHV8TDX8SUjw/CPR931a9zItx1+0W467xYjYC1AOFuTej9AMLdfAsHItwb2xOyZF1nuAePINxbE2mpLi0O33GvKo3JuJoy84PO4WcS7jm8OQVyaIS7bqMJd50XqxGwFiDcrQm9H0C4m2/hQIR7KpOR9mTnPe7BI7h1pqktISMqS8J73KMSkZIY97ib7xrPRKB/AcJdd3UQ7jovViNgLUC4WxN6P4BwN9/CgQj3nkeztKFVGtoSMrq6lF/AZL5VPBOBrAQI96yYuhcR7jovViNgLUC4WxN6P4BwN9/CgQr31c3t0tyRCg+sI5kOv2Um+GaZ4g3fJhN8OLWypNj8wHP0mdwqk6MbU0CHRbjrNptw13mxGgFrAcLdmtD7AYS7+RYOVLgH3ySzrjXR74GNri6T2rKY+YHn6DMJ9xzdmAI6LMJdt9mEu86L1QhYCxDu1oTeDyDczbdwoMJ94+9y7+vo4tHg3feI+YHn6DMJ9xzdmAI6LMJdt9mEu86L1QhYCxDu1oTeDyDczbdwoMLd/Ij8fibh7vf+5cPRE+66XSTcdV6sRsBagHC3JvR+AOFuvoWEu7ldX88k3N16Mk0vQLjrzAh3nRerEbAWINytCb0fQLibbyHhbm5HuLu1Y5obAcJd50i467xYjYC1AOFuTej9AMLdfAsJd3M7wt2tHdPcCBDuOkfCXefFagSsBQh3a0LvBxDu5ltIuJvbEe5u7ZjmRoBw1zkS7jovViNgLUC4WxN6P4BwN99Cwt3cjnB3a8c0NwKEu86RcNd5sRoBawHC3ZrQ+wGEu/kWEu7mdoS7WzumuREg3HWOhLvOi9UIWAsQ7taE3g8g3M23kHA3tyPc3doxzY0A4a5zJNx1XqxGwFqAcLcm9H4A4a7bwvZkWlKZTPikFxetlY5UWj4wulqqSjt/k2lJcZFEI/n3y5F0Smar+TpIMzee5U6AcNdZEu46L1YjYC1AuFsTej+AcNdt4eJ1rdLUngiftKyxLfzPUVUlEtkQ62Nry6S6pDPieegECHedF6vdCxDuOlPCXefFagSsBQh3a0LvBxDuui1c2dwurR2p8EnvrWsJ/3NsTZkUbQj3+ooSKY9HdUNZHQoQ7lwIQy1AuOt2gHDXebEaAWsBwt2a0PsBhLv5Fi5c3ijBTTM7jKzqDnfzaTyTcOcaGGoBwl23A4S7zovVCFgLEO7WhN4PINzNt5BwN7fr65mEu1tPpukFCHedGeGu82I1AtYChLs1ofcDCHfzLSTcze0Id7d2THMjQLjrHAl3nRerEbAWINytCb0fQLjrtnBVc7ts+FIZWd3cHj55eEVcRDq/SWZEZYluIKu7BXjHnYthqAUId90OEO46L1YjYC1AuFsTej+AcNdt4cIVTZLpKvc+nsr97jrPjVcT7uZ2PNONAOGucyTcdV6sLnCBlxe8LdfcfI+89e5SGT2iTi4+63g5YO+P9FJZ8Ma7MvumebJmXZOUlsTl4rOmy7577BKuI9wL/CISEcJddw1s/I57X8/kHXedJ+Fu7sUz3QsQ7jpTwl3nxeoCFgje8Zs6/SK5cMZn5IiD95Knnn1Rvjz7dnnm4e9JSXzT75A+6vNXyFmnHCWHTd1Dgoj/3HnXylMPzpHyslLCvYCvoa5TJ9y5CHJFgHfcc2UnCvc4CHfd3hPuOi9WF7BAW3uHPP7U83L0oft0K0w5ZIY8PO9aGT9mRPdfCwJ/l6lfkD//6rtSV1MV/vW9j5op9956hWy/zVjCvYCvIcKdzc81AcI913ak8I6HcNftOeGu82I1AqFAIpGUhx79s9w3/4/y4A9mSzRatInM6Rd9Rw7efzc54egD5V8vvyZf/ead8tuffFtixVHCnWuIW2W4BnJGgHDPma0o2AMh3HVbT7jrvFiNgPzpry/IuVd8V0bV18mcb5wrk3ea0Etl4ZuL5LQLvxX+SvaW1na54cqzZeq+U8J1Pe9xr6ngV7UX2mVVUVocnnJzW7LQTp3zzTGBoohISSza/Ztpc+zwOJw8FGhoTmxyVoS7bpMJd50XqxEIBZKplDz/wgK59Jt3yP23XyVjR9d3y7R3JOSIz10mX7/oVPn47pPDD7KedsG35N5bL5etx43qFe5dEQdt4Qh0/bDW83/ACkeAM80VgWhRRMpKorK+lR8ic2VP8v04er5hQbjrdpxw13mxuoAFVq9tlGf/8Wr4wdSux+cv+JZMP/IT4YdQux7/ff1/ctalN8nTD93S/de+eMn1ctQhe8tRh+zDrTIFfA11nTofTuUiyBUBbpXJlZ0o3OMg3HV7T7jrvFhdwAINTc1y0PSL5KZZM8Ovdgxuhwm+LeYnt10hkyaMl0eefFb2nLKzxOMxmfqZC+WHN35Fdtl5e1m5ep1M+8KVctcNl8gHJm1DuBfwNUS4s/m5JkC459qOFN7xEO66PSfcdV6sLnCBv/z933LTHQ/IkuWrpba6Us44+Ug59vD9QpX9pp0nc2afI1Mm7yBPP/uS3PKDX4b3twcfXD3luEPCD6oGD77HvcAvIr7HnQsghwQI9xzajAI9FMJdt/GEu86L1QhYCxDu1oTeD+BWGe+3MG9OgHDPm6309kQId93WEe46L1YjYC1AuFsTej+AcPd+C/PmBAj3vNlKb0+EcNdtHeGu82I1AtYChLs1ofcDCHfvtzBvToBwz5ut9PZECHfd1hHuOi9WI2AtQLhbE3o/gHD3fgvz5gQI97zZSm9PhHDXbR3hrvNiNQLWAoS7NaH3Awh377cwb06AcM+brfT2RAh33dYR7jovViNgLUC4WxN6P4Bw934L8+YECPe82UpvT4Rw120d4a7zYjUC1gKEuzWh9wMId++3MG9OgHDPm6309kQId93WEe46L1YjYC1AuFsTej+AcPd+C/PmBAj3vNlKb0+EcNdtHeGu82I1AtYChLs1IQMQQAABBPJEgHDXbSThrvNiNQLWAoS7NSEDEEAAAQTyRIBw120k4a7zYjUC1gKEuzUhAxBAAAEE8kSAcNdtJOGu82I1AtYChLs1IQMQQAABBPJEgHDXbSThrvNiNQIIIIAAAggggAACQyJAuA8JOy+KAAIIIIAAAggggIBOgHDXebEaAQQQQAABBBBAAIEhESDch4SdF0UAAQQQQAABBBBAQCdAuOu8WI0AAghYCdz100dk3gOPSzKVksOm7ilXnHeyRKNFVjN5MgJagY6OhOx6yAyJxYq7n3rgPruE00CWAAAJaElEQVTKTbNmakexHgEEBlGAcB9EbF4KAQQKW+Bv//yPfO07P5R5t1wmNVUVcvZXb5bDpu4hn/301MKG4ewHXWDVmgY5+rQr5Jlf3zbor80LIoCAuQDhbm7HMxFAAAGVwOyb75ExI4fJjJOOCJ/3p7++EL77fvecr6rmsBgBW4G3310a/uD42M++YzuK5yOAwCAKEO6DiM1LIYBAYQucfvF35ISjD5SD99sthAji6bQLvy1PPTinsGE4+0EX+Pd/3pRzv/ZdmbD1GHn97fdkx+22kqsuOlW23Wr0oB8LL4gAAtkLEO7ZW7ESAQQQsBI4aeY1cuYpR8p+e344nLNk2Sr59Be+Js89+n2ruTwZAa3Am+8slnt++YScOO0gmbDVaLn9noflT8+8IPN/fI12FOsRQGAQBQj3QcTmpRBAoLAFvnjJ9XLMp/YL72sPHgvfXCRnfuVG3nEv7MsiJ84++LD0boeeIY/fd4OMGlGXE8fEQSCAQG8Bwp2rAgEEEBgkgW/ecq/UVlfKzNOmha/46B/+Lg/+9mn54U1fGaQj4GUQ6BRYuXqdNDQ2y8QJ48L/P5FIykc/eYY8/dAtUldTBRMCCOSoAOGeoxvDYSGAQP4J/Ovl1+Qr3/i+3PPdy6WiokzOuOQGmX7UJ+TYw/fLv5PljHJa4K//eEW+9u0fyr23XiGjRwyT2+f9Wv7v+Zfl/tuvyunj5uAQKHQBwr3QrwDOHwEEBlVg3i8elx/89BFJJFPy6U9+XC6d+VmJRCKDegy8GAKBwI/v/53c++AT0tbWIZM/sF344dRxo+vBQQCBHBYg3HN4czg0BBBAAAEEEEAAAQS6BAh3rgUEEEAAAQQQQAABBDwQINw92CQOEQEEEEAAAQQQQAABwp1rAAEEEEAAAQQQQAABDwQIdw82iUNEAAEEEEAAAQQQQIBw5xpAAAEEEEAAAQQQQMADAcLdg036//btP8brOQ7g+CtTxiSS6ai4KInWZqQYZX6Fw2yxi2FK+iFarV9Sq/VDKb9SrpyLxlSmH9IshM1aSCpqaGvTqF250qRS6Hb2/Wzftkphu7N7n8f9d91333t9H6/XH8+++54RCRAgQIAAAQIECAh3N0CAAAECBAgQIEAgAQHhnsCSjEiAAAECBAgQIEBAuLsBAgQIECBAgAABAgkICPcElmREAgQIECBAgAABAsLdDRAgQIAAAQIECBBIQEC4J7AkIxIgQIAAAQIECBAQ7m6AAAECBAgQIECAQAICwj2BJRmRAAECBAgQIECAgHB3AwQIECBAgAABAgQSEBDuCSzJiAQIECBAgAABAgSEuxsgQIAAgSQEftnza3Qq6hdvvTI+WhU2q/GZ/+vfV+MvyC8gQCB5AeGe/Aq9AAIECPw/BP7rkD5QWRlr1m2Mi9sUxkknnvD/QPYqCRCo1QLCvVavx3AECBAgkBf4N+FeVVUVVVURxx1XDyABAgTqjIBwrzOr9EIIECBQtwXy4T5xRK94ee7S2FxeEW1bnxu575sVnBEfrVgbU0rmRreiLjH9lUUxt2RUtG7ZPJ4tfTOWLPskdu3eG4XNm8bQh7tHx0vaZlh39R4Tt15/RXz+5YbY+N2WOHDgQAzuWxxdr+kQh/9H4ViPrdvyXh0BArVFQLjXlk2YgwABAgSOKZAP6XYXtowJwx+Mxqc2jOETSrPAzkX68pXrYsi4mdG1S4fofd9t0eS0U2Lxeytiatn8mD31sTi7aZOYs+iDKHv9nfh44dSoX//4KO47Nnbs3BUvTRkchS0KYs6iD2ParAXxyZIXYvfefYd8pv5Yj61Xzzv7zpcAgZoXEO41b+w3ECBAgEA1COTDffywnnHHTVdlz7hy7bfRY+CTsWLx9Fi/YVP0GfZ0LJv3VJzVtEn2899+/yN+3bc/TmvUMPv+51174srb+8eSVydGyxYFWbi3a1MYjw+4N/t57l38rncPzcK+QYP6R4T70R7bpHGjaniFnoIAAQLHFhDuLoQAAQIEkhDIh/ucklHRvu15h4T2wlnjomLHz9F/xHPx1YezDr6eXb/sjefK5seqLzfE/v2/Zf++tWJnLCgbG23Ob5GF+w2dL40exTdnP9u2fWdce+egeH/eU9Hw5JOOCPejPTb3br4vAgQI1LSAcK9pYc9PgAABAtUikA/3eTNHZ++S5762bN0eN3YfEu+8Nik2l2+PgaOnxRfvlh78fcOfKI3vt/wYz497JM44/dTYs3dfXH5L30PC/cbOl8UDxTf9o3A/2mOFe7Ws2JMQIPA3AsLdiRAgQIBAEgL5cH9yZO8ouq5TNnP+ozKrls6M1es2HhHuuajvdU9RdCvqnD3+szXfRM9Bk4V7Ehs3JAEChwsIdzdBgAABAkkI5MP9knatYvLIPtlHWXJ/jFpZWRmlUwbH8pXrjwj3+wdMjIIzT48nhveKTT+Ux5QZb8SnX3wd0yY8Gld3bJ99VMY77kms35AECESEcHcGBAgQIJCEQP4PS58Z0y9emL04yrdtj4suKIxJIx7K4vyvwj33B6uPT3wptlb8FBe2OifGD3swXnzt7fhg+eqYMWlQTC6ZK9yT2L4hCRDICQh3d0CAAAECBAgQIEAgAQHhnsCSjEiAAAECBAgQIEBAuLsBAgQIECBAgAABAgkICPcElmREAgQIECBAgAABAsLdDRAgQIAAAQIECBBIQEC4J7AkIxIgQIAAAQIECBAQ7m6AAAECBAgQIECAQAICwj2BJRmRAAECBAgQIECAgHB3AwQIECBAgAABAgQSEBDuCSzJiAQIECBAgAABAgSEuxsgQIAAAQIECBAgkICAcE9gSUYkQIAAAQIECBAgINzdAAECBAgQIECAAIEEBIR7AksyIgECBAgQIECAAAHh7gYIECBAgAABAgQIJCAg3BNYkhEJECBAgAABAgQICHc3QIAAAQIECBAgQCABAeGewJKMSIAAAQIECBAgQEC4uwECBAgQIECAAAECCQgI9wSWZEQCBAgQIECAAAECwt0NECBAgAABAgQIEEhAQLgnsCQjEiBAgAABAgQIEBDuboAAAQIECBAgQIBAAgLCPYElGZEAAQIECBAgQICAcHcDBAgQIECAAAECBBIQEO4JLMmIBAgQIECAAAECBIS7GyBAgAABAgQIECCQgIBwT2BJRiRAgAABAgQIECAg3N0AAQIECBAgQIAAgQQEhHsCSzIiAQIECBAgQIAAgT8BIEt4uMHy+vEAAAAASUVORK5CYII="
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "originalKey": "b9127c73-b1b6-4e46-b593-43068d5015a8",
        "showInput": false,
        "code_folding": [],
        "hidden_ranges": []
      },
      "source": [
        "### Appendix: Using `ScalarizedPosteriorTransform`\n",
        "\n",
        "Using the `ScalarizedPosteriorTransform` abstraction, the functionality of `ScalarizedUpperConfidenceBound` implemented above can be easily achieved in just a few lines of code. `PosteriorTransform`s can be used with both the MC and analytic acquisition functions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "originalKey": "79a4b36f-4b14-4a62-9dc6-883931ceb5d3",
        "code_folding": [],
        "hidden_ranges": [],
        "collapsed": false,
        "requestMsgId": "490a2b3a-24f9-4b49-84f1-7a3851d5944f",
        "executionStopTime": 1668651301442,
        "executionStartTime": 1668651301431,
        "customOutput": null
      },
      "source": [
        "from botorch.acquisition.objective import ScalarizedPosteriorTransform\n",
        "from botorch.acquisition.analytic import UpperConfidenceBound\n",
        "\n",
        "pt = ScalarizedPosteriorTransform(weights=torch.tensor([0.1, 0.5]))\n",
        "SUCB = UpperConfidenceBound(gp, beta=0.1, posterior_transform=pt)"
      ],
      "execution_count": 16,
      "outputs": []
    }
  ]
}