From 86b3aa0876a23bf849ba8a831ff062d2366e571f Mon Sep 17 00:00:00 2001 From: Meraldo Antonio Date: Fri, 22 Nov 2024 17:05:18 +0800 Subject: [PATCH] Added example notebook --- ...bayesian_conjugate_linear_regression.ipynb | 1011 +++++++++++++++++ 1 file changed, 1011 insertions(+) create mode 100644 examples/05_bayesian_conjugate_linear_regression.ipynb diff --git a/examples/05_bayesian_conjugate_linear_regression.ipynb b/examples/05_bayesian_conjugate_linear_regression.ipynb new file mode 100644 index 000000000..de4dad6b6 --- /dev/null +++ b/examples/05_bayesian_conjugate_linear_regression.ipynb @@ -0,0 +1,1011 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "b7799052", + "metadata": {}, + "source": [ + "# Goal" + ] + }, + { + "cell_type": "markdown", + "id": "7ab018f8", + "metadata": {}, + "source": [ + "This notebook walks you through the theory and assumptions behind bayesian linear regression.\n", + "Afterwards, it shows how this inference can be done using `skpro`'s `BayesianConjugateLinearRegressor` class." + ] + }, + { + "cell_type": "markdown", + "id": "524e1891", + "metadata": {}, + "source": [ + "# Introduction" + ] + }, + { + "cell_type": "markdown", + "id": "9b104e0f", + "metadata": {}, + "source": [ + "## Bayesian linear regression" + ] + }, + { + "cell_type": "markdown", + "id": "e8d3059c", + "metadata": {}, + "source": [ + "**Bayesian linear regression** is a probabilistic approach to linear regression where prior beliefs about the model parameters are combined with observed data to compute posterior distributions. Unlike traditional linear regression, which provides point estimates for parameters, Bayesian linear regression offers distributions, capturing uncertainty in both parameters and predictions.\n", + "\n", + "In this notebook, we will specifically focus on **Bayesian Linear Regression**, assuming a **multivariate normal distribution** as the prior for the regression coefficients, leveraging its properties as a **conjugate prior** to simplify inference." + ] + }, + { + "cell_type": "markdown", + "id": "5e513bb4", + "metadata": {}, + "source": [ + "## Conjugate Prior" + ] + }, + { + "cell_type": "markdown", + "id": "030bf15e", + "metadata": {}, + "source": [ + "The use of **conjugate priors** simplifies the Bayesian inference. Conjugacy ensures that the posterior distribution belongs to the same family as the prior, making it analytically tractable. \n", + "\n", + "\n", + "In our case, as we are using a Gaussian prior and we are dealing with a Gaussian likelihood, the posterior remains Gaussian, allowing for straightforward updates to the mean and covariance of the distribution. This efficiency avoids the need for computationally intensive methods like Monte Carlo Markov Chain sampling." + ] + }, + { + "cell_type": "markdown", + "id": "4c900f26", + "metadata": {}, + "source": [ + "# Theory" + ] + }, + { + "cell_type": "markdown", + "id": "f070390e", + "metadata": {}, + "source": [ + "\n", + "In the sections that follow, we will delve into the key distributions that play a crucial role in Bayesian inference. These include:\n", + "\n", + "1. **The Likelihood Function**: This captures the relationship between the observed data and the model parameters\n", + "\n", + "2. **The Prior Distribution**: The multivariate normal prior represents our initial beliefs about the regression coefficients before observing any data. \n", + "\n", + "3. **The Posterior Distribution**: By combining the prior and likelihood using Bayes' theorem, we compute the posterior, which reflects our updated beliefs about the coefficients after accounting for the observed data.\n", + "\n", + "4. **The Predictive Distribution**: Finally, we use the posterior to make predictions on new data, incorporating the uncertainty in the coefficients to provide confidence intervals around the predictions." + ] + }, + { + "cell_type": "markdown", + "id": "979d683f", + "metadata": {}, + "source": [ + "## Likelihood" + ] + }, + { + "cell_type": "markdown", + "id": "8e3c81d4", + "metadata": {}, + "source": [ + "### Single target data point $t$" + ] + }, + { + "cell_type": "markdown", + "id": "fdd1ff4c", + "metadata": {}, + "source": [ + "In Bayesian Linear Regression, we assume that the target variable $t$ is generated by a deterministic function $y(\\mathbf{x}, \\mathbf{w})$, which represents the model's prediction based on the input features $\\mathbf{x}$ and the regression coefficients $\\mathbf{w}$. This deterministic prediction is subject to additive Gaussian noise, $ epsilon$, which accounts for uncertainty not captured by the deterministic function. Mathematically, this relationship is expressed as:\n", + "\n", + "$$\n", + "t = y(\\mathbf{x}, \\mathbf{w}) + \\epsilon,\n", + "$$\n", + "\n", + "where $\\epsilon \\sim \\mathcal{N}(0, \\beta^{-1})$ follows a Gaussian distribution with zero mean and variance $ \\beta^{-1}$. \n", + "\n", + "\n", + "We could also reframe this by assuming that $t$ is given probabilistically by a Gaussian distribution:\n", + "\n", + "$$\n", + "p(t | \\mathbf{x}, \\mathbf{w}, \\beta) = \\mathcal{N}(t | y(\\mathbf{x}, \\mathbf{w}), \\beta^{-1})\n", + "$$\n" + ] + }, + { + "cell_type": "markdown", + "id": "9c9023f8", + "metadata": {}, + "source": [ + "### A set of data points $\\mathbf{t}$" + ] + }, + { + "cell_type": "markdown", + "id": "c397f304", + "metadata": {}, + "source": [ + "\n", + "Now let's consider a data set of inputs $ \\mathbf{X} = \\{ \\mathbf{x}_1, \\ldots, \\mathbf{x}_N \\} $ with corresponding target values $ t_1, \\ldots, t_N $. We group the target variables $\\{ t_n \\}$ into a column vector that we denote by $\\mathbf{t}$. \n", + "\n", + "We assume that these data points are drawn independently from the above distribution. With this assumption, we proceed to construct the following expression for the likelihood function of the whole dataset:\n", + "\n", + "$$\n", + "\\begin{aligned}\n", + "p(\\mathbf{t} | \\mathbf{X}, \\mathbf{w}, \\beta) &= \\prod_{n=1}^N \\mathcal{N}(t_n | \\mathbf{w}^T \\mathbf{x}_n, \\beta^{-1}) \\\\\n", + "&\\propto \\exp \\left( -\\frac{\\beta}{2} \\sum_{n=1}^N (t_n - \\mathbf{w}^T \\mathbf{x}_n)^2 \\right)\n", + "\\\\\n", + "&\\propto \\exp \\left( -\\frac{\\beta}{2} \\|\\mathbf{t} - \\mathbf{X} \\mathbf{w}\\|^2 \\right)\n", + "\\end{aligned} \n", + "$$\n", + "\n", + "\n", + "Note: since the data matrix $\\mathbf{X} $ always appears in the set of conditioning variables, from this point onwards, we will drop the explicit \"$\\mathbf{X}$\" from expressions such as $ p(\\mathbf{t} | \\mathbf{X}, \\mathbf{w}, \\beta) $ to keep the notation uncluttered.\n" + ] + }, + { + "cell_type": "markdown", + "id": "7daa89c8", + "metadata": {}, + "source": [ + "## Prior" + ] + }, + { + "cell_type": "markdown", + "id": "4c0e34da", + "metadata": {}, + "source": [ + "We will first introduce a **prior** over the model parameters $\\mathbf{w}$, which represents our initial belief about their values before observing any data.\n", + "\n", + "In our framework, we treat the noise precision parameter $\\beta$ as a known constant. This simplifies the model while still allowing us to capture uncertainty in the regression coefficients $\\mathbf{w}$.\n", + "\n", + "As we saw above, the likelihood function $p(\\mathbf{t} | \\mathbf{w}, \\beta)$ is Gaussian. To achieve conjugacy and facilitate computation, we choose a multivariate Gaussian distribution as the prior for $\\mathbf{w}$, given by:\n", + "\n", + "\n", + "$$\n", + "\\begin{aligned}\n", + "p(\\mathbf{w}) &= \\mathcal{N}(\\mathbf{w} | \\mathbf{m}_0, \\mathbf{S}_0) \\\\\n", + "&\\propto \\exp \\left( -\\frac{1}{2} (\\mathbf{w} - \\mathbf{m}_0)^T \\mathbf{S}_0^{-1} (\\mathbf{w} - \\mathbf{m}_0) \\right)\n", + "\\end{aligned}\n", + "$$\n", + "\n", + "\n", + "where:\n", + "- $\\mathbf{m}_0$ is the prior mean vector of the regression coefficients; Its shape is $(D, 1)$, where $D$ is the number of features or coefficients.\n", + "- $\\mathbf{S}_0$ is the prior covariance matrix; its shape is $(D, D)$.\n" + ] + }, + { + "cell_type": "markdown", + "id": "5654d185", + "metadata": {}, + "source": [ + "**Simplification**\n", + "\n", + "To further simplify treatment, we shall use a particular form of a multivariate Gaussian prior.\n", + "\n", + "The prior we'll use is a **zero-mean isotropic Gaussian**, where the prior mean $\\mathbf{m}_0$ is zero and the prior covariance $\\mathbf{S}_0$ is isotropic (i.e. the distribution has the same variance in all directions). \n", + "\n", + "This distribution is governed by a single scalar precision parameter $\\alpha$ so that:\n", + "\n", + "$$\n", + "p(\\mathbf{w} | \\alpha) = \\mathcal{N}(\\mathbf{w} | 0, \\alpha^{-1} \\mathbf{I})\n", + "$$\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "3674c1ab", + "metadata": {}, + "source": [ + "## Posterior" + ] + }, + { + "cell_type": "markdown", + "id": "b691065c", + "metadata": {}, + "source": [ + "**General form**\n", + "\n", + "According to the Bayes formula, the **posterior** is proportional to the product of the likelihood and the prior:\n", + "
\n", + "\n", + "$$\n", + "p(\\mathbf{w} | \\mathbf{t}) \\propto \\exp \\left( -\\frac{\\beta}{2} \\|\\mathbf{t} - \\mathbf{X} \\mathbf{w}\\|^2 \\right) \\exp \\left( -\\frac{1}{2} (\\mathbf{w} - \\mathbf{m}_0)^T \\mathbf{S}_0^{-1} (\\mathbf{w} - \\mathbf{m}_0) \\right)\n", + "$$\n", + "\n", + "After expanding the terms in the exponents and completing the square, we obtain a posterior that's also a multivariate Gaussian:\n", + "\n", + "$$\n", + "p(\\mathbf{w} | \\mathbf{t}) = \\mathcal{N}(\\mathbf{w} | \\mathbf{m}_N, \\mathbf{S}_N)\n", + "$$\n", + "\n", + "where:\n", + "- $\\mathbf{S}_N$ is the posterior covariance. Its inverse (posterior precision) is given by:\n", + " $$\n", + " \\mathbf{S}_N^{-1} = \\mathbf{S}_0^{-1} + \\beta \\mathbf{X}^T \\mathbf{X}\n", + " $$\n", + "- $\\mathbf{m}_N$ is the posterior mean, given by:\n", + " $$\n", + " \\mathbf{m}_N = \\mathbf{S}_N \\left( \\mathbf{S}_0^{-1} \\mathbf{m}_0 + \\beta \\mathbf{X}^T \\mathbf{t} \\right)\n", + " $$\n", + "\n", + "\n", + "**Simplification**\n", + "\n", + "As mentioned above, we'll use a special case with the following parameters for our Gaussian prior:\n", + "- prior precision $\\mathbf{S}_0^{-1} = \\alpha \\mathbf{I}$ \n", + "- prior mean $\\mathbf{m}_0 = \\mathbf{0}$ \n", + "\n", + "For this special case, the posterior precision simplifies to:\n", + "$$\n", + "\\mathbf{S}_N^{-1} = \\alpha \\mathbf{I} + \\beta \\mathbf{X}^T \\mathbf{X}\n", + "$$\n", + "\n", + "The posterior mean simplifies to:\n", + "$$\n", + "\\mathbf{m}_N = \\beta \\mathbf{S}_N \\mathbf{X}^T \\mathbf{t}\n", + "$$\n", + "\n", + "**Intuition**\n", + "\n", + "1. **Posterior Precision $\\mathbf{S}_N^{-1}$**:\n", + "\n", + " - The posterior precision $\\mathbf{S}_N^{-1}$ is the sum of the prior-derived precision and data-derived precision.\n", + " - The prior precision $\\mathbf{S}_0^{-1} = \\alpha \\mathbf{I}$ reflects initial uncertainty in the weights $\\mathbf{w}$.\n", + " - The data precision $\\beta \\mathbf{X}^T \\mathbf{X}$ reflects the amount of information provided by the observed data $\\mathbf{X}$, adjusted by the noise precision $\\beta$.\n", + "\n", + "\n", + "\n", + "2. **Posterior Mean ($\\mathbf{m}_N$)**:\n", + " - Since we assume the prior mean $\\mathbf{m}_0$ to be 0, the posterior mean comes exclusively from the data\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "id": "5557b78a", + "metadata": {}, + "source": [ + "## Posterior Predictive" + ] + }, + { + "cell_type": "markdown", + "id": "a7550422", + "metadata": {}, + "source": [ + "Our ultimate goal is to get the posterior predictive distribution of a new target $t$ given a new input $\\mathbf{x}$:\n", + "\n", + "$$\n", + "p(t | \\mathbf{x}, \\mathbf{X}, \\mathbf{t}) = \\mathcal{N}(t | m(\\mathbf{x}), s^2(\\mathbf{x}))\n", + "$$\n", + "\n", + "As the notation suggests, this distribution depends on the training data ($\\mathbf{X}$ and $\\mathbf{t}$) used to fit the model.\n", + "\n", + "This posterior predictive distribution is a univariate Gaussian with mean $m(\\mathbf{x})$ and variance $s^2(\\mathbf{x})$, both of which depend on the given input $\\mathbf{x}$.\n", + "\n", + "\n", + "### **Predictive Mean**\n", + "\n", + "The predictive mean $m(\\mathbf{x})$ is given by:\n", + "$$\n", + "\\begin{aligned}\n", + "m(\\mathbf{x}) &= \\mathbf{x}^T \\beta S_N \\mathbf{X}^T \\mathbf{t} \\\\\n", + "&= \\mathbf{x}^T \\mathbf{m}_N\n", + "\\end{aligned}\n", + "$$\n", + "\n", + "We note that this predictive mean is very simple: it is simply a projection of incoming data point $\\mathbf{x}$ onto the posterior mean $\\mathbf{m}_N$.\n", + "\n", + "\n", + "### **Predictive Variance**\n", + "\n", + "The predictive variance $s^2(\\mathbf{x})$ is given by:\n", + "$$\n", + "s^2(\\mathbf{x}) = \\beta^{-1} + \\mathbf{x}^T S_N \\mathbf{x}\n", + "$$\n", + "\n", + "\n", + "Predictive variance quantifies model confidence, increasing in regions far from the training data or in uncertain directions. From the formula, we note that this uncertainty depends on the **position** and **direction** of the incoming data point $\\mathbf{x}$:\n", + "\n", + "- If it's close to fitted training data: $\\mathbf{x}^T S_N \\mathbf{x}$ is small near training data, where the model is confident.\n", + "- On the other hand, if it is far from fitted training data: $\\mathbf{x}^T S_N \\mathbf{x}$ grows as $\\mathbf{x}$ moves away, reflecting increased uncertainty.\n", + "- Lastly, larger $\\|\\mathbf{x}\\|$ increases $\\mathbf{x}^T S_N \\mathbf{x}$, leading to higher variance.\n" + ] + }, + { + "cell_type": "markdown", + "id": "51538300", + "metadata": {}, + "source": [ + "# Application" + ] + }, + { + "cell_type": "markdown", + "id": "c66c7cc1", + "metadata": {}, + "source": [ + "The above framework is implemented by the `BayesianConjugateLinearRegressor` class from `skpro`. In this section, we'll take a look at its usage." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "b942bdfb", + "metadata": {}, + "outputs": [], + "source": [ + "import pandas as pd\n", + "from sklearn.datasets import load_diabetes\n", + "from sklearn.model_selection import train_test_split\n", + "\n", + "from skpro.regression.bayesian.bayesian_conjugate import (\n", + " BayesianConjugateLinearRegressor,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "17863b2a", + "metadata": {}, + "source": [ + "## Data" + ] + }, + { + "cell_type": "markdown", + "id": "fe8ca402", + "metadata": {}, + "source": [ + "We will first load our dataset using the `load_diabetes` function from `sklearn`. \n", + "We will then split this dataset into " + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "6affb203", + "metadata": {}, + "outputs": [], + "source": [ + "X, y = load_diabetes(return_X_y=True, as_frame=True)\n", + "X = X.iloc[:10]\n", + "y = y.iloc[:10]\n", + "y = pd.DataFrame(y)\n", + "\n", + "X_train_update, X_test, y_train_update, _ = train_test_split(X, y, random_state=42)\n", + "X_train, X_update, y_train, y_update = train_test_split(\n", + " X_train_update, y_train_update, random_state=42\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "da4bcded", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
agesexbmibps1s2s3s4s5s6
3-0.089063-0.044642-0.011595-0.0366560.0121910.024991-0.0360380.0343090.022688-0.009362
20.0852990.0506800.044451-0.005670-0.045599-0.034194-0.032356-0.0025920.002861-0.025930
40.005383-0.044642-0.0363850.0218720.0039350.0155960.008142-0.002592-0.031988-0.046641
9-0.070900-0.0446420.039062-0.033213-0.012577-0.034508-0.024993-0.0025920.067737-0.013504
6-0.0454720.050680-0.047163-0.015999-0.040096-0.0248000.000779-0.039493-0.062917-0.038357
\n", + "
" + ], + "text/plain": [ + " age sex bmi bp s1 s2 s3 \\\n", + "3 -0.089063 -0.044642 -0.011595 -0.036656 0.012191 0.024991 -0.036038 \n", + "2 0.085299 0.050680 0.044451 -0.005670 -0.045599 -0.034194 -0.032356 \n", + "4 0.005383 -0.044642 -0.036385 0.021872 0.003935 0.015596 0.008142 \n", + "9 -0.070900 -0.044642 0.039062 -0.033213 -0.012577 -0.034508 -0.024993 \n", + "6 -0.045472 0.050680 -0.047163 -0.015999 -0.040096 -0.024800 0.000779 \n", + "\n", + " s4 s5 s6 \n", + "3 0.034309 0.022688 -0.009362 \n", + "2 -0.002592 0.002861 -0.025930 \n", + "4 -0.002592 -0.031988 -0.046641 \n", + "9 -0.002592 0.067737 -0.013504 \n", + "6 -0.039493 -0.062917 -0.038357 " + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "X_train" + ] + }, + { + "cell_type": "markdown", + "id": "f4123ed8", + "metadata": {}, + "source": [ + "## Instantiation and Fitting" + ] + }, + { + "cell_type": "markdown", + "id": "f59facf5", + "metadata": {}, + "source": [ + "We first instantiate a `BayesianConjugateLinearRegressor` model object with `alpha` = 0.5 and `beta` = 3" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "b28a1a65", + "metadata": {}, + "outputs": [], + "source": [ + "model = BayesianConjugateLinearRegressor(alpha=0.5, beta=3)" + ] + }, + { + "cell_type": "markdown", + "id": "3f615ed5", + "metadata": {}, + "source": [ + "After performing `.fit`, we will have access to a posterior distribution and its parameters (`mu` and `cov`)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "c9680aac", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
BayesianConjugateLinearRegressor(alpha=0.5, beta=3)
Please rerun this cell to show the HTML repr or trust the notebook.
" + ], + "text/plain": [ + "BayesianConjugateLinearRegressor(alpha=0.5, beta=3)" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model.fit(X_train, y_train)" + ] + }, + { + "cell_type": "markdown", + "id": "7f3b069b", + "metadata": {}, + "source": [ + "We see that the prior `mu` is zero, as explained above in our assumption." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "ef93bcde", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.],\n", + " [0.],\n", + " [0.],\n", + " [0.],\n", + " [0.],\n", + " [0.],\n", + " [0.],\n", + " [0.],\n", + " [0.],\n", + " [0.]])" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model._prior_mu" + ] + }, + { + "cell_type": "markdown", + "id": "5c5813a0", + "metadata": {}, + "source": [ + "Meanwhile, the prior covariance is the identity matrix times 1/`alpha`:" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "8f0f40b4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[2., 0., 0., 0., 0., 0., 0., 0., 0., 0.],\n", + " [0., 2., 0., 0., 0., 0., 0., 0., 0., 0.],\n", + " [0., 0., 2., 0., 0., 0., 0., 0., 0., 0.],\n", + " [0., 0., 0., 2., 0., 0., 0., 0., 0., 0.],\n", + " [0., 0., 0., 0., 2., 0., 0., 0., 0., 0.],\n", + " [0., 0., 0., 0., 0., 2., 0., 0., 0., 0.],\n", + " [0., 0., 0., 0., 0., 0., 2., 0., 0., 0.],\n", + " [0., 0., 0., 0., 0., 0., 0., 2., 0., 0.],\n", + " [0., 0., 0., 0., 0., 0., 0., 0., 2., 0.],\n", + " [0., 0., 0., 0., 0., 0., 0., 0., 0., 2.]])" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model._prior_cov" + ] + }, + { + "cell_type": "markdown", + "id": "9e846dc7", + "metadata": {}, + "source": [ + "As expected, the posterior has the same shape as the prior:" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "def45929", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(10, 1)" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model._posterior_mu.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "d58d2904", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(10, 10)" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "model._posterior_cov.shape" + ] + }, + { + "cell_type": "markdown", + "id": "a13eaf0a", + "metadata": {}, + "source": [ + "## Prediction" + ] + }, + { + "cell_type": "markdown", + "id": "1ffa6620", + "metadata": {}, + "source": [ + "We can then use our fitted model to perform prediction.\n", + "The resulting prediction is an instance of `skpro`'s `Normal` distribution with the same size as our incoming data `X_test`." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "3016ce84", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
agesexbmibps1s2s3s4s5s6
80.0417080.0506800.061696-0.040099-0.0139530.006202-0.028674-0.002592-0.0149600.011349
1-0.001882-0.044642-0.051474-0.026328-0.008449-0.0191630.074412-0.039493-0.068332-0.092204
5-0.092695-0.044642-0.040696-0.019442-0.068991-0.0792880.041277-0.076395-0.041176-0.096346
\n", + "
" + ], + "text/plain": [ + " age sex bmi bp s1 s2 s3 \\\n", + "8 0.041708 0.050680 0.061696 -0.040099 -0.013953 0.006202 -0.028674 \n", + "1 -0.001882 -0.044642 -0.051474 -0.026328 -0.008449 -0.019163 0.074412 \n", + "5 -0.092695 -0.044642 -0.040696 -0.019442 -0.068991 -0.079288 0.041277 \n", + "\n", + " s4 s5 s6 \n", + "8 -0.002592 -0.014960 0.011349 \n", + "1 -0.039493 -0.068332 -0.092204 \n", + "5 -0.076395 -0.041176 -0.096346 " + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "X_test" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "63fd427e", + "metadata": {}, + "outputs": [], + "source": [ + "y_test_pred_proba = model.predict_proba(X_test)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "edb7ce41", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
Normal(columns=Index(['target'], dtype='object'),\n",
+       "       index=Index([8, 1, 5], dtype='int64'),\n",
+       "       mu=[[-4.390691198701689], [6.064281031619252], [35.57270620266341]],\n",
+       "       sigma=[[0.5949424812347218], [0.6186268611569907], [0.6419937217502341]])
Please rerun this cell to show the HTML repr or trust the notebook.
" + ], + "text/plain": [ + "Normal(columns=Index(['target'], dtype='object'),\n", + " index=Index([8, 1, 5], dtype='int64'),\n", + " mu=[[-4.390691198701689], [6.064281031619252], [35.57270620266341]],\n", + " sigma=[[0.5949424812347218], [0.6186268611569907], [0.6419937217502341]])" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "y_test_pred_proba" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "866a9363", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(3, 1)" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "y_test_pred_proba.shape" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "41628aef", + "metadata": {}, + "outputs": [], + "source": [ + "y_test_pred = model.predict(X_test)" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "66d94ef7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
target
8-4.390691
16.064281
535.572706
\n", + "
" + ], + "text/plain": [ + " target\n", + "8 -4.390691\n", + "1 6.064281\n", + "5 35.572706" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "y_test_pred" + ] + } + ], + "metadata": { + "hide_input": false, + "kernelspec": { + "display_name": "pymc_env", + "language": "python", + "name": "pymc_env" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.3" + }, + "toc": { + "base_numbering": 1, + "nav_menu": {}, + "number_sections": true, + "sideBar": true, + "skip_h1_title": false, + "title_cell": "Table of Contents", + "title_sidebar": "Contents", + "toc_cell": false, + "toc_position": {}, + "toc_section_display": true, + "toc_window_display": false + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}