01.2_simple_rnn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Practicum AI Logo image](https://github.com/PracticumAI/practicumai.github.io/blob/main/images/logo/PracticumAI_logo_250x50.png?raw=true)\n",
    "***\n",
    "# *Practicum AI:* RNN - Simple RNN\n",
    "\n",
    "This exercise was adapted from Baig et al. (2020) <i>The Deep Learning Workshop</i> from <a href=\"https://www.packtpub.com/product/the-deep-learning-workshop/9781839219856\">Packt Publishers</a> (Exercise 5.03, page 243)\n",
    "\n",
    "In this exercise, we will build a simple RNN model using Keras. The model will consist of a reshaped layer followed by a SimpleRNN layer, and a dense layer for prediction. We will use the formatted trainX and trainY data, as well as initialized layers from Keras.\n",
    "\n",
    "<div style=\"padding: 10px;margin-bottom: 20px;border: thin solid #30335D;border-left-width: 10px;background-color: #fff\"><strong>Note:</strong> The setup code below comes from the data visualize exercise and must be run prior to this one. </div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd, numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>Close</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Volume</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1/17/2020</td>\n",
       "      <td>138.31</td>\n",
       "      <td>136.54</td>\n",
       "      <td>138.330</td>\n",
       "      <td>136.16</td>\n",
       "      <td>5623336</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1/16/2020</td>\n",
       "      <td>137.98</td>\n",
       "      <td>137.32</td>\n",
       "      <td>138.190</td>\n",
       "      <td>137.01</td>\n",
       "      <td>4320911</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1/15/2020</td>\n",
       "      <td>136.62</td>\n",
       "      <td>136.00</td>\n",
       "      <td>138.055</td>\n",
       "      <td>135.71</td>\n",
       "      <td>4045952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1/14/2020</td>\n",
       "      <td>135.82</td>\n",
       "      <td>136.28</td>\n",
       "      <td>137.139</td>\n",
       "      <td>135.55</td>\n",
       "      <td>3683458</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1/13/2020</td>\n",
       "      <td>136.60</td>\n",
       "      <td>135.48</td>\n",
       "      <td>136.640</td>\n",
       "      <td>135.07</td>\n",
       "      <td>3531572</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Date   Close    Open     High     Low   Volume\n",
       "0  1/17/2020  138.31  136.54  138.330  136.16  5623336\n",
       "1  1/16/2020  137.98  137.32  138.190  137.01  4320911\n",
       "2  1/15/2020  136.62  136.00  138.055  135.71  4045952\n",
       "3  1/14/2020  135.82  136.28  137.139  135.55  3683458\n",
       "4  1/13/2020  136.60  135.48  136.640  135.07  3531572"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inp0 = pd.read_csv('data/AAPL.csv')\n",
    "inp0.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts_data = inp0.Close.values.reshape(-1,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Preparing the Data for Stock Price Prediction\n",
    "\n",
    "In exercise `01.1_data_visualize`, we visualized the dataset.  Now, we need to prepare the data for the model. \n",
    "\n",
    "**Step 1,  create a train_test split of the data.** \n",
    "\n",
    "With time-series data, we can't simply select random data points for our training and testing sets. Instead, we must maintain the sequence of the data. A common approach for handling time-series data is to use the first portion of the data for training and the last portion for testing. In this case, we will use the first 75% of the records as the training data and the last 25% as the test data. It is important to preserve the temporal order of the data when working with time-series data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_recs = int(len(ts_data) * 0.75)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1885, 629)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_data = ts_data[:train_recs]\n",
    "test_data = ts_data[train_recs:]\n",
    "len(train_data), len(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Step 2, scale the stock data.**\n",
    "\n",
    "To scale the data so values are between 0 and 1 (inclusive), we use the `MinMaxScaler` from sklearn. This scaler maps the highest value in the data to 1. We then fit and transform the scaler on the training data but only transform the test data. This ensures that the scaling parameters learned from the training data are applied to the test data, allowing for a realistic evaluation of the model's performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "scaler = MinMaxScaler()\n",
    "train_scaled = scaler.fit_transform(train_data)\n",
    "test_scaled = scaler.transform(test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Step 3, format the data to get \"features\" for each instance.**\n",
    "\n",
    "To predict the next value, we need to define a *lookback period*, which is the number of previous days' data that we want to use as input. We can define a function that returns the target value (in this case, the stock price for a particular day) and the daily values in the *lookback period*. This function can be used to create training and testing sets that include the appropriate number of input days and corresponding target values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_lookback(inp, look_back):\n",
    "    y = pd.DataFrame(inp)\n",
    "    dataX = [y.shift(i) for i in range(1, look_back + 1)]\n",
    "    dataX = pd.concat(dataX, axis = 1)\n",
    "    dataX.fillna(0, inplace = True)\n",
    "    \n",
    "    return dataX.values, y.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we define a lookback period and apply the function to our data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "look_back = 10\n",
    "trainX, trainY = get_lookback(train_scaled, look_back = look_back)\n",
    "testX, testY = get_lookback(test_scaled, look_back = look_back)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1885, 10), (629, 10))"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainX.shape, testX.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As expected, each example includes 10 features, corresponding to the past 10 days of data. We have this historical data available for both the training and testing sets.\n",
    "\n",
    "Now, we can move on to building and training a model using this prepared data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise 5.03 (Student)\n",
    "\n",
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. Import the requisite libraries\n",
    "\n",
    "```python\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers \\\n",
    "import SimpleRNN, Activation, Dropout, Dense, Reshape\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. Instantiate a sequential model\n",
    "\n",
    "```python\n",
    "model = Sequential()\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. Add a reshape layer\n",
    "\n",
    "```python\n",
    "model.add(Reshape((look_back,1), input_shape = (look_back,)))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. Add a simple RNN layer with 32 neurons and specify input shape\n",
    "\n",
    "```python\n",
    "model.add(SimpleRNN(32, input_shape=(look_back, 1)))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. Add a dense layer\n",
    "\n",
    "```python\n",
    "model.add(Dense(1))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 6. Add an activation layer with a linear function\n",
    "\n",
    "```python\n",
    "model.add(Activation('linear'))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 7. Compile the model with an Adam optimizer\n",
    "\n",
    "```python\n",
    "model.compile(loss = 'mean_squared_error', optimizer = 'adam')\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 8. Print a summary of the model\n",
    "\n",
    "```python\n",
    "model.summary()\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The architecture of our single-layer (plain) RNN model is now complete.  This is a relatively simple model compared to those we have built for image data in the past.  But how will it perform?\n",
    "\n",
    "Let's see how it does with our current task. By evaluating the model's performance, we can determine whether this simple architecture is sufficient or if we need to consider a more complex model.\n",
    "\n",
    "***\n",
    "####  Model Training and Performance Evaluation\n",
    "\n",
    "It's time to fit the model to the data.  To do that, we'll set batch size to 1, validation split to 10%, and train the model for three epochs. Experiment with the number of epochs to see which produces the best results.\n",
    "\n",
    "Train the model using the `fit()` method.\n",
    "\n",
    "```python\n",
    "model.fit(trainX, trainY, epochs = 3, batch_size = 1, verbose = 2, validation_split = 0.1)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The loss appears to be quite low. With training complete, we need to evaluate the model's performance on the training and testing sets. To do this, we define two functions: one to print the root mean squared error (RMS) on the training and testing sets, and another to plot the predictions for the test data alongside the original values in the data.  The RMS error and prediction visualization should provide valuable insights into the model's accuracy and reliability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "def get_model_perf(model_obj):\n",
    "    score_train = model_obj.evaluate(trainX, trainY, verbose=0)\n",
    "    print('Train RMSE: %.2f RMSE' % (math.sqrt(score_train)))\n",
    "    score_test = model_obj.evaluate(testX, testY, verbose=0)\n",
    "    print('Test RMSE: %.2f RMSE' % (math.sqrt(score_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "get_model_perf(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The RMS error values appear to be relatively low.  Even so, we probably ought to visually assess the model's performance by comparing the actual values to the predicted values for the test period.  Doing so will give us a more intuitive understanding of how well the model predicts stock prices.  By plotting actual vs predicted values on the same graph, we can quickly identify discrepancies and see whether the model is making reasonable predictions.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_pred(model_obj):\n",
    "    testPredict = scaler.inverse_transform(model_obj.predict(testX))\n",
    "    \n",
    "    pred_test_plot = ts_data.copy()\n",
    "    pred_test_plot[:train_recs+look_back,:] = np.nan\n",
    "    pred_test_plot[train_recs+look_back:,:] = testPredict[look_back:]\n",
    "    \n",
    "    plt.plot(ts_data)\n",
    "    plt.plot(pred_test_plot, \"--\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function first makes predictions on the test data. Since our data has been scaled, we need to apply the inverse transform to get it back to its original scale before plotting it. The function then plots actual values as a solid line and predicted values as dotted lines. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "plt.figure(figsize=[10,5])\n",
    "plot_pred(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot visualizes the predictions made by the model (dotted lines) alongside the actual values (solid lines). At first glance, the model appears to be doing a good job.  But to get a better understanding of the data, let's zoom in on a specific section of the plot to see individual points more clearly. \n",
    "\n",
    "To focus on a specific range of plot indices, we can use the `xlim` function to set the x-axis limits of the plot. For example, to focus on indices 2300-2500, you can use the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "plot_pred(model)\n",
    "plt.xlim(2300, 2500) # set the x-axis limits\n",
    "plt.ylim(120, 150)   # set the y-axis limits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After zooming in, the results look good. The model has captured most of the variations in the data. It's impressive that a single RNN layer with just 32 neurons achieves does so well.  Indeed, you might be pleasantly surprised by the efficacy of RNNs on this task.  The ability of RNNs to process sequential data and capture temporal dependencies makes them well-suited for time series prediction tasks.\n",
    "\n",
    "#### Summary\n",
    "\n",
    "In this exercise, we constructed a basic RNN model to predict stock prices. We also saw that even a simple model can have strong predictive power with sequence prediction tasks. By training the model on historical stock data, we were able to make relatively accurate predictions about future prices. This demonstrates the potential of RNNs for time series prediction tasks and the importance of considering temporal dependencies when working with sequential data."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Tensorflow-2.4.1",
   "language": "python",
   "name": "tensorflow-2.4.1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}