1.1-tcad-preprocessing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TCAD Preprocessing\n",
    "\n",
    "TCAD provides an export files with the data in their database as zip files. These must be downloaded and parsed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "import tcad.tparser as tcp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download TCAD Files\n",
    "\n",
    "The TCAD Appraisal Export can be found publically on the public information section of the website under resources: https://traviscad.org/publicinformation\n",
    "\n",
    "In this notebook, we use the \"2023 Certified Export (July)\" file. Download and extract contents of the zip file to the raw folder.\n",
    "\n",
    "Direct download link: https://traviscad.org/wp-content/largefiles/2023%20Certified%20Appraisal%20Export%20Supp%200_07232022.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Deprecated\n",
    "# import tcad.downloader as tcd\n",
    "\n",
    "# tcd.download()\n",
    "# tcd.extract()\n",
    "\n",
    "# Provide the directory where you extracted the files here.\n",
    "RAW_TCAD_DIR = 'data/raw/2023_Certified_Appraisal_Export_Supp_0_07232022'\n",
    "\n",
    "# Where you want the files from this notebook to be saved.\n",
    "PROCESSED_TCAD_DIR = 'data/processed/TCAD'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Layout File"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also in the public information section, there is the \"Export Layouts\" file. This files serves the techical documentation or data dictionary for the database export since the TCAD export consists of a number of text files that are in the fixed width format (not comma or tab). The layout file tells you the start and end position for the fields in each row. This file is automatically downloaded for parsing purposes.\n",
    "\n",
    "Direct download link: https://traviscad.org/wp-content/largefiles/Legacy8.0.25-Export-Layouts-07242023.zip"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# tcp.download_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improvement Info"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Improvement info contains overview information for each property like the type of building as well as the total value of buildings on the site. There is one row per property. This function is used here for demonstration purposes. It's is already called from within parse_improvement_info and does not need to be called separately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Field Name</th>\n",
       "      <th>Datatype</th>\n",
       "      <th>Start</th>\n",
       "      <th>End</th>\n",
       "      <th>Length</th>\n",
       "      <th>Description</th>\n",
       "      <th>col_spec</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>prop_id</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>12</td>\n",
       "      <td>Property ID</td>\n",
       "      <td>(0, 12)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>prop_val_yr</td>\n",
       "      <td>numeric(4)</td>\n",
       "      <td>13</td>\n",
       "      <td>16</td>\n",
       "      <td>4</td>\n",
       "      <td>Year</td>\n",
       "      <td>(12, 16)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>imprv_id</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>17</td>\n",
       "      <td>28</td>\n",
       "      <td>12</td>\n",
       "      <td>Improvement ID</td>\n",
       "      <td>(16, 28)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>imprv_type_cd</td>\n",
       "      <td>varchar(10)`</td>\n",
       "      <td>29</td>\n",
       "      <td>38</td>\n",
       "      <td>10</td>\n",
       "      <td>Improvement Type</td>\n",
       "      <td>(28, 38)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>imprv_type_desc</td>\n",
       "      <td>varchar(25)</td>\n",
       "      <td>39</td>\n",
       "      <td>63</td>\n",
       "      <td>25</td>\n",
       "      <td>Improvement Description</td>\n",
       "      <td>(38, 63)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>imprv_state_cd</td>\n",
       "      <td>varchar(5)</td>\n",
       "      <td>64</td>\n",
       "      <td>68</td>\n",
       "      <td>5</td>\n",
       "      <td>State Code</td>\n",
       "      <td>(63, 68)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>imprv_homesite</td>\n",
       "      <td>varchar(1)</td>\n",
       "      <td>69</td>\n",
       "      <td>69</td>\n",
       "      <td>1</td>\n",
       "      <td>\"Y\" indicates Homesite improvement, \"N\" non Ho...</td>\n",
       "      <td>(68, 69)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>imprv_val</td>\n",
       "      <td>numeric(14)</td>\n",
       "      <td>70</td>\n",
       "      <td>83</td>\n",
       "      <td>14</td>\n",
       "      <td>Improvement Value</td>\n",
       "      <td>(69, 83)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>imprv_homesite_pct</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>84</td>\n",
       "      <td>98</td>\n",
       "      <td>15</td>\n",
       "      <td>Homesite Apportionment Percentage</td>\n",
       "      <td>(83, 98)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>omitted</td>\n",
       "      <td>varchar(1)</td>\n",
       "      <td>99</td>\n",
       "      <td>99</td>\n",
       "      <td>1</td>\n",
       "      <td>Omitted Improvement (\"Y\" or \"N\")</td>\n",
       "      <td>(98, 99)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>omitted_imprv_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>100</td>\n",
       "      <td>114</td>\n",
       "      <td>15</td>\n",
       "      <td>Omitted Value</td>\n",
       "      <td>(99, 114)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Field Name      Datatype  Start  End  Length  \\\n",
       "0              prop_id       int(12)      1   12      12   \n",
       "1          prop_val_yr    numeric(4)     13   16       4   \n",
       "2             imprv_id       int(12)     17   28      12   \n",
       "3        imprv_type_cd  varchar(10)`     29   38      10   \n",
       "4      imprv_type_desc   varchar(25)     39   63      25   \n",
       "5       imprv_state_cd    varchar(5)     64   68       5   \n",
       "6       imprv_homesite    varchar(1)     69   69       1   \n",
       "7            imprv_val   numeric(14)     70   83      14   \n",
       "8   imprv_homesite_pct   numeric(15)     84   98      15   \n",
       "9              omitted    varchar(1)     99   99       1   \n",
       "10   omitted_imprv_val   numeric(15)    100  114      15   \n",
       "\n",
       "                                          Description   col_spec  \n",
       "0                                         Property ID    (0, 12)  \n",
       "1                                                Year   (12, 16)  \n",
       "2                                      Improvement ID   (16, 28)  \n",
       "3                                    Improvement Type   (28, 38)  \n",
       "4                             Improvement Description   (38, 63)  \n",
       "5                                          State Code   (63, 68)  \n",
       "6   \"Y\" indicates Homesite improvement, \"N\" non Ho...   (68, 69)  \n",
       "7                                   Improvement Value   (69, 83)  \n",
       "8                   Homesite Apportionment Percentage   (83, 98)  \n",
       "9                    Omitted Improvement (\"Y\" or \"N\")   (98, 99)  \n",
       "10                                      Omitted Value  (99, 114)  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tcp.load_improvement_info_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prop_id</th>\n",
       "      <th>prop_val_yr</th>\n",
       "      <th>imprv_id</th>\n",
       "      <th>imprv_type_cd</th>\n",
       "      <th>imprv_type_desc</th>\n",
       "      <th>imprv_state_cd</th>\n",
       "      <th>imprv_homesite</th>\n",
       "      <th>imprv_val</th>\n",
       "      <th>imprv_homesite_pct</th>\n",
       "      <th>omitted</th>\n",
       "      <th>omitted_imprv_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225541</td>\n",
       "      <td>00</td>\n",
       "      <td>Detail Only</td>\n",
       "      <td>F1</td>\n",
       "      <td>N</td>\n",
       "      <td>12278</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>32</td>\n",
       "      <td>RESTAURANT</td>\n",
       "      <td>F1</td>\n",
       "      <td>N</td>\n",
       "      <td>176710</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100012</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225543</td>\n",
       "      <td>08</td>\n",
       "      <td>APARTMENT 100+</td>\n",
       "      <td>B1</td>\n",
       "      <td>N</td>\n",
       "      <td>38413766</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100012</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225544</td>\n",
       "      <td>24</td>\n",
       "      <td>COMMERCIAL SPACE CONDOS</td>\n",
       "      <td>F1</td>\n",
       "      <td>N</td>\n",
       "      <td>1432068</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>100012</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225545</td>\n",
       "      <td>19</td>\n",
       "      <td>SPECIAL (NODEPR)</td>\n",
       "      <td>B1</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>422189</th>\n",
       "      <td>977351</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807305</td>\n",
       "      <td>11</td>\n",
       "      <td>MOHO SINGLE PP</td>\n",
       "      <td>M1</td>\n",
       "      <td>N</td>\n",
       "      <td>11648</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>422190</th>\n",
       "      <td>977368</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807423</td>\n",
       "      <td>12</td>\n",
       "      <td>MOHO DOUBLE PP</td>\n",
       "      <td>M1</td>\n",
       "      <td>N</td>\n",
       "      <td>107388</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>422191</th>\n",
       "      <td>977370</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807440</td>\n",
       "      <td>12</td>\n",
       "      <td>MOHO DOUBLE PP</td>\n",
       "      <td>M1</td>\n",
       "      <td>Y</td>\n",
       "      <td>129739</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>422192</th>\n",
       "      <td>977394</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807584</td>\n",
       "      <td>11</td>\n",
       "      <td>MOHO SINGLE PP</td>\n",
       "      <td>M1</td>\n",
       "      <td>Y</td>\n",
       "      <td>38485</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>422193</th>\n",
       "      <td>977397</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807600</td>\n",
       "      <td>12</td>\n",
       "      <td>MOHO DOUBLE PP</td>\n",
       "      <td>M1</td>\n",
       "      <td>N</td>\n",
       "      <td>52360</td>\n",
       "      <td>100.0</td>\n",
       "      <td>N</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>422194 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        prop_id  prop_val_yr  imprv_id imprv_type_cd          imprv_type_desc  \\\n",
       "0        100008         2023   5225541            00              Detail Only   \n",
       "1        100008         2023   5225542            32               RESTAURANT   \n",
       "2        100012         2023   5225543            08           APARTMENT 100+   \n",
       "3        100012         2023   5225544            24  COMMERCIAL SPACE CONDOS   \n",
       "4        100012         2023   5225545            19         SPECIAL (NODEPR)   \n",
       "...         ...          ...       ...           ...                      ...   \n",
       "422189   977351         2023   5807305            11           MOHO SINGLE PP   \n",
       "422190   977368         2023   5807423            12           MOHO DOUBLE PP   \n",
       "422191   977370         2023   5807440            12           MOHO DOUBLE PP   \n",
       "422192   977394         2023   5807584            11           MOHO SINGLE PP   \n",
       "422193   977397         2023   5807600            12           MOHO DOUBLE PP   \n",
       "\n",
       "       imprv_state_cd imprv_homesite  imprv_val  imprv_homesite_pct omitted  \\\n",
       "0                  F1              N      12278               100.0       N   \n",
       "1                  F1              N     176710               100.0       N   \n",
       "2                  B1              N   38413766               100.0       N   \n",
       "3                  F1              N    1432068               100.0       N   \n",
       "4                  B1              N          0               100.0       N   \n",
       "...               ...            ...        ...                 ...     ...   \n",
       "422189             M1              N      11648               100.0       N   \n",
       "422190             M1              N     107388               100.0       N   \n",
       "422191             M1              Y     129739               100.0       N   \n",
       "422192             M1              Y      38485               100.0       N   \n",
       "422193             M1              N      52360               100.0       N   \n",
       "\n",
       "        omitted_imprv_val  \n",
       "0                       0  \n",
       "1                       0  \n",
       "2                       0  \n",
       "3                       0  \n",
       "4                       0  \n",
       "...                   ...  \n",
       "422189                  0  \n",
       "422190                  0  \n",
       "422191                  0  \n",
       "422192                  0  \n",
       "422193                  0  \n",
       "\n",
       "[422194 rows x 11 columns]"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tcp.parse_improvement_info(f'{RAW_TCAD_DIR}/IMP_INFO.txt',\n",
    "                           export_file=f'{PROCESSED_TCAD_DIR}/IMP_INFO.parquet')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improvement Details"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Improvement details tracks the changes to the property overtime in terms of its features (mainly the floors). There can be multiple rows per property (i.e. one row for each floor of a house and the year it was built and the floor area). In addition to floors, \"HVAC\" is also a possible entry of interest. However, this data is dependent on the accuracy of records.\n",
    "\n",
    "For instance, the moontower apartment complex has all its floor area assigned one floor. With the super-coop building in austin, there are rows for floors 3-5, but floors 1 and 2 are combined into a row named \"ADDL\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prop_id</th>\n",
       "      <th>prop_val_yr</th>\n",
       "      <th>imprv_id</th>\n",
       "      <th>imprv_det_id</th>\n",
       "      <th>Imprv_det_type_cd</th>\n",
       "      <th>Imprv_det_type_desc</th>\n",
       "      <th>Imprv_det_class_cd</th>\n",
       "      <th>yr_built</th>\n",
       "      <th>depreciation_yr</th>\n",
       "      <th>imprv_det_area</th>\n",
       "      <th>imprv_det_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225541</td>\n",
       "      <td>41038597</td>\n",
       "      <td>551</td>\n",
       "      <td>PAVED AREA</td>\n",
       "      <td>AA</td>\n",
       "      <td>1984</td>\n",
       "      <td>1984</td>\n",
       "      <td>17100</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>41038598</td>\n",
       "      <td>1ST</td>\n",
       "      <td>1st Floor</td>\n",
       "      <td>C</td>\n",
       "      <td>2013</td>\n",
       "      <td>2013</td>\n",
       "      <td>2986</td>\n",
       "      <td>161525</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>41038599</td>\n",
       "      <td>501</td>\n",
       "      <td>CANOPY</td>\n",
       "      <td>A</td>\n",
       "      <td>2013</td>\n",
       "      <td>2013</td>\n",
       "      <td>402</td>\n",
       "      <td>3036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>41038600</td>\n",
       "      <td>611</td>\n",
       "      <td>TERRACE</td>\n",
       "      <td>CA</td>\n",
       "      <td>2013</td>\n",
       "      <td>2013</td>\n",
       "      <td>402</td>\n",
       "      <td>760</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>41038601</td>\n",
       "      <td>511</td>\n",
       "      <td>DECK</td>\n",
       "      <td>WA</td>\n",
       "      <td>2013</td>\n",
       "      <td>2013</td>\n",
       "      <td>850</td>\n",
       "      <td>1369</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3150901</th>\n",
       "      <td>977351</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807305</td>\n",
       "      <td>45612257</td>\n",
       "      <td>1ST</td>\n",
       "      <td>1st Floor</td>\n",
       "      <td>MH5</td>\n",
       "      <td>1995</td>\n",
       "      <td>1995</td>\n",
       "      <td>896</td>\n",
       "      <td>11648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3150902</th>\n",
       "      <td>977368</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807423</td>\n",
       "      <td>45613519</td>\n",
       "      <td>1ST</td>\n",
       "      <td>1st Floor</td>\n",
       "      <td>MH8</td>\n",
       "      <td>2019</td>\n",
       "      <td>2019</td>\n",
       "      <td>1440</td>\n",
       "      <td>107388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3150903</th>\n",
       "      <td>977370</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807440</td>\n",
       "      <td>45613646</td>\n",
       "      <td>1ST</td>\n",
       "      <td>1st Floor</td>\n",
       "      <td>MH8</td>\n",
       "      <td>2020</td>\n",
       "      <td>2020</td>\n",
       "      <td>1790</td>\n",
       "      <td>129739</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3150904</th>\n",
       "      <td>977394</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807584</td>\n",
       "      <td>45615062</td>\n",
       "      <td>1ST</td>\n",
       "      <td>1st Floor</td>\n",
       "      <td>MH6</td>\n",
       "      <td>2001</td>\n",
       "      <td>2001</td>\n",
       "      <td>1190</td>\n",
       "      <td>38485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3150905</th>\n",
       "      <td>977397</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807600</td>\n",
       "      <td>45615198</td>\n",
       "      <td>1ST</td>\n",
       "      <td>1st Floor</td>\n",
       "      <td>MH7</td>\n",
       "      <td>2002</td>\n",
       "      <td>2002</td>\n",
       "      <td>1120</td>\n",
       "      <td>52360</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3150906 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         prop_id  prop_val_yr  imprv_id  imprv_det_id Imprv_det_type_cd  \\\n",
       "0         100008         2023   5225541      41038597               551   \n",
       "1         100008         2023   5225542      41038598               1ST   \n",
       "2         100008         2023   5225542      41038599               501   \n",
       "3         100008         2023   5225542      41038600               611   \n",
       "4         100008         2023   5225542      41038601               511   \n",
       "...          ...          ...       ...           ...               ...   \n",
       "3150901   977351         2023   5807305      45612257               1ST   \n",
       "3150902   977368         2023   5807423      45613519               1ST   \n",
       "3150903   977370         2023   5807440      45613646               1ST   \n",
       "3150904   977394         2023   5807584      45615062               1ST   \n",
       "3150905   977397         2023   5807600      45615198               1ST   \n",
       "\n",
       "        Imprv_det_type_desc Imprv_det_class_cd  yr_built  depreciation_yr  \\\n",
       "0                PAVED AREA                 AA      1984             1984   \n",
       "1                 1st Floor                  C      2013             2013   \n",
       "2                    CANOPY                  A      2013             2013   \n",
       "3                   TERRACE                 CA      2013             2013   \n",
       "4                      DECK                 WA      2013             2013   \n",
       "...                     ...                ...       ...              ...   \n",
       "3150901           1st Floor                MH5      1995             1995   \n",
       "3150902           1st Floor                MH8      2019             2019   \n",
       "3150903           1st Floor                MH8      2020             2020   \n",
       "3150904           1st Floor                MH6      2001             2001   \n",
       "3150905           1st Floor                MH7      2002             2002   \n",
       "\n",
       "         imprv_det_area  imprv_det_val  \n",
       "0                 17100              0  \n",
       "1                  2986         161525  \n",
       "2                   402           3036  \n",
       "3                   402            760  \n",
       "4                   850           1369  \n",
       "...                 ...            ...  \n",
       "3150901             896          11648  \n",
       "3150902            1440         107388  \n",
       "3150903            1790         129739  \n",
       "3150904            1190          38485  \n",
       "3150905            1120          52360  \n",
       "\n",
       "[3150906 rows x 11 columns]"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tcp.parse_improvement_details(f'{RAW_TCAD_DIR}/IMP_DET.txt',\n",
    "                              export_file=f'{PROCESSED_TCAD_DIR}/IMP_DET.parquet')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improvement Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Improvement features is another file that contains overall details about the house. Strangely, each field seems to have its own rows so there are multiple per property. Information of interest here may be the roof shape/material and foundation type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Field Name</th>\n",
       "      <th>Datatype</th>\n",
       "      <th>Start</th>\n",
       "      <th>End</th>\n",
       "      <th>Length</th>\n",
       "      <th>Description</th>\n",
       "      <th>col_spec</th>\n",
       "      <th>dtype</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>prop_id</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>12</td>\n",
       "      <td>Property ID</td>\n",
       "      <td>(0, 12)</td>\n",
       "      <td>int64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>prop_val_yr</td>\n",
       "      <td>numeric(4)</td>\n",
       "      <td>13</td>\n",
       "      <td>16</td>\n",
       "      <td>4</td>\n",
       "      <td>Year</td>\n",
       "      <td>(12, 16)</td>\n",
       "      <td>int16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>imprv_id</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>17</td>\n",
       "      <td>28</td>\n",
       "      <td>12</td>\n",
       "      <td>Improvement ID</td>\n",
       "      <td>(16, 28)</td>\n",
       "      <td>int64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>imprv_det_id</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>29</td>\n",
       "      <td>40</td>\n",
       "      <td>12</td>\n",
       "      <td>Improvement Detail ID</td>\n",
       "      <td>(28, 40)</td>\n",
       "      <td>int64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>imprv_attr_id</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>41</td>\n",
       "      <td>52</td>\n",
       "      <td>12</td>\n",
       "      <td>Improvement Attribute ID</td>\n",
       "      <td>(40, 52)</td>\n",
       "      <td>int64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>imprv_attr_desc</td>\n",
       "      <td>varchar(25)</td>\n",
       "      <td>53</td>\n",
       "      <td>77</td>\n",
       "      <td>25</td>\n",
       "      <td>Attribute Description</td>\n",
       "      <td>(52, 77)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>imprv_attr_cd</td>\n",
       "      <td>varchar(10)</td>\n",
       "      <td>78</td>\n",
       "      <td>87</td>\n",
       "      <td>10</td>\n",
       "      <td>Attribute Code</td>\n",
       "      <td>(77, 87)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Field Name     Datatype  Start  End  Length               Description  \\\n",
       "0          prop_id      int(12)      1   12      12               Property ID   \n",
       "1      prop_val_yr   numeric(4)     13   16       4                      Year   \n",
       "2         imprv_id      int(12)     17   28      12            Improvement ID   \n",
       "3     imprv_det_id      int(12)     29   40      12     Improvement Detail ID   \n",
       "4    imprv_attr_id      int(12)     41   52      12  Improvement Attribute ID   \n",
       "5  imprv_attr_desc  varchar(25)     53   77      25     Attribute Description   \n",
       "6    imprv_attr_cd  varchar(10)     78   87      10            Attribute Code   \n",
       "\n",
       "   col_spec  dtype  \n",
       "0   (0, 12)  int64  \n",
       "1  (12, 16)  int16  \n",
       "2  (16, 28)  int64  \n",
       "3  (28, 40)  int64  \n",
       "4  (40, 52)  int64  \n",
       "5  (52, 77)   None  \n",
       "6  (77, 87)   None  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tcp.load_improvement_features_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prop_id</th>\n",
       "      <th>prop_val_yr</th>\n",
       "      <th>imprv_id</th>\n",
       "      <th>imprv_det_id</th>\n",
       "      <th>imprv_attr_id</th>\n",
       "      <th>imprv_attr_desc</th>\n",
       "      <th>imprv_attr_cd</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>41038598</td>\n",
       "      <td>35044442</td>\n",
       "      <td>Floor Factor</td>\n",
       "      <td>1ST</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>41038598</td>\n",
       "      <td>35923380</td>\n",
       "      <td>Foundation</td>\n",
       "      <td>SLAB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>41038598</td>\n",
       "      <td>36271876</td>\n",
       "      <td>Grade Factor</td>\n",
       "      <td>A</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100008</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225542</td>\n",
       "      <td>41038598</td>\n",
       "      <td>37645636</td>\n",
       "      <td>Shape Factor</td>\n",
       "      <td>I</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>100012</td>\n",
       "      <td>2023</td>\n",
       "      <td>5225543</td>\n",
       "      <td>41038605</td>\n",
       "      <td>35346999</td>\n",
       "      <td>Floor Factor</td>\n",
       "      <td>1ST</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3362247</th>\n",
       "      <td>977341</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807214</td>\n",
       "      <td>45611448</td>\n",
       "      <td>40098776</td>\n",
       "      <td>Shape Factor</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3362248</th>\n",
       "      <td>977345</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807244</td>\n",
       "      <td>45611722</td>\n",
       "      <td>40099051</td>\n",
       "      <td>Shape Factor</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3362249</th>\n",
       "      <td>977346</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807254</td>\n",
       "      <td>45611800</td>\n",
       "      <td>40099161</td>\n",
       "      <td>Floor Factor</td>\n",
       "      <td>1ST</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3362250</th>\n",
       "      <td>977346</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807254</td>\n",
       "      <td>45611800</td>\n",
       "      <td>40099162</td>\n",
       "      <td>Shape Factor</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3362251</th>\n",
       "      <td>977397</td>\n",
       "      <td>2023</td>\n",
       "      <td>5807600</td>\n",
       "      <td>45615198</td>\n",
       "      <td>40102603</td>\n",
       "      <td>Shape Factor</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3362252 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         prop_id  prop_val_yr  imprv_id  imprv_det_id  imprv_attr_id  \\\n",
       "0         100008         2023   5225542      41038598       35044442   \n",
       "1         100008         2023   5225542      41038598       35923380   \n",
       "2         100008         2023   5225542      41038598       36271876   \n",
       "3         100008         2023   5225542      41038598       37645636   \n",
       "4         100012         2023   5225543      41038605       35346999   \n",
       "...          ...          ...       ...           ...            ...   \n",
       "3362247   977341         2023   5807214      45611448       40098776   \n",
       "3362248   977345         2023   5807244      45611722       40099051   \n",
       "3362249   977346         2023   5807254      45611800       40099161   \n",
       "3362250   977346         2023   5807254      45611800       40099162   \n",
       "3362251   977397         2023   5807600      45615198       40102603   \n",
       "\n",
       "        imprv_attr_desc imprv_attr_cd  \n",
       "0          Floor Factor           1ST  \n",
       "1            Foundation          SLAB  \n",
       "2          Grade Factor             A  \n",
       "3          Shape Factor             I  \n",
       "4          Floor Factor           1ST  \n",
       "...                 ...           ...  \n",
       "3362247    Shape Factor             R  \n",
       "3362248    Shape Factor             R  \n",
       "3362249    Floor Factor           1ST  \n",
       "3362250    Shape Factor             R  \n",
       "3362251    Shape Factor             R  \n",
       "\n",
       "[3362252 rows x 7 columns]"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tcp.parse_improvement_features(f'{RAW_TCAD_DIR}/IMP_ATR.txt',export_file=f'{PROCESSED_TCAD_DIR}/IMP_ATR.parquet')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Property Details"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The property details files is where the bulk of the TCAD records data is stored. Data of interest here includes the property address, property value as well as other locational or tax related data. Due to how large and sparse the data was, many columns were dropped to improve performance. For instance, the addresses of the property owner, appraisal agent, insurances company as well as serveral various tax exemption information did not seem relevant. Of about 440 columns, 57 were kept."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Field Name</th>\n",
       "      <th>Datatype</th>\n",
       "      <th>Start</th>\n",
       "      <th>End</th>\n",
       "      <th>Length</th>\n",
       "      <th>Description</th>\n",
       "      <th>col_spec</th>\n",
       "      <th>dtype</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>prop_id</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>12</td>\n",
       "      <td>Property ID</td>\n",
       "      <td>(0, 12)</td>\n",
       "      <td>int64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>prop_type_cd</td>\n",
       "      <td>char(5)</td>\n",
       "      <td>13</td>\n",
       "      <td>17</td>\n",
       "      <td>5</td>\n",
       "      <td>Property Type Code:\\nR = Real\\nP = Business Pe...</td>\n",
       "      <td>(12, 17)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>prop_val_yr</td>\n",
       "      <td>numeric(5)</td>\n",
       "      <td>18</td>\n",
       "      <td>22</td>\n",
       "      <td>5</td>\n",
       "      <td>Appraisal or Tax Year</td>\n",
       "      <td>(17, 22)</td>\n",
       "      <td>int16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>geo_id</td>\n",
       "      <td>char(50)</td>\n",
       "      <td>547</td>\n",
       "      <td>596</td>\n",
       "      <td>50</td>\n",
       "      <td>Geographic ID</td>\n",
       "      <td>(546, 596)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>partial_owner</td>\n",
       "      <td>char(1)</td>\n",
       "      <td>679</td>\n",
       "      <td>679</td>\n",
       "      <td>1</td>\n",
       "      <td>Partial Owner ('T' or 'F')\\nThis indicates if ...</td>\n",
       "      <td>(678, 679)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>udi_group</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>680</td>\n",
       "      <td>691</td>\n",
       "      <td>12</td>\n",
       "      <td>This field indicates the UDI group that a part...</td>\n",
       "      <td>(679, 691)</td>\n",
       "      <td>int64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>situs_street_prefx</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>1040</td>\n",
       "      <td>1049</td>\n",
       "      <td>10</td>\n",
       "      <td>Situs/Location Street Prefix</td>\n",
       "      <td>(1039, 1049)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>situs_street</td>\n",
       "      <td>char(50)</td>\n",
       "      <td>1050</td>\n",
       "      <td>1099</td>\n",
       "      <td>50</td>\n",
       "      <td>Situs/Location Street</td>\n",
       "      <td>(1049, 1099)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>situs_street_suffix</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>1100</td>\n",
       "      <td>1109</td>\n",
       "      <td>10</td>\n",
       "      <td>Situs/Location Street Suffix</td>\n",
       "      <td>(1099, 1109)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>situs_city</td>\n",
       "      <td>char(30)</td>\n",
       "      <td>1110</td>\n",
       "      <td>1139</td>\n",
       "      <td>30</td>\n",
       "      <td>Situs/Location City</td>\n",
       "      <td>(1109, 1139)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>situs_zip</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>1140</td>\n",
       "      <td>1149</td>\n",
       "      <td>10</td>\n",
       "      <td>Situs/Location Zip</td>\n",
       "      <td>(1139, 1149)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>legal_desc</td>\n",
       "      <td>char(255)</td>\n",
       "      <td>1150</td>\n",
       "      <td>1404</td>\n",
       "      <td>255</td>\n",
       "      <td>Legal Description</td>\n",
       "      <td>(1149, 1404)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>legal_desc2</td>\n",
       "      <td>char(255)</td>\n",
       "      <td>1405</td>\n",
       "      <td>1659</td>\n",
       "      <td>255</td>\n",
       "      <td>Additional Legal Description</td>\n",
       "      <td>(1404, 1659)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>legal_acreage</td>\n",
       "      <td>numeric(16)</td>\n",
       "      <td>1660</td>\n",
       "      <td>1675</td>\n",
       "      <td>16</td>\n",
       "      <td>Legal Acreage (4 Decimals)</td>\n",
       "      <td>(1659, 1675)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>abs_subdv_cd</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>1676</td>\n",
       "      <td>1685</td>\n",
       "      <td>10</td>\n",
       "      <td>Abstract/Subdivision code</td>\n",
       "      <td>(1675, 1685)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>hood_cd</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>1686</td>\n",
       "      <td>1695</td>\n",
       "      <td>10</td>\n",
       "      <td>Neighborhood code</td>\n",
       "      <td>(1685, 1695)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>block</td>\n",
       "      <td>char(50)</td>\n",
       "      <td>1696</td>\n",
       "      <td>1745</td>\n",
       "      <td>50</td>\n",
       "      <td>Block</td>\n",
       "      <td>(1695, 1745)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>tract_or_lot</td>\n",
       "      <td>char(50)</td>\n",
       "      <td>1746</td>\n",
       "      <td>1795</td>\n",
       "      <td>50</td>\n",
       "      <td>Tract/lot</td>\n",
       "      <td>(1745, 1795)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>land_hstd_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>1796</td>\n",
       "      <td>1810</td>\n",
       "      <td>15</td>\n",
       "      <td>Land Homestead Value</td>\n",
       "      <td>(1795, 1810)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>land_non_hstd_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>1811</td>\n",
       "      <td>1825</td>\n",
       "      <td>15</td>\n",
       "      <td>Land Non-Homestead Value</td>\n",
       "      <td>(1810, 1825)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>imprv_hstd_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>1826</td>\n",
       "      <td>1840</td>\n",
       "      <td>15</td>\n",
       "      <td>Improvement Homestead Value</td>\n",
       "      <td>(1825, 1840)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>imprv_non_hstd_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>1841</td>\n",
       "      <td>1855</td>\n",
       "      <td>15</td>\n",
       "      <td>Improvement Non-Homestead value</td>\n",
       "      <td>(1840, 1855)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>appraised_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>1916</td>\n",
       "      <td>1930</td>\n",
       "      <td>15</td>\n",
       "      <td>Appraised Value</td>\n",
       "      <td>(1915, 1930)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>ten_percent_cap</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>1931</td>\n",
       "      <td>1945</td>\n",
       "      <td>15</td>\n",
       "      <td>Ten Percent Homestead Cap Adjustment</td>\n",
       "      <td>(1930, 1945)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>assessed_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>1946</td>\n",
       "      <td>1960</td>\n",
       "      <td>15</td>\n",
       "      <td>Assessed Value (Appraised - Cap)</td>\n",
       "      <td>(1945, 1960)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>deed_book_id</td>\n",
       "      <td>char(20)</td>\n",
       "      <td>1994</td>\n",
       "      <td>2013</td>\n",
       "      <td>20</td>\n",
       "      <td>Deed Book ID</td>\n",
       "      <td>(1993, 2013)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>deed_book_page</td>\n",
       "      <td>char(20)</td>\n",
       "      <td>2014</td>\n",
       "      <td>2033</td>\n",
       "      <td>20</td>\n",
       "      <td>Deed Book Page</td>\n",
       "      <td>(2013, 2033)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>deed_dt</td>\n",
       "      <td>char(25)</td>\n",
       "      <td>2034</td>\n",
       "      <td>2058</td>\n",
       "      <td>25</td>\n",
       "      <td>Deed Date</td>\n",
       "      <td>(2033, 2058)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>mortage_co_name</td>\n",
       "      <td>char(70)</td>\n",
       "      <td>2071</td>\n",
       "      <td>2140</td>\n",
       "      <td>70</td>\n",
       "      <td>Name of Mortgage Company</td>\n",
       "      <td>(2070, 2140)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>en_exempt</td>\n",
       "      <td>char(1)</td>\n",
       "      <td>2724</td>\n",
       "      <td>2724</td>\n",
       "      <td>1</td>\n",
       "      <td>Energy  Exemption (‘T’ or ‘F’)</td>\n",
       "      <td>(2723, 2724)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>pc_exempt</td>\n",
       "      <td>char(1)</td>\n",
       "      <td>2728</td>\n",
       "      <td>2728</td>\n",
       "      <td>1</td>\n",
       "      <td>Pollution Control Exemption (‘T’ or ‘F’)</td>\n",
       "      <td>(2727, 2728)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>so_exempt</td>\n",
       "      <td>char(1)</td>\n",
       "      <td>2729</td>\n",
       "      <td>2729</td>\n",
       "      <td>1</td>\n",
       "      <td>Solar Exemption (‘T’ or ‘F’)</td>\n",
       "      <td>(2728, 2729)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>imprv_state_cd</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>2732</td>\n",
       "      <td>2741</td>\n",
       "      <td>10</td>\n",
       "      <td>State Code that is on the improvements</td>\n",
       "      <td>(2731, 2741)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>land_state_cd</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>2742</td>\n",
       "      <td>2751</td>\n",
       "      <td>10</td>\n",
       "      <td>State Code that is on the land</td>\n",
       "      <td>(2741, 2751)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>personal_state_cd</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>2752</td>\n",
       "      <td>2761</td>\n",
       "      <td>10</td>\n",
       "      <td>State Code that is on the personal property</td>\n",
       "      <td>(2751, 2761)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>land_acres</td>\n",
       "      <td>numeric(20)</td>\n",
       "      <td>2772</td>\n",
       "      <td>2791</td>\n",
       "      <td>20</td>\n",
       "      <td>Sum of the acres based on land segments</td>\n",
       "      <td>(2771, 2791)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>productivity_use_code</td>\n",
       "      <td>char(3)</td>\n",
       "      <td>4048</td>\n",
       "      <td>4050</td>\n",
       "      <td>3</td>\n",
       "      <td>Productivity Use Code</td>\n",
       "      <td>(4047, 4050)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>prop_owner_sequence</td>\n",
       "      <td>char(40)</td>\n",
       "      <td>4051</td>\n",
       "      <td>4090</td>\n",
       "      <td>40</td>\n",
       "      <td>Unique identifier for the property owner combi...</td>\n",
       "      <td>(4050, 4090)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>late_freeport_penalty</td>\n",
       "      <td>int(12)</td>\n",
       "      <td>4115</td>\n",
       "      <td>4126</td>\n",
       "      <td>12</td>\n",
       "      <td>Late freeport penalty</td>\n",
       "      <td>(4114, 4126)</td>\n",
       "      <td>int64</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>dba</td>\n",
       "      <td>char(40)</td>\n",
       "      <td>4136</td>\n",
       "      <td>4175</td>\n",
       "      <td>40</td>\n",
       "      <td>Doing business as</td>\n",
       "      <td>(4135, 4175)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>market_value</td>\n",
       "      <td>numeric(14)</td>\n",
       "      <td>4214</td>\n",
       "      <td>4227</td>\n",
       "      <td>14</td>\n",
       "      <td>Property market value</td>\n",
       "      <td>(4213, 4227)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>ref_id1</td>\n",
       "      <td>char(25)</td>\n",
       "      <td>4410</td>\n",
       "      <td>4434</td>\n",
       "      <td>25</td>\n",
       "      <td>Property Reference ID 1</td>\n",
       "      <td>(4409, 4434)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>ref_id2</td>\n",
       "      <td>char(25)</td>\n",
       "      <td>4435</td>\n",
       "      <td>4459</td>\n",
       "      <td>25</td>\n",
       "      <td>Property Reference ID 2</td>\n",
       "      <td>(4434, 4459)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>situs_num</td>\n",
       "      <td>char(15)</td>\n",
       "      <td>4460</td>\n",
       "      <td>4474</td>\n",
       "      <td>15</td>\n",
       "      <td>Situs Number</td>\n",
       "      <td>(4459, 4474)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>situs_unit</td>\n",
       "      <td>char(5)</td>\n",
       "      <td>4475</td>\n",
       "      <td>4479</td>\n",
       "      <td>5</td>\n",
       "      <td>Situs Unit #</td>\n",
       "      <td>(4474, 4479)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>entities</td>\n",
       "      <td>char(140)</td>\n",
       "      <td>5202</td>\n",
       "      <td>5341</td>\n",
       "      <td>140</td>\n",
       "      <td>Entities list</td>\n",
       "      <td>(5201, 5341)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>eco_exempt</td>\n",
       "      <td>char(1)</td>\n",
       "      <td>5342</td>\n",
       "      <td>5342</td>\n",
       "      <td>1</td>\n",
       "      <td>eco exemption</td>\n",
       "      <td>(5341, 5342)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>dataset_id</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>5343</td>\n",
       "      <td>5357</td>\n",
       "      <td>15</td>\n",
       "      <td>Unique number that identifies this export</td>\n",
       "      <td>(5342, 5357)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>deed_num</td>\n",
       "      <td>char(50)</td>\n",
       "      <td>5358</td>\n",
       "      <td>5407</td>\n",
       "      <td>50</td>\n",
       "      <td>Deed Number</td>\n",
       "      <td>(5357, 5407)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>exemption_percentage</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>5417</td>\n",
       "      <td>5431</td>\n",
       "      <td>15</td>\n",
       "      <td>Exemption percentage (string representing a fl...</td>\n",
       "      <td>(5416, 5431)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>sic_code</td>\n",
       "      <td>char(10)</td>\n",
       "      <td>5873</td>\n",
       "      <td>5882</td>\n",
       "      <td>10</td>\n",
       "      <td>SIC Code (new position in file due to change i...</td>\n",
       "      <td>(5872, 5882)</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>omitted_imprv_hstd_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>7294</td>\n",
       "      <td>7308</td>\n",
       "      <td>15</td>\n",
       "      <td>Omitted Improvement Homestead Value</td>\n",
       "      <td>(7293, 7308)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>omitted_imprv_non_hstd_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>7309</td>\n",
       "      <td>7323</td>\n",
       "      <td>15</td>\n",
       "      <td>Omitted Improvement Non-Homestead Value</td>\n",
       "      <td>(7308, 7323)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>pp_late_interstate_allocation_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>8588</td>\n",
       "      <td>8602</td>\n",
       "      <td>15</td>\n",
       "      <td>Personal Property interstate allocation value ...</td>\n",
       "      <td>(8587, 8602)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>appraised_val_reflecting_productivity_loss</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>8603</td>\n",
       "      <td>8617</td>\n",
       "      <td>15</td>\n",
       "      <td>Appraised Value reflecting Productivity Loss</td>\n",
       "      <td>(8602, 8617)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>55</th>\n",
       "      <td>assessed_val_reflecting_productivity_loss</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>8618</td>\n",
       "      <td>8632</td>\n",
       "      <td>15</td>\n",
       "      <td>Assessed Value reflecting Productivity Loss</td>\n",
       "      <td>(8617, 8632)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>56</th>\n",
       "      <td>late_correction_val</td>\n",
       "      <td>numeric(15)</td>\n",
       "      <td>8998</td>\n",
       "      <td>9012</td>\n",
       "      <td>15</td>\n",
       "      <td>Late correction value subject to penalty</td>\n",
       "      <td>(8997, 9012)</td>\n",
       "      <td>float32</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    Field Name     Datatype  Start   End  \\\n",
       "0                                      prop_id      int(12)      1    12   \n",
       "1                                 prop_type_cd      char(5)     13    17   \n",
       "2                                  prop_val_yr   numeric(5)     18    22   \n",
       "3                                       geo_id     char(50)    547   596   \n",
       "4                                partial_owner      char(1)    679   679   \n",
       "5                                    udi_group      int(12)    680   691   \n",
       "6                           situs_street_prefx     char(10)   1040  1049   \n",
       "7                                 situs_street     char(50)   1050  1099   \n",
       "8                          situs_street_suffix     char(10)   1100  1109   \n",
       "9                                   situs_city     char(30)   1110  1139   \n",
       "10                                   situs_zip     char(10)   1140  1149   \n",
       "11                                  legal_desc    char(255)   1150  1404   \n",
       "12                                 legal_desc2    char(255)   1405  1659   \n",
       "13                               legal_acreage  numeric(16)   1660  1675   \n",
       "14                                abs_subdv_cd     char(10)   1676  1685   \n",
       "15                                     hood_cd     char(10)   1686  1695   \n",
       "16                                       block     char(50)   1696  1745   \n",
       "17                                tract_or_lot     char(50)   1746  1795   \n",
       "18                               land_hstd_val  numeric(15)   1796  1810   \n",
       "19                           land_non_hstd_val  numeric(15)   1811  1825   \n",
       "20                              imprv_hstd_val  numeric(15)   1826  1840   \n",
       "21                          imprv_non_hstd_val  numeric(15)   1841  1855   \n",
       "22                               appraised_val  numeric(15)   1916  1930   \n",
       "23                             ten_percent_cap  numeric(15)   1931  1945   \n",
       "24                                assessed_val  numeric(15)   1946  1960   \n",
       "25                                deed_book_id     char(20)   1994  2013   \n",
       "26                              deed_book_page     char(20)   2014  2033   \n",
       "27                                     deed_dt     char(25)   2034  2058   \n",
       "28                             mortage_co_name     char(70)   2071  2140   \n",
       "29                                   en_exempt      char(1)   2724  2724   \n",
       "30                                   pc_exempt      char(1)   2728  2728   \n",
       "31                                   so_exempt      char(1)   2729  2729   \n",
       "32                              imprv_state_cd     char(10)   2732  2741   \n",
       "33                               land_state_cd     char(10)   2742  2751   \n",
       "34                           personal_state_cd     char(10)   2752  2761   \n",
       "35                                  land_acres  numeric(20)   2772  2791   \n",
       "36                       productivity_use_code      char(3)   4048  4050   \n",
       "37                         prop_owner_sequence     char(40)   4051  4090   \n",
       "38                       late_freeport_penalty      int(12)   4115  4126   \n",
       "39                                         dba     char(40)   4136  4175   \n",
       "40                                market_value  numeric(14)   4214  4227   \n",
       "41                                     ref_id1     char(25)   4410  4434   \n",
       "42                                     ref_id2     char(25)   4435  4459   \n",
       "43                                   situs_num     char(15)   4460  4474   \n",
       "44                                  situs_unit      char(5)   4475  4479   \n",
       "45                                    entities    char(140)   5202  5341   \n",
       "46                                  eco_exempt      char(1)   5342  5342   \n",
       "47                                  dataset_id  numeric(15)   5343  5357   \n",
       "48                                    deed_num     char(50)   5358  5407   \n",
       "49                        exemption_percentage  numeric(15)   5417  5431   \n",
       "50                                    sic_code     char(10)   5873  5882   \n",
       "51                      omitted_imprv_hstd_val  numeric(15)   7294  7308   \n",
       "52                  omitted_imprv_non_hstd_val  numeric(15)   7309  7323   \n",
       "53           pp_late_interstate_allocation_val  numeric(15)   8588  8602   \n",
       "54  appraised_val_reflecting_productivity_loss  numeric(15)   8603  8617   \n",
       "55   assessed_val_reflecting_productivity_loss  numeric(15)   8618  8632   \n",
       "56                         late_correction_val  numeric(15)   8998  9012   \n",
       "\n",
       "    Length                                        Description      col_spec  \\\n",
       "0       12                                        Property ID       (0, 12)   \n",
       "1        5  Property Type Code:\\nR = Real\\nP = Business Pe...      (12, 17)   \n",
       "2        5                              Appraisal or Tax Year      (17, 22)   \n",
       "3       50                                      Geographic ID    (546, 596)   \n",
       "4        1  Partial Owner ('T' or 'F')\\nThis indicates if ...    (678, 679)   \n",
       "5       12  This field indicates the UDI group that a part...    (679, 691)   \n",
       "6       10                       Situs/Location Street Prefix  (1039, 1049)   \n",
       "7       50                              Situs/Location Street  (1049, 1099)   \n",
       "8       10                       Situs/Location Street Suffix  (1099, 1109)   \n",
       "9       30                                Situs/Location City  (1109, 1139)   \n",
       "10      10                                 Situs/Location Zip  (1139, 1149)   \n",
       "11     255                                  Legal Description  (1149, 1404)   \n",
       "12     255                       Additional Legal Description  (1404, 1659)   \n",
       "13      16                         Legal Acreage (4 Decimals)  (1659, 1675)   \n",
       "14      10                          Abstract/Subdivision code  (1675, 1685)   \n",
       "15      10                                  Neighborhood code  (1685, 1695)   \n",
       "16      50                                              Block  (1695, 1745)   \n",
       "17      50                                          Tract/lot  (1745, 1795)   \n",
       "18      15                               Land Homestead Value  (1795, 1810)   \n",
       "19      15                           Land Non-Homestead Value  (1810, 1825)   \n",
       "20      15                        Improvement Homestead Value  (1825, 1840)   \n",
       "21      15                    Improvement Non-Homestead value  (1840, 1855)   \n",
       "22      15                                    Appraised Value  (1915, 1930)   \n",
       "23      15               Ten Percent Homestead Cap Adjustment  (1930, 1945)   \n",
       "24      15                   Assessed Value (Appraised - Cap)  (1945, 1960)   \n",
       "25      20                                       Deed Book ID  (1993, 2013)   \n",
       "26      20                                     Deed Book Page  (2013, 2033)   \n",
       "27      25                                          Deed Date  (2033, 2058)   \n",
       "28      70                           Name of Mortgage Company  (2070, 2140)   \n",
       "29       1                     Energy  Exemption (‘T’ or ‘F’)  (2723, 2724)   \n",
       "30       1           Pollution Control Exemption (‘T’ or ‘F’)  (2727, 2728)   \n",
       "31       1                       Solar Exemption (‘T’ or ‘F’)  (2728, 2729)   \n",
       "32      10             State Code that is on the improvements  (2731, 2741)   \n",
       "33      10                     State Code that is on the land  (2741, 2751)   \n",
       "34      10        State Code that is on the personal property  (2751, 2761)   \n",
       "35      20            Sum of the acres based on land segments  (2771, 2791)   \n",
       "36       3                              Productivity Use Code  (4047, 4050)   \n",
       "37      40  Unique identifier for the property owner combi...  (4050, 4090)   \n",
       "38      12                              Late freeport penalty  (4114, 4126)   \n",
       "39      40                                  Doing business as  (4135, 4175)   \n",
       "40      14                              Property market value  (4213, 4227)   \n",
       "41      25                            Property Reference ID 1  (4409, 4434)   \n",
       "42      25                            Property Reference ID 2  (4434, 4459)   \n",
       "43      15                                       Situs Number  (4459, 4474)   \n",
       "44       5                                       Situs Unit #  (4474, 4479)   \n",
       "45     140                                      Entities list  (5201, 5341)   \n",
       "46       1                                      eco exemption  (5341, 5342)   \n",
       "47      15          Unique number that identifies this export  (5342, 5357)   \n",
       "48      50                                        Deed Number  (5357, 5407)   \n",
       "49      15  Exemption percentage (string representing a fl...  (5416, 5431)   \n",
       "50      10  SIC Code (new position in file due to change i...  (5872, 5882)   \n",
       "51      15                Omitted Improvement Homestead Value  (7293, 7308)   \n",
       "52      15            Omitted Improvement Non-Homestead Value  (7308, 7323)   \n",
       "53      15  Personal Property interstate allocation value ...  (8587, 8602)   \n",
       "54      15       Appraised Value reflecting Productivity Loss  (8602, 8617)   \n",
       "55      15        Assessed Value reflecting Productivity Loss  (8617, 8632)   \n",
       "56      15           Late correction value subject to penalty  (8997, 9012)   \n",
       "\n",
       "      dtype  \n",
       "0     int64  \n",
       "1      None  \n",
       "2     int16  \n",
       "3      None  \n",
       "4      None  \n",
       "5     int64  \n",
       "6      None  \n",
       "7      None  \n",
       "8      None  \n",
       "9      None  \n",
       "10     None  \n",
       "11     None  \n",
       "12     None  \n",
       "13  float32  \n",
       "14     None  \n",
       "15     None  \n",
       "16     None  \n",
       "17     None  \n",
       "18  float32  \n",
       "19  float32  \n",
       "20  float32  \n",
       "21  float32  \n",
       "22  float32  \n",
       "23  float32  \n",
       "24  float32  \n",
       "25     None  \n",
       "26     None  \n",
       "27     None  \n",
       "28     None  \n",
       "29     None  \n",
       "30     None  \n",
       "31     None  \n",
       "32     None  \n",
       "33     None  \n",
       "34     None  \n",
       "35  float32  \n",
       "36     None  \n",
       "37     None  \n",
       "38    int64  \n",
       "39     None  \n",
       "40  float32  \n",
       "41     None  \n",
       "42     None  \n",
       "43     None  \n",
       "44     None  \n",
       "45     None  \n",
       "46     None  \n",
       "47  float32  \n",
       "48     None  \n",
       "49  float32  \n",
       "50     None  \n",
       "51  float32  \n",
       "52  float32  \n",
       "53  float32  \n",
       "54  float32  \n",
       "55  float32  \n",
       "56  float32  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tcp.load_property_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prop_id</th>\n",
       "      <th>prop_type_cd</th>\n",
       "      <th>prop_val_yr</th>\n",
       "      <th>geo_id</th>\n",
       "      <th>partial_owner</th>\n",
       "      <th>udi_group</th>\n",
       "      <th>situs_street_prefx</th>\n",
       "      <th>situs_street</th>\n",
       "      <th>situs_street_suffix</th>\n",
       "      <th>situs_city</th>\n",
       "      <th>...</th>\n",
       "      <th>dataset_id</th>\n",
       "      <th>deed_num</th>\n",
       "      <th>exemption_percentage</th>\n",
       "      <th>sic_code</th>\n",
       "      <th>omitted_imprv_hstd_val</th>\n",
       "      <th>omitted_imprv_non_hstd_val</th>\n",
       "      <th>pp_late_interstate_allocation_val</th>\n",
       "      <th>appraised_val_reflecting_productivity_loss</th>\n",
       "      <th>assessed_val_reflecting_productivity_loss</th>\n",
       "      <th>late_correction_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100008</td>\n",
       "      <td>R</td>\n",
       "      <td>2023</td>\n",
       "      <td>0100030105</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>LAMAR</td>\n",
       "      <td>BLVD</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>2014035621TR</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100012</td>\n",
       "      <td>R</td>\n",
       "      <td>2023</td>\n",
       "      <td>0100030109</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>LAMAR</td>\n",
       "      <td>BLVD</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>2011172140TR</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100015</td>\n",
       "      <td>R</td>\n",
       "      <td>2023</td>\n",
       "      <td>0100030112</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>LAMAR</td>\n",
       "      <td>BLVD</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>2001096152TR</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100018</td>\n",
       "      <td>R</td>\n",
       "      <td>2023</td>\n",
       "      <td>0100030115</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>LAMAR</td>\n",
       "      <td>BLVD</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>2014043470TR</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>100020</td>\n",
       "      <td>R</td>\n",
       "      <td>2023</td>\n",
       "      <td>0100030201</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>S</td>\n",
       "      <td>LAMAR</td>\n",
       "      <td>BLVD</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>2007035338TR</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>477141</th>\n",
       "      <td>977395</td>\n",
       "      <td>P</td>\n",
       "      <td>2023</td>\n",
       "      <td>NaN</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>KEVIN</td>\n",
       "      <td>LN</td>\n",
       "      <td>AUSTIN</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>477142</th>\n",
       "      <td>977396</td>\n",
       "      <td>P</td>\n",
       "      <td>2023</td>\n",
       "      <td>NaN</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>VARIOUS LOCATIONS</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>477143</th>\n",
       "      <td>977397</td>\n",
       "      <td>MH</td>\n",
       "      <td>2023</td>\n",
       "      <td>0259410216</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>KILLINGSWORTH</td>\n",
       "      <td>LN</td>\n",
       "      <td>PFLUGERVILLE</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>477144</th>\n",
       "      <td>977401</td>\n",
       "      <td>P</td>\n",
       "      <td>2023</td>\n",
       "      <td>NaN</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>PANTHER</td>\n",
       "      <td>LN</td>\n",
       "      <td>PFLUGERVILLE</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>477145</th>\n",
       "      <td>977402</td>\n",
       "      <td>P</td>\n",
       "      <td>2023</td>\n",
       "      <td>NaN</td>\n",
       "      <td>F</td>\n",
       "      <td>0</td>\n",
       "      <td>E</td>\n",
       "      <td>STASSNEY</td>\n",
       "      <td>LN</td>\n",
       "      <td>AUSTIN</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>477146 rows × 57 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        prop_id prop_type_cd  prop_val_yr      geo_id partial_owner  \\\n",
       "0        100008            R         2023  0100030105             F   \n",
       "1        100012            R         2023  0100030109             F   \n",
       "2        100015            R         2023  0100030112             F   \n",
       "3        100018            R         2023  0100030115             F   \n",
       "4        100020            R         2023  0100030201             F   \n",
       "...         ...          ...          ...         ...           ...   \n",
       "477141   977395            P         2023         NaN             F   \n",
       "477142   977396            P         2023         NaN             F   \n",
       "477143   977397           MH         2023  0259410216             F   \n",
       "477144   977401            P         2023         NaN             F   \n",
       "477145   977402            P         2023         NaN             F   \n",
       "\n",
       "        udi_group situs_street_prefx       situs_street situs_street_suffix  \\\n",
       "0               0                  S              LAMAR                BLVD   \n",
       "1               0                  S              LAMAR                BLVD   \n",
       "2               0                  S              LAMAR                BLVD   \n",
       "3               0                  S              LAMAR                BLVD   \n",
       "4               0                  S              LAMAR                BLVD   \n",
       "...           ...                ...                ...                 ...   \n",
       "477141          0                NaN              KEVIN                  LN   \n",
       "477142          0                NaN  VARIOUS LOCATIONS                 NaN   \n",
       "477143          0                NaN      KILLINGSWORTH                  LN   \n",
       "477144          0                NaN            PANTHER                  LN   \n",
       "477145          0                  E           STASSNEY                  LN   \n",
       "\n",
       "          situs_city  ... dataset_id      deed_num  exemption_percentage  \\\n",
       "0                NaN  ...          0  2014035621TR                     0   \n",
       "1                NaN  ...          0  2011172140TR                     0   \n",
       "2                NaN  ...          0  2001096152TR                     0   \n",
       "3                NaN  ...          0  2014043470TR                     0   \n",
       "4                NaN  ...          0  2007035338TR                     0   \n",
       "...              ...  ...        ...           ...                   ...   \n",
       "477141        AUSTIN  ...          0           NaN                     0   \n",
       "477142           NaN  ...          0           NaN                     0   \n",
       "477143  PFLUGERVILLE  ...          0           NaN                     0   \n",
       "477144  PFLUGERVILLE  ...          0           NaN                     0   \n",
       "477145        AUSTIN  ...          0           NaN                     0   \n",
       "\n",
       "        sic_code omitted_imprv_hstd_val omitted_imprv_non_hstd_val  \\\n",
       "0            NaN                      0                          0   \n",
       "1            NaN                      0                          0   \n",
       "2            NaN                      0                          0   \n",
       "3            NaN                      0                          0   \n",
       "4            NaN                      0                          0   \n",
       "...          ...                    ...                        ...   \n",
       "477141       NaN                      0                          0   \n",
       "477142       NaN                      0                          0   \n",
       "477143       NaN                      0                          0   \n",
       "477144       NaN                      0                          0   \n",
       "477145       NaN                      0                          0   \n",
       "\n",
       "       pp_late_interstate_allocation_val  \\\n",
       "0                                      0   \n",
       "1                                      0   \n",
       "2                                      0   \n",
       "3                                      0   \n",
       "4                                      0   \n",
       "...                                  ...   \n",
       "477141                                 0   \n",
       "477142                                 0   \n",
       "477143                                 0   \n",
       "477144                                 0   \n",
       "477145                                 0   \n",
       "\n",
       "       appraised_val_reflecting_productivity_loss  \\\n",
       "0                                               0   \n",
       "1                                               0   \n",
       "2                                               0   \n",
       "3                                               0   \n",
       "4                                               0   \n",
       "...                                           ...   \n",
       "477141                                          0   \n",
       "477142                                          0   \n",
       "477143                                          0   \n",
       "477144                                          0   \n",
       "477145                                          0   \n",
       "\n",
       "        assessed_val_reflecting_productivity_loss  late_correction_val  \n",
       "0                                               0                    0  \n",
       "1                                               0                    0  \n",
       "2                                               0                    0  \n",
       "3                                               0                    0  \n",
       "4                                               0                    0  \n",
       "...                                           ...                  ...  \n",
       "477141                                          0                    0  \n",
       "477142                                          0                    0  \n",
       "477143                                          0                    0  \n",
       "477144                                          0                    0  \n",
       "477145                                          0                    0  \n",
       "\n",
       "[477146 rows x 57 columns]"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tcp.parse_property_details(f'{RAW_TCAD_DIR}/PROP.txt',export_file=f'{PROCESSED_TCAD_DIR}/PROP.parquet')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "main",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.1"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}