nerds_analysis_full.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## NERD$ DATA ANALYSIS—Bexar County, Texas<br>\n",
    "#### Description\n",
    "This project is intended to gauge whether there are funding discrepancies within Bexar County Independent School Districts when it comes to per-pupil spending and economic disadvantage rates. If a discrepancy is detected (low correlation coefficient…threshold TBD…or there are outlier schools on the resulting scatterplot, we can work with an education reporter to investigate further.    <br>\n",
    "\n",
    "#### Data Sources\n",
    "- The NERD$ data come from Georgetown University's [National Education Resource Database on School Spending Organization](https://georgetown.app.box.com/s/1dknmu4bjltrehzdygh63xnzebcki4ni/file/1060770984817). <br>\n",
    "- The Economic Disadvantage Data comes from [Square Meals.org](https://data.texas.gov/dataset/School-Nutrition-Programs-Contact-Information-and-/jezb-2499) <br>\n",
    "\n",
    "___\n",
    " "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import requisite libraries: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import Data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "nerds_full = pd.read_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data/pass2/NERDS/TX_1819_all_NERD$.csv\")\n",
    "ed_full = pd.read_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data/pass2/TxDA/ED_2018-19_TxDA.csv\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Clean Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nerds_full.dropna()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we merge them together into a file called `complete_data`: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "complete_data = pd.merge(nerds_full, ed_full, on=['distname', 'schoolname'])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Output a single master table:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "complete_data.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/complete_data.csv\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check for the school district names for filtering: "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we filter the individual districts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alamo_hts_isd = complete_data[complete_data.distname == \"ALAMO HEIGHTS ISD\"]\n",
    "east_central_isd = complete_data[complete_data.distname == \"EAST CENTRAL ISD\"]\n",
    "edgewood_isd = complete_data[complete_data.distname == \"EDGEWOOD ISD\"]\n",
    "ft_sam_houston_isd = complete_data[complete_data.distname == \"FT SAM HOUSTON ISD\"]\n",
    "harlandale_isd = complete_data[complete_data.distname == \"HARLANDALE ISD\"]\n",
    "judson_isd = complete_data[complete_data.distname == \"JUDSON ISD\"]\n",
    "lackland_isd = complete_data[complete_data.distname == \"LACKLAND ISD\"]\n",
    "north_east_isd = complete_data[complete_data.distname == \"NORTH EAST ISD\"]\n",
    "randolph_field_isd = complete_data[complete_data.distname == \"RANDOLPH FIELD ISD\"]\n",
    "san_antonio_isd = complete_data[complete_data.distname == \"SAN ANTONIO ISD\"]\n",
    "south_san_antonio_isd = complete_data[complete_data.distname == \"SOUTH SAN ANTONIO ISD\"]\n",
    "southside_isd = complete_data[complete_data.distname == \"SOUTHSIDE ISD\"]\n",
    "southwest_isd = complete_data[complete_data.distname == \"SOUTHWEST ISD\"]\n",
    "boerne_isd = complete_data[complete_data.distname == \"BOERNE ISD\"]\n",
    "comal_isd = complete_data[complete_data.distname == \"COMAL ISD\"]\n",
    "medina_valley_isd = complete_data[complete_data.distname == \"MEDINA VALLEY ISD\"]\n",
    "northside_isd = complete_data[complete_data.distname == \"NORTHSIDE ISD\"]\n",
    "scuc_isd = complete_data[complete_data.distname == \"SCHERTZ-CIBOLO-U CITY ISD\"]\n",
    "somerset_isd = complete_data[complete_data.distname == \"SOMERSET ISD\"]\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(edgewood_isd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alamo_hts_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/alamo_hts.csv\")\n",
    "east_central_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/east_central.csv\")\n",
    "edgewood_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/edgewood.csv\")\n",
    "ft_sam_houston_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/ft_sam_houston.csv\")\n",
    "harlandale_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/harlandale.csv\")\n",
    "judson_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/judson.csv\")\n",
    "lackland_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/lackland.csv\")\n",
    "north_east_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/north_east.csv\")\n",
    "randolph_field_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/randolph.csv\")\n",
    "san_antonio_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/sa_isd.csv\")\n",
    "south_san_antonio_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/south_sa.csv\")\n",
    "southside_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/southside.csv\")\n",
    "southwest_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/southwest.csv\")\n",
    "boerne_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/boerne.csv\")\n",
    "comal_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/comal.csv\")\n",
    "medina_valley_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/medina.csv\")\n",
    "northside_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/northside.csv\")\n",
    "scuc_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/scuc.csv\")\n",
    "somerset_isd.to_csv(\"/Users/wratko/Documents/projects/tx-school_funding/data_output/somerset.csv\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12 (main, Apr  5 2022, 01:53:17) \n[Clang 12.0.0 ]"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "23c9b771267491a6d8b0211014be734acb79cef3e9f9f53b1bbe656671aa7312"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}