Merge pull request CodeForPhilly#13 from CodeForPhilly/mapbox-upload

mapbox upload and awkde

Pipfile

@@ -19,6 +19,7 @@ esridump = "*"
 sqlalchemy = "*"
 psycopg2-binary = "*"
 geoalchemy2 = "*"
+mapbox = "*"
 
 [dev-packages]
 

README.md

@@ -1,14 +1,17 @@
 # Clean & Green Philly
+
 Philadelphia has a gun violence problem. This dashboard aims to help solve it by empowering community groups to carry out cleaning and greening interventions in the vacant properties where they can have the biggest impact.
 
 # Motivation
+
 Philadelphia has a gun violence problem. Homicides have been on the rise since 2013. The past three years—2020, 2021, and 2022—have been the deadliest on record, with a high of 562 homicides in 2021. Community members need solutions, but many city-run initiatives are frustratingly slow or inadequate. Nearly 80% of the city’s anti-violence spending focuses on long-term violence reduction without any clear, immediate impact.
 
 Research shows that greening and cleaning vacant and abandoned parcels is one of the most impactful, cost-effective interventions available to reduce gun violence in a neighborhood. Drs. Eugenia South and Charles Branas have led several studies that [demonstrate that greening vacant lots in Philadelphia reduced gun violence by as much as 29% in the surrounding area](https://www.pnas.org/doi/10.1073/pnas.1718503115). Similarly, cleaning and lightly repairing vacant houses led a 13% drop in gun assaults compared to nearby blocks. These “greening and cleaning” interventions not only reduce gun violence but also provide other benefits, such as reducing the urban heat island effect, lowering residents’ stress levels, and contributing to lower levels of depression among residents.
 
 There are roughly 40,000 vaccant properties in Philadelphia. Identifying the highest-priority vacant properties will allow community groups to invest their limited resources where they will have the biggest impact. Combining various public data, this dashboard helps users identify the properties that are ideal for them to intervene in. It also offers additional information about each property to facilitate interventions and indicates the best possible route to cleaning up the property.
 
 # Current Draft
+
 You can view the [most recent prototype of the website and dashboard](https://nlebovits.github.io/dashboard_demo_website/more_info.html), which was created in Quarto.
 
 Currently we are in the process of user testing a newer prototype, which looks like this:
@@ -18,7 +21,9 @@ Currently we are in the process of user testing a newer prototype, which looks l
 ![User prototype map](https://github.com/CodeForPhilly/vacant-lots-proj/assets/111617674/8cbf0b06-b299-49cd-8f9f-bbb714e55b44)
 
 # Current Tasks
+
 At the moment, we are working on:
+
 - User testing with the prototype
 - Finishing the Python script that creates the underlying dataset
 
@@ -27,16 +32,17 @@ Current tasks for the Python script are visible under the Issues section.
 In September, we hope to begin turning our attention to building the dashboard and website, which will be built in React JS.
 
 # Looking For
+
 - Front end engineer w/mapping experience
 - Python data engineer
 
 # Languages/Frameworks
+
 - React
 - Nextjs
 - Tailwind
 - Python (data engineering)
 
-
 # Setup Instructions
 
 1. Install [pyenv](https://github.com/pyenv/pyenv) (or [pyenv-win](https://github.com/pyenv-win/pyenv-win) for Windows) for Python version management
@@ -56,24 +62,35 @@ In September, we hope to begin turning our attention to building the dashboard a
    `CREATE EXTENSION postgis;`
 5. Set your database connection string to an environment variable `VACANT_LOTS_DB`
 
+In `/config/config`, set `FORCE_RELOAD` = `False` to read "cached" data in postgres instead of downloading new data.
+
 # awkde
 
-We are using the [awkde package](https://github.com/mennthor/awkde) to create the Adaptive Width KDE. It is not available through pip. Install directly in the `src` folder using their instructions.
+We are using the [awkde package](https://github.com/mennthor/awkde) to create the Adaptive Width KDE. It is not available through pip. To install, navigate to `src/awkde` and run
+
+```
+pip install -e ./awkde
+```
 
 # Usage Instructions
 
 1. Activate the pipenv shell: `pipenv shell`
 2. Move the to src folder `cd src`
 3. Run the main script `python script.py`
 
+# Mapbox
+
+To upload to Mapbox through the API you'll need a key with `upload` write access, saved to the environment variable `CFP_MAPBOX_TOKEN_UPLOADER`
+
 # Contribution Instructions
 
 1. Create a fork of this repository and work from branches within your fork. When those changes are ready for review, create a pull request from fork:branch to upstream:main
-2. Before committing changes, format your code to maintain a consistent codebase: 
-    ```
-    pipenv shell
-    black .
-    ```
+2. Before committing changes, format your code to maintain a consistent codebase:
+   ```
+   pipenv shell
+   black .
+   ```
 
 # License
+
 MIT © Code for Philly

src/awkde/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2017 Thorben Menne
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

src/awkde/README.md

@@ -0,0 +1,65 @@
+# Adaptive Width KDE with Gaussian Kernels
+
+## Installation
+
+This uses the awesome [pybind11](https://github.com/pybind/pybind11) package which makes creating C++ bindings super convenient.
+Only the evaluation is written in a small C++ snippet to speed it up, the rest is a pure python implementation.
+
+The setup is taken from the example at https://github.com/pybind/python_example
+Just clone the repository and invoke `pip`:
+
+```bash
+git clone https://github.com/mennthor/awkde
+pip install [--user] [-e] ./awkde
+```
+Note: `--user` and `-e` are optional. Use `--user` to install in your user site-package directory (recommended) and `-e` if you want to only link the files to the site-package directories, so that when you make any changes in your local copy, you don't have to reinstall all the time.
+
+Try to run the `example/examply.py`:
+
+```
+cd awkde/example
+python example.py
+```
+
+You should get this plot in the examples folder (you need matplotlib for the plot):
+
+![example plot](example/example.png)
+
+
+## Algorithm
+
+The unweighted kernel density estimator is defined as
+
+![kernel density formula](tex/kernel_dens.png)
+
+where the product `h * lambda` takes the role of a local `sigma`.
+
+The kernel bandwith is choosen locally to account for variations in the
+density of the data.
+Areas with large density gets smaller kernels and vice versa.
+This smoothes the tails and gets high resolution in high statistics regions.
+The local bandwidth paramter is defined as
+
+![kernel density formula](tex/lambda.png)
+
+where
+
+![kernel density formula](tex/log_g.png)
+
+is some normalization and `^f(X_i)` the KDE estimate at the data point `X_i`.
+The local bandwidth is multiplied to the global bandwidth for each kernel.
+
+Furthermore different scales in data is accounted for by scaling it via its
+covariance matrix to an equal spread.
+First a global kernel bandwidth is applied to the transformed data and then
+based on that density a local bandwidth parameter is applied.
+
+All credit for the method goes to [1] and to S. Schoenen and L. Raedel for
+huge parts of the implementation.
+For information on Silverman or Scott rule, see [2] or [3].
+
+
+## References
+- [1] B. Wang and X. Wang, "Bandwidth Selection for Weighted Kernel Density Estimation", Sep. 2007, DOI: 10.1214/154957804100000000.
+- [2] D.W. Scott, "Multivariate Density Estimation: Theory, Practice, and Visualization", John Wiley & Sons, New York, Chicester, 1992.
+- [3] B.W. Silverman, "Density Estimation for Statistics and Data Analysis", Vol. 26, Monographs on Statistics and Applied Probability, Chapman and Hall, London, 1986.

src/awkde/awkde.egg-info/PKG-INFO

@@ -0,0 +1,8 @@
+Metadata-Version: 2.1
+Name: awkde
+Version: 0.1
+Summary: Adaptive width gaussian KDE
+Home-page: https://github.com/mennthor/awkde
+Author: Thorben Menne
+Author-email: [email protected]
+License-File: LICENSE

src/awkde/awkde.egg-info/SOURCES.txt

@@ -0,0 +1,13 @@
+LICENSE
+README.md
+pyproject.toml
+setup.py
+awkde/__init__.py
+awkde/awkde.py
+awkde/tools.py
+awkde.egg-info/PKG-INFO
+awkde.egg-info/SOURCES.txt
+awkde.egg-info/dependency_links.txt
+awkde.egg-info/requires.txt
+awkde.egg-info/top_level.txt
+cpp/backend.cpp

src/awkde/awkde.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

src/awkde/awkde.egg-info/requires.txt

@@ -0,0 +1,5 @@
+numpy
+scipy
+scikit-learn
+pybind11
+future

src/awkde/awkde.egg-info/top_level.txt

@@ -0,0 +1 @@
+awkde

src/awkde/awkde/__init__.py

@@ -0,0 +1,10 @@
+# coding: utf-8
+
+from __future__ import print_function
+from __future__ import division
+from __future__ import absolute_import
+from future import standard_library
+
+standard_library.install_aliases()
+
+from .awkde import GaussianKDE