Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hotel motel discovery #25

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
b2ed783
Merge pull request #16 from uchicago-dsi/dev
LaunaG Mar 29, 2024
d8688fa
Initial EDA for foot traffic in Hilo
Apr 2, 2024
0d77d52
opening webpages with seleniunwire then parsing with bs4
jchristensonCHI Apr 2, 2024
a269349
functions to crawl through hotel lst for cities
jchristensonCHI Apr 3, 2024
423cf94
EDA of high foot traffic areas - related same day
lydia-l3 Apr 3, 2024
6376e06
created crawling functions with Smartproxy to circumvent blocking and…
jchristensonCHI Apr 3, 2024
0426ec1
Merge pull request #19 from uchicago-dsi/dev
LaunaG Apr 3, 2024
e6ffb1a
minor edit to EDA notebook
lydia-l3 Apr 3, 2024
9083a84
Merge branch 'dev' into hotel_motel_discovery
jchristensonCHI Apr 3, 2024
f1c8099
Merge branch 'dev' into week-3-EDA-anuj
Apr 3, 2024
25b478d
deleted copied block
jchristensonCHI Apr 3, 2024
0b3d239
added .env file and setup
jchristensonCHI Apr 7, 2024
dfbd33a
added functions to get the Tripadvisor Location Search -> Find Search
jchristensonCHI Apr 7, 2024
a3af984
added functions to create the Tripadvisor location hotel list urls
jchristensonCHI Apr 8, 2024
886d9d7
EDA experimentation update
Apr 8, 2024
7777b90
Streamlining initialization through city name and working on Smartpro…
jchristensonCHI Apr 10, 2024
9cc17bd
restructured dataset so each row represents one
lydia-l3 Apr 10, 2024
0b00bbe
Viability analysis, visualization of Fema, and pull script
Kevin2330 Apr 10, 2024
ec29763
Scaffolds proof-of-concept
LaunaG Apr 12, 2024
5d813ff
Functions
Apr 14, 2024
bd8af2d
Week 5 update
Kevin2330 Apr 17, 2024
66988e8
Creation of boundingboxes, initialization via the geojson files, new …
jchristensonCHI Apr 17, 2024
d11aff6
removed accidentially created notebook
jchristensonCHI Apr 17, 2024
bdd959e
EDA functions added in EDA.py and DBSCAN.py
lydia-l3 Apr 17, 2024
8191d9e
functions added to EDA.py and DBSCAN.py
lydia-l3 Apr 17, 2024
f0dff10
moved hilo_full_patterns.parquet to data folder
lydia-l3 Apr 17, 2024
35cce68
Updates layout and begins scaffolding project form page
LaunaG Apr 18, 2024
ee79dd4
Removes erroneous notebook and scaffolds page form structure
LaunaG Apr 18, 2024
014cb01
DBSCAN split into three functions using seasonal subsets of data
lydia-l3 Apr 18, 2024
880cd35
moved EDA notebook
lydia-l3 Apr 18, 2024
646e677
DBSCAN/HDBSCAN tinkering before TA session
lydia-l3 Apr 19, 2024
88cb96a
Week 6 updates
Apr 22, 2024
0d2b1f2
script added
Kevin2330 Apr 23, 2024
9f3c59b
changed
Kevin2330 Apr 23, 2024
b5c0ae4
Currently indefinitely recursing through the bounding box
jchristensonCHI Apr 23, 2024
8f94bc8
HDBSCAN stopping point - algorithm works
lydia-l3 Apr 24, 2024
5242494
Creates initial workflow for creating and editing design projects. In…
LaunaG Apr 24, 2024
259eddb
exploration
Kevin2330 Apr 24, 2024
7ca8fd9
hilo related_brands_footpaths.py functions
lydia-l3 Apr 24, 2024
d3158de
DBSCAN and related brands updates
lydia-l3 Apr 26, 2024
99ae00b
temp commit
LaunaG Apr 26, 2024
7820c2d
April 30 update
Apr 30, 2024
dd53a15
The code now implements geotagging and duplicate removal, however thi…
jchristensonCHI Apr 30, 2024
11d5283
jupyter notebook showing results
jchristensonCHI Apr 30, 2024
ce482b7
changed max limit, corrected bounding box dimensions, added miles for…
jchristensonCHI Apr 30, 2024
2a47546
edited related brands functions to vary route
lydia-l3 May 1, 2024
0b3be39
nearest-neighbor
Kevin2330 May 1, 2024
d843828
Results checks via .ipynb, global variable use for better data collec…
jchristensonCHI May 1, 2024
77a3f39
adjustments
May 1, 2024
a713328
renamed DBSCAN notebook and script to HDBSCAN and updated HDBSCAN not…
lydia-l3 May 1, 2024
f011538
cleaned up all functions in python scripts, HDBSCAN switched to foliu…
lydia-l3 May 1, 2024
c05d50a
replaced old HDBSCAN.ipynb notebook with new one - contains all 12 ge…
lydia-l3 May 1, 2024
1d68e49
remove data files from repo (approved by Anuj)
sarahwalker10 May 2, 2024
70464ff
move .py script to utils folder (approved by Anuj)
sarahwalker10 May 2, 2024
c3a1210
added returns and output to docstrings of functions and restored the …
lydia-l3 May 2, 2024
1b039e5
added returns and output to function docstrings
lydia-l3 May 2, 2024
53fc3ea
update requirements.txt (includes both Lydia and Anuj's packages)
sarahwalker10 May 3, 2024
81fe07a
Merge remote-tracking branch 'refs/remotes/origin/high-foot-traffic-e…
sarahwalker10 May 3, 2024
34474d7
update requirements.txt (includes both Lydia and Anuj's packages)
sarahwalker10 May 3, 2024
e431409
Finalized dual pathway for data collection and created data visualiza…
jchristensonCHI May 7, 2024
00c7bb2
minor fixes
jchristensonCHI May 7, 2024
bcea444
added .pys to repository
jchristensonCHI May 7, 2024
933a354
formatting
jchristensonCHI May 7, 2024
186e9cb
formatting
jchristensonCHI May 7, 2024
2dd0bcd
script-update
Kevin2330 May 7, 2024
908d6ea
update readme
Kevin2330 May 7, 2024
35efd5c
Galveston added
Kevin2330 May 7, 2024
0487ea3
Adds category-related tables, enables retrieval of indoor bin locatio…
LaunaG May 8, 2024
02c79bb
Light fixes ahead of standardization
LaunaG May 8, 2024
1d261f8
Merge pull request #18 from dsi-clinic/week-3-EDA-anuj
LaunaG May 8, 2024
707960b
Merge branch 'dev' of github.com:dsi-clinic/2024-winter-perpetual int…
LaunaG May 8, 2024
cd004fd
Merges with dev ahead of standardization; temporarily removes conflic…
LaunaG May 8, 2024
8585852
Restores EDA notebook
LaunaG May 8, 2024
7d3b937
Merge pull request #21 from dsi-clinic/high-foot-traffic-eda-ll
LaunaG May 8, 2024
dcd2e3a
Merge branch 'dev' into residential_buildings_exploration
LaunaG May 8, 2024
37fe69b
Merge pull request #23 from dsi-clinic/residential_buildings_exploration
LaunaG May 8, 2024
077fd72
Merges with dev ahead of standardization
LaunaG May 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -174,4 +174,4 @@ cache/
pgdata/

# data
data
data
5 changes: 5 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,8 @@ run-notebooks:
$(project_name)-notebooks jupyter lab \
--port=8888 --ip='*' --NotebookApp.token='' \
--NotebookApp.password='' --no-browser --allow-root

run-full-stack:
cd $(current_abs_path) && \
. ./set_architecture.sh && \
docker compose --profile full-stack up --build
46 changes: 46 additions & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
version: "3.4"


services:
# POSTGIS
postgis:
Expand Down Expand Up @@ -31,6 +33,32 @@ services:
- pgadmin
env_file:
- .env
ports:
- "8080:8080"

# DJANGO-Q2
django-q2:
restart: "no"
environment:
- DJANGO_SECRET_KEY=local
- ENV=DEV
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_HOST=postgis
- POSTGRES_CHECK_TIMEOUT=120
build:
context: pipeline
command: ["bash", "setup.sh", "--use_djangoq"]
volumes:
- ./pipeline:/perpetual/pipeline
- ./data:/perpetual/data
depends_on:
- pipeline
env_file:
- .env
profiles:
- full-stack

# PGADMIN DATABASE GUI
pgadmin:
Expand All @@ -49,3 +77,21 @@ services:
- "443:443"
volumes:
- ./server.local.json:/pgadmin4/servers.json
logging:
driver: none

# WEB
web:
build:
context: web
depends_on:
- postgis
restart: always
ports:
- 3000:3000
volumes:
- ./web:/app
- /app/node_modules
- /app/.next
profiles:
- full-stack
551 changes: 551 additions & 0 deletions notebooks/DBSCAN.ipynb

Large diffs are not rendered by default.

Loading
Loading