Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scrape geoportal for shn, add SHN flag on stops table #3529

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

Scrape geoportal for shn, add SHN flag on stops table #3529

wants to merge 20 commits into from

Commits on Nov 22, 2024

  1. scrape geoportal for shn, add flag on stops table

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    3b8ad97 View commit details
    Browse the repository at this point in the history

  2. successful initial scrape of geoportal endpoint

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    9c47850 View commit details
    Browse the repository at this point in the history

  3. functioning json format and external table creation, need to modify c… 

    …oordinates
    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f305060 View commit details
    Browse the repository at this point in the history

  4. shn external tables

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    61acfe9 View commit details
    Browse the repository at this point in the history

  5. add initial source, staging, and docs for state geoportal scrape

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    4bb445a View commit details
    Browse the repository at this point in the history

  6. add flag for shn to dim_stops_latest

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f6f2bd4 View commit details
    Browse the repository at this point in the history

  7. readd templating for buckets and remove hardcode schema

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    84574ce View commit details
    Browse the repository at this point in the history

  8. refactor operator to be more agnostic for data inputs

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f1b6e0b View commit details
    Browse the repository at this point in the history

  9. add handling for state_highway_network table

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f079625 View commit details
    Browse the repository at this point in the history

  10. fix comment

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    284ed97 View commit details
    Browse the repository at this point in the history

  11. remove local variables and comments to allow for a production merge w… 

    …hen ready
    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    95fb806 View commit details
    Browse the repository at this point in the history

  12. use stop_id for calculations

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f85538b View commit details
    Browse the repository at this point in the history

  13. Apply suggestions from code review 

    Co-authored-by: Mjumbe Poe <[email protected]>
    @charlie-costanzo @mjumbewu
    charlie-costanzo and mjumbewu committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    0202bf2 View commit details
    Browse the repository at this point in the history

  14. restore source table

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a0a8ec3 View commit details
    Browse the repository at this point in the history

  15. remove hardcoded source, remove old comments, add comment decribing b… 

    …uffer distance
    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    7507c99 View commit details
    Browse the repository at this point in the history

  16. remove hardcoded bucket

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    57eb225 View commit details
    Browse the repository at this point in the history

  17. refactor how we keep final columns, dynamically rename columns

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    3162f1b View commit details
    Browse the repository at this point in the history

  18. restore _gtfs_key over stop_id in shn geo calculation

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    8c985ed View commit details
    Browse the repository at this point in the history

  19. checking before refactor

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    a81b205 View commit details
    Browse the repository at this point in the history

  20. revisions based on Mjumbe's review

    @charlie-costanzo
    charlie-costanzo committed Nov 22, 2024
    Configuration menu
    Copy the full SHA
    f0247fe View commit details
    Browse the repository at this point in the history