Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement API design with Postgres Backend #8

Merged
merged 16 commits into from
Jul 19, 2024
Merged

Conversation

zacdezgeo
Copy link
Collaborator

I implemented the first version of the API design with some small data processing scripts for ETL purposes in this PR:

  • Add summary response with mocked data: Initial setup to return mocked data in the summary response.
  • Add data ingestion: Implemented data ingestion to support data operations.
  • Adapt API to read data: Modified the API to utilize the new data ingestion logic for enhanced data retrieval performance.
  • Fix and clean up H3 generation functionality: Improve the H3 index generation logic for better accuracy and performance.
  • Add tests on API: Introduced initial tests for API endpoints to ensure reliability and correctness.
  • Adapt API for storing data within PostgreSQL: Transitioned to PostgreSQL for data storage, added functionality to load sample data for NYC, and created a visualization notebook with Lonboard for quick quality assurance.
  • Add configuration variable for table name: Made the table name configurable through environment variables for flexibility.
  • Add unit tests for db_utils and update existing API tests: Refactored API logic to use utility functions from db_utils.py, updated existing tests, and added new unit tests for database utility functions using pytest and unittest.mock.
  • Fix definition of environment variables error: Resolved an issue with the order of environment variable definitions, ensuring they are loaded correctly. Added a notebook visualization that includes the available fields endpoint.

zacdezgeo added 10 commits July 5, 2024 12:09
Create fastapi application skeleton with some initial description of the summary endpoint. Includes field valdiation for aoi polygon.
Load sample data in NYC for development. Modify api to use postgres. Visualization notebook with lonboard for quick QA.
Modified app/routers/api.py to utilize get_available_fields and get_summaries from db_utils.py, updated tests/test_api.py to align with the refactored API logic, added app/utils/db_utils.py with utility functions for database operations, including get_available_fields and get_summaries, and added tests/test_db_utils.py with unit tests for the new database utility functions using pytest and unittest.mock to ensure functionality without a real database connection.
Add notebook visualization that includes the available fields endpoint. Fix discovered bug in order of environment variables defined before being loaded from env file.
@zacdezgeo zacdezgeo self-assigned this Jul 17, 2024
@zacdezgeo zacdezgeo merged commit 7b6f354 into main Jul 19, 2024
2 checks passed
@zacdezgeo zacdezgeo deleted the feature/api-design branch July 19, 2024 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants