Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/h3ronpy #75

Merged
merged 22 commits into from
Nov 21, 2024
Merged

Feature/h3ronpy #75

merged 22 commits into from
Nov 21, 2024

Conversation

zacdezgeo
Copy link
Collaborator

What I Changed

  1. Refactored generate_h3_geometries function:

    • Added support for handling H3 hexagon IDs in uint64 format and converting them to geometries (either polygons or points).
    • Fixed the logic to correctly handle H3 ID to geometry conversion, using cells_to_wkb_polygons for polygons and cells_to_wkb_points for centroids.
    • Improved error handling by raising proper exceptions if invalid geometry types or pyarrow Scalar types are encountered.
  2. Adjusted aggregate and summaries methods in StatsTable:

    • Integrated cells_to_string to convert H3 IDs before database operations.
    • Added a ValueError check if the query returns no rows (to catch invalid fields).
    • Refined SQL query generation to ensure it properly interacts with H3 IDs stored as hexagons.
  3. Unit Tests:

    • Updated the test cases in test_h3_utils.py and test_module.py to reflect the changes in how geometries are generated.
    • Added assertions for error handling (e.g., invalid field names in aggregate should now raise a ValueError).
    • Updated tests to include both polygon and point geometry handling.
  4. Error Fixes:

    • Resolved the TypeError: Expected pyarrow Buffer, but got LargeBinaryScalar by ensuring proper conversion from pyarrow Scalar types.
    • Fixed SQL query results processing to handle and print row outputs when appropriate.

How to Test It

  1. Unit Tests:

    • Run pytest on the following test files:
      • tests/test_module.py
      • tests/test_h3_utils.py
      • tests/test_api.py
    • These tests will cover:
      • H3 ID generation
      • Geometry conversion for both polygons and points
      • Database interactions via the StatsTable methods
      • Error handling for invalid field names and geometries
  2. Notebooks:

    • Test the updated geometry generation functionality by running the relevant notebooks, such as notebooks/user-docs/space2stats_api_bug.ipynb, which shows how point geometries aren't being generated properly.

Other Notes

  • notebooks/user-docs/space2stats_api_bug.ipynb shows an example where point geometries are not being generated correctly. Further investigation is needed into the issue with point geometries and H3 ID conversion.

@zacdezgeo zacdezgeo self-assigned this Oct 3, 2024
@zacdezgeo
Copy link
Collaborator Author

@kylebarron; some weird behavior with points I might need help with:
image

h3ronpy currently has a bug with cells_to_wkb_points which returns matching coordinates.
@zacdezgeo zacdezgeo marked this pull request as ready for review October 3, 2024 23:45
zacdezgeo and others added 5 commits October 4, 2024 14:11
* create stac catalog from sample kenya data

* testing metadata

* update metadata files

* typo

* Update catalog with link to self and title based on stac-check best practice

* Move METADATA to space2stats_ingest

* fix sources link

* rewrite catalog with self link and item titles

* Rename duplicated variable gdf to adm_gdf

* Corrected the color breaks

* Adapt based on h3ronpy and adapt colormap

* Change formatting, ensure clear runs, and remove unused imports

* Add nbqa pre-commit for notebooks

---------

Co-authored-by: Andres Chamorro <[email protected]>
@Gabe-Levin
Copy link
Collaborator

Gabe-Levin commented Oct 14, 2024

@zacharyDez I ran the test files specified above and they are all passing. However, I couldn't find this notebook: notebooks/user-docs/space2stats_api_bug.ipynb

Has it been removed on purpose, is it still relevant?

@zacdezgeo
Copy link
Collaborator Author

Yes! It's been fixed in the new release of h3ronpy, so I removed the notebook.

@zacdezgeo zacdezgeo mentioned this pull request Nov 13, 2024
Copy link

github-actions bot commented Nov 21, 2024

PR Deployment Details:
🚀 PR deployed to https://yugjnm7wph.execute-api.us-east-1.amazonaws.com/

@zacdezgeo zacdezgeo temporarily deployed to Space2Stats API Dev November 21, 2024 16:03 — with GitHub Actions Inactive
@zacdezgeo zacdezgeo merged commit f559e92 into main Nov 21, 2024
3 checks passed
@zacdezgeo zacdezgeo temporarily deployed to Space2Stats API Dev November 21, 2024 16:10 — with GitHub Actions Inactive
@zacdezgeo zacdezgeo temporarily deployed to Space2Stats API Dev November 21, 2024 16:13 — with GitHub Actions Inactive
@zacdezgeo zacdezgeo deleted the feature/h3ronpy branch November 21, 2024 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants