Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Functions python refactoring #850

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

jcpitre
Copy link
Contributor

@jcpitre jcpitre commented Dec 5, 2024

Summary:

Refactored the functions-python build so it can get packages from api/src and not only functions-python folder.
Introduced include_api_folders in function_config.json to specify the folder from api/src that should be copied to the build. e.g. this was added to batch_datasets/function_config.json:

  "include_api_folders": ["database_gen"],

Refactored python functions so they can be self-contained, i.e. a functions-python function (e.g. batch_datasets) will contain all packages it needs to run and test. batch_datasets can we loaded directly in the IDE (e.g. Pycharm) and be managed in isolation.

To accomplish this, you need to run this:

scripts/function-python-setup.sh --function_name batch_datasets

or

scripts/function-python-setup.sh --all

This will create folder shared and test_shared in the function/src folder. These folders will contain links to the necessary packages.

Running api-tests.sh --folder functions-python will also call function-python-setup.sh for you before running the tests.

This change was done in prevision of #779 that will need packages coming from api/src

Expected behavior:

Build should be the same as before.
tests should run the same as before

Testing tips:

Tested running the build-tests.yml action on GH and made sure the tests run properly.
Also ran some of the python-functions with function-python-run.sh

Please make sure these boxes are checked before submitting your pull request - thanks!

  • Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
  • Add or update any needed documentation to the repo
  • Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
  • Linked all relevant issues
  • Include screenshot(s) showing how this pull request works and fixes the issue(s)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant