Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support nds power run on notebook #145

Draft
wants to merge 2 commits into
base: dev
Choose a base branch
from
Draft

Support nds power run on notebook #145

wants to merge 2 commits into from

Conversation

res-life
Copy link

@res-life res-life commented Mar 28, 2023 •
edited
Loading

Support nds power run on a notebook

  • Put the parameters file of power run into configs_in_zip folder
  • Put the stream file into configs_in_zip folder
  • Put the Spark property file into configs_in_zip folder
  • Package the Python files and above 2 files into a zip file
  • Run in the notebook:
spark.sparkContext.addPyFile("s3://path/to/this/zip")
nds_power.run_query_stream_in_zip()

Support power run on a notebook
5e422ea 
Signed-off-by: Chong Gao <[email protected]>

Copyright

Signed-off-by: Chong Gao <[email protected]>

Refactor; Support spark.properties; Add more info into readme.md

Update doc
@res-life res-life changed the title Support nds power run on EKS Support nds power run on notebook Apr 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@res-life