Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intake v2 #22

Closed
wants to merge 15 commits into from
Closed

Intake v2 #22

wants to merge 15 commits into from

Conversation

ahuang11
Copy link
Collaborator

No description provided.

@ahuang11 ahuang11 requested a review from droumis February 21, 2024 00:10
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link

github-actions bot commented Feb 21, 2024

👋 Thanks for opening this PR! The Cookbook will be automatically built with GitHub Actions. To see the status of your deployment, click below.
🔍 Git commit SHA: 55a06c3
✅ Deployment Preview URL: In Progress

@droumis
Copy link
Member

droumis commented Feb 28, 2024

Any further additions from Martin's revision?

@ahuang11
Copy link
Collaborator Author

Yes, I'm not sure if Martin's require a new version of intake or not

@ahuang11
Copy link
Collaborator Author

Added stuff from ProjectPythia/cookbook-template#160

@droumis
Copy link
Member

droumis commented Mar 15, 2024

@ahuang11 I'm seeing Panel/Bokeh error in every notebook in the preview https://projectpythia.org/landsat-ml-cookbook/_preview/22/notebooks/1.1_Data_Ingestion-Intake.html

But I'm not seeing this error when I run the notebook locally with an env created with the environment.yml file, which results in:
bokeh 3.3.4
panel 1.3.8

@droumis
Copy link
Member

droumis commented Mar 15, 2024

as soon as these errors are resolved, let's merge

@ahuang11
Copy link
Collaborator Author

What version of intake do you have?

@droumis
Copy link
Member

droumis commented Mar 15, 2024

intake 2.0.3

@ahuang11
Copy link
Collaborator Author

The errors will be resolved when intake/intake#791 is released.

environment.yml Outdated Show resolved Hide resolved
environment.yml Outdated Show resolved Hide resolved
@ahuang11
Copy link
Collaborator Author

ahuang11 commented Mar 20, 2024

Okay new issues that I'll have to look into later..
image

Probably have to completely refactor Preprocessing - Regrid as the dataset likely doesn't exist anymore, or requires a AWS key.
image

@droumis
Copy link
Member

droumis commented Mar 20, 2024

@sandhujasmine do you remember where this dataset might have been moved to? I think I remember you working to host it elsewhere at some point (maybe?).

@droumis
Copy link
Member

droumis commented Mar 20, 2024

@ahuang11, I'm not sure much if any of the regrid notebook makes sense anymore if someone is going to use the planetary computer data source. As described at the top of it, the regrid notebook was intended as a continuation of the original intake notebook which required additional processing before analysis because it loaded from a different data source. Now that the new intake notebook is just grabbing data from planetary computer, the intake notebook is kind of just a side note about another way to load from the same data source. Although it was a pretty big effort to create, if you don't think there is anything useful left in the regrid notebook then I'd maybe vote to just remove it.

@sandhujasmine
Copy link
Collaborator

@sandhujasmine do you remember where this dataset might have been moved to? I think I remember you working to host it elsewhere at some point (maybe?).

Hi @droumis - I did move the landsat data to the holoviz s3 bucket; see this PR: https://github.com/holoviz-topics/examples/pull/274/files - I think this is the correct dataset but let me know if I misunderstood.

environment.yml Outdated Show resolved Hide resolved
@ahuang11
Copy link
Collaborator Author

ahuang11 commented Mar 27, 2024

Wooh the intake notebook successfully ran in the action; will investigate the other notebook tomorrow.
image

@ahuang11
Copy link
Collaborator Author

So one other thing I wanted to add + demo was to_yaml_file and open_catalog but currently experiencing issues exporting. At the moment to move this forward, I think we could add it in a separate PR and delete 2.0 regrid notebook?

environment.yml Outdated Show resolved Hide resolved
@droumis
Copy link
Member

droumis commented Apr 2, 2024

@ahuang11 ping me when you want me to take a look at any of your changes! I'm guessing that the build error is preventing the 'Deployment Preview URL' .

@ahuang11 ahuang11 mentioned this pull request Apr 4, 2024
@ahuang11 ahuang11 closed this Apr 4, 2024
github-actions bot pushed a commit that referenced this pull request Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants