Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CMR and access metrics #58

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

abarciauskas-bgse
Copy link
Contributor

@abarciauskas-bgse abarciauskas-bgse commented Apr 10, 2023

This PR adds a notebook for generating s3 access metrics + plots using Athena and other notebooks for generating metrics for CMR queries.

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@abarciauskas-bgse abarciauskas-bgse marked this pull request as ready for review April 13, 2023 23:09
@abarciauskas-bgse abarciauskas-bgse changed the title Ab/add athena s3 access metrics Ab/add CMR and access metrics Apr 13, 2023
@abarciauskas-bgse abarciauskas-bgse changed the title Ab/add CMR and access metrics Add CMR and access metrics Apr 13, 2023
@abarciauskas-bgse
Copy link
Contributor Author

I think after we merge this, we should immediately remove the CMR queries since MAAP CMR is going away. But I'm going to share those with the CMR team.

@@ -0,0 +1,1062 @@
{
Copy link
Collaborator

@wildintellect wildintellect Apr 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

client references a global variable without passing to the function or being part of a class


Reply via ReviewNB

@@ -0,0 +1,1062 @@
{
Copy link
Collaborator

@wildintellect wildintellect Apr 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line #42.            if len(results_list) == no_results:

Can you comment what this no_results does, not particularly clear from the code.


Reply via ReviewNB

@@ -0,0 +1,1062 @@
{
Copy link
Collaborator

@wildintellect wildintellect Apr 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line #52.            # Boreal TIF files were not stored in sub-directory

Should we fix that? Ticket elsewhere...


Reply via ReviewNB

@@ -0,0 +1,1062 @@
{
Copy link
Collaborator

@wildintellect wildintellect Apr 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think 2.0 version of this should cache the results, so the query can optionally be rerun, but doesn't necessarily need to be if the data's already been pulled. (Not a blocker for the PR)


Reply via ReviewNB

@@ -0,0 +1,304 @@
{
Copy link
Collaborator

@wildintellect wildintellect Apr 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make a markdown section that explains what the purpose of this notebook is?

"Analyze CMR logs for Collection search using Collection Concept ID, ie finding a collection"


Reply via ReviewNB

@@ -0,0 +1,300 @@
{
Copy link
Collaborator

@wildintellect wildintellect Apr 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make a markdown section that explains what the purpose of this notebook is?

"Analyze CMR logs of collection granules(items)"


Reply via ReviewNB

@@ -0,0 +1,359 @@
{
Copy link
Collaborator

@wildintellect wildintellect Apr 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make a markdown section that explains what the purpose of this notebook is?

"Analyze CMR logs for use of attribute search parameters by users"


Reply via ReviewNB

@@ -0,0 +1,405 @@
{
Copy link
Collaborator

@wildintellect wildintellect Apr 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make a markdown section that explains what the purpose of this notebook is?

"Analyze CMR logs for Collection search using Collection Short Name, ie finding a collection"


Reply via ReviewNB

@abarciauskas-bgse
Copy link
Contributor Author

Thanks for reviewing @wildintellect - just noting that I plan to address these comments late this week or next week

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants