Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import Utility for Evaluations #7

Merged
merged 8 commits into from
Oct 3, 2024
Merged

Import Utility for Evaluations #7

merged 8 commits into from
Oct 3, 2024

Conversation

bbengfort
Copy link
Contributor

@bbengfort bbengfort commented Oct 2, 2024

Scope of changes

Updates the models with fields recommended by @pdeziel and creates an import utility to load the data.

Type of change

  • bug fix
  • new feature
  • documentation
  • other (describe)

Acceptance criteria

Can we start uploading data into the app after this PR is merged and the app is deployed?

Author checklist

  • I have manually tested the change and/or added automation in the form of unit tests or integration tests
  • I have updated the dependencies list
  • I have added new test fixtures as needed to support added tests
  • I have added or updated the documentation
  • Check this box if a reviewer can merge this pull request after approval (leave it unchecked if you want to do it yourself)

Reviewer(s) checklist

  • Any new user-facing content that has been added for this PR has been QA'ed to ensure correct grammar, spelling, and understandability.
  • To the best of my ability, I believe that this PR represents a good solution to the specified problem and that it should be merged into the main code base.
  • These fields are sufficient for us to start uploading model data today.

@bbengfort bbengfort requested a review from pdeziel October 2, 2024 18:56
Copy link
Contributor

@pdeziel pdeziel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I added a few more suggestions to make the LLM definition more complete. So is the remaining work to add a JSON utility to upload the models, prompts, and responses in a combined JSON file?

parley/models.py Outdated Show resolved Hide resolved
parley/models.py Show resolved Hide resolved
Co-authored-by: Patrick Deziel <[email protected]>
Signed-off-by: Benjamin Bengfort <[email protected]>
@bbengfort
Copy link
Contributor Author

Thanks, I added a few more suggestions to make the LLM definition more complete. So is the remaining work to add a JSON utility to upload the models, prompts, and responses in a combined JSON file?

Yes, I'm working on that uploader right now.

@bbengfort
Copy link
Contributor Author

@pdeziel I added those fields to the models and also added validation fields for the prompt and responses.

@bbengfort bbengfort merged commit 318f95f into main Oct 3, 2024
6 checks passed
@bbengfort bbengfort deleted the sc-27684 branch October 3, 2024 20:54
Copy link

sentry-io bot commented Oct 5, 2024

Suspect Issues

This pull request was deployed and Sentry observed the following issues:

  • ‼️ TypeError: QuerySet.exists() got an unexpected keyword argument 'model' /upload/ View Issue
  • ‼️ ModelEvaluation.DoesNotExist: ModelEvaluation matching query does not exist. /upload/ View Issue
  • ‼️ FieldError: Cannot resolve keyword 'label' into field. Choices are: created, evaluation, evaluation_id, exclu... /upload/ View Issue
  • ‼️ IntegrityError: duplicate key value violates unique constraint "sensitive_term_key" /upload/ View Issue

Did you find this useful? React with a 👍 or 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants