Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Archiving all runs history of the Finesse Benchmark Tool to Datahub #4

Open
3 tasks
Tracked by #1
ibrahim-kabir opened this issue Feb 28, 2024 · 1 comment · May be fixed by #5
Open
3 tasks
Tracked by #1

Archiving all runs history of the Finesse Benchmark Tool to Datahub #4

ibrahim-kabir opened this issue Feb 28, 2024 · 1 comment · May be fixed by #5
Assignees

Comments

@ibrahim-kabir
Copy link
Collaborator

ibrahim-kabir commented Feb 28, 2024

Summary

The Finesse benchmark tool currently saves test data locally, which hinders archiving all runs history for comparison with historical performance data. Moreover, not all data is captured in the CSV or Markdown files, as there is an overflow of responses from Finesse. Hence, it would be beneficial to migrate this data storage to a database solution.

Tasks

  • Create a schema on Datahub
  • Modify the script to connect to Datahub and save data on table
  • Test the migration process thoroughly with test.

Acceptance Criteria

  • Ensure all test data from Finesse benchmarking is successfully migrated to a PostgreSQL database.
  • Develop a script or tool to automate the process of saving data to the PostgreSQL table.
  • Verify that all data is accurately captured and stored in the database without any loss or truncation.
  • Implement error handling mechanisms to address any potential issues during data migration.
  • Conduct thorough testing to validate the functionality and reliability of the PostgreSQL database for storing Finesse benchmark test data.
  • Document the migration process and provide clear instructions for future maintenance and updates.
@ibrahim-kabir ibrahim-kabir changed the title Save all the test data on the PostgreSQL table Save all the test data on a PostgreSQL table Feb 28, 2024
@ibrahim-kabir ibrahim-kabir changed the title Save all the test data on a PostgreSQL table Save all the test data on Datahub Feb 28, 2024
@ibrahim-kabir ibrahim-kabir self-assigned this Feb 29, 2024
ibrahim-kabir added a commit that referenced this issue Feb 29, 2024
@ibrahim-kabir ibrahim-kabir changed the title Save all the test data on Datahub Preventing data loss by transitioning Finesse Benchmark Tool local data storage to Datahub Feb 29, 2024
@ibrahim-kabir ibrahim-kabir linked a pull request Feb 29, 2024 that will close this issue
9 tasks
@rngadam
Copy link

rngadam commented Mar 1, 2024

I wouldn't express it primarily as a "data loss risk" (information has no business value), more about archiving all runs history to enable comparison with historical performance data.

ibrahim-kabir added a commit that referenced this issue Mar 1, 2024
@ibrahim-kabir ibrahim-kabir moved this to In Progress in Finesse Mar 5, 2024
@ibrahim-kabir ibrahim-kabir changed the title Preventing data loss by transitioning Finesse Benchmark Tool local data storage to Datahub Archiving all runs history of the Finesse Benchmark Tool to Datahub Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

2 participants