Migration to Datahub for Finesse Benchmark Tool

[ibrahim-kabir]

Summary

Currently, the tool saves test data locally, which poses risks of data loss and limits the comprehensiveness of captured data. By using locust-dashboard, which inherently offers postgresql storage functionalities, we aim to ensure data preservation and efficient management. It offers various tables containing essential data such as the number of users and machines utilized for stressing the API. Additionally, it allows for the incorporation of custom metrics.

Tasks

• Create a schema on DataHub for Finesse Benchmark Tool data storage.
• Configure the schema to function with a local PostgreSQL database.
• Integrate custom metrics into the data storage process.
  • Identify the run_id attribute
  • Find the getter of the run_id
  • Modify the request table to add an accuracy score and a response uuid
  • Create a table for the Finesse response
  • Connect data storage system to datahub
• Connect the data storage system to DataHub.

Acceptance Criteria

• Schema designed to accommodate various data types and metrics.
• Data successfully stored and retrievable from the local PostgreSQL database.
• Custom metrics, including user and machine counts, stress levels on APIs, successfully incorporated into the data storage.
• Data seamlessly transmitted from the local PostgreSQL database to DataHub for comprehensive management and analysis.