Charter 28: Explore production summary table with reference to the database to discover accuracy

Charter:

• Explore production summary table with reference to the database to discover accuracy

Testers:

• Mike Walker-Rose

Date:

• 20/08/24

Duration:

• 45 - 90 mins

Scope:

• UI summary table on prod

References:

• UI
• prod database

Test Approach:

• Query the prod forecast database for a specific city and base time
• Query the prod in situ database for a specific city across the different measurement dates for a forecast window of 1
• Manually calculate the average in situ values for each in situ measurement date
• Compare to the forecasts for the window
• compare to the table in the UI

Notes

• Time Range: 18 Aug 12:00 - 19 Aug 12:00 UTC
• Relevant forecast valid times for this period
• Forecast
  • PM2.5 recorded- 28.9, this value is a value from the range, is assigned the correct timestamp and the value relates to AQI 4 which is the highest reading bracket, NOTE - it is not the highest value, could be that the biggest measurement discrep is at a different reading
  • PM10 recorded- 50.7, this value is a value from the range, is assigned the correct timestamp and the value relates to AQI 4 which is the highest reading bracket, NOTE - it is not the highest value, could be that the biggest measurement discrep is at a different reading
  • NO2 recorded - 4.3, this value is a value from the range, is assigned the correct timestamp and the value relates to AQI 1 which is the highest reading bracket, NOTE - it is not the highest value, could be that the biggest measurement discrep is at a different reading
  • O3 recorded - 54.3, this value is a value from the range, is assigned the correct timestamp and the value relates to AQI 2 which is the highest reading bracket, NOTE - it is not the highest value, could be that the biggest measurement discrep is at a different reading
  • SO2 recorded - 1.2, this value is a value from the range, is assigned the correct timestamp and the value relates to AQI 1 which is the highest reading bracket, NOTE - it is not the highest value, could be that the biggest measurement discrep is at a different reading

• retried this as the database data did not include some data from 1.5hrs prior to first measurement date which is needed, still got some variance as described below

Conclusion:

• Seemingly random which value is when more than one in the same bracket
• a variance between expected and actual values based on the database data of up to 4%
• Raised a possible bug ticket here