-
Notifications
You must be signed in to change notification settings - Fork 1
CAMS Schema
Maxwell Nyamunda edited this page May 23, 2024
·
3 revisions
- _id : unique id given to a document. Useful for tracking changes made to the document.
- name: This is one of the 153 city names given by ECMWF
- forecast_valid_time : This is the time of which the forecast corresponds to e.g. the levels are predicted to so like so at on 2024-05-25T00:00:00.000+00:00 (Start of the day) These increment in 3 hours. So should ideally only display 00:00, 03:00, 06:00, 09:00, 12:00, 15:00, 18:00, 21:00, 23:00.
- last_modified_time : This refers to the last time something was changed in the document. Let's say today is Monday and we have predications for the entire week. The forecast we get today may be different CAMS new forecast release. We though AQI would be 5 on a Friday for London, but this new forecast is showing 3. Therefore our document for London would update and our last_modified_time would change to whenever that new document was uploaded.
- created_time : This one is fairly straight forward. I run the program at XX:XX time and the new documents upserted will also convey the same time. This field should stay the same as nothing is getting replaced but rather updated.
- location_type : CAMS has various location types. In our program we are using cities as opposed to countries.
- This is how we are currently assigning what a "city" is. With a latitude and longitude.
- forecast_base_time : This is the time at which CAMS released the forecast. It can either be at midnight or midday. We are only using midnight.
- forecast_range : This is like a time marker in the forecast. So essentially in the screenshot above that forecast is predicted for 96 hours ahead of the forecast_base_time Which adds up as we have a 4 day gap between forecast_base_time and forecast_valid_time. (Basically the hourly difference between the two)
- overall_aqi_level : This is what will be displayed for a location overall. It will always show the highest value of our 5 individual pollutants. Since all of our individual pollutants are 1, the overall_aqi_level is also 1. If they were 1, 3, 4, 1, 1, then our overall_aqi_level would resolve to 4.
- source : This is used to identify which source the data is coming from. In this case it is CAMS.
- aqi_level: This is the level of each individual pollutant. Which is measured and determined by the chart below.
- value: These are the measured values of each pollutant. These are standardised and then used to determine the aqi_level of each pollutant.
Getting Started and Overview
- Product Description
- Roles and Responsibilities
- User Roles and Goals
- Architectural Design
- Iterations
- Decision Records
- Summary Page Explanation
- Deployment Guide
- Working Practices
- Q&A
Investigations and Notebooks
- CAMs Schema
- Exploratory Notebooks
- Forecast ETL Process
- In Situ air pollution data sources
- Notebook: OpenAQ data overview
- Notebook: Unit conversion
- Data Archive Considerations
Manual Test Charters
- Charter 1 (Comparing ECMWF forecast to database values)
- Charter 2 (Backend performance)
- Charter 3 (Forecast range implementation)
- Charter 4 (In situ bad data)
- Charter 5 (Filtering ppm units)
- Charter 7 (Forecast API input validation)
- Charter 8 (Forecast API database sizes)
- Charter 9 (Measurements summary API input validation)
- Charter 10 (Seeding bad data)
- Charter 11 ()Measurements API input validation
- Charter 12 (Validating echart plot accuracy)
- Charter 13 (Explore UI after data outage)
- Charter 14 (City page address)
- Charter 15 (BugFix diff 0 calculation)
- Charter 16 (City page chart data mocking)
- Charter 17 (Summary table logic)
- Charter 18 (AQI chart colour banding)
- Charter 19 (City page screen sizes)
- Charter 20 (Date picker)
- Charter 21 (Graph consistency)
- Charter 22 (High measurement values)
- Charter 23 (ppm -> µg m³)
- Charter 24 (Textures API input validation)
- Charter 25 (Graph line colours)
- Charter 26 (Fill in gaps in forecast)
- Charter 27 (Graph behaviour with mock data)
- Charter 28 (Summary table accuracy)
- Re‐execute: Charter 28
- Charter 29 (Fill in gaps in situ)
- Charter 30 (Forecast window)
- Charter 31 (UI screen sizes)