This piece of documentation explains how recent changes to the list of JSON data files, and the NLP metrics that may result from the definition of new objects, get propagated into the database.
Given that the data-ingestion and NLP metrics computing pipelines may run on a dedicated cluster isolated from the rest of the project infrastructure (based on GitHub / GCP), the propagation of changes will happen asynchronously.
This asynchronous propagation will occur when the cron specification for the different pipelines is set, and considering the average amount of time they need to finish.
The current pipelines set up on a cron-job are:
- Data-static pipeline (private). Cron specification: TBA.
- NLP-metrics pipeline (private). Cron specification: TBA.