- Scrap tabular data from given URL(https://mcxlive.org/)
- Store data in database of your choice(postgres/mongo dB)
- Generate API from stored data ( json Response)
- Run scheduler(cron job) for above scrapper(every 1 hour)
- Design CI/CD pipeline
The execution of cronjob is done by the following commands:
crontab -e
0 * * * * cron projects/DevOps/cron.sh
enter crontab -l
to see the added cron job.\
Json output after scraping:
Output on django server after retrieving stored data in postgresql.