Extract / Transform / Load Scripts for databases used in Dataviva Project.
The sources of Dataviva can be found at Dataviva GitHub.
Databases are released by Brazil government as raw data and there is some analysis to extract the information needed for the project.
For each database we have a Wiki documentation that explain all the ETL process.
- You'll need libmagic for this project:
Mac installation
brew install libmagicUbuntu (BSD Linux) installation
apt-get install libmagic- Install requirements
pip install -r requirements.txtSee the Wiki for more information.