Skip to content

Latest commit

 

History

History
137 lines (107 loc) · 3.81 KB

README.md

File metadata and controls

137 lines (107 loc) · 3.81 KB
Raw

Datawarehouse

Contributors Forks Stargazers Issues LinkedIn


Sensor Data ELT

A fully dockerized ELT pipeline project, using Airflow, PostgreSQL, DBT, and Redash.
Report Bug · Request Feature

Table of Contents
  1. About The Project
  2. Getting Started
  3. Contact
  4. Acknowledgements

About The Project

Fully dockerized scalable ELT pipeline using PostgreSQL, Airflow, DBT and Redash.

Built With

Tech Stack used in this project

Getting Started

Prerequisites

Make sure you have docker installed on local machine.

  • Docker
  • DockerCompose

Installation

  1. Clone the repo and Navigate 
    git clone https://github.com/sel6/Datawarehouse.git
Navigate to airflow/dags
  2. Run 
     docker-compose -f docker-compose.yaml up --build
  3. Open Airflow web browser 
    Navigate to `http://localhost:8081/` on the browser
activate and trigger the `create_connection` dag
activate and trigger the `create_new_db` dag
activate and trigger the `create_new_table` dag
activate and trigger the `fill_table` dag
  4. Access redash dashboard 
    Navigate to `http://localhost:5011/` on the browser
  5. Access your postgres database 
    Navigate to `http://localhost:5432/` on the browser
use `warehouse` for db name
use `warehouse` for username
use `warehouse` for password

Contact

Selam Ayehubirhan - [email protected]

Acknowledgements