Skip to content

Commit

Permalink
Remove Angellist mention
Browse files Browse the repository at this point in the history
  • Loading branch information
pedrogk committed Sep 7, 2024
1 parent 95e9f6e commit 9a058a6
Showing 1 changed file with 9 additions and 9 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ images:
- /images/sessions/2024/from-tech-specs.jpg
---

There are many Airflow tutorials. However, many don’t show the full process of sourcing, transforming, testing, alerting, documenting, and finally supplying data. This talk with go over how to piece together an end-to-end Airflow project that transforms raw data to be consumable by the business. It will include how various technologies can all be orchestrated by Airflow to satisfy the needs of analysts, engineers, and business stakeholders. Each step and technology mentioned will be something that we at AngelList use, and code snippets will be sprinkled throughout so that attendees can implement this project within their organizations.
The talk will be divided into the following sections:
1. Introduction: Introducing the business problem and how we came up with the solution design
2. Data sourcing: Fetching and storing API data using basic operators and hooks
3. Transformation and Testing: How to use dbt to build and test models based on the raw data
4. Alerting: Alerting the necessary parties when any part of this DAG fails using Elementary and Slack
5. Documentation and Consumption: How to sync data documentation to Metabase and dbt docs for easy consumption and lineage
There are many Airflow tutorials. However, many don’t show the full process of sourcing, transforming, testing, alerting, documenting, and finally supplying data. This talk with go over how to piece together an end-to-end Airflow project that transforms raw data to be consumable by the business. It will include how various technologies can all be orchestrated by Airflow to satisfy the needs of analysts, engineers, and business stakeholders.

The talk will be divided into the following sections:

Introduction: Introducing the business problem and how we came up with the solution design
Data sourcing: Fetching and storing API data using basic operators and hooks
Transformation and Testing: How to use dbt to build and test models based on the raw data
Alerting: Alerting the necessary parties when any part of this DAG fails using Slack
Consumption: How to make dynamic data accessible to business stakeholders

0 comments on commit 9a058a6

Please sign in to comment.