Repositories list

• ethereum-etl-airflow

  Public
  Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee

  cryptoetlcryptocurrency+ 10data-engineeringdata-analyticsweb3apache-airflowblockchain-analyticson-chain-analysisethereum+ 3

  Python

  •

  MIT License

  •192•407•25•3•Updated Nov 20, 2024Nov 20, 2024

• etl-rust

  Public
  Rust

  •

  MIT License

  •0•6•0•0•Updated Nov 5, 2024Nov 5, 2024

• aptos-etl

  Public
  Rust

  •

  MIT License

  •0•4•1•0•Updated Nov 4, 2024Nov 4, 2024

• solana-etl

  Public
  Rust

  •

  MIT License

  •7•22•0•0•Updated Sep 27, 2024Sep 27, 2024

• polygon-etl

  Public
  ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

  gcppolygoncryptocurrency+ 6maticnetworkmatic-networkbigqueryairflowetldata-engineering

  Python

  •

  MIT License

  •76•101•13•0•Updated Sep 10, 2024Sep 10, 2024

• ethereum-etl

  Public
  Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

  awscsverc20-tokens+ 11blockchain-analyticserc721exportbigquerysqletlethereum+ 4

  Python

  •

  MIT License

  •849•3k•116•35•Updated Aug 13, 2024Aug 13, 2024

• blockchain-etl-dataflow

  Public
  Dataflow pipelines for Blockchain ETL. Connects Pub/Sub topics with BigQuery tables.

  Java

  •

  MIT License

  •8•8•0•1•Updated Aug 12, 2024Aug 12, 2024

• public-datasets

  Public
  The list of public blockchain datasets in BigQuery

  data-sciencecryptogcp+ 15google-cloudpolygoncryptocurrencydata-engineeringdata-analyticsweb3google-cloud-platform+ 8

  MIT License

  •21•223•4•0•Updated Jun 26, 2024Jun 26, 2024

• awesome-etl-tools

  Public
  Awesome ETL tools for blockchain data

  MIT License

  •2•9•0•0•Updated Apr 15, 2024Apr 15, 2024

• ethereum-etl-postgres

  Public
  ETL for moving Ethereum data to PostgreSQL database

  postgressqletl+ 2ethereumpostgresql

  Shell

  •

  MIT License

  •63•143•4•1•Updated Apr 11, 2024Apr 11, 2024

• bitcoin-etl

  Public
  ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ

  cryptoetlgcp+ 15cryptocurrencydata-engineeringdashdata-analyticsapache-beamweb3zcash+ 8

  Python

  •

  MIT License

  •123•410•23•4•Updated Apr 5, 2024Apr 5, 2024

• abi-parser

  Public
  Web app which parses smart contracts and outputs queries and tables for Ethereum-ETL

  googlecryptogcp+ 9google-cloudcryptocurrencydata-engineeringdata-analyticsweb3google-cloud-platformblockchain-analytics+ 2

  JavaScript

  •

  MIT License

  •15•6•0•2•Updated Apr 2, 2024Apr 2, 2024

• solana-etl-airflow

  Public
  ETL for Solana. Contributions are welcome. Join the Telegram channel https://t.me/joinchat/GsMpbA3mv1OJ6YMp3T5ORQ

  Python

  •

  MIT License

  •15•42•3•2•Updated Dec 19, 2023Dec 19, 2023

• pubsub-to-firestore-cloud-function

  Public
  Google Cloud Function that pulls messages from Pub/Sub and saves them in Firestore

  JavaScript

  •

  MIT License

  •0•0•0•1•Updated Dec 4, 2023Dec 4, 2023

• blockchain-etl-common

  Public
  Common utils for blockchain-etl

  Python

  •

  MIT License

  •5•5•0•2•Updated Nov 21, 2023Nov 21, 2023

• blockchain-terraform-deployment

  Public
  Template repository for deploying https://github.com/blockchain-etl/blockchain-terraform

  cryptogoogle-cloudcryptocurrency+ 7web3google-container-enginekubernetesterraformgcpgkegoogle-cloud-platform

  HCL

  •

  MIT License

  •5•6•0•1•Updated Nov 7, 2023Nov 7, 2023

• ordinals-etl

  Public
  Python

  •

  MIT License

  •2•5•0•0•Updated Sep 19, 2023Sep 19, 2023

• abi

  Public
  EVM public good - pull requests welcome for any ABI from any EVM

  MIT License

  •1•5•1•0•Updated Jul 7, 2023Jul 7, 2023

• iotex-etl

  Public
  ETL (extract, transform and load) tools for ingesting IoTeX blockchain data to Google BigQuery and Pub/Sub

  sqliotexblockchain-data+ 1bigquery

  Python

  •

  MIT License

  •7•5•0•5•Updated Jul 5, 2023Jul 5, 2023

• abi-functions

  Public
  MIT License

  •2•7•0•2•Updated Jun 16, 2023Jun 16, 2023

• tezos-etl

  Public
  Python scripts for ETL (extract, transform and load) jobs for Tezos blocks, balance updates, and operations

  bigquerycsvsql+ 3blockchaincryptocurrencytezos

  Python

  •

  MIT License

  •3•13•1•1•Updated May 22, 2023May 22, 2023

• eos-etl

  Public
  ETL scripts for EOS.

  cryptogcpgoogle-cloud+ 14cryptocurrencydata-engineeringdata-analyticsapache-beamweb3google-cloud-platformgoogle-bigquery+ 7

  Python

  •

  MIT License

  •7•9•0•2•Updated May 22, 2023May 22, 2023

• tezos-etl-airflow

  Public
  Airflow DAGs for exporting and loading the Tezos blockchain data to Google BigQuery

  Python

  •

  MIT License

  •1•2•2•1•Updated May 1, 2023May 1, 2023

• zilliqa-etl

  Public
  Python scripts for ETL (extract, transform and load) jobs for Zilliqa blockchain data

  Python

  •

  MIT License

  •7•3•3•1•Updated Mar 29, 2023Mar 29, 2023

• ethereum2-etl

  Public
  Python scripts for ETL (extract, transform and load) jobs for Ethereum 2.0 beacon blocks, attestations, deposits, slashings, validators, committees. Data is available in Google BigQuery

  Python

  •

  MIT License

  •12•71•7•1•Updated Mar 29, 2023Mar 29, 2023

• band-etl

  Public
  ETL (extract, transform and load) tools for ingesting Band Protocol blockchain data to Google BigQuery and Pub/Sub

  Python

  •

  MIT License

  •3•5•3•1•Updated Mar 29, 2023Mar 29, 2023

• ethers.js-bigquery

  Public
  ethers.js library, compiled for use in Google BigQuery

  data-sciencecryptogcp+ 11google-cloudcryptocurrencydata-engineeringdata-analyticsweb3google-cloud-platformblockchain-analytics+ 4

  JavaScript

  •

  MIT License

  •11•40•0•10•Updated Mar 2, 2023Mar 2, 2023

• hedera-etl

  Public
  ETL scripts for Hedera Hashgraph

  cryptoetlgoogle-cloud+ 14cryptocurrencydata-engineeringdata-analyticsapache-beamweb3google-cloud-platformgoogle-bigquery+ 7

  Java

  •

  Apache License 2.0

  •3•11•6•0•Updated Feb 14, 2023Feb 14, 2023

• docker-bitcoind

  Public
  💰 Bitcoind Docker image that runs the Bitcoin node in a container for easy deployment

  kubernetescryptocryptocurrency+ 1web3

  Dockerfile

  •

  MIT License

  •408•3•0•0•Updated Jan 12, 2023Jan 12, 2023

• twitter-bot-cloud-function

  Public
  Google Cloud Function for tweeting Blockchain ETL alerts

  JavaScript

  •

  MIT License

  •0•3•0•3•Updated Dec 10, 2022Dec 10, 2022