Home

Workflow, https://github.com/n8n-io/n8n

-- Ubuntu 18.04, Hash sum mismatch
sudo rm -rf /var/lib/apt/lists/*
sudo apt-get update -o Acquire::CompressionTypes::Order::=gz
sudo apt-get update && sudo apt-get upgrade

Procella: Unifying serving and analytical data at YouTube, https://ai.google/research/pubs/pub48388/

https://www.slideshare.net/AmazonWebServices/big-data-analytics-architectural-patterns-and-best-practices-ant201r1-aws-reinvent-2018

https://github.com/Integerous/goQuality-dev-contents

InnerSource - http://paypal.github.io/InnerSourceCommons/assets/files/AdoptingInnerSource.pdf

Js Chart - https://blog.bitsrc.io/11-javascript-charts-and-data-visualization-libraries-for-2018-f01a283a5727

Metadata and Data lineage

https://engineering.linkedin.com/blog/2019/data-hub

https://medium.com/airbnb-engineering/democratizing-data-at-airbnb-852d76c51770

https://eng.uber.com/databook/

• Netflix, metacat

  • https://github.com/Netflix/metacat
  • https://medium.com/netflix-techblog/metacat-making-big-data-discoverable-and-meaningful-at-netflix-56fb36a53520

• https://medium.com/netflix-techblog/building-and-scaling-data-lineage-at-netflix-to-improve-data-infrastructure-reliability-and-1a52526a7977

• LinkedIn, WhereHows

  • https://github.com/linkedin/WhereHows
  • https://engineering.linkedin.com/blog/2016/03/open-sourcing-wherehows--a-data-discovery-and-lineage-portal

• Lyft, Amundsen

  • https://eng.lyft.com/amundsen-lyfts-data-discovery-metadata-engine-62d27254fbb9
  • https://www.youtube.com/watch?v=m1B-ptm0Rrw
  • https://www.datacouncil.ai/hubfs/DataEngConf/Data%20Council/Slides%20SF%2019/Amundsen%20-%20A%20Data%20Discovery%20Platform%20from%20Lyft.pdf

---

https://medium.com/@garyogasawara/hadoop-performance-benchmark-results-comparing-on-premise-s3-vs-hdfs-cf7a9ea3baa3

http://info.bluedata.com/rs/693-TGY-247/images/IDC-Perspective-Decoupling-Compute-Storage.pdf

https://github.com/rstacruz/cheatsheets

Apache Airflow:

• https://github.com/etsy/boundary-layer

zeppelin, centos7, 971748464

pypi mirror, https://stackoverflow.com/questions/50348248/creating-a-full-replica-offline-copy-of-the-public-pypi-repository

• https://github.com/pypa/bandersnatch

Readings:

• Architecting Modern Data Platforms, https://learning.oreilly.com/library/view/architecting-modern-data/9781491969267/
• Stream Processing with Apache Flink, https://learning.oreilly.com/library/view/stream-processing-with/9781491974285/
• Learning Spark Streaming, https://learning.oreilly.com/library/view/learning-spark-streaming/9781491944233/
• http://book.mixu.net/distsys/