Home

Notes

---

Workflow...

• Spring boot + Cadence. https://github.com/anarsultanov/examples/tree/master/cadence-saga
• https://github.com/frtelg/cadence-spring-boot

https://www.snowflake.com/wp-content/uploads/2019/06/Snowflake_SIGMOD.pdf http://cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf

• https://docs.microsoft.com/en-us/azure/architecture/patterns/cqrs
• https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing
• https://dzone.com/articles/microservices-with-cqrs-and-event-sourcing

Mongoose storage performance testing tools, https://github.com/emc-mongoose

Linux LVM:

• https://www.howtogeek.com/howto/40702/how-to-manage-and-use-lvm-logical-volume-management-in-ubuntu/

https://github.com/ExpediaGroup/stream-registry

FLOW:

• https://www.ververica.com/blog/?hs_preview=OZYfhRDp-24407619564

https://github.com/tiangolo/fastapi

Workflow, https://github.com/n8n-io/n8n

-- Ubuntu 18.04, Hash sum mismatch
sudo rm -rf /var/lib/apt/lists/*
sudo apt-get update -o Acquire::CompressionTypes::Order::=gz
sudo apt-get update && sudo apt-get upgrade

Procella: Unifying serving and analytical data at YouTube, https://ai.google/research/pubs/pub48388/

https://www.slideshare.net/AmazonWebServices/big-data-analytics-architectural-patterns-and-best-practices-ant201r1-aws-reinvent-2018

https://github.com/Integerous/goQuality-dev-contents

InnerSource - http://paypal.github.io/InnerSourceCommons/assets/files/AdoptingInnerSource.pdf

Js Chart - https://blog.bitsrc.io/11-javascript-charts-and-data-visualization-libraries-for-2018-f01a283a5727

Metadata and Data lineage

https://engineering.linkedin.com/blog/2019/data-hub

https://medium.com/airbnb-engineering/democratizing-data-at-airbnb-852d76c51770

https://eng.uber.com/databook/

• https://github.com/MarquezProject/marquez

• Netflix, metacat

  • https://github.com/Netflix/metacat
  • https://medium.com/netflix-techblog/metacat-making-big-data-discoverable-and-meaningful-at-netflix-56fb36a53520

• https://medium.com/netflix-techblog/building-and-scaling-data-lineage-at-netflix-to-improve-data-infrastructure-reliability-and-1a52526a7977

• LinkedIn, WhereHows

  • https://github.com/linkedin/WhereHows
  • https://engineering.linkedin.com/blog/2016/03/open-sourcing-wherehows--a-data-discovery-and-lineage-portal

• LinkedIn

  • https://github.com/linkedin/datahub

• Lyft, Amundsen

  • https://eng.lyft.com/amundsen-lyfts-data-discovery-metadata-engine-62d27254fbb9
  • https://www.youtube.com/watch?v=m1B-ptm0Rrw
  • https://www.datacouncil.ai/hubfs/DataEngConf/Data%20Council/Slides%20SF%2019/Amundsen%20-%20A%20Data%20Discovery%20Platform%20from%20Lyft.pdf

https://medium.com/wbaa/facilitating-data-discovery-with-apache-atlas-and-amundsen-631baa287c8b

• https://github.com/rsyi/metaframe

• https://marquezproject.github.io/marquez/

• https://atlas.apache.org/#/

---

https://medium.com/@garyogasawara/hadoop-performance-benchmark-results-comparing-on-premise-s3-vs-hdfs-cf7a9ea3baa3

http://info.bluedata.com/rs/693-TGY-247/images/IDC-Perspective-Decoupling-Compute-Storage.pdf

https://github.com/rstacruz/cheatsheets

Apache Airflow:

• https://github.com/etsy/boundary-layer

zeppelin, centos7, 971748464

pypi mirror, https://stackoverflow.com/questions/50348248/creating-a-full-replica-offline-copy-of-the-public-pypi-repository

• https://github.com/pypa/bandersnatch

Readings:

• Architecting Modern Data Platforms, https://learning.oreilly.com/library/view/architecting-modern-data/9781491969267/
• Stream Processing with Apache Flink, https://learning.oreilly.com/library/view/stream-processing-with/9781491974285/
• Learning Spark Streaming, https://learning.oreilly.com/library/view/learning-spark-streaming/9781491944233/
• http://book.mixu.net/distsys/