AWS Glue Series by JA (johnny-alin-data.com)

AWS Certificate

• AWS Certified Data Engineer – Associate
• AWS Certified Cloud Practitioner

=====================================================================================

• Unlocking the Power of AWS Glue Crawlers for Effective ETL Processes Part 1

• Unlocking the Power of AWS Glue Crawlers for Effective ETL Processes Part 2

• Unlocking the Power of AWS Glue ETL: Transforming and Organizing Data for Analysis (Part 3)

• Optimizing AWS Glue ETL Jobs: Transitioning to Separate S3 Buckets for Raw and Analytics Layers (Part 4)

• Automating ETL with AWS Glue: Setting Up Daily Schedules and Auto-Crawlers (Part 5)

• Visualizing Your AWS Data Lake with Amazon QuickSight (Part 6)

     ___        ______     ____ _                 _  ___  
    / \ \      / / ___|   / ___| | ___  _   _  __| |/ _ \ 
   / _ \ \ /\ / /\___ \  | |   | |/ _ \| | | |/ _` | (_) |
  / ___ \ V  V /  ___) | | |___| | (_) | |_| | (_| |\__, |
  

  // __/_/ |/ _||___/ _,|_,| //

---

---

• 2024-05-28 updated gmaing.script to generated yyyymmdd __ name data
• 2024-07-21 updated the article-part1 for genrated data and upload process note

Hi there! Welcome to AWS Cloud9!

To get started, create some files, play with the terminal, or visit https://docs.aws.amazon.com/console/cloud9/ for our documentation.

Happy coding!

Johnny Updated Article One Generated Data

---

Run python script to generated one day with specific rows you want

python generate_data.py 2024-01-01 2000

Run the script to auto gnerated the Range of the days and random rows-range

source run_generator_domain_game_slot_with_argument.sh

Run Auto Upload Data to S3

source run_data_etl_to_s3_raw.sh

Part 2 later ...