AI Agents A-Z

This repository contains code for the O'Reilly Live Online Training for AI Agents A-Z

This course provides a comprehensive guide to understanding, implementing, and managing AI agents both at the prototype stage and in production. Attendees will start with foundational concepts and progressively delve into more advanced topics, including various frameworks like CrewAI, LangChain, and AutoGen as well as building agents from scratch using powerful prompt engineering techniques. The course emphasizes practical application, guiding participants through hands-on exercises to implement and deploy AI agents, evaluate their performance, and iterate on their designs. We will go over key aspects like cost projections, open versus closed source options, and best practices are thoroughly covered to equip attendees with the knowledge to make informed decisions in their AI projects.

Setup Instructions

Using Python 3.11 Virtual Environment

At the time of writing, we need a Python virtual environment with Python 3.11.

Option 1: Python 3.11 is Already Installed

Step 1: Verify Python 3.11 Installation

python3.11 --version

Step 2: Create a Virtual Environment

python3.11 -m venv .venv

This creates a .venv folder in your current directory.

Step 3: Activate the Virtual Environment

• macOS/Linux:

  source .venv/bin/activate

• Windows:

  .venv\Scripts\activate

You should see (.venv) in your terminal prompt.

Step 4: Verify the Python Version

python --version

Step 5: Install Packages

pip install -r requirements.txt

Step 6: Deactivate the Virtual Environment

deactivate

---

Option 2: Install Python 3.11

If you don’t have Python 3.11, follow the steps below for your OS.

macOS (Using Homebrew)

brew install [email protected]

Ubuntu/Debian

sudo apt update
sudo apt install python3.11 python3.11-venv

Windows (Using Windows Installer)

1. Go to Python Downloads.
2. Download the installer for Python 3.11.
3. Run the installer and ensure "Add Python 3.11 to PATH" is checked.

Verify Installation

python3.11 --version

Notebooks

In the activated environment, run

python3 -m jupyter notebook

• Using 3rd party agent frameworks

  • Intro to CrewAI - An introductory notebook for CrewAI

    • See the streamlit directory for an example of deploying crew on a streamlit app

  • Intro to OpenAI Swarm - An introductory notebook for OpenAI's Swarm

  • Intro to LangGraph - An introductory notebook for LangGraph

• Evaluating Agents

  • Evaluating Agent Output with Rubrics - Exploring a rubric prompt to evaluate generative output. This notebook also notes positional biases when choosing between agent responses.

    • Advanced - Evaluating Alignment - A longer notebook doing a much more in depth analysis on how an LLM can judge agent's responses

  • Evaluating Tool Selection - Calculating the accuracy of tool selection between different LLMs and quantifying the positional bias present in auto-regressive LLMs

• Building our own agents

  • First Steps with our own Agent - Working towards building our own agent framework

  • See Squad Goals for a very simple example of my own agent framework

    • Intro to Squad Goals - using my own framework to do some basic tasks

• Modern Agent Paradigms

  • Plan & Execute Agents - Plan & Execute Agents use a planner to create multi-step plans with an LLM and an executor to complete each step by invoking tools.

  • Reflection Agents - Reflection Agents combine a generator to perform tasks and a reflector to provide feedback and guide improvements.

Instructor

Sinan Ozdemir is the Founder and CTO of LoopGenius where he uses State of the art AI to help people run digital ads on Meta, Google, and more. Sinan is a former lecturer of Data Science at Johns Hopkins University and the author of multiple textbooks on data science and machine learning. Additionally, he is the founder of the recently acquired Kylie.ai, an enterprise-grade conversational AI platform with RPA capabilities. He holds a master’s degree in Pure Mathematics from Johns Hopkins University and is based in San Francisco, CA.