The sixth lesson of the open-source PhiloAgents course: a free course on building gaming simulation agents that transform NPCs into human-like characters in an interactive game environment.
A 6-module journey, where you will learn how to:
Create AI agents that authentically embody historical philosophers.
Master building real-world agentic applications.
Architect and implement a production-ready RAG, LLM, and LLMOps system.
Welcome to Lesson 6 of the PhiloAgents open-source course, where you will learn to architect and build a production-ready gaming simulation agent that transforms NPCs into human-like characters in an interactive game environment.
Our philosophy is that we learn by doing. No procrastination, no useless “research,” just jump straight into it and learn along the way. Suppose that’s how you like to learn. This course is for you.
During the first five lessons, we discussed in depth what it takes to build a production-ready AI agent, from creating the agent itself to wrapping it up in a backend and frontend architecture to serve it as a game and implement the LLMOps layer.
Still, in the world of AI, we are bombarded with the latest models, tools, and algorithms, but often forget what matters: building software that works.
Thus, we aim to conclude this series by returning to the fundamentals. To understand how to structure a Python project and use the right development tools (e.g., uv, ruff, Make) like a senior software engineer would. Additionally, we will explore how to containerize the project using Docker, as a senior DevOps engineer would.
These are all essential skills you will need in any software project, whether you're building AI applications around LLMs, Agents, RAG, or any other type of AI models. Thus, these are crucial skills for:
development speed and experience;
ease of deploying your app to the cloud;
making your project future-proof;
moving away from “it works on my machine” to “it works everywhere”.
We want you to understand the importance of structuring your Python module and using necessary tools, such as `uv`, `Ruff`, and `Docker`.
Additionally, as Docker containerizes your application into distinct components (backend, frontend, database), mirroring how it would appear in the cloud, we aim to explain what happens behind the scenes. Once a user enters a query in the Game UI, what happens infrastructure-wise until the answer reaches you?
To understand this, we must delve into networking, architecture design, and Docker principles.
Let’s get started. Enjoy!
Podcast version of the lesson
0:00
-6:20
Table of contents:
Structuring Python projects like a PRO
Exploring modern Python tooling
Setting up a future-proof local infrastructure using Docker
Opening up the Docker image
Deploying the PhiloAgent to the cloud
Running the code
1. Structuring Python projects like a PRO
Let’s begin by looking at the Python project structure. We designed the repository as a monolith. Why is that?
Remember that we have a frontend and backend infrastructure. Thus, instead of having two separate repositories for the front-end and back-end, we kept both in a single monolithic repository. We did that for ease of use. Depending on the size of the project and team sizes, you might want to divide them or not.
A typical structure resembles the one shown in Figure 2, where you divide your core components into separate folders: `philoagents-api` (backend) and `philoagents-ui` (frontend). For example, if we move one of the folders out of the project, it remains usable and makes sense on its own. This is called a modular monolith.
Figure 2: Repository folder structure
Along with these folders, we have some documentation files (README.md, INSTALL_AND_USAGE, LICENSE, etc.) and the infrastructure files (docker-compose.yml) used to glue everything together into a cohesive system that is easy to run with something as simple as:
make infrastructure-up
As we don’t care about the game UI (the frontend), we will focus solely on the Python project, located under `philoagents-api`, as illustrated in Figure 3.
Figure 3: Python project folder structure
Let’s focus on the key folders:
`src`: Where we place all the Python modules (in our use case, we have only one, which is `philoagents`). Usually, you want to structure your Python modules as standard Python packages that you are used to installing as `pip install …`, which means they are modular, hence the name `Python modules`. You don’t place any entry points here, just core classes and functions that define your business logic.
`tools`: This directory contains all the Python scripts used as entry points for our project. These are usually CLI commands that collect inputs and use the `philoagents` Python module to call specific pieces of logic, as seen below:
import asyncio
from functools import wraps
import click
from philoagents.application.conversation_service.generate_response import (
get_streaming_response,
)
from philoagents.domain.philosopher_factory import PhilosopherFactory
@click.command()
@click.option(
"--philosopher-id",
type=str,
required=True,
help="ID of the philosopher to call.",
)
@click.option(
"--query",
type=str,
required=True,
help="Query to call the agent with.",
)
@async_command
async def main(philosopher_id: str, query: str) -> None:
"""CLI command to query a philosopher.
Args:
philosopher_id: ID of the philosopher to call.
query: Query to call the agent with.
"""
philosopher_factory = PhilosopherFactory()
philosopher = philosopher_factory.get_philosopher(philosopher_id)
async for chunk in get_streaming_response(
messages=query,
philosopher_id=philosopher_id
):
print(f"\033[32m{chunk}\033[0m", end="", flush=True)
if __name__ == "__main__":
main()
As you can see, zero logic is implemented here. We just initialize some Python classes and call some Python functions, and that’s it! In this way, we maintain the Python module's modularity.
`notebooks`: This is where we store all the Jupyter Notebooks. From the folder structure point of view, it’s similar to the Python scripts from the `tools` folder, but it uses Notebooks instead of scripts. One key observation is that we DON’T keep the logic in the Notebooks; we just use them to explore the data, algorithms, or document workflows by leveraging the Python module. Taking this approach, Notebooks are amazing assets for exploration while keeping the code clean and maintainable.
`data`: Here we place any data we need locally for fine-tuning, evaluation, or RAG. Typically, this is stored in the cloud and downloaded only when needed.
A few last remarks on the `src` folder: We don’t place the code directly into `src` but rather create a new folder with the name of the Python module, which is `philoagents` in our use case, as seen in Figure 4.
Figure 4: `src` folder structure
In this way, we can easily manage multiple Python modules within the same codebase and use constructs like `from philoagents import X` instead of `from src import X`, making our code appear more professional and resembling a Python package.
As you can see in the screenshots above, we had many other files that we haven’t explored. Most of them are used by various tools required to manage the Python project. Let’s explore that further.
2. Exploring modern Python tooling
Let’s dig into the de facto tools used in 2025 to manage modern Python projects.
First, let’s start with the latest kid on the block: uv. An extremely fast Python package and project manager, written in Rust. If you've worked with Python before, you're likely familiar with pip for installing packages, venv for managing virtual environments, and pyenv for managing multiple Python versions. Also, if you are not new to the Python ecosystem, you probably know about Poetry, the king of Python project management until 2025.
Well, `uv` took that crown as it’s a super-fast, all-in-one replacement for everything listed above. As it’s written in Rust, it installs and manages your dependency tree much quicker, 10-100x faster than pip or Poetry (no more 30-minute waits to install your Python projects).
TL/DR All you need it’s `uv` to install and manage your Python projects.
Still, keeping your code clean and consistent is crucial, and that's where ruff comes in. It's a utility tool, written in Rust, acting as both a linter (checking your code for errors and potential problems using the PEP8 standard as reference) and a formatter (automatically styling your code to match agreed-upon conventions). Previously, you might have needed several tools like Flake8, isort, and Black. ruff combines everything into one.
Writing code is subjective. Everyone has their style or is too lazy to follow PEP8 conventions. Thus, by using a formatter and linter, you standardize how the code should be written, helping keep it organized, clean, maintainable, and easy to read (yes, that’s super important for the future of you and others!).
Like this, there will be no discussions on how the code should look between the engineers, as the rules are encoded into the linter and formatter.
Figure 5: Python project folder structure
Next, let's talk about Make. You might have noticed a file named Makefile in the project root. Make isn't a Python tool specifically. It's a classic utility for automating tasks. We use it to create simple shortcuts for common development commands. Instead of remembering and typing out long commands to run tests, build a Docker image, or start the application, you can define these tasks in the Makefile. Then, you just run simple commands like make test or make infrastructure-up.
Finally, any application must manage configuration settings like API keys, database connection details, or model parameters. Hardcoding these values is not a good idea. That's where Pydantic Settings helps. It's a library that lets you define your application's configuration using Pydantic models, which you might already use for data validation. It can automatically load settings from environment variables or special .env files, validate their types, and make them easily accessible in your code.
These tools work together to make building, managing, and running the Python backend, or any other Python application, a breeze.
3. Setting up a future-proof local infrastructure using Docker
We haven’t explicitly mentioned Docker, as we want to dedicate a few sections to it. Why? Because it’s the most essential component for developing any software, not only Python projects. Docker is everywhere, including Node, Java, Rust, you name it!
Docker is a software platform that allows you to build, test, and deploy applications quickly by packaging software into standardized units called containers, including everything needed to run, such as libraries, system tools, code, and runtime.
As shown in Figure 6, we utilized Docker to containerize all our core services: the backend, frontend, and MongoDB database.
Figure 6: Backend-frontend architecture using Docker
Let’s understand why containerizing everything with Docker (known as Dockerizing) is so powerful. Remember the well-known issue when building software with “It works on my machine”? Well, Docker solves that problem.
By using Docker, we isolate the entire application from the OS (e.g., Linux), system dependencies (e.g., Python version, CUDA kernels, PDF renderers, etc.), and application dependencies (the actual Python dependencies, such as LangGraph, FastAPI, or PyTorch). In other words, Docker is a lightweight virtual machine that encapsulates your entire application into a Docker image.
Why can’t we package everything, including the backend, frontend, and database, into a single Docker image when working locally? The second benefit of containerization is that it allows us to mimic the production environment locally. When we ship the application, the backend, frontend, and database reside on different machines, sometimes even in completely different cloud environments. Thus, by separating the Docker images, deployment becomes 10x easier.
To intuitively understand the local Docker infrastructure, let’s walk you step-by-step through what happens since a user asks a question in the game, until it gets a response from the philosopher (as seen in the video below and Figure 6):
The Phaser game UI is hosted as a web server in the `philoagents-ui` Docker container. When we enter `http://localhost:8080/` in the browser, it serves the HTML, CSS and JS code, which is loaded in our browser.
Next, when we talk to a PhiloAgent, we open up a WebSocket connection between the host browser and the FastAPI backend, hosted as a web server in the `philoagents-api` Docker container. The FastAPI application is a uvicorn web server, which knows how to handle all the network requirements such as proxies, load balancers and scaling.
When we ask the PhiloAgent a question, it’s passed from the browser (not the UI Docker) to the backend Docker container as an HTTP request, which is immediately processed under the `@app.websocket("/ws/chat")` API endpoint.
The backend passes the query to the LangGraph and Groq implementation (the core agentic logic behind the PhiloAgent) and starts doing the agentic RAG stuff we discussed in previous lessons.
Once the PhiloAgent agentic module generates a token, it’s passed immediately to the WebSocket, which the frontend application receives.
The frontend shows each token received through the WebSocket immediately (token-by-token) until the whole answer is generated.
If we ask another question, we repeat steps 3 to 6, or if we walk away from the PhiloAgent, we close the WebSocket connection.
With this architecture, we can easily take the frontend and backend Docker images, push them to a Docker registry (e.g., AWS ECR), where we can load them on a dedicated web server that knows how to scale horizontally and vertically, such as AWS EKS, ECS, or Beanstalk.
When working locally, we use Docker Compose to connect all the Docker images to the same network. Like Docker with dependencies, Docker Compose helps you encapsulate your whole application network and separate it from the rest of your local applications.
To fully understand this, let’s take a look at the `docker-compose.yml` file used inside the PhiloAgents course:
As you can see, we have the `local_dev_atlas` (MongoDB), `api` (backend), and `ui` (frontend) services, which communicate with each other through the `philoagents-network`network.
The most important attribute of the network you have to care about is the port. When you do "8080:8080," it maps the port from the internal network of the Docker container to your host port (your PC, laptop, etc.). If you already have an application running on port 8080, this will throw an error. That’s why you always have to be aware of the ports used on your machine.
By running the following Docker command:
docker compose up --build -d
# or use our Makefile and run "make infrastructure-up"
You will build (or pull) the three Docker images. Then, based on these images, you will create and run an instance of each, known as a Docker container. You have only one Docker image (the blueprint, similar to a class in Object-Oriented Programming), and you can have multiple Docker containers based on that image.
Figure 7 shows the Docker containers running after using the docker compose command from above. Note how each has a container ID that is uniquely attributed at run time. Also, note the ports column, which maps the ports from inside the container to the host machine as follows: “<host_machine>:<container>”.
Figure 7: Docker containers
Now, let’s open up the `philoagents-course-api` Docker image and see how we defined it.
4. Opening up the Docker image
We’ll break it down into logical steps, so you can see how each part contributes to building and running your app smoothly inside a container.
We start by choosing a slimmed-down Python base image (a Linux container containing Python and the bare minimum of system dependencies) and making sure `uv` (our Python project manager) is installed:
FROM python:3.11-slim
# Install uv.
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
Next, we set up the working directory and copy over the uv.lock, pyproject.toml, and README.md files required by `uv` to install the Python dependencies:
Now, we install all the required Python packages. The `uv sync --frozen --no-cache` command ensures that the exact versions specified in uv.lock are installed, and it skips using any cached packages to guarantee a clean, reproducible environment:
RUN uv sync --frozen --no-cache
Now it’s time to bring in the actual application code:
Finally, we define the command that will run when the container starts. This command launches the FastAPI application using uvicorn, pointing it to your main API file and setting it to listen on all network interfaces at port 8000:
Why did we copy and install the Python dependencies before copying the application code?
Docker caches each command independently (RUN, COPY, etc.). As the code changes extremely often, but the dependencies do not, and they take the longest to run, this separation allows us to rebuild Docker images significantly faster. This makes developing and testing code in containers fast, saving you much time.
The last thing we want to explain is how to take the local Docker setup and deploy it to the cloud.
5. Deploying the PhiloAgent to the cloud
As shown in Figure 8, there are six components to consider when deploying the application to the cloud.
Because we use Docker, we can easily replicate the local environment in the cloud. This is powerful because we already know that the local environment works. Through Docker, we package everything into a Docker image, push the image to a Docker registry (e.g., AWS ECR), and then pull it and use it in any desired environment. This is what most CI/CD pipelines do.
Figure 8: Architecting agentic RAG systems using Docker
Now, the question is, “Where can we run these Docker images to make them accessible to our users?” Let’s explore this for all six components.
Backend (Agentic RAG FastAPI Server): Shipped as a web server, usually under an HTTP server such as nginx, and deployed to services such as AWS EKS, AWS ECS, AWS Beanstalk, which can run the service 24/7 (or their GCP, Azure equivalents). The server listens constantly and responds to HTTP requests (API calls).
Frontend (Game UI): The frontend will be shipped similarly to the backend. The key difference is that a frontend web server must only serve the necessary HTML, CSS, and JavaScript code, which will be loaded into your browsers. More concretely, when you type “https://philoagents.com”, it will query the web server for the Phaser.js code, load it into your browser, and run it on your local computer. The API calls will be done directly from your browser. That’s why serving frontend applications can be done by a light server, as the computation is done on your side, whereas for the backend, the calculation is done on the server.
Database (Memory): The database is a little different. In our use case, the MongoDB database can be hosted on AWS, or we can leverage MongoDB Atlas, a fully managed cloud database service that handles all the infrastructure issues. Hosting document databases can get complex and time-consuming. Thus, we chose to use MongoDB Atlas, their fully managed solution that simplifies deploying, scaling, and managing the database. It supports vector and text indexes required for hybrid retrieval during RAG.
Observability pipeline: The observability pipeline centers around Opik, an open-source LLMOps framework. Thus, we could easily take their software and deploy it ourselves as a Docker image (or with a similar containerization logic), which could also sit in our AWS ECS, EKS, or Beanstalk cluster. Still, to simplify the initial deployment and get the application off the ground, we can use their fully managed service(with a generous free tier) and deploy it later in our AWS infrastructure.
Feature pipelines (RAG, Eval Dataset Generation): The feature pipelines are offline pipelines, which do not have to run 24/7, unlike all the other services. They will be scheduled, triggered manually, or by other events. Thus, we can deploy them on serverless infrastructure such as AWS Lambda or AWS SageMaker, which specializes in ML pipelines. In more mature projects, MLOps frameworks such as ZenML or Metaflow handle offline ML pipelines.
LLM API: We added this as a trap. Groq is a completely external API. Thus, we don’t have to care about deploying it. That’s the beauty of using APIs: we externalize all the infrastructure and model hassle, while focusing on our application.
Note that we provided only vendor-locked examples on where to run each component. Another option, which is cloud agnostic, is to deploy everything on Kubernetes, which natively knows how to run and scale Docker containers. For example, AWS EKS is just a self-hosted version of Kubernetes, or AWS ECS is AWS’s implementation of Kubernetes. In reality, hosting your own K8s cluster is not feasible if you don’t have a dedicated team to manage it. Thus, going with managed versions of K8s is usually the way to go.
A final thing to notice is the importance of open-source vs. closed-source software. There is a gradient between these two, but it can be split into three main categories:
100% closed source: You are completely locked in once you use it.
Fully managed open-source: Many companies build an open-source product and offer a managed version. This makes sense because hosting it yourself at scale can be time-consuming and complex. The good news is that you are not locked in, as you can always switch from their fully managed version to something you host. For example, all the tools presented in this course follow this structure: MongoDB, ZenML, Comet, and Opik.
100% open-source: You can build whatever you want with the software (except repacking and selling it yourself).
With all this in mind, let’s see how to run the code.
6. Running the code
We use Docker, Docker Compose, and Make to run the entire infrastructure, such as the game UI, backend, and MongoDB database.
Thus, to spin up the code, everything is as easy as running:
make infrastructure-up
But before spinning up the infrastructure, you have to fill in some environment variables, such as Groq’s API Key, and make sure you have all the local requirements installed.
Our GitHub repository has step-by-step setup and running instructions (it’s easy—probably a 5-minute setup).
You can also follow the first video lesson, where Miguel explains the setup and installation instructions step-by-step. Both are valid options; choose the one that suits you best.
After going through the instructions, type in your browser http://localhost:8080/, and it’s game on!
You will see the game menu from Figure 9, where you can find more details on how to play the game, or just hit “Let’s Play!” to start talking to your favorite philosopher!
Figure 9: Philoagents game menu.
For more details on installing and running the PhiloAgents game, go to our GitHub.
With the sixth lesson, we wrapped up the PhiloAgents open-source course.
During this lesson, we looked into how to structure Python projects like a PRO and what tools to use as an AI engineer in 2025 when working with Python.
Then we took a deep dive into our Docker setup, understood why it’s so critical for any software, and ultimately explored how to further take it and deploy the entire project to cloud vendors such as AWS.
If you read the whole PhiloAgents series, we sincerely appreciate that you trusted us with your time and attention. We hope that you enjoyed the series and learned a lot from it!
If you haven’t read the whole course, you can continue your learning journey by digging deeper into the PhiloAgent AI system design, agentic RAG implementation, and LLMOps layer by:
Watching Miguel’s end-to-end video on YouTube (if you prefer video).
Keep in mind that the written and video lessons are complementary. We recommend going through both to get the whole experience, such as the fundamentals, system design, tooling, and code understanding.
If you have questions or need clarification, feel free to ask in the comments below or the GitHub issues section.
💻 Explore all the lessons and the code in our freely available GitHub repository.
Free open-source courses:Master production AI with our end-to-end open-source courses, which reflect real-world AI projects and cover everything from system architecture to data collection, training and deployment.
A16z-Infra. (n.d.). GitHub - a16z-infra/ai-town: A MIT-licensed, deployable starter kit for building and customizing your own version of AI town - a virtual town where AI characters live, chat and socialize. GitHub. https://github.com/a16z-infra/ai-town
OpenBMB. (n.d.). GitHub - OpenBMB/AgentVerse: 🤖 AgentVerse 🪐 is designed to facilitate the deployment of multiple LLM-based agents in various applications, which primarily provides two frameworks: task-solving and simulation. GitHub. https://github.com/OpenBMB/AgentVerse
Sponsors
Thank our sponsors for supporting our work — this course is free because of them!
Images
If not otherwise stated, all images are created by the author.
Amazing article!! So sad the series is over 😢😢😢
Appreciate all of works your guys did.