Paul: Today’s spotlight: Paolo Perrone, master of turning tech into scroll-stopping content. This one’s packed, let’s go 👀 ↓
About two weeks before a big product deadline, my prototype agent broke in the worst way.
It looked fine. It fetched data, called tools, and even explained its steps. But under the hood, it was bluffing. No real state, no memory, no reasoning. Just looping prompts pretending to be smart.
I only noticed when an edge case completely threw it off. That’s when it hit me: I hadn’t built an agent. I’d built a fancy prompt chain.
Fixing it meant redesigning the whole thing — not just chaining calls, but managing state, decisions, and long-term flow. Once that clicked, everything got simpler. The code, the logic, the results.
That’s what this guide is about: breaking agent design into five practical levels of difficulty — each with working code.
Whether you’re just starting out or trying to scale real-world tasks, this will help you avoid the traps I fell into and build agents that actually work.
The levels are:
Level 1: Agent with Tools and Instructions
Level 2: Agent with Knowledge and Memory
Level 3: Agent with Long-Term Memory and Reasoning
Level 4: Multi-Agent Teams
Level 5: Agentic Systems
Alright, let’s dive in.
Level 1: Agent with Tools and Instructions
This is the basic setup — an LLM that follows instructions and calls tools in a loop. When people say, “agents are just LLMs plus tool use,” they’re talking about this level (and revealing how far they’ve explored).
Instructions tell the agent what to do. Tools let it take action — fetching data, calling APIs, or triggering workflows. It’s simple, but already powerful enough to automate some tasks.
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.duckduckgo import DuckDuckGoTools
agno_assist = Agent(
name="Agno AGI",
model=0penAIChat(id="gpt-4.1"),
description=dedent("""\
You are "Agno AGI, an autonomous AI Agent that can build agents using the Agno)
framework. Your goal is to help developers understand and use Agno by providing
explanations, working code examples, and optional visual and audio explanations
of key concepts."""),
instructions="Search the web for information about Agno.",
tools=[DuckDuckGoTools()],
add_datetime_to_instructions=True,
markdown=True,
)
agno_assist.print_response("What is Agno?", stream=True)Level 2: Agent with Knowledge and Memory
Most tasks require information the model doesn’t have. You can’t stuff everything into context, so the agent needs a way to fetch knowledge at runtime — this is where agentic RAG or dynamic few-shot prompting comes in.
Search should be hybrid (full-text + semantic), and reranking is a must. Together, hybrid search + reranking is the best plug-and-play setup for agentic retrieval.
Storage gives the agent memory. LLMs are stateless by default; storing past actions, messages, and observations makes the agent stateful — able to reference what’s happened so far and make better decisions.
... imports
# You can also use https://docs.agno.com/llms-full.txt for the full documentation
knowledge_base = UrlKnowledge(
urls=["https://docs.agno.com/introduction.md"],
vector_db=LanceDb(
uri="tmp/lancedb",
table_name="agno_docs",
search_type=SearchType.hybrid,
embedder=0penAIEmbedder(id="text-embedding-3-small"),
reranker=CohereReranker(model="rerank-multilingual-v3.0"),
),
)
storage = SqliteStorage(table_name="agent_sessions", db_file="tmp/agent.db")
agno_assist = Agent(
name="Agno AGI",
model=OpenAIChat(id="gpt-4.1"),
description=...,
instructions=...,
tools=[PythonTools(), DuckDuckGoTools()],
add_datetime_to_instructions=True,
# Agentic RAG is enabled by default when 'knowledge' is provided to the Agent.
knowledge=knowledge_base,
# Store Agent sessions in a sqlite database
storage=storage,
# Add the chat history to the messages
add_history_to_messages=True,
# Number of history runs
num_history_runs=3,
markdown=True,
)
if __name_ == "__main__":
# Load the knowledge base, comment after first run
# agno_assist.knovledge.load(recreate=True)
agno _assist.print_response("What is Agno?", stream=True)Level 3: Agent with Long-Term Memory and Reasoning
Memory lets agents recall details across sessions — like user preferences, past actions, or failed attempts — and adapt over time. This unlocks personalization and continuity. We’re just scratching the surface here, but what excites me most is self-learning: agents that refine their behavior based on past experiences.
Reasoning takes things a step further.
It helps the agent break down problems, make better decisions, and follow multi-step instructions more reliably. It’s not just about understanding — it’s about increasing the success rate of each step. Every serious agent builder needs to know when and how to apply it.
... imports
knowledge_base = ...
memory = Memory(
# Use any model for creating nemories
model=0penAIChat(id="gpt-4.1"),
db=SqliteMemoryDb(table_name="user_menories", db_file="tmp/agent.db"),
delete_memories=True,
clear_memories=True,
)
storage =
agno_assist = Agent(
name="Agno AGI",
model=Claude (id="claude-3-7-sonnet-latest"),
# User for the memories
user_id="ava",
description=...,
instructions=...,
# Give the Agent the ability to reason
tools=[PythonTools(), DuckDuckGoTools(),
ReasoningTools(add_instructions=True)],
...
# Store memories in a sqlite database
memory=memory,
# Let the Agent manage its menories
enable_agentic_memory=True,
)
if __name__ == "__main__":
# You can comment this out after the first run and the agent will remember
agno_assist.print_response("Always start your messages with 'hi ava'", stream=True)
agno_assist.print_response("What is Agno?", stream=True)Level 4: Multi-Agent Teams
Agents are most effective when they’re focused — specialized in one domain with a tight toolset (ideally under 10). To tackle more complex or broad tasks, we combine them into teams. Each agent handles a piece of the problem, and together they cover more ground.
But there’s a catch: without strong reasoning, the team leader falls apart on anything nuanced. Based on everything I’ve seen so far, autonomous multi-agent systems still don’t work reliably. They succeed less than half the time — which isn’t good enough.
That said, some architectures make coordination easier. Agno, for example, supports three execution modes — coordinate, route, and collaborate — along with built-in memory and context management. You still need to design carefully, but these building blocks make serious multi-agent work more feasible.
... imports
web agent = Agent(
name="Web Search Agent",
role="Handle web search requests",
model= OpenAIChat(id="gpt-4o-mini"),
tools=[DuckDuckGoTools()],
instructions="Always include sources",
)
finance_agent= Agent(
name="Finance Agent",
role="Handle financial data requests",
model=OpenAIChat(id="gpt-4o-mini"),
tools=[YFinanceTools()],
instructions=[
"You are a financial data specialist. Provide concise and accurate data.",
"Use tables to display stock prices, fundamentals (P/E, Market Cap)",
],
)
team_leader = Team (
name="Reasoning Finance Team Leader",
mode="coordinate",
model=Claude(id="claude-3-7-sonnet-latest"),
members=[web_agent, finance_agent],
tools=[ReasoningTools(add_instructions=True)],
instructions=[
"Use tables to display data",
"Only output the final answer, no other text.",
],
show_members_responses=True,
enable_agentic_context=True,
add_datetime_to_instructions=True,
success_criteria="The team has successfully completed the task.",
)
if __name__ == "__main__":
team_leader.print_response(
"""\
Analyze the impact of recent US tariffs on market performance across
these key sectors:
- Steel & Aluminum: (X, NUE, AA)
- Technology Hardware: (AAPL, DELL, HPQ)
For each sector:
1. Compare stock performance before and after tariff implementation
2. Identify supply chain disruptions and cost impact percentages
3. Analyze companies' strategic responses (reshoring, price adjustments, supplier
diversification)""",
stream=True,
stream_intermediate_steps=True,
show_full_reasoning=True,
)Level 5: Agentic Systems
This is where agents go from being tools to infrastructure. Agentic Systems are full APIs — systems that take in a user request, kick off an async workflow, and stream results back as they become available.
Sounds clean in theory. In practice, it’s hard. Really hard.
You need to persist state when the request comes in, spin up a background job, track progress, and stream output as it’s generated. Websockets can help, but they’re tricky to scale and maintain. Most teams underestimate the backend complexity here.
This is what it takes to turn agents into real products. At this level, you’re not building a feature — you’re building a system.
From Demo Fails to Real Wins: Key Lessons in Agent Design
Building AI agents isn’t about chasing hype or stacking features — it’s about getting the fundamentals right. Each level, from basic tool use to fully asynchronous agentic systems, adds power only when the underlying architecture is sound.
Most failures don’t come from missing the latest framework. They come from ignoring the basics: clear boundaries, solid reasoning, effective memory, and knowing when to let humans take the wheel.
If you start simple, build up with purpose, don’t overcomplicate upfront and add complexity only when it solves a real problem, you won’t just build something cool — you’ll build something that works.
See you next week.
👋 I’d love your feedback to help improve Decoding ML.
Share what you want to see next or your top takeaway. I read and reply to every comment!
Struggling to grow your audience as a Tech Professional?
The Tech Audience Accelerator is the go-to newsletter for tech creators serious about growing their audience. You’ll get the proven frameworks, templates, and tactics behind my 30M+ impressions (and counting).
Whenever you’re ready, there are 3 ways we can help you:
Perks: Exclusive discounts on our recommended learning resources
(books, live courses, self-paced courses and learning platforms).
The LLM Engineer’s Handbook: Our bestseller book on teaching you an end-to-end framework for building production-ready LLM and RAG applications, from data collection to deployment (get up to 20% off using our discount code).
Free open-source courses: Master production AI with our end-to-end open-source courses, which reflect real-world AI projects and cover everything from system architecture to data collection, training and deployment.
Images
If not otherwise stated, all images are created by the author.
Thanks, Paolo, for contributing another excellent piece!
Thanks for the solution design considerations and services to the same for the good 😊