If AI and AI Agents still feel confusing, this is a good place to start. And if you already know the basics, consider this a quick recap of the concepts, architecture, and engineering ideas worth revisiting.
To understand AI Agents, we first need to understand the difference between traditional software, LLMs, and agents.
Traditional software is mostly deterministic. We define the rules, write the logic, and the system follows those instructions.
An LLM is different. Instead of explicitly programming every response, we give it context and a prompt, and it generates an answer based on what it has learned.
An AI Agent takes this one step further.
Instead of simply answering:
“Here is how you can do this.”
An agent can work toward a goal by deciding what steps to take, using tools, observing the results, and adapting its approach.
A simple way to remember it:
Traditional software → Follow the rules
LLM → Generate an answer
AI Agent → Achieve a goal
Why are AI Agents becoming important?
The bigger shift actually started with foundation models and LLMs.
Earlier ML systems often required data collection, manual labeling, feature engineering, and model training for specific problems.
With pre-trained foundation models, developers can now build applications on top of existing models.
The focus is increasingly moving from:
“How do I train the model?”
to:
“How do I build a reliable system around the model?”
That brings context, tools, retrieval, workflows, orchestration, observability, and evaluation into the picture.
So, what can AI Agents actually do?
There isn't just one type of agent.
We can broadly think about agents such as conversational assistants, business automation agents, research agents, data and analytics agents, coding agents, domain-specific agents, and multi-agent systems.
A research agent, for example, might gather information from multiple sources, extract relevant facts, and synthesize the findings.
A coding agent might help generate code, debug issues, refactor a project, or review changes.
A data agent might generate SQL, execute analysis, work with Python, and produce insights or visualizations.
A multi-agent system can go even further by allowing specialized agents to collaborate and delegate different parts of a problem.
But here's the important engineering question:
Do we actually need an agent?
Not every AI problem needs one.
If the logic is predictable, deterministic code may be enough.
If the process has a known sequence of steps, a workflow may be better.
If the main problem is giving an LLM access to private or external information, RAG may be the right approach.
Agents become useful when the path to completing a task is not completely predictable and the system needs to make decisions along the way.
So the progression is roughly:
Code → Workflow → RAG → Agent
The more autonomy we introduce, the more complexity we also introduce.
And that complexity matters.
An agent may need multiple model calls, tool calls, retries, and reasoning steps.
That means:
More steps → More tokens → More latency → More cost → More opportunities for failure
This is why building an agent isn't simply about connecting an LLM to a few tools.
A production-ready agent needs things like cost monitoring, modular components, retries, termination limits, logging, observability, and human approval for sensitive actions.
For example, an agent might be allowed to analyze a payment, but actually executing that payment may require human approval.
The same applies to actions such as production deployments or other irreversible operations.
Where do frameworks fit?
Frameworks such as LangGraph, CrewAI, AutoGen, and the OpenAI Agent SDK can help developers build and orchestrate agentic systems.
But the framework itself isn't the most important part.
The real architecture is about bringing together:
Model + Context + Tools + Workflow + Memory + Guardrails + Observability
The framework is simply one layer that helps us manage those components.
The mental model
If you're completely new to AI, remember these three sentences:
LLM:
“Give me a question, and I'll generate an answer.”
Workflow:
“Give me a task, and I'll follow these predefined steps.”
Agent:
“Give me a goal, and I'll figure out the steps, use the available tools, observe what happens, and adapt.”
That's the fundamental shift behind agentic AI:
From generating answers → to taking actions.
And perhaps the most important lesson is not how to build the most autonomous agent.
It's knowing when you don't need one.
Sometimes the best AI architecture is still good old deterministic code.
These notes are based on the “What are AI Agents in depth” discussion by Hitesh Choudhary
0 Comments