Every architectural era is defined by where the bottleneck sits. When you fix a bottleneck, you don’t get a better version of the old thing - you get a whole new discipline. That’s exactly the arc Agentic AI has walked in about four years: four bottlenecks, four disciplines.

Told through one running analogy - a brilliant new hire who starts by being handed a note and ends up running a factory floor.

The journey at a glance

1. Prompt Engineering - “say it better”

The idea. You’ve hired someone extraordinarily well-read but with total amnesia: they know everything about the world, nothing about your world, and forget it all the moment the conversation ends.

The craft. So at first you obsess over how you word the note you hand them:

• “You are an expert lawyer.”
• Show three examples.
• “Think step by step.”
• “Return JSON only.”

Rephrasing the same request could swing quality dramatically - and that felt like magic.

Why it stalled. It’s brittle: change a word and the output changes. And more fundamentally, no note is ever long enough to contain what the model needs to do a real job. The wording was never the bottleneck - the model’s ignorance of your business was.

Bottleneck: Instructions (the note)

2. Context Engineering - “give it the right material”

The shift. Stop polishing the note; start managing what the model actually knows the moment it answers - feeding it your documents (RAG), past context (memory), and live data (tools).

The engineering problem. The context window is a small desk with a hard size limit. Every decision is a trade-off: what goes on the desk, in what order, summarised how much, refreshed how often.

• Too little → it hallucinates.
• Too much → it gets distracted (and expensive).
• Anyone who’s built a search-ranking pipeline recognises this instantly - it’s relevance engineering wearing a new hat.

Why it stalled. You now get a very well-informed single answer. But real work isn’t a single answer. It’s an attempt, a mistake, a correction, another attempt.

Bottleneck: Single-shot output (one answer)

3. Loop Engineering - “let it work, not just answer”

The idea. The model stops replying and starts working:

think → act → observe → think again - until the job is done, or you stop it.

The employee is no longer answering a question. They’ve been handed a task: they’ll open the file, run the query, notice a number looks wrong, re-run it, and come back when it’s finished.

What engineers build here.

• Stopping conditions, step & token budgets
• Retries with backoff
• Idempotency for actions with side effects
• Sandboxing, so a bad step can’t do real damage
• Evaluation of the whole trajectory, not just the final message

Why it stalled. One loop is a single thread - linear, opaque, hard to parallelise and govern. Over long runs the desk gets cluttered and the agent drifts. And when an auditor asks “why did it approve that refund?”, “it thought about it for 40 steps” is not an acceptable answer in an enterprise.

Bottleneck: Single thread of reasoning (one loop)

4. Graph Engineering - “design the org chart, not the employee”

The realisation. Real work was never a loop - it’s a graph. You stop trying to build one genius who does everything and start designing a workflow.

A planner (orchestrator) routes to a tool executor, a database lookup, and an evaluator that checks progress - with a human approval step before anything final.

• Nodes are stations: an agent, a function, a DB call, a human.
• Edges are routing rules. State travels along the edges.
• You get branching, parallel fan-out & fan-in, checkpoints, resume-ability, and humans as first-class steps.

Two kinds of “graph engineering” - people use one term for two things:

1. Execution graph (workflow topology): LangGraph & similar orchestration frameworks.
2. Knowledge graph (domain model): GraphRAG - entities & relationships so retrieval can traverse connections (e.g. “which contracts are affected by this vendor’s change?”).

Bottleneck: Un-governed, non-deterministic workflows that are hard to audit at scale

The big picture - it’s a layering chain

Each stage didn’t replace the last - it absorbed it.

• Prompts didn’t die - they live inside the nodes.
• Context engineering didn’t die - it decides what each node sees.
• Loops didn’t die - a node with real autonomy still runs one internally.
• What changed at each step is the unit of design - from the sentence, to the working set, to the trajectory, and now to the topology.

The honest tension at the frontier

Graphs give you governance and repeatability. But over-specify the graph and you’ve rebuilt a rigid business-process workflow with an expensive LLM bolted on - you’ve paid for intelligence and then forbidden it.

The current art is knowing which parts of your process deserve a hard-wired edge (predictable, auditable) and which deserve a node that’s allowed to think.

The one line to remember

The field didn’t get better at prompting - it kept moving the problem up a level: from wording the request, to feeding the model, to letting it work, to architecting the whole system.

A step-by-step guide. Everything runs on your own hardware - no API keys leave the building, no data goes to a cloud model. We climb one rung at a time: talk to the model -> give it a tool -> let it run the loop -> give it memory -> watch what it does -> put a UI on it.

Stack: Python · Ollama (local model) · LangChain · LangGraph · LangSmith · Open WebUI Tested against: LangGraph v1.0 (stable), langchain-ollama, current LANGSMITH_* tracing vars.

The mental model

Each tool does exactly one job. Conflating them is the most common source of confusion:

The #1 catch: the model must natively support tool calling. A chat model that doesn’t will silently never call your tools. Stick to llama3.1, qwen2.5 / qwen3, or mistral.

Step 0 - Prerequisites & install

Install Ollama (macOS/Linux/Windows) from ollama.com, then pull a tool-capable model:

ollama pull llama3.1        # solid tool-caller, ~4.7GB
# ollama pull qwen2.5        # also excellent at tool calling
ollama serve                 # starts the local API at http://localhost:11434

Set up Python:

python -m venv .venv && source .venv/bin/activate
pip install -U langchain langchain-ollama langgraph langsmith

Sanity check the model is up:

ollama run llama3.1 "Say hello in five words."

Step 1 - Talk to the local model (no agent yet)

Before anything clever, prove the brain works. This is the same thing Open WebUI does under the hood.

from langchain_ollama import ChatOllama

llm = ChatOllama(
    model="llama3.1",
    base_url="http://localhost:11434",
    temperature=0,
)

print(llm.invoke("Explain what an AI agent is in one sentence.").content)

Important Point: right now this is a pure LLM - smart, but no hands and no memory (rung 1 of the LLM->agent ladder). It can only produce text.

Step 2 - Give it a tool (and watch it decide to use one)

Do not skip straight to the agent abstraction. Show the raw mechanic first, or students will treat the loop as magic.

Define tools with the @tool decorator. The docstring is not optional - the model reads it to decide when and how to call the tool.

from langchain_core.tools import tool
from langchain_ollama import ChatOllama
from langchain_core.messages import HumanMessage

@tool
def multiply(a: int, b: int) -> int:
    """Multiply two integers together.

    Args:
        a: the first integer
        b: the second integer
    """
    return a * b

@tool
def word_count(text: str) -> int:
    """Count the number of words in a piece of text.

    Args:
        text: the text to count words in
    """
    return len(text.split())

llm = ChatOllama(model="llama3.1", temperature=0)
llm_with_tools = llm.bind_tools([multiply, word_count])

response = llm_with_tools.invoke("What is 23 times 17?")
print(response.tool_calls)
# -> [{'name': 'multiply', 'args': {'a': 23, 'b': 17}, 'id': '...', 'type': 'tool_call'}]

Notice: the model didn’t compute the answer - it emitted a structured request to call multiply. Now you run the tool and feed the result back, by hand, so the full round-trip is visible:

tools_by_name = {"multiply": multiply, "word_count": word_count}
messages = [HumanMessage("What is 23 times 17?")]

ai_msg = llm_with_tools.invoke(messages)   # model decides
messages.append(ai_msg)

for call in ai_msg.tool_calls:             # we execute
    selected = tools_by_name[call["name"]]
    tool_msg = selected.invoke(call)       # returns a ToolMessage
    messages.append(tool_msg)

final = llm_with_tools.invoke(messages)    # model reads the result, answers
print(final.content)                       # -> "23 times 17 is 391."

Important point: that four-step dance - decide -> execute -> feed back -> answer - is the entire agent loop. Everything after this just automates it.

Step 3 - Let LangGraph run the loop (this is the agent)

Now hand the loop to LangGraph’s prebuilt ReAct agent. It does exactly what you just did by hand, repeatedly, until the task is done.

from langgraph.prebuilt import create_react_agent
from langchain_ollama import ChatOllama

llm = ChatOllama(model="llama3.1", temperature=0)

agent = create_react_agent(
    model=llm,
    tools=[multiply, word_count],
    prompt="You are a helpful assistant. Use tools when they help.",
)

for chunk in agent.stream(
    {"messages": [{"role": "user",
                   "content": "What is 23 times 17, and how many words are in that question?"}]},
    stream_mode="updates",
):
    print(chunk)

You’ll see it reason, call multiply, observe the result, call word_count, observe again, then compose a final answer - the perceive -> reason -> act -> observe -> repeat loop, automated. This is the moment it becomes an agent.

create_react_agent lives in langgraph.prebuilt. If a tutorial you find online uses set_entry_point() or hand-rolls a StateGraph for a basic agent, it predates v1.0 - you don’t need that for a simple agent.

Step 4 - Give it memory

Right now each call starts from zero (a colleague with amnesia). Add a checkpointer and a thread_id, and the agent remembers within a conversation thread - this is short-term / working memory.

from langgraph.checkpoint.memory import MemorySaver
from langgraph.prebuilt import create_react_agent

checkpointer = MemorySaver()   # in-memory; swap for a DB-backed saver in production
agent = create_react_agent(
    model=llm,
    tools=[multiply, word_count],
    checkpointer=checkpointer,
)

config = {"configurable": {"thread_id": "user-123"}}

agent.invoke({"messages": [{"role": "user", "content": "My name is John."}]}, config)
resp = agent.invoke({"messages": [{"role": "user", "content": "What's my name?"}]}, config)
print(resp["messages"][-1].content)   # -> recalls "John"

Important point: the thread_id is the memory boundary. Same id -> same conversation. For persistence across restarts, swap MemorySaver for a database-backed checkpointer (Postgres/SQLite savers exist) - that’s the jump from working memory to long-term memory.

Step 5 - Turn on the eyes (LangSmith observability)

An autonomous system you can’t see is a liability. LangSmith traces every reasoning step and tool call - no code changes needed, LangChain/LangGraph auto-instrument once the environment variables are set:

export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=lsv2_...            # from smith.langchain.com
export LANGSMITH_PROJECT=onprem-agent-demo
export LANGSMITH_ENDPOINT=https://api.smith.langchain.com

Re-run any script above, then open the project in LangSmith. You’ll see the full run tree: the prompt sent to the model, each tool call with its arguments and output, and the final answer nested inside.

Important point: this is the “log every tool call” discipline made real. When an agent misbehaves, the trace is the only way you’ll reconstruct what it actually did.

On-prem note: vanilla LangSmith is a hosted (cloud) service - the traces leave your network. For a truly air-gapped setup you have two honest options: self-hosted LangSmith (available on the enterprise plan, runs in your own cluster), or export traces via OpenTelemetry into an observability stack you already run. If “on-prem” is a hard requirement, decide this before you standardise on cloud LangSmith.

Step 6 - Put a face on it (Open WebUI)

6a - Chat with the raw model immediately. Open WebUI is a ChatGPT-style UI that auto-detects Ollama:

docker run -d -p 3000:8080 \
  --add-host=host.docker.internal:host-gateway \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  -v open-webui:/app/backend/data \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

Open http://localhost:3000 and your local models are already in the dropdown. This gives you a friendly frontend for the model - but note it’s talking to the raw model, not your agent (no tools, no memory logic).

6b - Expose the agent through the same UI. Open WebUI speaks the OpenAI API protocol, so wrap your LangGraph agent behind a minimal OpenAI-compatible endpoint:

# agent_server.py - expose the LangGraph agent as an OpenAI-compatible endpoint
from fastapi import FastAPI
from pydantic import BaseModel
import time, uuid

from langgraph.prebuilt import create_react_agent
from langchain_ollama import ChatOllama
# from your_tools import multiply, word_count   # reuse the tools from Step 2

llm = ChatOllama(model="llama3.1", temperature=0)
agent = create_react_agent(model=llm, tools=[multiply, word_count])

app = FastAPI()

class Message(BaseModel):
    role: str
    content: str

class ChatRequest(BaseModel):
    model: str
    messages: list[Message]
    stream: bool = False

@app.get("/v1/models")
def list_models():
    return {"object": "list",
            "data": [{"id": "onprem-agent", "object": "model", "owned_by": "you"}]}

@app.post("/v1/chat/completions")
def chat(req: ChatRequest):
    result = agent.invoke({"messages": [m.model_dump() for m in req.messages]})
    answer = result["messages"][-1].content
    return {
        "id": f"chatcmpl-{uuid.uuid4().hex}",
        "object": "chat.completion",
        "created": int(time.time()),
        "model": req.model,
        "choices": [{
            "index": 0,
            "message": {"role": "assistant", "content": answer},
            "finish_reason": "stop",
        }],
    }

Run it and register it in Open WebUI:

pip install fastapi uvicorn
uvicorn agent_server:app --host 0.0.0.0 --port 8000

Then register it in Open WebUI (steps as of v0.9.x):

1. Avatar (bottom-left) -> Admin Panel
2. Settings tab -> Connections
3. In the OpenAI API section, click the + (plus) icon
4. URL: http://host.docker.internal:8000/v1 - use host.docker.internal, not localhost, since Open WebUI runs in Docker and the shim runs on the host
5. API Key: any non-empty dummy string (e.g. sk-noauth) - the field is required but the shim doesn’t check it
6. Click Verify Connection, then Save

Your agent - tools, memory, and all - now appears as a selectable “model” (onprem-agent) in the chat dropdown.

Where this maps to the LLM->Agentic ladder

The pitfalls to call out

• Model choice is destiny. Small or non-tool-tuned models call tools poorly or not at all. Start with llama3.1 or qwen2.5; only shrink the model once the logic works.
• Docstrings are prompts. The @tool docstring and type hints are what the model reads to decide when and how to call it. Vague docstring -> wrong tool calls.
• Show the manual loop before the abstraction. Step 2 by hand, then Step 3. Students who skip this treat create_react_agent as a black box.
• thread_id is the memory key. Reuse it to continue a conversation; change it to start fresh.
• Set a recursion limit. Agents can loop. agent.invoke(..., {"recursion_limit": 10}) stops runaway reasoning.
• Observability isn’t optional for production. If it can act autonomously, you must be able to see what it did - decide your on-prem tracing story early.

A plain-language walkthrough of the core concepts: what an agent is, how it thinks and works, and what makes it reliable in production.

1. Agentic AI - Concepts

An LLM on its own just answers questions. An agent is what you get when you point that brain at a goal and let it act on its own: it looks at the situation, thinks, makes a plan, does something, checks the result, and adjusts - with barely any hand-holding.

The core loop is the whole idea:

perceive -> reason -> act -> learn -> repeat

Why now? Because the brains (LLMs) finally got good enough at reasoning, the tools to wire them up matured, and businesses genuinely want work done without a human in every loop.

2. Agent Architecture - Building Blocks

This is what’s actually inside an agent. Inputs come in (a user, sensors, data, events), and the agent runs them through five parts:

• Perception - understand what’s being asked
• Reasoning - the LLM decides what to do
• Planning - break it into steps
• Memory - hold context, short and long term
• Action / Tools - actually go do it

Underneath sits a knowledge store it can pull facts from. This panel is just naming the organs of the thing - the same loop from panel 1, drawn as machinery.

3. Types of Agents

Not all agents are equally clever, and that’s on purpose - you match the type to the job.

The lesson: more autonomy isn’t always better - pick the simplest type that does the job.

4. Agent Workflow - Deep Dive

This zooms into one full pass of the loop, step by step:

1. Goal - get the objective from a user or system
2. Perception - gather context from memory, tools, environment
3. Reasoning - figure out intent and constraints
4. Planning - break the goal into actionable steps
5. Action - execute a step using tools / APIs
6. Observation - look at the result and feedback
7. Reflection & Learning - update memory, improve the plan

…then repeat until the goal is achieved.

Made concrete: the goal is “analyze sales data and email me insights,” so it fetches the data, thinks, plans four steps, runs Python, checks “was the email actually sent?”, and stores what it learned.

The observation and reflection steps are what separate a real agent from a script - it notices when something went wrong and corrects.

5. Memory in Agentic AI

Memory is what turns a goldfish into a colleague.

• Short-term (working) - the current conversation; lives in the context window
• Long-term (persistent) - stored outside the model in databases, survives across sessions
• Episodic - “what happened last time”; past experiences it can learn from
• Semantic - plain facts and domain knowledge

Because you can’t cram everything into the context window, there are strategies to manage it: summarize old chats, embed + retrieve only the relevant bits, use a memory graph of related facts, or a hybrid of these.

This is usually where real agent projects succeed or fail - get memory wrong and the agent feels forgetful and unreliable.

6. Tools & Actions

An agent that can only talk is a chatbot; tools are what let it do things in the real world - call APIs, run code, search, read/write databases, send Slack messages, drive a browser.

The pattern is always the same:

LLM decides it needs a tool -> calls it with inputs -> gets the output -> continues

Best practices (the part experience teaches you the hard way):

• Describe your tools clearly, with proper schemas
• Validate inputs and handle errors
• Limit what the agent is allowed to touch, for safety
• Log every single tool call - because when an autonomous system misbehaves, the logs are the only way you’ll ever understand what it did

The thread tying it together

Memory so it doesn’t forget, and disciplined tool use so it doesn’t go off the rails - that’s what carries an agent from an impressive demo to something you can actually trust in production.

Think of it as hiring someone brilliant and slowly making them actually useful at your company. Each stage below is one thing you give them.

1. LLM - the brilliant new hire on day one

They’ve read everything, they’re sharp, they can talk about anything. But they know nothing about your company, they can’t pick up a phone, and they forget you the moment you leave the room.

Pure thinking, no doing.

2. + RAG - give them the filing cabinet

Before answering, they go look it up in your documents. Now instead of “here’s how expense policies generally work,” you get “here’s your expense policy, updated last March.”

Same brain, but grounded in your reality instead of vague memory.

3. + Tool calling - give them hands

Until now they could only talk. Now they can act: call an API, run a query, send an email, execute code.

This is the biggest leap in the whole chain - the moment the system stops being an advisor and starts being a worker.

4. + Memory - give them a notebook

They remember you. Your preferences, what you asked last week, which approach failed. Without this, every conversation starts from zero - like a colleague with amnesia.

With it, they get better at working with you specifically.

5. = AI Agent - that’s the employee

Nothing new here. It’s just the name for the sum: a brain, that reads your docs, has hands, and remembers. Give it a goal instead of a script, and it figures out the steps itself.

That’s the definition of “agent” - you say what, not how.

6. + Agentic AI - now it’s a team, not a person

One person can’t do everything. So agents hire agents: a planner breaks the work down, specialists handle pieces, someone reviews the output and sends it back if it’s wrong.

The self-correction loop matters more than the delegation - a team that checks its own work is what makes multi-step tasks survivable.

7. + Skills & Hooks - the company handbook and the standard procedures

• Skills are packaged know-how: “here’s exactly how we do a client onboarding” - so the agent doesn’t reinvent it badly each time.
• Hooks are the rules that fire automatically at certain moments: before you deploy, run the tests; after any refund, log it.

Together: less improvisation, more institutional muscle memory.

8. + Governance & Observability - badge, camera, and audit trail

Everything above makes it powerful. This makes it employable. You can see what it did and why (traces), stop it doing things it shouldn’t (guardrails), and prove any of it to an auditor later.

Skip this and you have an impressive demo. Include it and you have production.

The through-line

Most teams get very excited about the middle and discover the hard way that the last one is what actually determines whether the thing ever leaves the lab.

That’s it. That’s the whole trick. Everything else - writing code, answering questions, drafting emails - falls out of doing that one thing extremely well, billions of times over.

The key parts, in plain language:

Where the knowledge comes from

Nobody taught it grammar or facts by hand. It was shown an enormous amount of text - books, websites, code - with words hidden, and asked to guess them. Get it wrong, adjust slightly. Repeat trillions of times. To guess well, it had to absorb spelling, grammar, facts, tone, reasoning patterns. Prediction was the exercise; understanding-ish behaviour was the side effect.

How you get whole answers

It picks a word, adds it to the sentence, then predicts the next one, over and over. Your question is just the opening - it keeps completing until it decides it’s done.

Why it makes things up

It’s always producing the most plausible-sounding continuation, not looking anything up. “Plausible” and “true” usually overlap. Sometimes they don’t - and it sounds equally confident either way.

An analogy that holds up well: it’s like someone who has read every book in the world but remembers none of them specifically - only a deep instinct for how sentences tend to go. Ask them anything and they’ll answer fluently from that instinct. Usually right. Occasionally confidently wrong.