AI Development

What Is an AI Agent? 2026 Guide to How Agents Work, Tool Calling, Function Calling, and JSON

JSONSort Published 2026-09-05 Updated 2026-09-05 About 16 min read

Every AI product page in 2026 says “Agent.” Most of them collapse a chatbot, a model that can call functions, and a runtime that can finish a multi-step workflow into one word. This article starts from the definition: what an AI Agent is, how the loop runs, how Tool Calling differs from Function Calling, and why every layer lands on JSON. When you finish, you should be able to draw a debuggable call chain—not a longer prompt.

Quick overview

Idea	One line	JSON lands on
AI Agent	A runtime that plans, calls tools, and hands the result to the next step	Workflow state / handoff
Tool Calling	The general name for “the model wants an external capability”	`tool_calls`
Function Calling	The vendor-API implementation of that call	`tools[]` / `arguments`
JSON	The only stable, validatable object between steps	Schema / payload

Keep the layers: Agent is the runtime, Tool Calling is the capability, Function Calling is the API frame, JSON is the shared contract.

What is an AI Agent?

An AI Agent is not “a model that chats better.” The model only generates text (or a structured object). The agent is the runtime that puts the model in a loop: observe state, plan the next step, call an external tool, digest the result, then continue, hand off, or stop.

A shippable agent in 2026 is usually a trio:

The model: reason, pick a tool, emit Structured Outputs.
The host: parse call intent, run the real API / database / file op, return the result, block illegal JSON.
The contract: tool parameters, tool results, handoff objects, workflow state—almost all JSON.

Without a host, the model can only suggest. Without a contract, a multi-step flow treats a half-right object as fact. The product boundary sits on the host, not on the model card.

How is that different from a chatbot or a plain LLM?

Three labels get pasted onto one feature. Production has to split them:

Plain LLM: text in, text out. No tools, no cross-step state.
Chatbot: an LLM wired to a conversation UI. Output is for humans; the user retries failures.
AI Agent: output is for a program to execute. Tool results, routing, and eval scores are objects, not prose.

A simple test: if the next step must be executed by code—not understood by a person—you need an agent, not a clever dialog. Execution belongs to the host; the model only proposes call intent.

How does an AI Agent work in 2026?

The loop is still observe → plan → act → observe. What changed is that every hop is typed, instead of “the model mentioned a refund.”

Observe: fold the user message, the last tool result, and the current JSON state into context.
Plan: the model answers, calls a tool, or hands an object to another agent.
Act: the host runs tool_calls, or switches agents from handoff JSON.
Observe again: tool results return as role: tool; the state object updates before the next turn.

The host still does four things the model will not do for you: inject tools[], parse arguments a second time, validate with Schema, and short-circuit on failure instead of “try a longer prompt.”

The production stack has already converged. The Assistants API retired on 26 August 2026; new code uses Responses API + Agents SDK. DevDay 2026 predictions writes the same trend as three layers: Structured Outputs shape the object, Function Calling expresses the call, the agent runtime orchestrates both.

Do not ask the chat history to remember progress. Write the current step as recoverable JSON:

{
  "step": 2,
  "agent": "billing",
  "last_tool": "lookup_order",
  "state": {
    "order_id": "A-1042",
    "status": "paid",
    "amount": 99.00
  },
  "next": "create_refund"
}

Format those objects locally with format and syntax-check them, then JSON Diff them against last week’s working definition. Do not paste tool responses that carry user data or secrets onto a public JSON site.

What is Tool Calling?

Tool Calling is the capability-level name: at the right moment the model emits a machine-readable “I need an external capability,” instead of pretending in prose that it already queried the database. It is not tied to one vendor’s field names.

At least three 2026 implementations all get called Tool Calling: Function Calling on OpenAI-compatible APIs, vendor Tool Use (Anthropic and others), and MCP’s tools/list + tools/call. MCP Tool Schema vs Function Calling drew the boundary: MCP supplies tools across hosts; the API frame lets the model express this hop’s intent.

For developers, Tool Calling answers “can the model reach out?” It does not answer “who validates parameters and results after the hand reaches out?” That is Schema and the host.

What is Function Calling?

Function Calling (OpenAI docs now mostly say tools) is Tool Calling’s concrete shape inside a vendor Chat / Responses API. You inject function definitions; if the model decides to call, the response contains tool_calls.

A typical OpenAI-style tool definition looks like this. Note that parameters is already a JSON Schema subset:

{
  "type": "function",
  "function": {
    "name": "lookup_order",
    "description": "Look up an order by ID",
    "parameters": {
      "type": "object",
      "properties": {
        "order_id": {
          "type": "string",
          "description": "Order ID, e.g. A-1042"
        }
      },
      "required": ["order_id"],
      "additionalProperties": false
    }
  }
}

The call intent usually comes back like this:

{
  "tool_calls": [{
    "id": "call_3c91",
    "type": "function",
    "function": {
      "name": "lookup_order",
      "arguments": "{\"order_id\":\"A-1042\"}"
    }
  }]
}

arguments is still stringified JSON. The host must JSON.parse again and validate order_id against the Schema. Skipping that step is the most common 2026 production incident. When the nesting gets ugly, format and syntax-check them first so you can see the brackets and escapes.

How do Tool Calling and Function Calling differ—and how do you stack them?

They are not competing products. They sit on different layers:

Tool Calling: the idea. Any “model emits intent, host executes” counts.
Function Calling: the API frame. Fields are tools[], tool_calls, role: tool.
MCP Tool Schema: the supply protocol. The field is inputSchema, forwarded over stdio / SSE to an MCP Server.

The right stack: one Schema in the repo, exported as Function Calling parameters and MCP inputSchema. Do not keep a third “almost the same” parameter list in prose inside the prompt. Details in MCP Tool Schema vs Function Calling and Why AI Agents now need JSON Schema.

Why does every 2026 agent land on JSON?

Natural language is for talking to users. Every hop between machines must be validatable, Diff-able, and replayable. JSON carries five kinds of object:

Tool definitions: tools[] or MCP inputSchema.
Call intent: function name + arguments inside tool_calls.
Tool results: objects from queries, writes, and external APIs. This layer is the one most often missing.
Agent handoff: intent, confidence, who to route to. The Agents SDK routes on this object, not on the word “billing.”
Workflow state and evals: checkpoints, traces, eval fixtures. Long jobs resume from JSON; they do not guess progress from a chat log.

JSON Schema is the diameter of those objects. Without it, Function Calling, MCP, and A2A each drift into a “looks similar” copy. Why AI Agents now need JSON Schema turned Schema from documentation into a runtime gate; the syntax primer is the JSON Schema tutorial.

One full JSON pipeline

Take “look up an order → refund.” A conservative, testable 2026 chain looks like this:

The host injects Schemas for lookup_order / create_refund into tools[] and turns on strict: true.
The model returns tool_calls; arguments is still a string.
The host JSON.parses and validates against the same Schema; on failure it short-circuits and never touches billing.
The real lookup runs and returns an order object.
The host posts role: tool—preferably schema-valid JSON, not a paragraph.
The model either calls another tool or emits a Structured Output for the user or a downstream agent.
To change agents, it writes a handoff object instead of saying “please ask billing.”

A handoff object should stay this stable:

{
  "handoff": {
    "from": "triage",
    "to": "billing_agent",
    "intent": "refund",
    "order_id": "A-1042",
    "confidence": 0.91
  }
}

When several agents must discover each other, A2A Extensions puts capabilities on the Agent Card and negotiates with extension headers. It does not replace the pipeline above; it only dispatches across agents.

What should you pick in 2026?

A single app, a closed internal loop, fewer than ten tools: Function Calling is enough. Put Schema and strict: true in CI first.
An IDE / platform / third-party tools you must plug in: MCP supplies the tools; the API layer still uses Function Calling for this hop.
Heterogeneous agents that must discover and negotiate extensions: add A2A instead of inventing another private envelope.
On every path: write tool parameters, tool results, and handoffs as JSON Schema, then export adapters.

Do not wait for a keynote to “unify the protocol.” The stable 2026 move is layers, not betting on one winner.

Common pitfalls

Treating an agent as a chatty ChatGPT. A dialog can demo; production needs a recoverable state machine.
Treating Tool Calling and Function Calling as rival products. One is the idea, one is the API frame. MCP is a third supply layer.
Skipping validation of arguments. Stringified JSON, numbers-as-strings, and extra fields slipping into billing all happen on this hop.
Free-text tool results. The next hop will treat “looks successful” as the order status.
Stuffing 50 tools into one request. Huge schemas are expensive and raise mis-calls. Past ten tools, add routing or retrieval.

FAQ

Does an agent have to use Function Calling?

No. Function Calling is the most common API frame, but MCP Tool Use and vendor Tool Use are still Tool Calling. What matters is host execution plus Schema validation—not whether the field is named tools or inputSchema.

Is Tool Calling just the new name for Function Calling?

Some docs mix the two. The cleaner split: Tool Calling is the umbrella; Function Calling is the OpenAI-compatible tools / tool_calls shape. Draw them as separate boxes and most arguments disappear.

Is JSON mode enough if I skip Schema?

JSON mode only guarantees “it looks like JSON,” not legal fields, types, or enums. 2026 Structured Outputs pin the diameter with json_schema + strict. On a multi-step agent, JSON mode fails the eval set.

Should I wait for DevDay 2026?

No. 29 September may deepen Structured Outputs and call plans. It will not change “write the Schema first.” Get tools and handoffs into git now; after the keynote you only swap adapters.

How do I debug tool and handoff JSON?

Format it in the browser, check the missing required field, then Diff against the previous version. JSONSort runs locally and does not upload. Do not paste secret-bearing tool responses onto a public site.

Conclusion

An AI Agent is a runtime with tools and state, not a model that talks better. Tool Calling lets it reach out, Function Calling is the most common API shape of that hand, and JSON is the object that must match at every hop.

Start today with schemas/tools/lookup_order.json, not a longer prompt. Models will change and protocols will multiply. Stabilize the diameter first.