Why AI Agents Now Need JSON Schema: From Tool Calling to Structured Agent Workflows

Quick overview

Stage	Without Schema	With Schema
Single Tool Calling	Arguments are strings; types are luck; extra fields still run	parameters / inputSchema intercept before the call
Multi-step agents	Downstream parses prose; handoff depends on mentioning “billing”	Handoff objects carry output_type; routing is testable
Cross-protocol reuse	Separate copies for Function Calling, MCP, and A2A drift apart	One Schema in git, exported as three adapters

Up front: JSON Schema is not “documentation for the model.” It is the validator the agent host runs before every step, at every handoff, and in every eval.

Why “now,” not “always”?

Early Tool Calling survived because the chain was short: the model emitted a function name and some arguments, the host executed them, and the result went back into the conversation. Failures retried in view of the user. Schema, if it existed at all, lived in a README—nice to have.

In 2025–2026 the chain got longer. A ticket agent may classify first, then look up an order, then decide on a refund or a human handoff, then pass an object to another agent. If any hop is only “JSON-shaped,” later steps treat the error as fact. Schema becomes a runtime gate: if it fails, the workflow does not continue.

A second pressure comes from the protocol layer. As we wrote in MCP Tool Schema vs Function Calling: vendor APIs use parameters; MCP uses inputSchema. A2A Extensions put capabilities on the Agent Card. The three protocols do not share a runtime, but they all accept a JSON Schema subset. If you do not pick Schema first, you will maintain three “almost the same” definitions—and in week three discover that order_id is a string in one place and a number in another.

DevDay 2026 predictions put the same point more bluntly: Structured Outputs, Function Calling, and the agent runtime are collapsing into one JSON pipe. Schema is the pipe’s diameter. Leave it undefined, and every protocol you attach later leaks.

Tool Calling already looks “structured”—so where is the gap?

Function Calling already carries a Schema-like parameters object inside tools[]. Many teams conclude they are already structured. Production incidents usually fall into four buckets.

First, arguments is still a string. The model returns:

{
  "tool_calls": [{
    "id": "call_8f2a",
    "type": "function",
    "function": {
      "name": "create_refund",
      "arguments": "{\"order_id\":\"A-1042\",\"amount\":\"99.00\",\"reason\":\"late\"}"
    }
  }]
}

The host must JSON.parse again. Whether amount is a string or a number, whether reason is in the enum, whether an extra note can enter the billing system—if parameters is not enforced on the host, all of that depends on “the model’s mood today.” Writing a Schema does not make it take effect. strict: true only constrains the model side; the host still has to validate.

Second, tool results have no Schema. role: tool is usually a free string. The next model step “understands” that prose again. Query success, zero inventory, and insufficient permissions look different; a multi-step workflow will fork onto the wrong branch by hop three.

Third, types are too wide to filter anything. A priority that is only type: string lets the model emit "urgent!!!". A Schema without enum, required, and additionalProperties: false is a comment.

Fourth, a single successful call hides a workflow failure. Step one lookup_order adds a status: "ok"; the refund agent in step two treats status as the order status. A local happy-path run will not catch it; the regression set will. Schema’s value is here: it turns “the object this hop can hand to the next hop” into an assertable test, not a chat log.

When you debug nested JSON like this, first format and syntax-check it locally, then Diff it against last week’s working definition. Do not paste tool responses that contain user data or secrets onto a public JSON site.

JSON Schema actually plays four roles in an agent

Treat Schema only as “the tool-parameter spec” and you miss the three layers that actually grew in 2026. The full picture:

1. Tool inputs. Function Calling parameters, MCP inputSchema. They decide what the model “is allowed to request.”
2. Tool outputs. Return objects from queries, writes, and external APIs. They decide what the next hop “is allowed to trust.” This layer is the one most often missing.
3. Agent handoff. Classification, intent, confidence, who to route to. In the Agents SDK this is the routing input, not a chat summary.
4. Workflow state and evals. Checkpoints, traces, eval fixtures. Long jobs must resume from JSON, not guess progress from a conversation log.

Input Schema example (refund tool):

{
  "$id": "https://example.com/schemas/tools/create_refund.json",
  "type": "object",
  "properties": {
    "order_id": { "type": "string", "pattern": "^A-[0-9]{4,}$" },
    "amount": { "type": "number", "exclusiveMinimum": 0 },
    "reason": { "type": "string", "enum": ["late", "damaged", "duplicate", "other"] }
  },
  "required": ["order_id", "amount", "reason"],
  "additionalProperties": false
}

Handoff Schema example (triage → billing):

{
  "$id": "https://example.com/schemas/handoffs/triage_to_billing.json",
  "type": "object",
  "properties": {
    "intent": { "type": "string", "enum": ["refund", "exchange", "human"] },
    "order_id": { "type": "string" },
    "confidence": { "type": "number", "minimum": 0, "maximum": 1 },
    "from": { "const": "triage" },
    "to": { "const": "billing_agent" }
  },
  "required": ["intent", "order_id", "confidence", "from", "to"],
  "additionalProperties": false
}

What the two files share matters more than how they differ: both are objects, both use enums, both forbid extra fields, both belong in git, and both can run under ajv (or an equivalent) in CI. Change the model snapshot or switch MCP for Function Calling—the validator does not need a rewrite. For how to write the keywords, see the JSON Schema tutorial.

From a single call to a structured workflow: where Schema sits at the gates

Single Tool Calling has one gate: are the arguments valid? A structured agent workflow has at least four.

Gate 1: before tool selection. Past ten tools, stuffing the full Schema into one request is expensive and raises mis-calls. When you retrieve first, then call, the retrieval result itself should be a Schema-backed shortlist—not a model-generated tool blurb.

Gate 2: before execution. parse → validate → execute. On validate failure, return a structured error (missing field, wrong type, outside the enum). Do not pass the raw string to the next hop. This step also blocks extra keys from prompt injection.

Gate 3: before handoff. Triage must emit an object that matches the handoff Schema before billing starts. Write the route as intent === "refund" && confidence >= 0.8, not “the reply mentioned a refund.”

Gate 4: resume and evals. Long jobs write checkpoints as valid JSON:

{
  "run_id": "run_20260903_014",
  "step": "await_refund_ack",
  "state": {
    "order_id": "A-1042",
    "intent": "refund",
    "amount": 99.0
  },
  "schema": "https://example.com/schemas/state/refund_flow.json"
}

Eval sets are the same kind of object: given an input, assert the output passes the Schema and that key fields are equal. When an agent “got dumber,” Diff this week’s Schema and fixtures against last week’s before you blame the model. Many incidents are one extra required, or one missing enum value.

How one Schema feeds Function Calling, MCP, and A2A

Do not write a separate Schema per protocol. The recommended shape is one spec file, three projections:

Watch subset differences when you project: vendor strict mode often requires every property in required, and disables some oneOf / $ref. The spec file can be richer; the exporter trims per target, and CI runs a structural Diff of “spec vs projection” so nobody hand-edits one side.

MCP owns tool supply, Function Calling owns how the model expresses call intent, and A2A owns how agents discover and negotiate. Schema does not replace those three layers; it is the field dictionary they share. That matches the conclusion of the two earlier articles—except the “dictionary” is now a file in the repo, not a concept.

In practice: treat JSON Schema as the single source of truth

Things you can do immediately, without waiting for any keynote:

• Create schemas/tools/, schemas/handoffs/, and schemas/state/ in the repo. Give each file a stable $id and a version.
• Export Function Calling / MCP / Structured Outputs adapters with a script. Do not hand-edit fields in the adapter files.
• Validate in CI with ajv (or an equivalent): example payloads, historical fixtures, and production samples (after redaction) must pass the current Schema.
• Default new tools to additionalProperties: false and an explicit enum. When you need open extension, add a separate metadata object—do not loosen the root.
• Past ten tools, add routing or retrieval. Do not dump the whole catalog into one request.
• Accept arguments as either a string or an object on the host, so a vendor type tightening does not page the whole site.
• Inspect secret- or user-bearing payloads only in the browser: format, Diff, zero upload.

The directory can be thin. What matters is files first, then the prompt. “Please return JSON” in a prompt cannot fail in CI. A Schema can.

Common pitfalls

• Putting the Schema in the system prompt and calling it done. Model-side constraints and host-side validation are two jobs. Without host validation, the contract vanishes the moment strict is off or the snapshot changes.
• Validating inputs only, not outputs or handoffs. Multi-step agent incidents happen more often when “the previous step returned an unexpected shape.”
• Hand-writing a “close enough” Schema per protocol. They will fork by week three. Export; do not copy.
• Using a wide type: object to dodge design. An open object is no gate. For real extension, use a version or an explicit metadata field.
• Treating Schema as model training data. It exists first for validators and tests. Clear descriptions help tool selection; they do not replace enum and required.
• Changing fields on a visual canvas without writing them back to git. Nodes on a canvas are not the source of truth. Export Schema first; let the canvas read the files.

FAQ

Can Tool Calling ship without a Schema?

For a prototype, yes. When tools are fixed, steps are under three, and a human can see failures, string arguments are often enough. The moment you add branches, automatic retries, or a second agent, Schema-less errors propagate silently.

Will Structured Outputs or MCP replace JSON Schema?

No. They consume Schema; they are not another type system. The 2026 trend is more protocols citing the same Schema, not inventing a new field language.

Should I use draft-07 or 2020-12?

Follow your validator and the vendor subset. Agent work actually uses type, properties, required, enum, and additionalProperties. A subset that aligns CI and the model API matters more than chasing the latest draft.

Do private internal APIs still need MCP and A2A?

Not necessarily. A closed internal loop can use Function Calling + Schema. The value of leaving room for MCP / A2A is this: once the same Schema lives in the repo, adding a protocol later is an exporter, not a new field vocabulary.

How should I debug failed tool / handoff JSON?

Format it in the browser, check the Schema for the missing required, then Diff against the previous version. JSONSort does not upload data. Do not paste secret-bearing responses onto a public site.

Conclusion

AI agents “now” need JSON Schema not because models suddenly cannot read natural language, but because workflows got longer, protocols multiplied, and errors have to be stopped on the host. Tool Calling expresses call intent; Structured Outputs produce the final object; MCP and A2A handle supply and negotiation. What glues those layers together is the set of Schema files in the repo that you can validate, Diff, and export.

Starting today, write schemas/tools/create_refund.json before you write a longer prompt. Models will change and protocols will shift. The diameter should stabilize first.

Further reading

• What is JSON Schema? Usage and tutorial Keywords, required, and additionalProperties
• MCP Tool Schema vs Function Calling Protocol boundary for tool JSON
• A2A Extensions and the JSON protocol The third layer of multi-agent interop
• OpenAI DevDay 2026 key predictions How three layers become one JSON pipe
• JSONSort local toolbox Format, validate, Diff
• All blog posts More JSON and AI development notes

Changelog: 2026-09-03 initial release