AI Development

Why Do AI Models Keep Getting Bigger? Parameters, Tokens, Context Windows, and JSON

JSONSort Published 2026-09-05 Updated 2026-09-07 About 16 min read

Hundreds of billions of parameters and million-token windows get sold as progress itself. For JSON developers, bigger usually means the same order object costs more, truncates sooner, and is harder to emit as a legal structure. This article splits three mixed-up words—parameters, tokens, context window—and lands on how to handle JSON in 2026.

Quick overview

Idea	One line	JSON angle
Parameters	Learnable weights — a rough measure of capacity	Affects how stably the model emits legal JSON
Token	The unit used to measure and bill text	Braces, quotes, and repeated keys all cost money
Context Window	Max tokens a single request can hold	Fat JSON and multi-step tool results hit it first

Bigger ≠ better at JSON. Parameters are capacity, tokens are the bill, the context window is the bucket. When the bucket grows, the first move should not be pouring in a whole API response.

Why do large models keep getting larger?

The empirical rule is simple: with data and compute keeping up, more parameters and longer training usually improve the loss curve. Vendors therefore default to “bigger” — it is easier to sell on a leaderboard, in a demo, and on a purchase list.

From 2024 to 2026, “large” split into three axes. It is no longer just a parameter count:

Parameter scale: dense models still grow; MoE keeps total parameters high but activates only a slice per token.
Context window: 8K and 32K became 128K, then a million. Agents and long documents are the usual excuse.
Output constraints: Structured Outputs / JSON Schema force “bigger” to mean “a more stable object,” not a longer paragraph.

If you write JSON, the useful questions are not “how many B is this model?” They are: how stably it hits a Schema, the price per thousand tokens, and whether your tool result still fits in the window.

What are parameters?

Parameters are the weights left in the model after training. At inference they no longer update; they only compute. Counts such as 7B, 70B, 405B (billion) describe capacity — not how many characters this request read.

More parameters usually mean three things:

Better fit to complex constraints. Nested objects, enum, and required break less often.
Costlier, slower inference. The same JSON costs more latency and money per token on a larger model.
Not the same as a larger context window. A small model can sit on 128K; a huge one may still be 32K.

Product pages in 2026 like to print MoE “total” and “active” parameters side by side. Ask which one the bill uses and which one the latency uses. As the GPT-6 Astra guide put it: for JSON work, stable Structured Outputs often beat one denser size class.

What is a token?

A token is a piece cut by the tokenizer — not a character and not a word. English often lands near 1 token ≈ 4 characters. Chinese, code, and JSON punctuation fragment more, so the same visual length costs more.

APIs bill by token and truncate by token. The system prompt, tool definitions, chat history, tool-returned JSON, and the model’s JSON output all share one ledger.

JSON is expensive by nature. Braces, quotes, commas, indent spaces, and repeated keys can each take a token. This “full order” is easy to read and wasteful to send:

{
  "order": {
    "order_id": "A-1042",
    "customer": {
      "full_name": "Ada Lovelace",
      "email": "ada@example.com",
      "locale": "en-GB"
    },
    "line_items": [
      {"sku": "KB-1", "title": "Keyboard", "qty": 1, "price": 99.00}
    ],
    "shipping": {"city": "London", "method": "express"},
    "debug": {"trace_id": "tr_9f", "retries": 0}
  }
}

For the next agent hop, the same business usually only needs:

{
  "order_id": "A-1042",
  "amount": 99.00,
  "status": "paid"
}

Format it locally with formatting so you can see the shape, pull order_id / amount / status with JSONPath extract, then minify before the model sees it. Pretty-print to debug; slim down to infer. Those are not the same payload.

What is a context window?

The context window is the token ceiling of a single request. Some vendors count input and output in one window; others reserve a slice for output. Until the docs are explicit, budget as “input + reserved output ≤ window.”

Growing from 8K to 128K to a million does not automatically make JSON easier. It only makes “paste the whole response” look feasible. Feasible is not the same as cheap or stable.

In an agent loop the window is usually eaten like this:

{
  "context_window": 128000,
  "spent": {
    "system": 1200,
    "tools": 3400,
    "history": 18000,
    "tool_result_json": 42000
  },
  "reserved_output": 4000,
  "remaining": 59400
}

The table is illustrative, not a vendor invoice. The point: tool-result JSON is often fatter than the prompt. A multi-step the AI Agent guide that never projects or summarizes will truncate a half-object by hop three — and the next hop will treat it as a full order.

How do parameters, tokens, and the context window stack?

The three words get flattened into “the model is bigger.” Their jobs are different:

Parameters set the quality ceiling: can it follow the Schema, or will it improvise past additionalProperties?
Tokens set the bill and latency of each hop. Fat JSON inside the same window crowds out history and output headroom.
The context window sets how much one call can see. A larger bucket cannot fix “too few parameters to hold the constraint” or “tokens too expensive to retry.”

The default 2026 incident: you bought a large window, poured in 2MB of pretty-printed JSON, then blamed the model for “forgetting” earlier fields. It did not forget. You filled the window first.

What does that mean for JSON processing?

On a daily pipeline the three layers become very concrete:

Pretty-print for humans, compact for the model. Indentation pays off in an editor and is pure waste on the token bill.
Project instead of pasting. If a tool returns 80 fields, keep only the keys the next hop marks required.
Schema is retry insurance. Structured Outputs trade validation for fewer illegal JSON samples — fewer output tokens.
Stringified arguments pay twice in shape. Escaped quotes are hard to read and spend budget; the host still has to parse again.
Long context is not a database. Query with a tool; compare with a local Diff. Do not stuff two 5,000-line files into one message.

Why AI Agents now need JSON Schema turned Schema into a runtime gate; the syntax primer is the JSON Schema tutorial. When versions drift, JSON Diff last week’s working object against today’s rejection before you buy a larger model.

How to cut cost in 2026 (fix the JSON, then the model)

Write an allow-list of fields per agent step instead of JSON.stringify(fullResponse).
Slim the tool definitions too: keep constraints in description, delete prose that never changes a call.
Turn on strict: true / json_schema and spend validation instead of retries.
Keep history as a state object, not the full tool transcript. Checkpoints are small JSON, not the chat log.
Inspect and trim secrets and user fields locally. Do not upload them to a public JSON site, and do not send them to the model without a reason.

The order is: project → compact → Schema → then consider a smaller or larger model. Reverse it and you waste both the window and the parameters.

Common pitfalls

Treating parameter count as the context window. 70B and 128K are two line items, even on the same slide.
Estimating tokens from character counts. JSON punctuation and Chinese will top out an object that “does not look long.”
Stopping projection because the window grew. A million tokens are still billed. Fat JSON only lengthens the invoice.
Buying a larger model to fix illegal JSON. Check the Schema and the host validator first. Many “the model got worse” incidents are one extra required.
Shipping pretty JSON as the live payload. Editor spaces should not appear in the inference request.

FAQ

Do more parameters make JSON more legal?

They help; they do not guarantee. Without a Schema and a host check, a large model still drops fields and writes numbers as strings. Capacity lowers the accident rate; the gate decides whether the accident reaches the next hop.

Can I convert tokens and characters at 4:1?

As a rule of thumb for English prose, yes. Not for JSON, code, or Chinese. Use the vendor tokenizer or the official counting API. Do not bill from the editor’s character count.

Does the context window include the model’s output?

Most Chat / Responses APIs count output in the same budget or set a max output. Worst-case: input + reserved output ≤ window, so the last closing brace is not cut off.

I already have a 1M window — should I still trim JSON?

Yes. The window answers “does it fit,” not “can you afford it” or “is the next hop still stable.” Projection and minify stay the default.

How do I check locally whether a JSON blob will blow the context?

Format it to see the shape, project the fields, then count the string you will actually send. JSONSort formats, extracts, and diffs in the browser and does not upload.

Conclusion

Models grow because more parameters, longer windows, and more data still sell on evals. If you process JSON, parameters are capacity, tokens are the bill, and the context window is the bucket. Pouring pretty JSON into a bigger bucket is just another way to overspend.

Start today with “which keys may enter this hop,” then decide whether you need a larger model. Models will keep growing. The diameter should get smaller first.