AI Development

Can an AI Agent Browse, Write Code, and Run Tasks on Its Own? From Browser Agent to JSON Tool Calling

JSONSort Published 2026-09-18 Updated 2026-09-18 About 15 min read

Product pages say “browses, writes code, and runs tasks autonomously.” The more accurate engineering sentence is: the model only fills JSON; the host opens the browser, edits files, and calls APIs. This piece walks from Browser Agent to JSON Tool Calling and answers three questions: does it do the work itself? Who actually does it? Which object should you inspect?

At a glance

Idea	One line	Lands on
Browser Agent	Structured actions against a page or desktop	computer_use / browser JSON
Coding Agent	Read a repo, edit files, run commands	tool_use / Bash
Task Agent	Call business APIs, write state, hand off	tool_calls / arguments
JSON Tool Calling	The contract for names, arguments, and results	Schema / payload

Remember the split: the model proposes objects; the host executes side effects. Swap the brand; the JSON call chain stays.

It does the work. It does not do the work “itself”

A 2026 agent can open pages, fill forms, and export; read a repo, edit files, and run tests; look up an order, start a refund, and hand off. Those are real side effects, not chat bubbles.

The word “itself” puts the subject in the wrong place. A large model does not own a browser process, does not spawn a shell, and does not hit your intranet. It only emits a validatable object: a tool name, arguments, sometimes a next intent. What actually clicks is the host.

Browse: the model emits browser_navigate or computer_call; the host's browser sandbox runs it.
Write code: the model emits Read / Edit / Bash; the host's workspace runs it.
Run a task: the model emits tool_calls; the host calls an API or a workflow.

If you still collapse Agent, Tool Calling, and Function Calling into one word, start with AI Agent complete guide. This article covers one fact: three product shells, one JSON call chain.

Three shells, one tool table

The market names products Browser Agent, Coding Agent, Computer Use, Operator, ChatGPT Agent. The shells differ. The table does not: what it can do depends on which tools the host exposes.

Browser Agent: tabs, screenshots, clicks, DOM extract. Best demo; easiest permissions to underestimate.
Coding Agent: repo, patch, tests, terminal. It changes disk and processes.
Task Agent: orders, tickets, calendars, refunds. It changes business systems.

Do not buy capability by brand. Buy it by the tool table. No browser tool, it cannot go online. No Bash, it cannot ship a runnable patch. No business function, it cannot refund.

JSON Tool Calling is the shared skeleton

Tool Calling, Function Calling, and computer_use have different names and the same shape. One turn is usually four hops:

Declare: the host offers tools[] or inputSchema.
Propose: the model fills a tool name and an argument object. It does not execute.
Gate: the host validates, applies an allowlist, and decides whether to wait for a human.
Write back: a screenshot, stdout, or an API response becomes the next visible object.

That is why Schema is worth more than a prompt. The prompt influences what it fills; Schema and the host decide what can happen. Contract in Why AI needs JSON Schema; protocol boundary in MCP Tool Schema vs Function Calling.

Browser Agent: the model is not the one going online

A Browser Agent looks most like “it browses on its own.” A tab really moves on screen. The action is still an object.

A structured Browser Agent—Playwright-style, Claude's browser tools, many MCP browsers—emits a validatable navigation:

{
  "type": "tool_use",
  "id": "call_nav_1",
  "name": "browser_navigate",
  "input": {
    "url": "https://docs.example.com/api/orders",
    "wait_until": "domcontentloaded"
  }
}

OpenAI's computer use / Operator is closer to coordinate actions. The model sees a screenshot or an accessibility tree and emits click, type, scroll:

{
  "type": "computer_call",
  "call_id": "call_8c12",
  "action": {
    "type": "click",
    "x": 412,
    "y": 268
  }
}

Do not read either shape as “the model saw the page, so it clicked.” What entered context is pixels or a tree; what clicked is the host. Lock the arguments, not the copy:

URL allowlist: default to docs, status pages, and domains you declared.
No extra headers: Cookie and Authorization must not enter the argument object.
Extract with a Schema: export fields, not a whole HTML page.
Keep the session on the host: login state is sandbox policy, not model weights.

A tightened navigation Schema looks like this. additionalProperties: false is not pedantry—a wide object lets the model slip in an intranet URL or a script:

{
  "name": "browser_navigate",
  "description": "Open a URL in the isolated browser",
  "inputSchema": {
    "type": "object",
    "properties": {
      "url": { "type": "string", "format": "uri" },
      "wait_until": {
        "type": "string",
        "enum": ["load", "domcontentloaded"]
      }
    },
    "required": ["url"],
    "additionalProperties": false
  }
}

GPT-6 Astra guide writes browsing and computer use as one capability. In engineering they are still tool JSON. A wide browser tool hands the session to the loop; for the security surface see AI Agent security and JSON validation.

Coding Agent: the model is not the one writing to disk

AI Coding Agents and the JSON call chain already opened Claude Code and GPT Agent. One line to align with Browser Agent: writing code is also filling JSON.

{
  "type": "tool_use",
  "id": "call_read_1",
  "name": "Read",
  "input": {
    "path": "src/billing/refund.py"
  }
}

The model does not open() a file. The host reads it and writes a tool_result. Only then can the next turn emit Edit or Bash.

The difference from browsing is only the tool surface: one side is url / selector / screenshot; the other is path / command / patch. The loop is the same.

Running a task: side effects live in the API, not the weights

“Run a task” is the phrase product copy swallows first. Billing refunds, ticket routing, calendar writes—those are Task Agents. The shape is still Function Calling:

{
  "tool_calls": [{
    "id": "call_task_3",
    "type": "function",
    "function": {
      "name": "create_ticket",
      "arguments": "{\"title\":\"Refund A-1042\",\"priority\":\"high\"}"
    }
  }]
}

The danger here is not prose. It is a wide object. Amount caps, enum, and additionalProperties: false belong in the Schema, not in “please be careful.”

Multi-agent handoff is JSON too. A handoff object is more auditable than “please give this to billing.” Capture tools and workflows as Schema—see JSON Schema and AI Agent workflows.

One turn is four hops

Product pages say “completes the job autonomously.” In engineering, Browser, Coding, and Task all walk the same four hops:

Observe: a screenshot, repo state, and the last tool result enter context.
Propose: the model emits a tool name and an argument object.
Execute: the host opens a page, edits a file, or hits an API. Side effects happen here.
Write back: a DOM summary, stdout, or a response body becomes the next visible object.

Skip a gate and JSON is a command. Auto-mode means you pre-approved a class of objects. Swap the shell; the loop stays.

Three tables, one chain

The question is not who is smarter. It is which tool table the host hangs on the loop:

Axis	Browser Agent	Coding Agent	Task Agent
What it sees	Screenshot / DOM / a11y tree	Files, tests, terminal output	Business objects, last-hop state
What it emits	navigate / click / type	Read / Edit / Bash	lookup / create / handoff
Where side effects land	Browser sandbox, login state	Workspace disk, processes	Business APIs, downstream systems
What you lock	URL, Cookie, extract Schema	cwd, timeout, network gate	Amount, enum, required fields

Swap the brand; the loop stays. Stabilize the tool contract and the execution world, not a vendor demo reel.

Inspect the objects, not the demo video

After an agent “browses, codes, and runs the job,” do not review the last sentence that says “done.” Review the trail.

format tool_use / tool_calls / computer_call and check that the name and arguments are the hop you thought.
Use JSONPath extract to pull url, command, and name. Do not scan prose by eye.
JSON Diff against a trail that passed last week. If the loop drifted, look at the tool table and argument shape before you blame the model.
Screenshots, stdout, and API responses belong on the trail. A sentence that says “completed” does not.
Do not paste cookies, sessions, or intranet URLs into a public formatter. JSONSort runs in the browser locally; that is the right place to read these objects.

If the trail can be rebuilt, the session can be reviewed. That is the entry ticket into production, not a nice-to-have.

Pitfalls

Thinking the model opened the page. The host's browser process opened it.
Thinking the model executed the code. Execution lives on the host, not in the weights.
Watching the demo and skipping the tool JSON. The click or command may already have happened.
Giving the browser a blank check. No URL gate and no cookie isolation means the loop is a session leak.
Replacing Schema with a prompt. The contract must be validatable.
Treating three agents as three kinds of intelligence. They are three tool tables, not three brains.

FAQ

Can an AI agent really browse on its own now?

It can open pages, click, and extract. It does not own the browser. The model fills JSON; the host sandbox runs it. No browser tool, it cannot go online.

What is the difference between a Browser Agent and Computer Use?

Both operate a UI. A Browser Agent often uses selectors, navigation, and DOM extract; Computer Use / Operator more often uses screenshots and coordinates. Both are structured calls—not a model that grew hands.

Are writing code and running tasks a different protocol?

No. A Coding Agent's Read / Bash and a Task Agent's Function Calling share the same JSON call chain as a Browser Agent. What changes is the tool table and the blast radius.

Why talk about JSON instead of a smarter model?

Because side effects happen after the object is parsed. A smarter model can still fill the wrong URL or amount. Schema and host gates decide what can happen.

Should an agent touch production login state or a production database?

No. Let it draft in an isolated browser and workspace, and keep the trail. Login state, secrets, and release stay human strategic gates. The loop drafts; the gate ships.

Conclusion

“An AI agent browses, writes code, and runs tasks” is only half true. The true half is that side effects already happen. The false half is the subject. The subject is the host, the verb is execute, the object is JSON the model filled.

From today, argue less about whether it “does it itself.” Put the Browser, Coding, and Task tool tables, the Schema, and the session trail in the repo first. Models turn over; this JSON Tool Calling chain should stabilize first. How incidents grow from the same chain: Claude incidents and tool JSON.