Why AI Agents Can't Live Without JSON: Tool Calling, Function Calling, and MCP Data Flow

Function Calling / Tool Calling / MCP JSON-RPC — four round-trips / schema dialects / observability and isolation

Open Cursor, Claude Code, or any Agent framework debug log and almost every payload is JSON: tool catalogs are JSON Schema, model tool_calls are JSON, MCP Server frames are JSON-RPC 2.0. That is not accidental — Agents must bridge unpredictable natural language and executable program interfaces, and JSON is the interchange tape format both sides can parse reliably.

This article walks Function Calling → Tool Calling → MCP and maps the four JSON round-trips in a complete tool invocation. If you are deploying an MCP Server on a cloud Mac or picking frameworks from our GitHub Agent roundup, understanding this flow beats memorizing API docs when debugging.

Agents speak JSON almost everywhere

LLMs emit token streams; Shell, databases, and HTTP APIs need structured arguments. Early practice was prompting “reply in this JSON shape” — models often added trailing commas, dropped quotes, or wrapped Markdown fences, and parsers failed constantly.

Since 2023, major APIs moved structured actions into the protocol: the model no longer freely emits arbitrary JSON text; the server constrains output shape and the Host executes. The pipeline becomes:

• Declare tools with JSON Schema;
• Decide tool name and parameters inside a constrained grammar;
• Execute via local code or MCP Server;
• Write back results as JSON messages into context.

Design principle: Agent observability = logging, replaying, and diffing every JSON step. Without structured intermediates, automation is not auditable.

Function Calling: from natural language to structured actions

The OpenAPI-era function object

OpenAI added functions / tools to Chat Completions: you attach tool definitions in the request; the model returns function_call or tool_calls instead of text you must regex.

Typical request snippet

{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Weather in Tokyo?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Current weather by city",
      "parameters": {
        "type": "object",
        "properties": { "city": { "type": "string" } },
        "required": ["city"]
      }
    }
  }]
}

Hand-written JSON templates in prompts are replaced by protocol constraints; the Host validates arguments against schema then calls real functions. Function Calling fixes what the model says — not where tools run, how they are discovered, or how auth works. Tool Calling ecosystems and MCP handle that.

Tool Calling: four JSON round-trips per turn

1. Register — Host sends tools[] with JSON Schema to the model API;
2. Decide — API returns tool_calls with id, name, arguments (JSON string);
3. Execute — Host parses arguments and calls local code or MCP;
4. Write back — role: tool messages return results; model produces the user-facing answer.

{
  "tool_calls": [{
    "id": "call_abc123",
    "type": "function",
    "function": {
      "name": "get_weather",
      "arguments": "{\"city\":\"Tokyo\"}"
    }
  }]
}

arguments is a string, not a nested object — for streaming and SDK compatibility. Hosts must JSON.parse(arguments) before validation. Press Ctrl + Shift + I in DevTools and you will see the same structure in Network.

API schema dialects

OpenAI / compatible endpoints

tools + tool_choice; streaming uses delta.tool_calls chunks.

Anthropic

tools with input_schema; responses use tool_use blocks and tool_result replies.

Google Gemini

functionDeclarations and functionCall with different field names.

Ollama local

OpenAI-compatible tools; capability depends on model training.

Frameworks like LangGraph and DeepSeek Harness (see our Harness guide) compress these dialects into one Tool abstraction — still JSON underneath.

MCP: JSON-RPC standardizes the tool layer

Model Context Protocol (MCP) sits below Function Calling: how tool processes are discovered, parameterized, and reused across Hosts. Transport may be stdio or SSE, but messages are always JSON-RPC 2.0.

Hosts call tools/list at startup, translate catalogs into API tools[], then send tools/call after the model decides — two dialect conversions, one semantic pipeline.

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "tools/call",
  "params": {
    "name": "read_file",
    "arguments": { "path": "/tmp/demo.txt" }
  }
}

Full data flow: user input to tool execution

• Do logs capture each tools/list version and tool count?
• Is arguments schema-validated before execution?
• Are MCP timeouts mapped to readable tool messages?
• Are secrets stripped before logging?

Same idea as the Verify stage in our AI Coding Workflow: without parseable intermediates you cannot auto-verify the Agent actually called the right tool.

Why not Protobuf or XML?

• Model APIs are JSON-first; binary adds serialization layers;
• Debugging — engineers read parameters in logs;
• Schema ecosystem — JSON Schema and OpenAPI are de facto standards;
• Scripts and browsers — curl, n8n, front-end demos assume JSON.

Tradeoff: size and CPU. High-QPS setups cache internally and compress logs; large blobs go base64 inside JSON fields — not replacing the JSON envelope.

JSON costs and engineering fixes

Production tip: run MCP on an isolated cloud Mac — Cursor on your laptop sends JSON-RPC over SSH; Shell never lives on your daily machine.

Agents depend on JSON because natural-language decisions and programmatic execution need a human-readable, machine-validatable, pluggable contract. Function Calling defines what the model says; Tool Calling defines how the Host runs a turn; MCP defines how tool processes plug into the ecosystem — JSON is the thread through all three.