为什么工具是 agent 的分界线

语言模型本质上是一个预测 token 的机器，它不能自己查天气、读文件、执行代码。工具提供了这样一层接口：模型输出一段结构化的调用，外部程序真正执行，结果再回到上下文。

工具带来的收益可以分成两类，这个划分决定了你该不该为某件事写工具：

第二类常被忽略。模型当然「会」算数，但它要花很长的推理链，而计算器一次调用就能给出确定结果。能用确定性程序替代的概率性推理，通常都应该替换掉。

工具的主要类别

课堂上把常见工具归成几类：

• 文本回复：可以显式提供一个 finish 工具，也可以约定「不调用工具就结束循环」。
• 网页搜索 / 检索：把外部信息取回来再生成。课堂上有一个判断：RAG 不是过时了，而是普及到大家不再意识到自己在用它。
• 代码执行：计算、数据处理、验证。
• 图像生成：与文本工具不同，它提示了一个额外问题——模型需要自己改写查询，最终送给图像模型的描述往往和用户原话差别很大。
• 业务 API：下单、建购物车、调用企业内部系统。

时间轴

代码即动作：一个特殊的元工具

常规做法是准备几十个 API，模型每一步调用一个。代码则不同：它本身极其表达力强，可以把许多工具调用组织成一棵树，还能带循环、变量和第三方库。

课堂用同一个任务对比了两种写法：查四个国家的手机价格、汇率和税费，算出最划算的购买地。

• 逐步调用：查汇率 → 查价格 → 换算加税 → 再查汇率 → 再查价格……轮数很多。
• 写成一段程序：一次生成完整逻辑，环境执行后直接得到结果。

课程引用的研究显示，代码式工具调用能提高成功率、减少交互轮数，即使任务本身看起来不需要编程。

代价

代码是「高功率」动作，带来四类风险：

1. 可能卡死：无限循环需要外部超时机制。
2. 依赖供应链：引入的第三方库可能已被投毒，agent 被攻破会连带整个系统。
3. 难以约束：动作空间大，校验和可预测性都更差。
4. 资源影响大：CPU、内存、磁盘的消耗远高于一次 API 调用。

所以一旦允许代码执行，沙箱就不再是可选项，而是前置条件。

工具调用的机械过程

从特殊 token 到函数调用

模型看到的不是「函数」，而是一段带特殊 token 的文本。不同厂商的格式并不相同：Qwen 有自己的 tool call 标记，Mistral 有另一套，DeepSeek 用 DSML 风格的 function call。

这些特殊 token 通常不在预训练阶段引入，而是在中期训练时加入，再通过指令微调与强化学习让模型学会使用。实际工程中必须使用模型自带的 chat template（Hugging Face 的 apply_chat_template 就是做这件事），否则格式不匹配，性能会明显下降。

分发与执行

一次工具调用的完整链路是：

1. 模型输出调用（函数名 + 参数）；
2. 脚手架解析参数；
3. 按名称在注册表里找到对应函数；
4. 校验参数；
5. 执行；
6. 把结果作为观察写回上下文。

失败分两种：

• 校验失败：参数不符合 schema，或者调用了不存在的工具。
• 执行失败：参数形式合法但语义无效，例如一段能通过解析、运行时却报错的 Python。

每次调用会分配一个 ID，返回结果带着同一个 ID。这在并行调用时是必需的。还有一个容易踩的坑：某些 API（例如 Anthropic）要求历史里每个工具调用都有匹配的结果，否则整段历史会被判为非法。程序崩溃导致结果没写回时，恢复会话就会失败。

约束解码：让输出在语法上必然合法

模型不保证生成合法的 JSON。少一个右括号，解析器就会抛异常。手工补括号不是工程做法。

约束可以分三层：

JSON Schema 能表达这三层。它对应的形式语言不是正则文法，而是上下文无关文法：正则表达式可以用有限状态自动机匹配，JSON 的嵌套结构需要带栈的下推自动机。

解码时如何生效

以 XGrammar 这类实现为例：每生成一个 token，就用当前语法状态检查候选词表，合法 token 保留，非法 token 的 logit 设为负无穷，再重新归一化。这样采样只会在合法集合里进行。

工程上会预计算「永远合法」「永远非法」的 token，只对依赖栈状态的 token 现场判断，以降低开销。

它保证不了什么

约束解码只保证形式正确，不保证语义正确。一段语法合法的 Python 仍然可能是错程序；一个 schema 合法的城市名仍然可能不存在。

还有一个边界情况：当输出 token 预算耗尽时，模型可能停在一条合法路径的中间，而不是终止状态。此时语法没错，调用却是不完整的。

三种接入方式：REST、代码库与 MCP

REST API

最传统的方式。用 FastAPI 写一个后端，加一个 API key 做鉴权，就得到一个远程可调用的工具。它有两个附带好处：

• 可以限制谁能调用，避免把私有数据暴露给所有人；
• 框架会自动生成 JSON Schema，这份 schema 可以直接喂给模型。

如果 agent 有 bash，也可以直接用 curl 调接口。

本地函数库

在程序化工具调用里，工具就是一个 Python 模块，模型写代码 import 它。这种方式简单，但不可远程调用，也无法用于标准的 JSON 工具调用流程。

MCP：多一层凭据隔离

Model Context Protocol 刚出现时，课堂上的第一反应是「为什么需要它，直接调 API 不就行了」。后来认可的理由主要是安全分层：

• 上游凭据（例如 GitHub token）交给 MCP server；
• agent 只拿到 MCP API key。

这样即使 agent 把 key 泄露到公开仓库，泄露的也只是通往 MCP server 的钥匙，而不是 GitHub 账号本身。

MCP 还提供工具发现、标准传输（stdio 或 socket）、资源与提示等约定，并有官方注册表。fastmcp 的定位相当于 MCP 版的 FastAPI。

并行工具调用

如果多个调用彼此独立——查天气、读日历、查航班——就没有理由串行。做法是在一次回复里放多个工具调用块，由 asyncio.gather 一类机制并发执行。

前提是调用之间没有依赖。「先查客户 ID，再用它查订单，再退款」这种链条不能并行。

课堂上有一个反直觉的观察：更贵的模型按任务算可能更便宜。原因有两个——更强的模型更快选对方案；新模型更擅长并行调用。而并行能力的大幅提升，很大程度上是强化学习逼出来的：训练时对任务耗时施加强惩罚，模型就学会了一次发多个调用。

如何评估工具使用

端到端任务评估之前，先要评估工具调用本身。最常用的是 Berkeley Function Calling Leaderboard（BFCL），已迭代到第四版，覆盖单轮、多轮、agentic 工具调用与鲁棒性（参数幻觉、格式敏感性）。

评估维度可以拆成：

• 选对工具：有没有选到根本做不到该任务的工具；
• 参数正确：值、单位、类型是否正确；
• 顺序正确：多步调用是否按依赖关系执行；
• 端到端准确率；
• 效率：耗时、token、成本；
• 可靠性：超时是否重试、工具失败能否换路；
• 安全：面对对抗性请求会不会滥用工具。

同一个模型，不同供应商，差距巨大

OpenRouter 的工具调用错误率基准测试了同一个模型在不同推理供应商上的表现，错误率从约 0.01% 到 15% 不等。原因包括：

1. 量化程度：FP8 通常比 FP4 的错误率低得多，后者省钱但更伤质量；
2. 投机解码：无损版本不应改变结果，有损版本会；
3. 约束解码：有的供应商实现了语法约束，有的完全依赖模型自觉；
4. 服务可靠性：超时与宕机也会被计为失败。

这条结论对工程选型很实际：「用了同一个模型」不等于「得到同一个 agent」。

本讲结论

• 工具把模型的能力边界从「生成文本」扩展到「改变世界」。
• 代码是表达力最强的工具，也是风险最高的工具，必须配沙箱。
• 工具调用是一条完整链路：格式、解析、校验、分发、执行、结果回填。
• 约束解码能保证语法合法，但不能保证语义正确。
• MCP 的核心价值是凭据隔离与标准化，而不只是「又一个 API」。
• 并行调用能显著降低延迟，前提是调用之间没有依赖。
• 评估要同时看选择、参数、顺序、效率、可靠性与安全。

理解检验

1. 什么样的任务应该写成工具，而不是让模型「自己算」？

参考答案：先看工具带来的是哪一类收益。扩展能力：模型从根本上做不到的事——查四个月前的实时信息、读私有状态、修改外部环境，只能靠工具。提供便利：模型「能做」但概率性推理又慢又不稳，例如七位数乘法、精确统计、格式校验，确定性程序一次调用就能给出可靠结果。判断标准可以写成一句：只要存在能确定性完成该子任务的程序，就应优先用它替换模型的概率性推理。代价也要计入：工具越多，系统提示与动作空间越大，上下文与安全成本都上升。

2. 为什么 JSON Schema 需要下推自动机而不是有限状态自动机？

参考答案：JSON 允许对象与数组任意深度嵌套，一段文本是否合法取决于括号、花括号是否按正确顺序配对，这要求解码器记住「当前还有几层没有闭合」。有限状态自动机只有有限个状态，无法记录任意深度的嵌套；JSON 属于上下文无关语言，需要带栈的下推自动机。正则表达式对应有限状态自动机，只能描述无嵌套或固定层数的模式，所以约束解码的实现（如 XGrammar）按上下文无关文法维护栈状态。

3. 约束解码在什么情况下仍然会产生不完整的工具调用？

参考答案：主要情形是输出 token 预算耗尽。此时模型可能停在一条合法路径的中间——例如已经写出函数名与左括号，但还没到语法要求的终止状态。语法检查不会报错，因为到目前为止的每个 token 都合法，但这次调用缺少闭合括号或必填字段，无法执行。另一个相邻问题是约束只保证形式正确：schema 合法并不保证语义正确，例如枚举内的城市名仍然可能不存在、语法合法的 Python 仍然可能运行时报错。

4. MCP 相比直接给 agent 一个 GitHub token，安全性提升在哪里？

参考答案：核心是凭据分层隔离。直接给 token 时，agent 的上下文或代码里就带着上游凭据，一旦泄露到公开仓库或被提示注入诱导写出，攻击者拿到的是 GitHub 账号本身。经 MCP 时，上游凭据交给 MCP server 保管，agent 只持有 MCP 的 API key；泄露的只是通往 server 的钥匙，可以单独撤销、限权、审计，不影响 GitHub 账号。MCP 还顺带提供了工具发现、标准传输（stdio 或 socket）与资源约定。

5. 两个供应商都声称托管同一个模型，为什么工具调用错误率可能相差三个数量级？

参考答案：课堂列的差异来源有四条：(1) 量化程度——FP8 通常比 FP4 错误率低得多，后者便宜但更伤质量；(2) 投机解码——无损版本不应改变结果，有损版本会改变；(3) 约束解码——有的供应商实现了语法约束，有的完全依赖模型自觉；(4) 服务可靠性——超时与宕机会被计入失败。所以「用了同一个模型」不等于「得到同一个 agent」，选型时要看实际的工具调用错误率，而不是模型名字。

与后续课程的关系

这一讲建立了工具层。下一讲处理长上下文：当历史、工具定义与观察不断累积，如何让模型仍然保持一致。再往后是记忆与技能、规划，以及把工具调用用于训练与评估的完整闭环。

Why tools define the boundary

A language model predicts tokens. It cannot check the weather, read a file, or run code on its own. A tool is the interface that closes the gap: the model emits a structured call, an external program executes it, and the result returns to the context.

Tools pay off in two distinct ways, and the distinction tells you when a tool is worth writing:

The second category is easy to overlook. A model can "do" arithmetic, but only through a long chain of reasoning. A calculator returns a deterministic answer in one call. Whenever a deterministic program can replace probabilistic reasoning, it usually should.

The usual categories

• Text response. Either an explicit finish tool, or the convention that not calling a tool ends the loop.
• Web search and retrieval. Fetch outside information, then generate. The lecture's aside: RAG is not dead; it became so normal that people stopped noticing they use it.
• Code execution. Computation, data processing, verification.
• Image generation. Unlike text tools, the model rewrites the prompt substantially; the description sent to the image model often differs a lot from the user's words.
• Application APIs. Ordering, cart creation, internal business systems.

Timeline

Code as a meta-tool

The conventional design exposes dozens of APIs and the model calls one per step. Code is different: it is expressive enough to organize many calls into a single tree, with loops, variables, and libraries.

The lecture compares two ways to answer one question: find the cheapest country among four to buy a given phone, accounting for price, exchange rate, and tax.

• Step-by-step tool calls: look up the rate, look up the price, convert, add tax, look up the rate again, and so on for many turns.
• A single program: generate the whole procedure once, execute it, read the answer.

The cited work finds that code-style tool calling raises success rates and reduces the number of interaction turns, even on tasks that did not obviously need programming.

What it costs

Code is a high-powered action, and that brings four risks:

1. It can hang. Infinite loops require external timeouts.
2. Dependencies are an attack surface. A pulled-in library may be compromised, and then the agent and the host are compromised with it.
3. It is hard to constrain. A large action space makes validation and predictability harder.
4. Resource impact is larger. CPU, memory, and disk use far exceed a single API call.

Once code execution is allowed, a sandbox stops being optional and becomes a precondition.

The mechanics of a tool call

Special tokens, not function objects

The model does not see functions. It sees text with special tokens, and the format differs by vendor: Qwen has its own tool-call markers, Mistral another, DeepSeek uses a DSML-style function call.

These tokens are usually introduced during mid-training rather than pre-training, then reinforced through instruction tuning and RL. In practice you must use the model's own chat template (Hugging Face's apply_chat_template does exactly this). Mismatched templates noticeably degrade performance.

Dispatch and execution

A complete call passes through six stages:

1. the model emits a call (name plus arguments);
2. the harness parses the arguments;
3. the name is resolved in a registry of tools;
4. arguments are validated;
5. the function runs;
6. the result is appended to the context as an observation.

Failures divide into two kinds:

• Validation failure: arguments violate the schema, or the tool does not exist.
• Execution failure: the arguments are well formed but semantically wrong, such as Python that parses and then raises.

Every call is assigned an ID and its result carries the same ID, which is essential for parallel calls. One practical trap: some APIs (Anthropic's, for instance) reject a history in which a tool call has no matching result. If a crash loses the result, resuming the conversation fails.

Constrained decoding

Models do not guarantee valid JSON. A missing closing brace makes the parser throw, and patching brackets by hand is not engineering.

Constraints come in three layers:

JSON Schema expresses all three. The corresponding formal language is not regular: regular expressions are matched by finite-state automata, while nested JSON needs a pushdown automaton with a stack.

How it applies during generation

In an implementation such as XGrammar, every time the model proposes logits for the next token, the current grammar state checks the vocabulary. Legal tokens keep their scores, illegal tokens are set to negative infinity, and the distribution is renormalized. Sampling can only land on legal tokens.

For speed, tokens that are always legal or always illegal are precomputed; only tokens whose legality depends on the stack state are checked on the fly.

What it does not guarantee

Constraint decoding guarantees form, not meaning. A syntactically valid Python program can still be wrong. A schema-valid city name can still be fictitious.

There is also an edge case: when the output token budget runs out, the model may stop mid-path in a legal but non-terminal state. The syntax is fine; the call is incomplete.

Three ways to expose a tool

REST APIs

The traditional route. A FastAPI backend plus an API key gives a remotely callable tool. Two side benefits: access can be restricted, and the framework generates a JSON Schema that can be handed straight to the model. If the agent has bash, curl works too.

Local function libraries

In programmatic tool calling, a tool is a Python module the model imports in generated code. Simple, but not remotely callable and not usable through the standard JSON tool-call path.

MCP: a credential boundary

The first reaction to the Model Context Protocol was "why not just call the API". The accepted reason is security layering:

• the upstream credential (a GitHub token, say) goes to the MCP server;
• the agent receives only an MCP API key.

If the agent leaks its key into a public repository, the leak exposes a key to the MCP server rather than the GitHub account itself.

MCP also standardizes discovery, transport (stdio or socket), resources, and prompts, and has an official registry. fastmcp is positioned as FastAPI for MCP.

Parallel tool calls

When calls are independent — weather, calendar, flight status — there is no reason to serialize them. The model puts several call blocks in one response, and the harness runs them concurrently with something like asyncio.gather.

The precondition is no dependencies between calls. "Look up the customer ID, fetch their orders, refund one" cannot be parallelized.

A counterintuitive observation: a more expensive model can be cheaper per task. It picks the right approach faster, and newer models parallelize much better. The parallelization itself was largely forced by reinforcement learning: penalize wall-clock duration heavily and models learn to issue many calls at once.

Evaluating tool use

Before end-to-end agent evaluation comes tool-call evaluation. The standard is the Berkeley Function Calling Leaderboard, now in its fourth version, covering single-turn, multi-turn, agentic tool calling, and robustness (argument hallucination, format sensitivity).

Useful dimensions:

• Tool selection: did it pick a tool that can actually do the task?
• Arguments: are values, units, and types right?
• Ordering: do multi-step calls respect dependencies?
• End-to-end accuracy.
• Efficiency: latency, tokens, cost.
• Reliability: retries on timeout, fallback when a tool fails.
• Safety: resistance to adversarial requests to misuse tools.

Same model, different provider, very different results

An OpenRouter benchmark measured tool-call error rates for one model across inference providers. The spread ran from roughly 0.01% to 15%. Causes include:

1. Quantization: FP8 is typically much better than FP4, which saves money at a real quality cost.
2. Speculative decoding: lossless should not change results; lossy can.
3. Constrained decoding: some providers implement grammar constraints, others rely on the model alone.
4. Reliability: timeouts and outages also count as failures.

The practical lesson: "same model" does not mean "same agent".

Takeaways

• Tools extend the model from generating text to changing the world.
• Code is the most expressive tool and the riskiest; it requires a sandbox.
• A tool call is a full pipeline: format, parse, validate, dispatch, execute, return.
• Constrained decoding guarantees syntax, not semantics.
• MCP's core value is credential isolation and standardization, not merely another API.
• Parallel calls cut latency when calls are independent.
• Evaluate selection, arguments, ordering, efficiency, reliability, and safety.

Check your understanding

1. When should a task become a tool instead of something the model computes itself?

Reference answer: First ask which kind of benefit the tool provides. Extended capability: things the model fundamentally cannot do — fetch information from four months ago, read private state, modify an external environment. Convenience: things the model "can" do but does slowly and unreliably with probabilistic reasoning — seven-digit multiplication, exact statistics, format validation. The rule of thumb: whenever a deterministic program can perform the subtask, prefer it over the model's probabilistic reasoning. The cost also counts: more tools mean a larger system prompt and action space, raising both context and safety costs.

2. Why does JSON Schema require a pushdown automaton rather than a finite-state machine?

Reference answer: JSON permits objects and arrays nested to arbitrary depth, so whether a string is valid depends on brackets and braces closing in the right order. The decoder must remember how many levels are still open. A finite-state machine has only finitely many states and cannot record unbounded nesting; JSON is a context-free language, which needs a pushdown automaton with a stack. Regular expressions correspond to finite-state machines and can only describe patterns with no nesting or with fixed depth — which is why constrained-decoding engines such as XGrammar track grammar with a stack.

3. Under what condition does constrained decoding still produce an incomplete call?

Reference answer: Mainly when the token budget runs out. The model may stop in the middle of a valid path — say, after emitting the function name and an opening parenthesis but before the grammar's accepting state. Every token emitted so far is legal, so no syntax error is raised, yet the call lacks a closing bracket or required field and cannot execute. A related limitation: the constraint only guarantees form, not meaning — a schema-valid city name may not exist, and a syntactically valid Python snippet may still fail at runtime.

4. What security property does MCP add over handing the agent a GitHub token?

Reference answer: Layered credential isolation. With a raw token, the upstream credential sits in the agent's context or code; if it leaks into a public repository or is exfiltrated through prompt injection, the attacker obtains the GitHub account itself. With MCP, the upstream credential stays in the MCP server and the agent holds only an MCP API key. A leaked key is just a key to the server: it can be revoked, scoped and audited without exposing the GitHub account. MCP also standardizes tool discovery, transport (stdio or socket) and resource conventions.

5. Why can two providers of the "same model" differ by orders of magnitude in tool-call error rate?

Reference answer: The lecture lists four sources: (1) quantization — FP8 is typically far less error-prone than FP4, which is cheaper but degrades quality; (2) speculative decoding — lossless versions should not change results, lossy ones do; (3) constrained decoding — some providers enforce the grammar, others rely on the model behaving; (4) service reliability — timeouts and outages are counted as failures. So "same model" does not mean "same agent": choose by measured tool-call error rate, not by model name.

How this connects to the rest of the course

This lecture builds the tool layer. The next one handles long context: what happens when history, tool definitions, and observations keep accumulating. Then come memory and skills, planning, and the training and evaluation loop that turns tool use into a learned capability.