先区分三种「写代码」

1. 单步生成：给一段自然语言，生成一段程序。
2. 仓库级修改：读多个文件，修一个 issue 或加一个功能。
3. 完整软件开发：还包括审查、部署、监控、维护与需求。

这一讲主要覆盖第二类与第三类中的关键工程问题。第三类常被忽略：微软 2019 年的内部调查显示，开发者只有约 15% 的时间在写代码，25% 在开会与邮件，14% 在调试，8% 在跑测试。

时间轴

一个模型要会写代码，需要什么

单步能力可以拆成三块：

「编辑」不是免费的。模型需要在训练中见过大量「修改前 → 修改后」的样本，才能可靠地改代码。

预训练

代码必须在语料里，但不能只有代码——模型还要理解自然语言指令。课堂强调的细节：

• 空白有意义：Python 的缩进不能被归一化掉；
• 文件边界：如何表示一个文件的开始与结束；
• 块结构：Markdown 代码块在语料中要保持完整；
• API 与文档关系：代码与解释它的文字要一起出现。

数据清洗

代码的分布很特殊，清洗比文本模型更麻烦：

• 过滤语言与来源，去掉自动生成、编码错误、低价值的重复代码；
• 去重尤其重要：GitHub 上有大量 fork、vendored 库、几乎相同的文件与复制的题解；
• 记录许可证、源码日期、opt-out 与敏感数据。API key 泄露非常常见，可以用 TruffleHog 一类工具扫描。

分词

用 BPE 或任意子词方法都行，但代码有两个特殊要求：

1. 保留空白，否则缩进信息丢失；
2. 压缩高频片段，例如把多层缩进与换行合并成一个 token。

允许跨空白切词能得到更紧凑的表示（约省 40% token），但可能切出「import numpy as np 是一个 token」这种粒度，导致自动补全无法在中间停下。工程上要权衡。

模型还有明显的语言偏好：不指定语言时，大多数任务会写成 Python，做仪表盘则会选 React。偏好来自语言易写程度、流行度，以及 RL 阶段使用的解释器。

中期训练：把通用模型变成编码模型

Code Llama 与 Qwen2.5-Coder 的公开报告给出了两条经验：

1. 提高代码比例，但不能到 100%。Qwen2.5-Coder 的实验显示，纯代码继续训练会显著损害数学与问答能力；混入文本与数学后，代码能力只轻微下降，通用能力得以保留。
2. 上下文要延长。代码仓库往往超过 100 万 token，不能简单拼接所有文件。可行做法是按目录组织，或沿依赖图把相关文件放在一起。Qwen2.5-Coder 从 8K 训练到 128K，用的是 YaRN 式扩展。

填充式训练（FIM）

普通自回归模型只能看左边。但在编辑器里，补全往往发生在中间：

def is_positive(x):  # 这里要推断返回类型
    return x > 0

只看左侧，is_positive 可能是 bool、字符串或枚举；看右侧才能确定是 bool。

做法是把填充任务改写成自回归任务：给出前缀与后缀，用哨兵 token 标记待填区间，让模型生成中间内容。推理时把哨兵放在前缀与后缀之间即可。多个空洞可以用多个哨兵依次生成。

这类训练支持很多软件工程任务：文件中段补全、类型推断、生成注释、重命名变量。课堂引用的结果显示，训练过 FIM 的模型在中段补全上远好于「从左到右生成 + 重排」，而且不会牺牲普通的从左到右补全能力。

还能从代码里学到什么

CI 尤其值得强调：代码在某个版本报错、后续版本通过并合并，这段历史在 GitHub 上公开，几乎不需要自己构造。

如何评估生成的代码

与参考实现比较

BLEU 一类指标比较 token 与 n-gram 重合，但 >= 与 > 只差一个 token，行为却可能完全不同；反过来，写法不同但语义相同的代码又会被判低分。

改进方案：

• CodeBLEU：比较抽象语法树，并匹配语义数据流；
• CodeBERTScore：逐 token 嵌入，检查每个 token 在另一份实现里是否有对应项。

这类指标仍然脆弱，现在只作为辅助。

执行测试

主流做法是跑单元测试：全部通过得 1，否则得 0。好处是能接受任何正确的实现；坏处是写好测试极难。

两类错误都很常见：

• 假阳性：测试太松，错误实现也能通过；
• 假阴性：需求里没写清楚，但测试检查了（例如按钮的背景色），模型无从得知。

此外还需要隔离环境与依赖：通常用 Docker 预装依赖，但版本升级会让测试失效；大型仓库跑一次测试可能要好几分钟。

常见基准：HumanEval（OpenAI 人工编写）、CodeContests（竞赛题），指标是 pass@1 与 pass@k（生成 k 个，只要有一个通过）。

代码质量、运行效率、可维护性在非 agent 训练中大多没有被纳入奖励；NoFun 一类工作开始把运行时、资源占用、静态检查纳入评估。

用强化学习提升单步生成

有了测试就可以做 RL：采样回答、跑测试、按结果更新模型。这正是推理模型在竞赛编程上进步的主要方式之一；竞赛题通常有执行时限，超时直接不得分。

Agentic 编码：定位、编辑、验证

仓库级任务的标准循环：

1. 定位：找到需要修改的位置；
2. 编辑：修改代码；
3. 验证：跑测试，失败则带着错误信息回到第 1 步。

Agentless 提供了另一条路：先预测要改的文件，再预测类与函数，再定位到行，最后用单步模型生成补丁。它在某段时间里出奇地有效，因为当时的模型更擅长单步生成，而不擅长工具调用与验证循环。现在主流仍然是 agent。

只给一个 bash 工具够不够

mini-swe-agent 证明了「只给 bash」也能工作：

• 定位：grep、ripgrep、cat、find；
• 编辑：sed、一次性 Python 脚本、patch；
• 验证：pytest、构建命令。

但 bash 不是理想的编辑接口。用 sed 替换带大量反斜杠的内容时，转义很容易出错；每次还要在 sed、Python、patch 之间做选择。

几乎所有有竞争力的编码 agent 都额外提供一个专用文件编辑工具，支持整文件写入或搜索替换。

编辑格式比想象中重要

两种常见格式：

• 统一 diff：带行号，语义明确，但模型经常把行号数错；
• 搜索替换：给一段原文与替换文本，不含行号，更稳。

搜索替换的缺点是歧义：文件里有多处相同的 retries = 3 时无法确定改哪一处，模型必须多写几行上下文来消歧。总体收益仍然更大。

课堂引用 Aider 的编辑基准：GPT-4 Turbo 在统一 diff 上明显更好，因为训练数据里见得多。后来 Gemini 也因使用自家编辑格式而在很多 harness 上表现不佳。模型与 harness 的格式不匹配，会成为一个主要瓶颈。

定位代码

最常用的还是 ripgrep/grep 加沿调用关系阅读。LOK Agent 一类工作提供了沿依赖路径跳转的专用工具，但课堂的经验是：复杂工具往往打不过简单方法。

评估与训练环境

SWE-bench

构造方式：

1. 取一个 GitHub issue 作为问题；
2. 取 issue 解决前的仓库状态作为起点；
3. 取与该 issue 一起引入的测试作为判据。

补丁应用后跑测试，相关测试全过得 1。测试分两类：

• FAIL_TO_PASS：修复前失败、修复后通过，验证新功能；
• PASS_TO_PASS：修复前后都通过，防止回归。

完整测试套件太慢（Django、NumPy 这类仓库一次要五到十分钟），所以只跑一个子集。

一个可运行任务需要：起始文件、依赖、测试命令、issue 描述、检查项。

从评估集到训练集

• SWE-Gym：为 SWE-bench 风格任务额外构造训练环境。一个原本约 11% 成功率的模型，用 500 个样本训练后明显提升；生成多个输出再重排还能进一步提升。
• SWE-rebench：持续从新 PR 生成环境（课堂提到覆盖 2024 年 12 月到 2026 年 6 月），并提供训练集与开发集，是目前规模较大的公开编码 agent 数据。
• SWE-smith：不依赖 PR，而是对可运行代码做变异（mutation testing 的思路），保留测试、注入 bug，再让模型修复。数据几乎无限、成本低，但不如真实 issue 自然。

跨语言版本（Multi-SWE-bench、SWE-bench Multilingual）主要难点是各语言的工程约定不同，而现在 agent 自己就能处理这些差异。

多 harness 训练

同一个模型在不同 harness 上表现可能差很多，因为编辑格式与工具协议不同。因此模型厂商会在多个 harness 上训练；NVIDIA 的 Nemotron 报告提到训练时使用了五个以上 harness。

编码之外：软件开发的其余部分

内循环是修 bug、跑测试；外循环包括代码审查、部署、监控、维护、需求与规划。

已经出现的任务与基准：

• 代码审查：判断意见是否被采纳；
• 从前端 bug 修复：SWE-bench Multimodal；验证可以写 Playwright 脚本，也可以让 GUI agent 看页面；
• 从零建应用：需要好的验证器，难度在于评估；
• 重写二进制：ProgramBench，反推一个二进制程序并重新实现，长程且困难；
• 多 issue 连续开发：SWE Milestone，检查实现多个 issue 后仓库是否仍然可维护；
• 测试生成：SWT，基于变异测试判断测试是否真的能发现问题；
• 部署与运维：需要真实基础设施，目前还缺少成熟基准。

课堂上还有一个反直觉的观察：GLM 5.2 并非多模态模型，却在某个前端设计榜上排到第一。这说明「前端能力」并不必然依赖视觉输入，但对大多数任务，多模态仍然重要。

本讲结论

• 编码能力来自预训练语料、数据清洗、分词、长上下文与中期训练的共同作用。
• 纯代码继续训练会损害通用能力，混合数据更稳。
• FIM 让模型能补全代码中间部分，支撑类型推断、重命名等任务。
• 执行测试是主流评估方式，但测试本身的质量是最大瓶颈。
• Agent 循环是定位、编辑、验证；编辑工具的格式会显著影响成功率。
• SWE-bench 用 FAIL_TO_PASS 与 PASS_TO_PASS 定义成功；训练环境可以从 PR、变异或 agent rollout 生成。
• 软件开发远不止内循环；审查、部署、维护等外循环任务仍是开放问题。

理解检验

1. 为什么 Python 的空白不能在预训练时被归一化？

参考答案：Python 用缩进定义块结构，空白是语法的一部分，不是排版。把连续空白合并或去掉缩进，会让不同作用域的代码变成同一个结构，模型学到的就是错误的「语法到语义」映射，生成时也会写出语义不同的程序。同理需要保留的还有文件边界（如何表示一个文件的开始与结束）、Markdown 代码块的完整性，以及代码与解释它的文档并置出现的关系。

2. FIM 如何把「填中间」变成自回归任务？它支持哪些实际功能？

参考答案：做法是给模型前缀与后缀，用哨兵 token 标出待填区间，让模型像普通自回归一样生成中间内容；推理时把哨兵放在前缀与后缀之间即可。多个空洞可以用多个哨兵依次生成。它支持文件中段补全（编辑器里补全常发生在中间）、类型推断（return x > 0 的返回类型要靠右侧上下文确定）、生成注释与重命名变量等任务；训练过 FIM 的模型在中段补全上远好于「从左到右生成再重排」，且不牺牲普通补全能力。

3. 执行式评估的两类错误分别是什么？为什么测试质量是当前最大瓶颈？

参考答案：假阳性：测试太松，错误实现也能通过；假阴性：测试检查了需求里没写清楚的东西（例如按钮背景色），模型无从得知却被判错。测试质量是瓶颈，因为评估的全部效力都压在测试上——测试写得不全，正确实现被判错；测试太松，错误实现被判对；再叠加隔离环境、依赖版本升级导致测试失效、大仓库一次跑几分钟的成本，测试本身成了最难扩展的部分。

4. 为什么大多数编码 agent 除了 bash 还要一个专用编辑工具？搜索替换的歧义如何解决？

参考答案：bash 不是理想的编辑接口——用 sed 替换含大量反斜杠的内容时转义很容易出错，而且每次都要在 sed、一次性 Python 脚本与 patch 之间做选择。专用工具支持整文件写入或搜索替换，更可靠。搜索替换的歧义来自文件里多处相同的片段（例如多个 retries = 3）：模型必须多写几行上下文来指定唯一位置。相比统一 diff，搜索替换不依赖模型数行号，总体更稳。另一条经验是模型与 harness 的编辑格式必须匹配，格式不匹配会成为主要瓶颈。

5. SWE-bench 的 FAIL_TO_PASS 与 PASS_TO_PASS 分别防什么？

参考答案：FAIL_TO_PASS 是修复前失败、修复后必须通过的测试，用来验证新功能或修复确实生效——它防止「什么都没改也得分」。PASS_TO_PASS 是修复前后都通过的测试，用来防止回归——它防止「修好了这个、弄坏了别处」。两者合起来，一个补丁既要解决问题，又不能破坏既有行为。

与后续课程的关系

这一讲聚焦编码 agent。下一讲转向计算机使用 agent：如何理解屏幕、点击与输入。再往后是监督微调与强化学习，把本讲提到的 SWE 环境变成训练信号。

Three different things called "writing code"

1. Single-step generation: natural language in, a program out.
2. Repository-level change: read several files, fix an issue or add a feature.
3. Full software development: review, deployment, monitoring, maintenance, requirements.

This lecture focuses on the engineering behind the second and third. The third is often forgotten: a 2019 Microsoft survey found developers spend only about 15% of their time writing code, 25% in meetings and email, 14% debugging, and 8% running tests.

Timeline

What a model needs to write code

Single-step ability splits into three parts:

Editing is not free. A model needs many before/after examples in training to change code reliably.

Pre-training

Code must be in the corpus, but not only code — the model also needs to follow natural-language instructions. Details the lecture stresses:

• Whitespace carries meaning: Python indentation cannot be normalized away.
• File boundaries: the corpus must represent where files start and end.
• Block structure: Markdown code blocks must stay intact.
• API and documentation relationships: code should appear alongside text explaining it.

Data cleaning

Code has an unusual distribution, so cleaning is harder than for text:

• filter by language and source, removing autogenerated code, broken encodings, and low-value repetition;
• deduplicate aggressively: GitHub contains enormous numbers of forks, vendored libraries, near-identical files, and copied solutions;
• record licenses, source dates, opt-outs, and filter sensitive data. Leaked API keys are common; tools such as TruffleHog scan for them.

Tokenization

Any subword method works, but code imposes two requirements:

1. Preserve whitespace, or indentation information is lost.
2. Compress frequent spans, such as several levels of indentation plus a newline.

Allowing tokens to cross whitespace yields a more compact representation (about 40% fewer tokens) but can produce very long tokens — import numpy as np as a single unit — which prevents mid-line completion. There is a trade-off.

Models also have strong language biases: without a specified language, most tasks become Python, while dashboards become React. The bias comes from ease of writing, popularity, and the interpreter used during RL.

Mid-training: turning a general model into a coding model

The public reports from Code Llama and Qwen2.5-Coder give two lessons:

1. Raise the code share, but not to 100%. Qwen2.5-Coder found that training on pure code hurt math and question answering significantly; mixing text and math kept general ability with only a small code regression.
2. Extend context. Repositories often exceed a million tokens, so simply concatenating files does not work. Options include grouping by directory or following the dependency graph to keep related files together. Qwen2.5-Coder trained from 8K up to 128K using YaRN-style extension.

Fill-in-the-middle training

A plain autoregressive model sees only the left context, but editor completion happens in the middle:

def is_positive(x):  # infer the return type here
    return x > 0

From the left alone, is_positive could return bool, a string, or an enum; the right side settles it.

FIM rewrites the task as autoregressive generation: provide prefix and suffix, mark the hole with sentinel tokens, and generate the middle. At inference the sentinel sits between prefix and suffix. Multiple holes use multiple sentinels.

This supports file-middle completion, type inference, comment generation, and variable renaming. The cited results show FIM-trained models far outperform left-to-right generation with reranking on middle completion, without losing ordinary completion ability.

What else code can teach

CI deserves emphasis: a version fails, a later version passes and merges, and the whole history is public on GitHub. Almost no manual construction is required.

Evaluating generated code

Comparison with a reference

BLEU-style metrics compare token and n-gram overlap, but >= and > differ by one token and behave differently; conversely, different but equivalent implementations score low.

Improvements:

• CodeBLEU compares abstract syntax trees and matches semantic data flow.
• CodeBERTScore embeds tokens and checks that every token has a counterpart in the other implementation.

These remain fragile and are now auxiliary signals.

Execution-based evaluation

The mainstream approach runs unit tests: all pass gives 1, otherwise 0. It accepts any correct implementation, but writing good tests is extremely hard.

Two error types are common:

• False positives: tests are too weak and incorrect implementations pass.
• False negatives: the task description omits a requirement the test checks, such as a button's background color.

Isolation and dependencies also matter: Docker images preinstall packages, but upgrades break tests; running a large repository's suite can take minutes per example.

Common benchmarks include HumanEval (hand-written by OpenAI) and CodeContests (competitive programming), reported with pass@1 and pass@k.

Code quality, runtime efficiency, and maintainability were largely absent from non-agentic training rewards; work such as NoFun begins to score runtime, resource use, maintainability, and security.

Reinforcement learning for single-step generation

With tests, RL is straightforward: sample responses, run tests, update on the result. This is a major reason reasoning models improved at competitive programming, where execution timeouts make efficiency part of the score.

Agentic coding: localize, edit, verify

The standard loop for repository tasks:

1. Localize the place that needs changing;
2. Edit it;
3. Verify by running tests; on failure, use the error to return to step 1.

Agentless offers an alternative: predict the files, then classes and functions, then lines, and generate a patch with a single-step model. For a while it was surprisingly effective, because models were trained heavily for single-step resolution and less for tool calling and verification loops. Agents are now the norm.

Is a single bash tool enough?

mini-swe-agent shows that bash alone can work:

• localization with grep, ripgrep, cat, find;
• editing with sed, one-off Python scripts, patch;
• verification with pytest and build commands.

But bash is a poor editing interface. Replacing text full of backslashes with sed invites escaping errors, and the model must choose among sed, Python, and patch each time.

Nearly every competitive coding agent adds a dedicated file-editing tool supporting whole-file writes or search-and-replace.

Edit format matters more than expected

Two common formats:

• Unified diff: line-numbered and precise, but models frequently miscount lines.
• Search-and-replace: original text plus replacement, no line numbers, more robust.

Search-and-replace is ambiguous when the same retries = 3 appears several times; the model must include more context to disambiguate. The overall win still favors it.

The Aider edit benchmark found GPT-4 Turbo much better at unified diffs, simply because it had seen more of them. Gemini later underperformed on many harnesses because it used its own edit format. A mismatch between model and harness format becomes a major bottleneck.

Localizing code

The usual tools are ripgrep/grep plus reading along call chains. Work such as LOK Agent provides a dependency-path jumping tool, but the lecture's experience is that complex tools often lose to simple ones.

Evaluation and training environments

SWE-bench

Construction:

1. take a GitHub issue as the problem;
2. take the repository state before the issue was resolved as the starting point;
3. take the tests introduced with that issue as the check.

Apply the patch, run the tests, and score 1 if the relevant tests pass. Tests split into:

• FAIL_TO_PASS: fail before the fix and pass after, verifying the new behavior.
• PASS_TO_PASS: pass before and after, preventing regressions.

Running the full suite is too slow — Django or NumPy can take five to ten minutes per example — so a subset is selected.

A runnable task needs: starting files, dependencies, a test command, the issue text, and the checks.

From evaluation set to training set

• SWE-Gym builds additional training environments in the SWE-bench style. A model at roughly 11% success improved noticeably after 500 examples; generating multiple outputs and reranking improved it further.
• SWE-rebench continuously generates environments from new pull requests (the lecture mentions December 2024 through June 2026), with train and development splits — currently one of the larger public coding-agent datasets.
• SWE-smith does not need pull requests. It mutates runnable code in the spirit of mutation testing, keeps the tests, injects bugs, and asks the model to repair them. Data is nearly unlimited and cheap, but less natural than real issues.

Multilingual versions (Multi-SWE-bench, SWE-bench Multilingual) mostly contend with differing language conventions, which agents now handle well.

Multi-harness training

One model can perform very differently across harnesses because edit formats and tool protocols differ. Model providers therefore train on several harnesses; NVIDIA's Nemotron report mentions training on five or more.

Beyond fixing issues

The inner loop is bug fixing, testing, and iteration. The outer loop includes review, deployment, monitoring, maintenance, and requirements.

Tasks and benchmarks that now exist:

• Code review: measure whether comments are adopted;
• Front-end bug fixing: SWE-bench Multimodal, verified with Playwright scripts or a GUI agent;
• Building apps from scratch: the hard part is validation;
• Reimplementing binaries: ProgramBench, reverse-engineering a binary and rebuilding it — long-horizon and difficult;
• Many issues over time: SWE Milestone, checking that a repository stays maintainable after repeated changes;
• Test generation: SWT, using mutation testing to judge whether tests catch real breakage;
• Deployment and operations: needs real infrastructure and lacks mature benchmarks.

A counterintuitive observation: GLM 5.2, not a multimodal model, ranked first on a front-end design arena. Front-end skill does not strictly require visual input, though multimodality still matters for most tasks.

Takeaways

• Coding ability comes from pre-training data, cleaning, tokenization, long context, and mid-training together.
• Pure-code continued training damages general ability; mixed data is safer.
• FIM lets models complete the middle of code, enabling type inference and renaming.
• Execution-based tests are the main evaluation, and test quality is the biggest bottleneck.
• The agent loop is localize, edit, verify; the edit tool's format strongly affects success.
• SWE-bench defines success with FAIL_TO_PASS and PASS_TO_PASS; training environments can come from PRs, mutations, or agent rollouts.
• Software development extends far beyond the inner loop; review, deployment, and maintenance remain open problems.

Check your understanding

1. Why must Python whitespace not be normalized during pre-training?

Reference answer: Python uses indentation to define block structure, so whitespace is part of the grammar, not layout. Collapsing whitespace or stripping indentation merges blocks that belong to different scopes, teaching the model a wrong mapping from syntax to meaning; generation then produces programs with different semantics. The same applies to file boundaries (how the start and end of a file are represented), keeping Markdown code blocks intact, and placing code next to the prose that explains it.

2. How does FIM turn middle-filling into autoregressive generation, and what does it enable?

Reference answer: The model is given a prefix and a suffix, with sentinel tokens marking the region to fill, and it generates the middle autoregressively as usual; at inference the sentinel sits between prefix and suffix. Several holes can be generated with several sentinels. It enables mid-file completion (where editor completions usually happen), type inference (the return type of return x > 0 needs the right-hand context), docstring generation and variable renaming. Models trained with FIM are far better at mid-file completion than "generate left-to-right then rearrange", without losing ordinary completion ability.

3. What are the two error types in execution-based evaluation, and why is test quality the current bottleneck?

Reference answer: False positives — the tests are too weak, so a wrong implementation passes; false negatives — the tests check something the requirements never stated (a button's background colour), so a correct model is marked wrong. Test quality is the bottleneck because the entire evaluation rests on the tests: weak tests pass wrong code, incomplete tests fail right code, and add to that Docker isolation, dependency upgrades that break tests, and several minutes per run on large repositories. Writing good tests is the hardest part to scale.

4. Why do most coding agents add a dedicated editing tool to bash? How is search-and-replace ambiguity resolved?

Reference answer: bash is a poor editing interface — sed replacements with many backslashes are easy to escape wrongly, and the model must choose among sed, a one-off Python script and patch each time. A dedicated tool supporting whole-file writes or search-and-replace is more reliable. Search-and-replace is ambiguous when the same snippet occurs several times (multiple retries = 3): the model must include enough surrounding lines to identify one location. Compared with unified diffs, search-and-replace does not rely on counting line numbers and is generally steadier. A related lesson: the model and the harness must agree on the edit format, since a mismatch becomes a major bottleneck.

5. What do FAIL_TO_PASS and PASS_TO_PASS each protect against?

Reference answer: FAIL_TO_PASS tests fail before the fix and must pass after it, verifying that the feature or fix actually works — they prevent "scoring without changing anything". PASS_TO_PASS tests pass both before and after, guarding against regressions — they prevent "fixing this and breaking that". Together they require a patch both to solve the problem and to preserve existing behaviour.

How this connects to the rest of the course

This lecture focuses on coding agents. The next turns to computer-use agents: understanding screens, clicking, and typing. Then supervised fine-tuning and reinforcement learning turn the SWE environments described here into training signal.