第 08 章
正则表达式:把「格式」写成模式
从手工检查邮箱地址出发,逐步引入 re.search、量词、转义、锚点、字符集与捕获组;用校验邮箱、整理姓名格式、提取 Twitter 用户名三个例子,讲清正则如何验证、清洗与抽取数据,以及为什么复杂正则需要一步步搭。
Regular Expressions: Writing Format as a Pattern
Starting from hand-written email validation, the chapter introduces re.search, quantifiers, escaping, anchors, character classes and capture groups. Three examples — validating an email, normalising a name, extracting a Twitter handle — show how regex validates, cleans and extracts data, and why complex patterns must be built up in small steps.
篇内目录
- Starting without regex
- Timeline
- re.search: the smallest regex
- Basic symbols
- Why raw strings
- What the engine is doing
- Anchors: start and end only
- Character sets: stop using .
- Ranges and shorthands
- Case
- Groups: optional parts, alternatives, captures
- Optional parts
- Alternatives
- Non-capturing groups
- Example one: turning "Last, First" into "First Last"
- Example two: extracting a handle from a URL
- Three attempts
- Conditional re.search
- Why not write your own email regex
- A method for writing regex
- Takeaways
- Comprehension check
- How it connects
这是长视频(约 16 小时)的第 8 章,对应 CS50P 第 7 次讲座「Regular Expressions」,时间区间 8:32:47–10:37:47。正则表达式(regex)本质上是描述文本形状的模式。有了它,程序可以判断「这串字符像不像邮箱」、把杂乱的输入整理成统一格式,或者从一段文本里抽出想要的部分。
从「不用正则」开始
先写一个校验邮箱的程序,只用最基础的字符串操作:
email = input("What's your email? ").strip()
if "@" in email:
print("Valid")
else:
print("Invalid")
问题立刻出现:输入 @ 也会被判为合法。于是加一条「必须有点」:
if "@" in email and "." in email:
print("Valid")
输入 @. 仍然通过。继续收紧,用 split("@") 拆出用户名与域名:
username, domain = email.split("@")
if username and "." in domain:
print("Valid")
else:
print("Invalid")
if username 利用了 Python 的真值判断:空字符串为假。再把「有点」改成「以 .edu 结尾」:
if username and domain.endswith(".edu"):
print("Valid")
这时输入 malan@.edu 仍会通过——域名部分只有一个 .edu,前面没有学校名。每修一个洞,就冒出下一个洞。课程在这里停下,给出本节的核心判断:这种「一层层打补丁」的写法会迅速失控,应该改用能直接描述整体形状的工具。
时间轴
| 时间 | 内容 |
|---|---|
| 8:32:47 | 动机:用模式匹配校验用户输入 |
| 8:33:50 | 手写 validate.py:in、split、endswith 逐步加码 |
| 8:43:23 | 引入 re 库与 re.search |
| 8:46:57 | 基本符号:.、*、+、?、{m}、{m,n} |
| 8:53:04 | 有限状态机:re.search 如何从左到右做匹配 |
| 8:57:12 | 字面点号要转义 \.;原始字符串 r"..." |
| 9:05:46 | 锚点 ^ 与 $ |
| 9:10:55 | 字符集 [...] 与取反 [^...] |
| 9:18:23 | 字符范围 [a-zA-Z0-9_] 与简写 \w、\d、\s |
| 9:24:59 | 分组 (...) 与或 |;非捕获组 (?:...) |
| 9:28:43 | re.IGNORECASE 标志 |
| 9:35:01 | 用 (...)? 让子域名可选 |
| 9:45:22 | re.match、re.fullmatch |
| 9:46:09 | 清洗姓名:re.search 的捕获组 .group(1) |
| 10:02:28 | 海象运算符 := |
| 10:05:37 | 提取 Twitter 用户名:replace → removeprefix → re.sub → re.search |
| 10:30:53 | 非捕获组解决「组号错位」 |
re.search:最小的正则
import re
email = input("What's your email? ").strip()
if re.search(r"@", email):
print("Valid")
else:
print("Invalid")
re.search(pattern, string, flags=0) 在字符串里寻找符合 pattern 的第一处,找到就返回一个匹配对象(真值),找不到返回 None。这一版的判断力与 "@" in email 相同,但从此可以往 pattern 里写更复杂的东西。
基本符号
| 符号 | 含义 |
|---|---|
. | 任意一个字符,换行除外 |
* | 前面的东西重复 0 次或多次 |
+ | 前面的东西重复 1 次或多次 |
? | 前面的东西重复 0 次或 1 次(可选) |
{m} | 恰好重复 m 次 |
{m,n} | 重复 m 到 n 次 |
把 .*@.* 填进去,表示「@ 左边任意多个字符,右边任意多个字符」。它仍然接受单独一个 @,因为 * 允许零次。改成 .+.+,两侧就至少要有一个字符:
if re.search(r".+@.+", email):
为什么必须用原始字符串
想要求「以 .edu 结尾」,直接写 r".+@.+\.edu"。如果漏掉反斜杠,写成 .+@.+.edu,那么 .edu 里的 . 会被当成「任意字符」,于是 malan@harvard?edu 也能通过——这正是课上演示的坑。
反斜杠在普通字符串里还会被 Python 自己解释(例如 \n 是换行)。所以正则一律写成 raw string:r"...",让反斜杠原样传给正则引擎。
引擎在做什么
re.search 可以看成一个有限状态机:从起始状态出发,逐字符读取输入,按 pattern 转移状态;读完全部输入后若停在接受状态,就算匹配。.*@.* 的第一段是「读到 @ 之前一直停在原地」,第二段是「读到结尾都停在原地」;@ 是必须跨过的唯一一条边。理解这一点,就能理解为什么 * 允许「立刻跨过」——零次重复也是一次合法的转移。
锚点:只匹配开头与结尾
re.search(r".+@.+\.edu", email) 仍然接受一整句话:
My email address is malan@harvard.edu
因为模式只要求「某处存在这个形状」,前后多余的文字不影响结果。要限定整串必须完全符合,就加锚点:
if re.search(r"^.+@.+\.edu$", email):
^表示从字符串开头匹配;$表示匹配到字符串结尾(严格说是结尾换行之前)。
两者一起用,等于要求「整串就是这个形状」。
字符集:不再用 .
^.+@.+\.edu$ 还有一个漏洞:malan@@harvard.edu 也能通过,因为 . 可以匹配 @。这里需要表达「除 @ 以外的任意字符」:
if re.search(r"^[^@]+@[^@]+\.edu$", email):
方括号定义一个字符集。[^@] 是它的补集:任何不是 @ 的字符。于是字符串里只能有一个 @。
需要注意:^ 在方括号外表示开头,在方括号内表示取反。同一个符号,两种含义,这是正则最容易混淆的地方之一。
范围与简写
逐个列出允许的字符太啰嗦,可以用连字符表示范围:
r"^[a-zA-Z0-9_]+@[a-zA-Z0-9_]+\.[a-zA-Z0-9_]+\.edu$"
更常见的写法是用内置简写:
| 简写 | 含义 | 补集 |
|---|---|---|
\w | 字母、数字、下划线 | \W |
\d | 数字 0–9 | \D |
\s | 空白(空格、制表符等) | \S |
于是:
r"^\w+@\w+\.\w+\.edu$"
\w 不包含点号,所以 Gmail 地址里的 first.last@gmail.com 会被拒绝——这提醒我们:简写是省事,不是标准。真实邮箱允许的字符由 RFC 规定,比 \w 复杂得多。
大小写
re.search 的第三个参数是标志位。re.IGNORECASE 让匹配忽略大小写:
if re.search(r"^\w+@\w+\.edu$", email, re.IGNORECASE):
这样 MALAN@HARVARD.EDU 也能通过。另一种做法是把输入统一转成小写(email.lower()),但那会改变变量本身的值;标志位只影响匹配行为。
分组:可选、或、捕获
可选的部分
malan@cs50.harvard.edu 比 malan@harvard.edu 多一层子域名。用括号把「子域名 + 点」括起来,再加 ?:
r"^\w+@(\w+\.)?\w+\.edu$"
(\w+\.)? 整体出现 0 次或 1 次,于是两种地址都能通过。注意括号不能省:写成 \w+\.? 只让「点」可选,cs50 这段仍然缺失。
或
竖线表示「或」,括号用来限定「或」的范围:
r"^\w+@(\w+\.)?\w+\.(edu|com|org)$"
(edu|com|org) 表示顶级域名三者之一。
非捕获组
括号默认会捕获匹配到的内容,供之后提取。如果只是想分组、不想捕获,用 (?:...):
r"^https?://(?:www\.)?twitter\.com/(\w+)$"
这里 (?:www\.)? 只负责让 www. 可选,不占用捕获组编号;(\w+) 才是我们真正想要的用户名。
例子一:把「姓, 名」整理成「名 姓」
用户在表单里可能写成 Malan, David,也可能写成 David Malan。目标是统一成后者。
import re
name = input("What's your name? ").strip()
if matches := re.search(r"^(.+), ?(.+)$", name):
name = f"{matches.group(2)} {matches.group(1)}"
print(f"hello, {name}")
逐段看 pattern:
^(.+)捕获「第一个逗号之前的所有内容」,也就是姓;, ?匹配逗号,以及逗号后可有可无的一个空格;(.+)$捕获剩下的内容,也就是名。
matches.group(1) 是第一个括号,matches.group(2) 是第二个;group(0) 是整段匹配。注意组号从 1 开始,这是正则库的约定,与 Python 列表索引从 0 开始不同。
:= 是海象运算符:把右边赋给 matches,同时把它的真值作为 if 的条件。它等价于先赋值、再判断,只是合成了一行。
为什么用正则而不是 split(", ")?因为用户可能不写空格。split(", ") 遇到 Malan,David 会把整个字符串当成一个元素,last, first = ... 直接抛 ValueError;而 , ? 把空格变成了可选项。
补充:这个模式仍然处理不了 Malan, David, Jr. 这类含多个逗号的名字——(.+) 是贪婪的,会尽量多吃。真实场景要么限制输入,要么把规则写得更细。
例子二:从 URL 里提取用户名
输入 https://twitter.com/davidjmalan,要取出 davidjmalan。
三次尝试
第一次:replace
username = url.replace("https://twitter.com/", "")
只要协议变成 http、前面多一个 www.、或者用户写了「My username is https://…」,就失效。
第二次:removeprefix
username = url.removeprefix("https://twitter.com/")
removeprefix 只删开头,不会误删中间的内容,比 replace 安全。但协议、www 仍然必须完全一致。
第三次:re.sub
username = re.sub(r"^(https?://)?(www\.)?twitter\.com/", "", url, flags=re.IGNORECASE)
(https?://)?:整个协议部分可选,s?让https与http都行;(www\.)?:www.可选;\.:字面的点;- 末尾的
/:必须有。
re.sub(pattern, replacement, string) 把匹配到的部分替换掉,这里替换成空串。它比 replace 强在「模式」而不是「固定文本」。
但 re.sub 是无条件的:即使输入根本不是 Twitter URL,它也会返回原字符串,程序照样把一串垃圾当用户名打印出来。
用 re.search 加条件
import re
url = input("URL: ").strip()
if matches := re.search(
r"^https?://(?:www\.)?twitter\.com/([a-z0-9_]+)/?$",
url,
re.IGNORECASE,
):
print(f"Username: {matches.group(1)}")
(?:www\.)?是非捕获组:让www.可选,但不占用组号;([a-z0-9_]+)捕获用户名,只允许字母、数字、下划线(Twitter 的实际规则);/?$允许结尾多一个斜杠。
这里有一个课堂上演示过的陷阱:如果写成捕获组 (www\.)?,它会成为第 1 组,用户名就变成第 2 组,group(1) 取到的是 www. 或空。改成非捕获组 (?:...) 之后,组号回到预期。
re.match 与 re.fullmatch 是 re.search 的两个变体:前者自动从开头匹配(不必写 ^),后者还要求匹配到结尾(不必写 $)。用 re.fullmatch 时,模式可以省掉两个锚点。
为什么不自己写邮箱正则
课程展示了一段浏览器实际使用的邮箱正则,长度以屏幕计,充满了人们不熟悉但规范允许的字符。它说明两件事:
- 真实的标准远比想象复杂;
- 手写这种东西几乎必然出错。
结论是:对邮箱、URL 这类有公开标准的格式,优先用成熟库(例如 email_validator),正则留给「自己业务里的格式」,比如「学号」「房间号」「课程代码」。
写正则的方法论
课上反复强调的一点:不要一次写出复杂正则。正确做法是每次只加一个特征,运行、验证、再加下一个。原因是正则的符号密度极高,一旦出错,很难在几十个字符里定位。
这个建议不只适用于正则:凡是「表达式密集、错误信息模糊」的领域(正则、SQL、复杂条件判断),增量构建都优于一次性写完。
本章结论
- 正则描述「文本的形状」,
re.search判断某处是否匹配,re.fullmatch判断整串是否匹配。 .任意字符、*零或多次、+一次或多次、?可选、{m,n}次数范围。- 字面点号写
\.;正则字符串一律用r"..."。 ^/$锚定首尾;方括号内的^表示取反。[a-z0-9_]与\w等价;\w不含点号,所以不等于「合法邮箱字符」。( ... )既分组又捕获;(?: ... )只分组不捕获;|表示或。- 捕获组用
.group(n)取出,组号从 1 开始,group(0)是整段。 re.sub做模式化替换,re.split、re.findall分别做拆分与全局查找。- 标志位
re.IGNORECASE让匹配忽略大小写,不改变原字符串。 - 复杂正则要增量构建;有公开标准的格式优先用现成库。
理解检验
1. 为什么 .+@.+\.edu 会接受 malan@harvard?edu?怎样修正?
参考答案:模式里的 . 是「任意字符」的元字符,而 .edu 中的点没有转义,于是它匹配了 ?。修正方式是给这个点加反斜杠,写成 r".+@.+\.edu",让正则引擎把它当作字面的句点。补充:这也是为什么正则字符串要写成 raw string,否则反斜杠还可能先被 Python 的字符串转义吃掉。
2. .*@.* 与 .+@.+ 有什么区别?为什么前者会接受单独一个 @?
参考答案:* 表示前面的元素重复 0 次或多次,+ 表示 1 次或多次。用 .* 时,@ 左右都可以是空字符串,所以仅输入 @ 也满足模式;用 .+ 时两侧都至少需要一个字符,单独 @ 就不匹配。同理,\.? 只让点可选,而 (www\.)? 让整段可选,括号决定了「可选的单位」。
3. ^ 和 $ 各起什么作用?不加它们会发生什么?
参考答案:^ 要求从字符串开头开始匹配,$ 要求匹配延伸到字符串结尾。不加锚点时,re.search 只要在任意位置找到符合模式的子串就算成功,所以 My email is malan@harvard.edu 这样的整句也会被判定为「合法邮箱」。加上 ^...$ 之后,整串必须恰好是这个形状,多余的文字会导致匹配失败。
4. [^@]+ 与 \w+ 都能匹配用户名,它们的区别是什么?各自适合什么场景?
参考答案:\w 等价于 [a-zA-Z0-9_],只匹配字母、数字与下划线,不包含点号、空格和标点。[^@]+ 匹配除 @ 以外的任意字符,范围宽得多,包括空格和点号。前者适合「用户名只允许字母数字下划线」这类明确的业务规则(如 Twitter);后者适合「排除分隔符」的粗粒度约束。课堂上 [^@]+ 版本会接受带空格或点号的输入,而 \w+ 版本会拒绝 first.last@gmail.com 中的点——两者都不是完整的邮箱标准。
5. 在 ^https?://(?:www\.)?twitter\.com/([a-z0-9_]+)$ 中,为什么 www 用 (?:...) 而不是 (...)?如果用普通括号会怎样?
参考答案:普通括号会创建一个捕获组,并占用组号。这样 (www\.)? 成为第 1 组,用户名 ([a-z0-9_]+) 变成第 2 组;调用 group(1) 取到的是 www. 或空字符串,而不是用户名,程序就会输出错误结果。(?:...) 是非捕获组:它仍然把 www\. 当作一个整体来加 ?,但不参与编号,用户名保持在第 1 组。课堂上正是先写错、再用这个语法修正的。
6. 为什么用 name.split(", ") 整理「姓, 名」会崩溃?正则版本如何避免?
参考答案:split(", ") 要求逗号后恰好有一个空格。用户若输入 Malan,David(没有空格),整个字符串不会被拆开,返回只有一个元素的列表,而左边有 last, first 两个变量,解包时抛出 ValueError: not enough values to unpack。正则版本 ^(.+), ?(.+)$ 里的 ? 让空格成为可选项,因此 Malan, David 与 Malan,David 都能匹配,两个捕获组分别拿到姓和名。
7. 为什么不应该手写正则来校验邮箱地址?实际应该怎么做?
参考答案:邮箱的合法字符与结构由 RFC 标准规定,允许大量不常见字符,完整的正则非常长且难以维护,手写几乎必然引入错误(漏掉合法地址或放过非法地址)。课程展示的浏览器版本也只是「覆盖大多数错误」的简化版。实践中的做法是使用成熟库(如 email_validator)处理这类有公开标准的格式;正则留给业务自定义的格式,例如学号、房间号、课程代码。
与前后章的关系
第 7 章从文件里按逗号切分字段,本章把「切分」升级为「按模式识别」:CSV 是固定分隔符,正则可以描述变化的结构。第 9 章会把本章清洗后的数据装进类里——例如让 Student 对象自己负责校验姓名格式,而不是散落在脚本中。第 10 章的命令行工具也常需要正则来解析参数以外的自由文本。
This is chapter 8 of the long video (about 16 hours), corresponding to CS50P lecture 7, "Regular Expressions", covering 8:32:47–10:37:47. A regular expression (regex) is a description of the shape of text. With it a program can decide whether a string looks like an email address, normalise messy input into one format, or pull the part it wants out of a longer string.
Starting without regex
Begin with email validation built only from basic string operations:
email = input("What's your email? ").strip()
if "@" in email:
print("Valid")
else:
print("Invalid")
The flaw is immediate: typing @ alone counts as valid. Add a requirement for a dot:
if "@" in email and "." in email:
print("Valid")
@. still passes. Tighten it by splitting on the at-sign:
username, domain = email.split("@")
if username and "." in domain:
print("Valid")
else:
print("Invalid")
if username uses Python truthiness: an empty string is false. Finally require the domain to end with .edu:
if username and domain.endswith(".edu"):
print("Valid")
Now malan@.edu passes: the domain ends in .edu but has no school name. Every hole patched reveals another. The lecture stops here to make its central point: patching conditions one at a time gets out of hand quickly; use a tool that describes the whole shape at once.
Timeline
| Time | Topic |
|---|---|
| 8:32:47 | Motivation: pattern matching for user input |
| 8:33:50 | Hand-written validate.py: in, split, endswith |
| 8:43:23 | The re module and re.search |
| 8:46:57 | Basic symbols: ., *, +, ?, {m}, {m,n} |
| 8:53:04 | Finite state machines: how re.search matches left to right |
| 8:57:12 | Escaping a literal dot as \.; raw strings r"..." |
| 9:05:46 | Anchors ^ and $ |
| 9:10:55 | Character sets [...] and negation [^...] |
| 9:18:23 | Ranges [a-zA-Z0-9_] and shorthands \w, \d, \s |
| 9:24:59 | Groups (...), alternation |, non-capturing (?:...) |
| 9:28:43 | The re.IGNORECASE flag |
| 9:35:01 | Optional subdomain via (...)? |
| 9:45:22 | re.match and re.fullmatch |
| 9:46:09 | Cleaning a name with capture groups and .group(n) |
| 10:02:28 | The walrus operator := |
| 10:05:37 | Extracting a Twitter handle: replace → removeprefix → re.sub → re.search |
| 10:30:53 | Non-capturing groups fix the group-numbering bug |
re.search: the smallest regex
import re
email = input("What's your email? ").strip()
if re.search(r"@", email):
print("Valid")
else:
print("Invalid")
re.search(pattern, string, flags=0) looks for the first place matching the pattern. It returns a match object (truthy) on success and None otherwise. This version is no stronger than "@" in email, but the pattern slot can now hold much more.
Basic symbols
| Symbol | Meaning |
|---|---|
. | any single character except a newline |
* | zero or more repetitions of what precedes |
+ | one or more repetitions |
? | zero or one repetition (optional) |
{m} | exactly m repetitions |
{m,n} | between m and n repetitions |
.*@.* means "any characters, an at-sign, any characters". It still accepts a lone @, because * permits zero repetitions. Change to .+.+ and both sides need at least one character:
if re.search(r".+@.+", email):
Why raw strings
To require an ending of .edu, write r".+@.+\.edu". Drop the backslash and write .+@.+.edu, and the . before edu becomes "any character", so malan@harvard?edu passes — exactly the trap demonstrated in the lecture.
A backslash also gets interpreted by Python itself in ordinary strings (\n is a newline). So regex patterns are written as raw strings: r"...", which passes backslashes through untouched.
What the engine is doing
re.search behaves like a finite state machine: start in an initial state, read the input left to right, move between states according to the pattern, and accept if the final state after the last character is an accepting state. For .*@.*, the first part loops in place until an @ appears, the @ is the one edge that must be crossed, and the second part loops to the end. Understanding this explains why * allows "cross immediately" — zero repetitions is still a legal transition.
Anchors: start and end only
re.search(r".+@.+\.edu", email) still accepts a whole sentence:
My email address is malan@harvard.edu
The pattern only requires that the shape appears somewhere; surrounding text does not matter. Anchors force the whole string to match:
if re.search(r"^.+@.+\.edu$", email):
^matches at the start of the string;$matches at the end (technically just before a trailing newline).
Together they require the entire string to have exactly this shape.
Character sets: stop using .
^.+@.+\.edu$ still lets malan@@harvard.edu through, because . can match @. What is needed is "any character other than @":
if re.search(r"^[^@]+@[^@]+\.edu$", email):
Square brackets define a set of characters. [^@] is its complement: anything that is not @. Now only one at-sign can appear.
Careful: ^ outside brackets means "start of string", but inside brackets it means "negate". One symbol, two meanings — one of the most confusing corners of regex.
Ranges and shorthands
Listing every allowed character is tedious, so a hyphen expresses a range:
r"^[a-zA-Z0-9_]+@[a-zA-Z0-9_]+\.[a-zA-Z0-9_]+\.edu$"
The usual shorthand is shorter:
| Shorthand | Meaning | Complement |
|---|---|---|
\w | letters, digits, underscore | \W |
\d | digits 0–9 | \D |
\s | whitespace (space, tab, …) | \S |
So:
r"^\w+@\w+\.\w+\.edu$"
\w excludes the dot, so first.last@gmail.com is rejected — a reminder that shorthand is convenience, not a standard. Real email addresses allow characters defined by RFC, far beyond \w.
Case
The third argument of re.search is a set of flags. re.IGNORECASE makes matching case-insensitive:
if re.search(r"^\w+@\w+\.edu$", email, re.IGNORECASE):
Now MALAN@HARVARD.EDU passes. Lower-casing the input (email.lower()) would also work but changes the variable's value; a flag changes only how matching behaves.
Groups: optional parts, alternatives, captures
Optional parts
malan@cs50.harvard.edu has one more level than malan@harvard.edu. Wrap "subdomain plus dot" in parentheses and add ?:
r"^\w+@(\w+\.)?\w+\.edu$"
(\w+\.)? occurs zero or one time, so both forms pass. The parentheses cannot be omitted: \w+\.? would make only the dot optional and still require cs50.
Alternatives
A vertical bar means "or", and parentheses limit its scope:
r"^\w+@(\w+\.)?\w+\.(edu|com|org)$"
(edu|com|org) accepts any one of the three top-level domains.
Non-capturing groups
Parentheses normally capture what they match, so it can be extracted later. To group without capturing, use (?:...):
r"^https?://(?:www\.)?twitter\.com/(\w+)$"
Here (?:www\.)? merely makes www. optional without taking a group number; (\w+) is the only capture.
Example one: turning "Last, First" into "First Last"
A form may receive Malan, David or David Malan; the goal is always the second form.
import re
name = input("What's your name? ").strip()
if matches := re.search(r"^(.+), ?(.+)$", name):
name = f"{matches.group(2)} {matches.group(1)}"
print(f"hello, {name}")
Reading the pattern:
^(.+)captures everything before the first comma — the last name;, ?matches the comma and an optional single space;(.+)$captures the rest — the first name.
matches.group(1) is the first pair of parentheses, group(2) the second, and group(0) the whole match. Group numbering starts at 1 — a library convention that differs from Python's zero-based indexing.
:= is the walrus operator: assign to matches and use its truthiness in the if at the same time. It is the two-step version collapsed into one line.
Why not split(", ")? Because users may omit the space. With Malan,David, split(", ") returns a single element and last, first = ... raises ValueError; , ? makes the space optional.
Note: this pattern still fails on Malan, David, Jr. — (.+) is greedy and takes as much as it can. Real systems either constrain input or write finer rules.
Example two: extracting a handle from a URL
Given https://twitter.com/davidjmalan, produce davidjmalan.
Three attempts
First: replace
username = url.replace("https://twitter.com/", "")
This breaks as soon as the protocol is http, a www. appears, or the user writes "My username is https://…".
Second: removeprefix
username = url.removeprefix("https://twitter.com/")
removeprefix only removes from the start, so it cannot damage text in the middle — safer than replace. But protocol and www must still match exactly.
Third: re.sub
username = re.sub(r"^(https?://)?(www\.)?twitter\.com/", "", url, flags=re.IGNORECASE)
(https?://)?: the whole protocol optional, withs?allowing bothhttpandhttps;(www\.)?:www.optional;\.: a literal dot;- the final
/is required.
re.sub(pattern, replacement, string) replaces every match with the replacement (here, the empty string). It is stronger than replace because it matches a pattern, not fixed text.
But re.sub is unconditional: give it a non-Twitter URL and it returns the string unchanged, so the program confidently prints garbage as a username.
Conditional re.search
import re
url = input("URL: ").strip()
if matches := re.search(
r"^https?://(?:www\.)?twitter\.com/([a-z0-9_]+)/?$",
url,
re.IGNORECASE,
):
print(f"Username: {matches.group(1)}")
(?:www\.)?makeswww.optional without using a group number;([a-z0-9_]+)captures the handle, restricted to letters, digits and underscore (Twitter's actual rule);/?$tolerates one trailing slash.
This is where the lecture runs into the group-numbering bug: with a capturing (www\.)?, that group becomes number 1 and the handle becomes number 2, so group(1) yields www. or an empty string. Switching to (?:...) restores the expected numbering.
re.match and re.fullmatch are variants of re.search: match anchors at the start automatically (no ^ needed), and fullmatch also requires the end (no $ needed). With fullmatch, both anchors can be dropped.
Why not write your own email regex
The lecture shows the regex browsers actually use. It spans the screen and includes characters people rarely see but the standard permits. Two conclusions follow:
- The real standard is far more complex than intuition suggests;
- Hand-writing such a pattern is almost certain to introduce errors.
So: for formats with a public standard — email, URLs — use a mature library (for example email_validator). Keep regex for formats specific to your own application: student IDs, room numbers, course codes.
A method for writing regex
The lecture insists on one practice: never write a complicated pattern in one go. Add one feature, run it, verify, add the next. Regex is dense — a single wrong character among dozens is nearly impossible to find by staring.
This generalises beyond regex: in any dense-expression, vague-error domain — regex, SQL, long boolean conditions — incremental construction beats writing it all at once.
Takeaways
- A regex describes the shape of text;
re.searchfinds a match anywhere,re.fullmatchrequires the whole string. .any character,*zero or more,+one or more,?optional,{m,n}a repetition range.- A literal dot is
\.; write patterns as raw stringsr"...". ^/$anchor start and end;^inside brackets means negation.[a-z0-9_]equals\w;\wexcludes dots, so it is not "valid email characters".( ... )groups and captures;(?: ... )groups without capturing;|means or.- Captures are read with
.group(n); numbering starts at 1, andgroup(0)is the whole match. re.subperforms pattern-based replacement;re.splitandre.findallsplit and find all.- The
re.IGNORECASEflag makes matching case-insensitive without changing the string. - Build complex patterns incrementally; for standard formats, prefer an existing library.
Comprehension check
1. Why does .+@.+\.edu accept malan@harvard?edu, and how do you fix it?
Reference answer: . is the metacharacter for "any character", and the dot in .edu was left unescaped, so it matched ?. The fix is to write r".+@.+\.edu", escaping the dot so the engine treats it as a literal period. This is also why regex strings are raw strings: otherwise Python's own string escaping may consume the backslash before the engine sees it.
2. What is the difference between .*@.* and .+@.+? Why does the first accept a lone @?
Reference answer: * means zero or more repetitions, + means one or more. With .*, both sides can be empty, so the string @ satisfies the pattern. With .+, each side needs at least one character, so a lone @ fails. The same distinction explains \.? (only the dot optional) versus (www\.)? (the whole group optional): parentheses decide what the quantifier applies to.
3. What do ^ and $ do, and what happens without them?
Reference answer: ^ requires a match to begin at the start of the string, $ requires it to reach the end. Without anchors, re.search succeeds if the pattern appears anywhere, so My email is malan@harvard.edu is judged a valid email. With ^...$, the entire string must have exactly that shape and any extra text causes failure.
4. [^@]+ and \w+ can both match a username. How do they differ, and when is each appropriate?
Reference answer: \w equals [a-zA-Z0-9_] — letters, digits and underscore only, excluding dots, spaces and punctuation. [^@]+ matches anything except @, which is much broader and includes spaces and dots. The first suits an explicit business rule such as "handles contain only letters, digits and underscore" (Twitter's rule); the second suits a coarse constraint such as "exclude the separator". In the lecture the [^@]+ version accepts inputs with spaces or dots, while \w+ rejects first.last@gmail.com because of the dot; neither is the full email standard.
5. In ^https?://(?:www\.)?twitter\.com/([a-z0-9_]+)$, why is www wrapped in (?:...) rather than (...)? What goes wrong with ordinary parentheses?
Reference answer: Ordinary parentheses create a capture group and consume a group number. Then (www\.)? becomes group 1 and the handle ([a-z0-9_]+) becomes group 2, so group(1) returns www. or an empty string instead of the username, and the program prints the wrong thing. (?:...) is a non-capturing group: it still groups www\. so ? can apply to the whole thing, but it takes no number, leaving the handle in group 1. The lecture demonstrates exactly this mistake and then fixes it.
6. Why does name.split(", ") fail on "Last,First", and how does the regex version avoid it?
Reference answer: split(", ") requires exactly one space after the comma. Given Malan,David, nothing is split, a one-element list comes back, and unpacking into last, first raises ValueError: not enough values to unpack. The regex ^(.+), ?(.+)$ makes the space optional with ?, so both Malan, David and Malan,David match, and the two capture groups yield the last and first names.
7. Why should you not hand-write a regex to validate email addresses, and what should you do instead?
Reference answer: The legal characters and structure of email addresses are defined by RFC standards, permitting many uncommon characters. A complete regex is very long, hard to maintain, and almost certain to be wrong when written by hand — either rejecting valid addresses or accepting invalid ones. The browser regex shown in the lecture is itself a simplified version that catches most mistakes, not all. In practice, use a mature library such as email_validator for formats with public standards, and reserve regex for your own application's formats such as student IDs, room numbers or course codes.
How it connects
Chapter 7 split file rows on a fixed comma; this chapter upgrades splitting to pattern recognition — CSV has a fixed delimiter, while regex can describe varying structure. Chapter 9 puts the cleaned data into classes, so that a Student object can own its own name validation instead of scattering it across a script. Chapter 10's command-line tools often need regex as well, for free text that goes beyond the structured arguments.