一句话定位

上一讲说清了「概率是什么」，这一讲说清「知道了某件事之后，概率怎么变」。核心只有两个定义——条件概率与独立性——但它们推出的结论会反复与直觉冲突：医生会把 $8\%$ 读成 $75\%$，而一个看起来该等于 $3$ 的期望其实是 $3/2$。

冲突的来源不是公式难，而是样本空间被重新归一化之后，人的直觉没有跟着更新。这一讲用同一个飞镖盘把这件事讲到不能再清楚。

时间轴

复习：上一讲的四件工具

• 样本空间 $\Omega$：一切可能结果的集合，可以是无限的。
• 概率测度 $p$：满足 $p(x)\ge0$ 且 $\sum_{x\in\Omega}p(x)=1$。
• 均匀分布：在有限样本空间 $S$ 上，每个结果都取 $p(x)=1/|S|$。
• 事件：样本空间的子集 $T\subseteq\Omega$，其概率是其中各结果概率之和。

事件的并 $T_1\cup T_2$ 表示「至少发生一个」，交 $T_1\cap T_2$ 表示「两个都发生」。这一讲要在这两种组合之上再加一种：在已知某事件发生的条件下。

条件概率：把样本空间缩到 $B$ 里

定义

设 $P(B)>0$。在已知 $B$ 发生的条件下，$A$ 发生的条件概率定义为

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)} .$$

这个公式没有任何神秘之处：它只是把概率重新归一化到 $B$ 上。原来的测度把总质量 $1$ 分摊在整个 $\Omega$ 上；现在已知结果落在 $B$ 内，于是把 $B$ 内每个结果的概率都除以 $P(B)$，让新的总质量重新等于 $1$。

飞镖盘图像

把样本空间画成一个矩形，概率对应「随机投一支飞镖，落在某区域的概率等于该区域占的比例」。

• 无条件时，$P(A)$ 是区域 $A$ 占整个矩形的比例。
• 已知落在 $B$ 中后，整个「世界」缩成区域 $B$：$P(A\mid B)$ 是 $A\cap B$ 占 $B$ 的比例。

关键的一句是：重要的不是 $A$ 在 $S$ 中占多大，而是 $A\cap B$ 在 $B$ 中占多大。同样大小的 $A$，如果它大部分落在 $B$ 外，那么条件概率会很小；如果它大部分落在 $B$ 内，条件概率会很大。

独立性：条件不改变概率

定义与等价形式

两个事件 $A$、$B$ 称为独立，若

$$P(A\cap B)=P(A)\,P(B).$$

若 $P(B)>0$，两边除以 $P(B)$ 并用条件概率的定义，得到等价形式

$$P(A\mid B)=P(A).$$

两个条件完全等价，检查哪个方便就用哪个。它说的是：知道 $B$ 发生与否，不会改变对 $A$ 的判断。

例子：掷两次骰子

掷一枚公平骰子两次，样本空间是 $36$ 个有序对 $(x_1,x_2)$，每个结果的概率为 $1/36$。

定义

• $A$：$x_1$ 是奇数；
• $B$：$x_2\le 3$。

直观上它们独立，因为一个只看第一次，另一个只看第二次。计算确认：

$$P(A)=\frac{3}{6}=\frac12,\qquad P(B)=\frac{3}{6}=\frac12,$$

而 $A\cap B$ 由 $x_1\in\{1,3,5\}$、$x_2\in\{1,2,3\}$ 的所有组合构成，共 $3\times3=9$ 个结果，所以

$$P(A\cap B)=\frac{9}{36}=\frac14=\frac12\cdot\frac12=P(A)P(B).$$

乘积结构正是独立性的来源：两个事件分别约束不同的坐标，组合数自然相乘。

反直觉之一：$x_1-x_2=3$ 不独立

定义 $C$：$x_1-x_2=3$。满足条件的有

$$(4,1),\ (5,2),\ (6,3),$$

所以 $P(C)=3/36$。

而 $A\cap C$ 只有一个结果 $(5,2)$（$x_1=5$ 是奇数），故 $P(A\cap C)=1/36$。另一方面

$$P(A)P(C)=\frac12\cdot\frac{3}{36}=\frac{3}{72}=\frac{1}{24}\ne\frac{1}{36}.$$

所以 $A$ 与 $C$ 不独立。原因也直观：$C$ 同时涉及 $x_1$ 与 $x_2$，它与 $A$ 共享了对第一次投掷的依赖。

反直觉之二：把 $3$ 换成 $4$ 就独立了

定义 $D$：$x_1-x_2=4$。满足条件的是

$$(5,1),\ (6,2),$$

所以 $P(D)=2/36$。而 $A\cap D$ 只有 $(5,1)$，故 $P(A\cap D)=1/36$。检查：

$$P(A)P(D)=\frac12\cdot\frac{2}{36}=\frac{1}{36}=P(A\cap D).$$

于是 $A$ 与 $D$ 独立。

这个例子值得停一下：两个事件明明都依赖 $x_1$，却满足独立性的定义。它说明独立性是一个计算性质，不是「看起来有没有关系」。当直觉说不清时，只能代公式。

全概率公式：把难算的概率拆开

两分版本

对任意事件 $A$、$B$，

$$P(A)=P(A\cap B)+P(A\cap B^{c}),$$

其中 $B^{c}$ 是 $B$ 的补事件。理由只有一句话：$A$ 被分成「$A$ 且 $B$」与「$A$ 且非 $B$」两块，两块不相交，合起来正好是 $A$。

用条件概率改写每一项，得到

$$P(A)=P(A\mid B)P(B)+P(A\mid B^{c})P(B^{c}).$$

这在实用上极为有力：当 $P(A)$ 很难直接算，但「在各种情形下 $A$ 的概率」容易算时，就拆开算。

一般版本

设 $B_1,\dots,B_k$ 两两不交且并集为整个样本空间（这样的集合族称为一个分割），则

$$P(A)=\sum_{i=1}^{k}P(A\cap B_i)=\sum_{i=1}^{k}P(A\mid B_i)P(B_i).$$

两分版本就是取 $B_1=B$、$B_2=B^{c}$ 的特例。

树状图

全概率公式有一个几乎不需要记的图像：把随机过程想成一棵树。

1. 首先分出若干分支 $B_1,B_2,\dots$，走哪条分支的概率是 $P(B_i)$。
2. 在每个分支之后，事件 $A$ 发生的概率是 $P(A\mid B_i)$。
3. 一条完整路径的概率是沿途概率之积 $P(B_i)P(A\mid B_i)$。
4. 把所有以 $A$ 结尾的路径概率相加，就是 $P(A)$。

树的叶子是样本空间中的结果；树的第一层不是「最终结果」，而是「落在哪个区域」。这个视角的价值在于：它把「先发生什么、再发生什么」的因果顺序直接画了出来。

应用：Monty Hall

三门问题：一扇门后是汽车，另外两扇是山羊。你先选一扇门；主持人知道答案，从你没选的两扇中打开一扇有山羊的门；然后问你要不要换到剩下那扇没开的门。

结论：换，胜率 $2/3$；不换，胜率 $1/3$。

用树状图看，一切都很清楚。奖品在随机位置，所以

• 你初选到山羊的概率是 $2/3$；
• 你初选到汽车的概率是 $1/3$。

如果初选到山羊（概率 $2/3$），主持人没有选择：他只能打开另一扇山羊门。此时剩下的那扇门必然是汽车，换就必赢。

如果初选到汽车（概率 $1/3$），主持人可以打开任意一扇山羊门；换就必输。

把策略固定为「换」，则

$$P(\text{赢})=P(\text{初选山羊})\cdot 1+P(\text{初选汽车})\cdot 0=\frac23 .$$

树状图让「主持人没有选择」这一点变得显眼：他不是随机开门，而是被迫避开汽车。这正是很多人算错的地方——把主持人的行为当成与奖品位置无关的随机事件。

贝叶斯公式：把条件反过来

公式与推导

设 $P(A)>0$、$P(B)>0$，则

$$P(B\mid A)=\frac{P(A\mid B)P(B)}{P(A)} .$$

证明。 由定义 $P(A\mid B)=P(A\cap B)/P(B)$，两边乘以 $P(B)/P(A)$：

$$P(A\mid B)\cdot\frac{P(B)}{P(A)}=\frac{P(A\cap B)}{P(A)} .$$

右边正是 $P(B\mid A)$（因为 $A\cap B=B\cap A$，交运算与顺序无关）。$\square$

推导只有一行，但用法很讲究：它把「已知患病时检测阳性的概率」换成「已知阳性时真正患病的概率」，这两者常常相差一个数量级。

真实例子：钼靶检查

给定以下数据（课程强调这些数字都是真实的）：

• $40$–$50$ 岁女性中约 $1\%$ 患乳腺癌，即 $P(B)=0.01$，其中 $B$ 表示患病；
• 假阳性率 $10\%$，即 $P(A\mid B^{c})=0.10$，其中 $A$ 表示检测阳性；
• 假阴性率 $10\%$，即 $P(A^{c}\mid B)=0.10$，等价于灵敏度 $P(A\mid B)=0.90$。

问：检测阳性时，真正患病的概率是多少？

先算分母（检测阳性的总概率），用全概率公式：

$$P(A)=P(A\mid B)P(B)+P(A\mid B^{c})P(B^{c}) =0.90\times0.01+0.10\times0.99 =0.009+0.099=0.108 .$$

再用贝叶斯公式：

$$P(B\mid A)=\frac{0.90\times0.01}{0.108}=\frac{0.009}{0.108}\approx0.0833 .$$

答案约为 $8.3\%$，远不是 $90\%$。

为什么直觉会错

把人数摊开来看最清楚。假设有 $10{,}000$ 名女性：

阳性者共 $1{,}080$ 人，其中真正患病的只有 $90$ 人，比例约 $8.3\%$。

直觉的错误在于盯着「假阳性率只有 $10\%$」看，却忘了未患病的人数是患病者的 $99$ 倍。基数决定了：即使误报率很低，误报的绝对人数仍然可观。

课程提到，一项对数百名医生的调查显示，平均回答是 $75\%$——比正确答案高出一个数量级。这不是数学难题，而是没有把先验概率放进计算。

随机变量与期望

定义

事件是样本空间的子集；随机变量则是定义在样本空间上的函数

$$f:\Omega\to\mathbb{R},$$

它把每个结果映射成一个实数。

期望是随机变量的加权平均：

$$\mathbb{E}[f]=\sum_{x\in\Omega}p(x)\,f(x).$$

权重正是每个结果发生的概率。赌博的赔付、系统的寿命、抛硬币的次数，都可以写成随机变量。

例一：抛到正面为止

公平硬币一直抛，直到第一次出现正面。样本空间是所有形如

$$H,\ TH,\ TTH,\ TTTH,\dots$$

的序列，其中第 $i$ 个序列（$i$ 个反面后接一个正面）的概率是 $2^{-(i+1)}$。

令 $f$ 为抛掷的总次数，则

$$\mathbb{E}[f]=\sum_{k=1}^{\infty}k\cdot\frac{1}{2^{k}} .$$

这个和可以逐列求和：

$$\begin{aligned} &\frac12+\frac14+\frac18+\cdots=1,\\ &\frac14+\frac18+\cdots=\frac12,\\ &\frac18+\cdots=\frac14, \end{aligned}$$

各列之和依次是 $1,\ \frac12,\ \frac14,\dots$，总和

$$1+\frac12+\frac14+\cdots=2 .$$

所以抛到正面为止的期望次数是 $2$。也可以直接用公式 $\sum_{k\ge1}k r^{k-1}=1/(1-r)^2$，取 $r=1/2$ 得 $4$，再乘 $1/2$ 得 $2$。

例二：平均无故障时间

一台机器每天独立地以概率 $p$ 发生故障，令 $T$ 为它撑过的天数。则

$$P(T=1)=p,\qquad P(T=2)=(1-p)p,\qquad P(T=3)=(1-p)^2p,\ \dots$$

直接求和会很麻烦，但有一个递归技巧。把第一次观察拆开：

• 以概率 $p$，第一天就坏，$T=1$；
• 以概率 $1-p$，第一天没坏，那么整个过程重新开始，只是已经多花了一天，于是剩余寿命的期望仍是 $\mathbb{E}[T]$。

所以

$$\mathbb{E}[T]=p\cdot1+(1-p)\cdot(1+\mathbb{E}[T]).$$

整理：

$$\mathbb{E}[T]=1+(1-p)\mathbb{E}[T] \;\Longrightarrow\; p\,\mathbb{E}[T]=1 \;\Longrightarrow\; \mathbb{E}[T]=\frac1p .$$

这个「无记忆性」是几何分布的核心：过去没有坏，不会让明天更容易坏。对抛硬币例子取 $p=1/2$，立刻得到 $\mathbb{E}[f]=2$，与逐列求和一致。

指示函数与期望的线性性

把事件变成随机变量

对事件 $A$，定义指示函数

$$\mathbf{1}_A(x)= \begin{cases} 1, & x\in A,\\ 0, & x\notin A . \end{cases}$$

它的期望恰好是 $A$ 的概率：

$$\mathbb{E}[\mathbf{1}_A]=\sum_{x\in\Omega}p(x)\mathbf{1}_A(x)=\sum_{x\in A}p(x)=P(A).$$

因为求和里只有属于 $A$ 的项非零，其余全部消失。于是事件的概率问题可以翻译成随机变量的期望问题，两种语言随时互换。

线性性

对任意随机变量 $f,g$ 与常数 $a,b$，

$$\mathbb{E}[af+bg]=a\mathbb{E}[f]+b\mathbb{E}[g].$$

这条性质对任意随机变量成立，不要求独立。证明只是把求和拆开重排：

$$\sum_x p(x)\bigl(af(x)+bg(x)\bigr) =a\sum_x p(x)f(x)+b\sum_x p(x)g(x).$$

一个容易踩的坑

线性性给出

$$\mathbb{E}[\mathbf{1}_A+\mathbf{1}_B]=P(A)+P(B).$$

但这不等于 $P(A\cup B)$。原因是 $\mathbf{1}_A+\mathbf{1}_B$ 不再是 $0$–$1$ 指示函数：当结果落在 $A\cap B$ 中时，它取值为 $2$。也就是说，交集被数了两次。

正确的关系是

$$P(A\cup B)=P(A)+P(B)-P(A\cap B),$$

这正是下一讲容斥原理的起点。用期望的语言说：线性性永远成立，但「把指示函数相加」与「把事件取并」是两件不同的事。

Mossel 骰子悖论

问题

不断掷一枚公平骰子，直到出现 $6$ 为止，记 $T$ 为掷的总次数。已知所有掷出的点数都是偶数，问 $\mathbb{E}[T\mid \text{全为偶数}]$ 等于多少？

错误直觉：$3$

一个很自然的推理是：既然每次都是偶数，可能的点数只有 $2,4,6$，每次出现 $6$ 的概率是 $1/3$，所以期望次数是 $1/(1/3)=3$。

这个推理听起来无懈可击，但答案是错的。

正确计算：$3/2$

先算条件事件的概率。要使「所有掷出都是偶数」且总共掷 $k$ 次，前 $k-1$ 次必须都是 $2$ 或 $4$（每次概率 $2/6=1/3$），最后一次必须是 $6$（概率 $1/6$）。于是

$$P(T=k,\ \text{全为偶数})=\left(\frac13\right)^{k-1}\frac16 .$$

对所有 $k$ 求和：

$$P(\text{全为偶数})=\sum_{k=1}^{\infty}\left(\frac13\right)^{k-1}\frac16 =\frac16\cdot\frac{1}{1-\frac13} =\frac16\cdot\frac32=\frac14 .$$

条件分布因此是

$$P(T=k\mid \text{全为偶数}) =\frac{\left(\frac13\right)^{k-1}\frac16}{\frac14} =\frac23\left(\frac13\right)^{k-1}.$$

这是一个成功概率为 $2/3$ 的几何分布，期望为

$$\mathbb{E}[T\mid \text{全为偶数}]=\frac{1}{2/3}=\frac32 .$$

也可以直接用定义验证：

$$\mathbb{E}[T\cdot \mathbf{1}_{\text{全为偶数}}] =\sum_{k\ge1}k\left(\frac13\right)^{k-1}\frac16 =\frac16\cdot\frac{1}{(1-\frac13)^2} =\frac16\cdot\frac94=\frac38,$$

再除以 $P=1/4$ 得 $3/2$。

错在哪里

条件化不是「把每次掷骰的分布换成均匀的 $\{2,4,6\}$」。条件事件「所有掷出都是偶数」对序列长度有偏置：序列越长，满足条件的概率按 $(1/3)^{k-1}$ 衰减，因此长的序列被大幅压低权重。

从条件分布看得很清楚：$\mathbb{E}[T\mid\text{全为偶数}]=3/2<3$，而且

$$P(T=1\mid \text{全为偶数})=\frac{1/6}{1/4}=\frac23 .$$

也就是说，在已知全为偶数的条件下，第一次就掷出 $6$ 的概率高达 $2/3$——因为一旦第一掷是 $6$，实验立刻结束，「全为偶数」自动成立；而若第一掷是 $2$ 或 $4$，后面还要继续，且必须一直保持偶数。

这个例子说明：条件概率的正确性只能靠计算保证，直觉在这里系统性地偏向错误方向。

本节结论

• 条件概率 $P(A\mid B)=P(A\cap B)/P(B)$ 是在 $B$ 上重新归一化；看的是 $A\cap B$ 在 $B$ 中的比例。
• 独立性有两种等价写法：$P(A\cap B)=P(A)P(B)$ 或 $P(A\mid B)=P(A)$；它必须按定义验证，不能凭「看起来有没有关系」判断。
• 全概率公式把 $P(A)$ 按一个分割拆成 $\sum_i P(A\mid B_i)P(B_i)$，树状图是它的直观图像。
• 贝叶斯公式 $P(B\mid A)=P(A\mid B)P(B)/P(A)$ 把条件反过来；钼靶例子中，阳性后真正患病的概率只有约 $8\%$，因为先验只有 $1\%$。
• 随机变量是 $\Omega\to\mathbb R$ 的函数，期望是 $\sum p(x)f(x)$。
• 几何分布的期望是 $1/p$；抛硬币到正面为 $2$ 次，平均无故障时间为 $1/p$。
• $\mathbb{E}[\mathbf{1}_A]=P(A)$；期望的线性性无条件成立，但 $\mathbf{1}_A+\mathbf{1}_B$ 不是并集的指示函数——交集被重复计数。
• 条件期望可能严重偏离朴素直觉：Mossel 悖论的答案是 $3/2$，不是 $3$。

理解检验

1. 用飞镖盘解释：为什么 $P(A\mid B)$ 与 $A$ 在整个样本空间中占的比例无关？

参考答案：条件化把「世界」从 $\Omega$ 缩小成 $B$：已知结果落在 $B$ 内，就把 $B$ 的总概率重新归一化为 $1$。此时 $A$ 能发生的位置只剩下 $A\cap B$，所以

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)}=\frac{A\cap B\text{ 占 }B\text{ 的比例}}{1}.$$

若 $A$ 有一大块落在 $B$ 之外，那部分在条件化后被完全排除，不贡献任何概率；反过来，$A$ 在 $\Omega$ 中很小但几乎全在 $B$ 内时，条件概率可以很大。决定条件概率的是 $A\cap B$ 相对于 $B$ 的大小。

2. 掷两次公平骰子，$A=\{x_1\text{ 为奇数}\}$，$C=\{x_1-x_2=3\}$，$D=\{x_1-x_2=4\}$。验证 $A$ 与 $C$ 不独立、$A$ 与 $D$ 独立。

参考答案：$A$ 有 $18$ 个结果，$P(A)=1/2$。$C=\{(4,1),(5,2),(6,3)\}$，$P(C)=3/36$；$A\cap C=\{(5,2)\}$，$P(A\cap C)=1/36$。而 $P(A)P(C)=\frac12\cdot\frac{3}{36}=\frac1{24}\ne\frac1{36}$，故不独立。 $D=\{(5,1),(6,2)\}$，$P(D)=2/36$；$A\cap D=\{(5,1)\}$，$P(A\cap D)=1/36$。而 $P(A)P(D)=\frac12\cdot\frac{2}{36}=\frac1{36}$，恰好相等，故独立。两个事件都依赖 $x_1$，却仍可独立——独立性是计算性质，不是因果性质。

3. 写出全概率公式的一般形式，并说明对 $B_i$ 的两个要求各自起什么作用。

参考答案：若 $B_1,\dots,B_k$ 两两不交且 $\bigcup_i B_i=\Omega$，则

$$P(A)=\sum_{i=1}^{k}P(A\mid B_i)P(B_i).$$

「两两不交」保证 $A$ 被切成互不重叠的块 $A\cap B_i$，相加时不会重复计数；「并集为 $\Omega$」保证没有遗漏，$A$ 的每一部分都落在某个 $B_i$ 里。缺任何一个，等式都不成立：不交缺失会重复计，覆盖缺失会漏计。两分版本取 $B_1=B$、$B_2=B^c$。

4. 钼靶例子中，若把患病率改成 $10\%$（其余不变），$P(B\mid A)$ 变成多少？这说明什么？

参考答案：$P(A)=0.9\times0.1+0.1\times0.9=0.09+0.09=0.18$，于是

$$P(B\mid A)=\frac{0.9\times0.1}{0.18}=0.5 .$$

患病率从 $1\%$ 提高到 $10\%$，阳性后的患病概率从约 $8.3\%$ 升到 $50\%$。这说明同样的检测，结论强烈依赖先验概率：检测的灵敏度与假阳性率没有变，变的只是被检测人群中真正患病者的比例。忽略先验（基率）是医学判断中最常见的条件概率错误。

5. 为什么 $\mathbb{E}[\mathbf{1}_A+\mathbf{1}_B]=P(A)+P(B)$，却不能由此推出 $P(A\cup B)=P(A)+P(B)$？

参考答案：期望的线性性对任意随机变量成立，不需要独立，所以等式左边确实等于 $P(A)+P(B)$。但 $\mathbf{1}_A+\mathbf{1}_B$ 与 $\mathbf{1}_{A\cup B}$ 不是同一个随机变量：当结果落在 $A\cap B$ 时，前者取 $2$，后者取 $1$。因此 $P(A\cup B)=\mathbb{E}[\mathbf{1}_{A\cup B}]$ 并不等于 $\mathbb{E}[\mathbf{1}_A+\mathbf{1}_B]$。正确关系是

$$P(A\cup B)=P(A)+P(B)-P(A\cap B),$$

即容斥原理；差的那一项正是被重复计数的交集。

6. Mossel 悖论中，为什么「只剩 $2,4,6$ 三种可能，所以期望是 $3$」是错的？正确值 $3/2$ 的直觉解释是什么？

参考答案：错误在于把条件化当成「每次独立地在 $\{2,4,6\}$ 上均匀取值」。实际上条件事件是「整段序列的所有掷出都是偶数」，它的概率随序列长度按 $(1/3)^{k-1}$ 衰减，因此条件分布对短序列赋予了更大权重。精确计算：

$$P(T=k\mid\text{全偶})=\frac{(1/3)^{k-1}(1/6)}{1/4}=\frac23\left(\frac13\right)^{k-1},$$

这是成功概率 $2/3$ 的几何分布，期望 $3/2$。直觉解释：在「全为偶数」的条件下，第一掷就是 $6$ 的概率是 $P(T=1)/P(\text{全偶})=(1/6)/(1/4)=2/3$——第一掷为 $6$ 时实验立刻结束，条件自动满足；第一掷为 $2$ 或 $4$ 时还必须继续并一直保持偶数，路径越长越难。条件化系统性地偏向「早停」。

与前后课的关系

这一讲把「事件」这套语言用到了极限：条件、独立、分割、树状图、贝叶斯。下一讲用容斥原理处理「至少一个事件发生」的计数，正好接上本讲最后指出的问题——指示函数相加会重复计数交集。此后生成函数、尾界与编码理论都会反复调用本讲的工具，尤其是全概率公式与期望的线性性。

一句话记住这一讲：先验决定了后验的量级，而条件化会改变整个分布，不只是改变一个数字。

One-sentence orientation

The previous lecture said what probability is; this one says how probability changes once something is known. Everything rests on two definitions — conditional probability and independence — yet their consequences keep colliding with intuition: doctors read $8\%$ as $75\%$, and an expectation that looks like $3$ is actually $3/2$.

The collision is not caused by hard formulas. It is caused by renormalising the sample space while leaving human intuition un-updated. A single dartboard picture settles the matter.

Timeline

Review: four tools from last time

• Sample space $\Omega$: the set of all possible outcomes, possibly infinite.
• Probability measure $p$: $p(x)\ge0$ and $\sum_{x\in\Omega}p(x)=1$.
• Uniform distribution: on a finite $S$, every outcome gets $p(x)=1/|S|$.
• Event: a subset $T\subseteq\Omega$, with probability the sum of the probabilities of its outcomes.

The union $T_1\cup T_2$ means "at least one happens"; the intersection $T_1\cap T_2$ means "both happen". This lecture adds a third way to combine: given that something happened.

Conditional probability: shrink the world to $B$

Definition

For $P(B)>0$,

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)} .$$

There is nothing mysterious here: the probability is renormalised onto $B$. The original measure spread mass $1$ over $\Omega$; once the outcome is known to lie in $B$, every outcome inside $B$ is divided by $P(B)$ so that the new total mass is again $1$.

The dartboard picture

Draw the sample space as a rectangle, with probability corresponding to "a dart thrown at random lands in a region with probability proportional to its area".

• Unconditionally, $P(A)$ is the fraction of the whole rectangle covered by $A$.
• Given that the dart lies in $B$, the world shrinks to $B$: $P(A\mid B)$ is the fraction of $B$ covered by $A\cap B$.

The key sentence: what matters is not how much of $S$ that $A$ occupies, but how much of $B$ that $A\cap B$ occupies. A set $A$ of fixed size has a small conditional probability if most of it lies outside $B$, and a large one if most of it lies inside.

Independence: conditioning changes nothing

Definition and equivalent form

Events $A$ and $B$ are independent if

$$P(A\cap B)=P(A)\,P(B).$$

When $P(B)>0$, dividing by $P(B)$ and using the definition of conditional probability gives

$$P(A\mid B)=P(A).$$

The two conditions are equivalent; use whichever is convenient. In words: learning whether $B$ happened does not change your assessment of $A$.

Example: two rolls of a die

Roll a fair die twice. The sample space is the $36$ ordered pairs $(x_1,x_2)$, each with probability $1/36$.

Let

• $A$: $x_1$ is odd;
• $B$: $x_2\le 3$.

They look independent, since one constrains only the first roll and the other only the second. Compute:

$$P(A)=\frac{3}{6}=\frac12,\qquad P(B)=\frac{3}{6}=\frac12,$$

and $A\cap B$ consists of all pairs with $x_1\in\{1,3,5\}$ and $x_2\in\{1,2,3\}$, namely $3\times3=9$ outcomes, so

$$P(A\cap B)=\frac{9}{36}=\frac14=\frac12\cdot\frac12=P(A)P(B).$$

The product structure is exactly where independence comes from: the two events constrain different coordinates, so the counts multiply.

Counterintuitive case one: $x_1-x_2=3$ is not independent

Let $C$: $x_1-x_2=3$. The outcomes are

$$(4,1),\ (5,2),\ (6,3),$$

so $P(C)=3/36$. Now $A\cap C$ contains only $(5,2)$ (since $x_1=5$ is odd), so $P(A\cap C)=1/36$. But

$$P(A)P(C)=\frac12\cdot\frac{3}{36}=\frac{3}{72}=\frac{1}{24}\ne\frac{1}{36}.$$

Hence $A$ and $C$ are not independent. The reason is visible: $C$ involves both $x_1$ and $x_2$, and therefore shares the dependence on the first roll with $A$.

Counterintuitive case two: replace $3$ by $4$ and independence appears

Let $D$: $x_1-x_2=4$. The outcomes are

$$(5,1),\ (6,2),$$

so $P(D)=2/36$, while $A\cap D=\{(5,1)\}$, so $P(A\cap D)=1/36$. Now

$$P(A)P(D)=\frac12\cdot\frac{2}{36}=\frac{1}{36}=P(A\cap D).$$

So $A$ and $D$ are independent.

This deserves a pause: both events clearly depend on $x_1$, yet the definition is satisfied. Independence is a computational property, not a causal one. When intuition is unsure, substitute into the formula.

The law of total probability: split a hard probability

Two-part version

For any events $A$ and $B$,

$$P(A)=P(A\cap B)+P(A\cap B^{c}),$$

where $B^{c}$ is the complement of $B$. The reason is one sentence: $A$ splits into "$A$ and $B$" together with "$A$ and not $B$"; the pieces are disjoint and their union is $A$.

Rewriting each piece with conditional probability,

$$P(A)=P(A\mid B)P(B)+P(A\mid B^{c})P(B^{c}).$$

This is powerful in practice: when $P(A)$ is hard to compute directly but "the probability of $A$ within each scenario" is easy, split it.

General version

Let $B_1,\dots,B_k$ be pairwise disjoint with union the whole sample space (such a family is a partition). Then

$$P(A)=\sum_{i=1}^{k}P(A\cap B_i)=\sum_{i=1}^{k}P(A\mid B_i)P(B_i).$$

The two-part version is the special case $B_1=B$, $B_2=B^{c}$.

The tree diagram

The law has a picture that barely needs memorising: think of the random process as a tree.

1. The first level splits into branches $B_1,B_2,\dots$, taken with probabilities $P(B_i)$.
2. After each branch, $A$ happens with probability $P(A\mid B_i)$.
3. The probability of a complete path is the product $P(B_i)P(A\mid B_i)$.
4. Summing the probabilities of all paths that end in $A$ gives $P(A)$.

The leaves are outcomes; the first level is not "the final answer" but "which region the outcome falls in". The value of the picture is that it displays the causal order — what happens first, what happens next.

Application: Monty Hall

Three doors, a car behind one and goats behind the others. You pick a door; the host, who knows the answer, opens a different door revealing a goat; then you may switch to the remaining unopened door.

Result: switching wins with probability $2/3$; staying wins with probability $1/3$.

The tree makes it obvious. The prize is placed uniformly, so

• your first pick is a goat with probability $2/3$;
• your first pick is the car with probability $1/3$.

If your first pick is a goat (probability $2/3$), the host has no choice: he must open the only other goat door. The remaining door is then necessarily the car, so switching wins for sure.

If your first pick is the car (probability $1/3$), the host may open either goat door, and switching loses.

Fix the strategy "always switch". Then

$$P(\text{win})=P(\text{first pick goat})\cdot 1+P(\text{first pick car})\cdot 0=\frac23 .$$

The tree exposes the point people miss: the host is not opening a door at random — he is forced to avoid the car. Treating his action as an event independent of the prize location is precisely the error.

Bayes' rule: turn the conditional around

Formula and derivation

For $P(A)>0$ and $P(B)>0$,

$$P(B\mid A)=\frac{P(A\mid B)P(B)}{P(A)} .$$

Proof. By definition $P(A\mid B)=P(A\cap B)/P(B)$. Multiply both sides by $P(B)/P(A)$:

$$P(A\mid B)\cdot\frac{P(B)}{P(A)}=\frac{P(A\cap B)}{P(A)} .$$

The right-hand side is $P(B\mid A)$, since $A\cap B=B\cap A$. $\square$

The derivation is one line, but its use is subtle: it converts "the chance of a positive test given disease" into "the chance of disease given a positive test". Those two numbers often differ by an order of magnitude.

Real data: mammography

The lecture stresses that all numbers are real:

• about $1\%$ of women aged $40$–$50$ have breast cancer, so $P(B)=0.01$, where $B$ denotes disease;
• the false positive rate is $10\%$, i.e. $P(A\mid B^{c})=0.10$, where $A$ denotes a positive test;
• the false negative rate is $10\%$, i.e. $P(A^{c}\mid B)=0.10$, equivalently sensitivity $P(A\mid B)=0.90$.

Question: given a positive test, what is the probability of actually having the disease?

First compute the denominator with the law of total probability:

$$P(A)=P(A\mid B)P(B)+P(A\mid B^{c})P(B^{c}) =0.90\times0.01+0.10\times0.99 =0.009+0.099=0.108 .$$

Then Bayes:

$$P(B\mid A)=\frac{0.90\times0.01}{0.108}=\frac{0.009}{0.108}\approx0.0833 .$$

About $8.3\%$, not $90\%$.

Why intuition fails

Lay out the population. Among $10{,}000$ women:

There are $1{,}080$ positives, of whom only $90$ truly have the disease — about $8.3\%$.

The error is to stare at "false positives are only $10\%$" while forgetting that people without the disease outnumber those with it $99$ to $1$. The base rate decides: even a low error rate produces a large number of false alarms.

The lecture notes that a survey of hundreds of physicians averaged $75\%$ — an order of magnitude too high. This is not a hard calculation; it is a failure to put the prior into the computation.

Random variables and expectation

Definition

Events are subsets of the sample space; a random variable is a function on the sample space,

$$f:\Omega\to\mathbb{R},$$

assigning a real number to each outcome.

The expectation is the weighted average

$$\mathbb{E}[f]=\sum_{x\in\Omega}p(x)\,f(x),$$

with each outcome weighted by its probability. Payoffs, lifetimes and numbers of coin flips all become random variables.

Example one: flip until heads

Flip a fair coin until the first head. The sample space consists of

$$H,\ TH,\ TTH,\ TTTH,\dots$$

where the sequence with $i$ tails has probability $2^{-(i+1)}$.

Let $f$ be the total number of flips. Then

$$\mathbb{E}[f]=\sum_{k=1}^{\infty}k\cdot\frac{1}{2^{k}} .$$

Sum column by column:

$$\begin{aligned} &\frac12+\frac14+\frac18+\cdots=1,\\ &\frac14+\frac18+\cdots=\frac12,\\ &\frac18+\cdots=\frac14, \end{aligned}$$

so the column totals are $1,\ \frac12,\ \frac14,\dots$ and

$$1+\frac12+\frac14+\cdots=2 .$$

Hence the expected number of flips is $2$. The closed form $\sum_{k\ge1}k r^{k-1}=1/(1-r)^2$ with $r=1/2$ gives $4$, times $1/2$, again $2$.

Example two: mean time to failure

A machine fails independently each day with probability $p$. Let $T$ be the number of days it survives. Then

$$P(T=1)=p,\qquad P(T=2)=(1-p)p,\qquad P(T=3)=(1-p)^2p,\ \dots$$

Summing directly is painful, but a recursion settles it. Split on the first day:

• with probability $p$ it fails immediately, $T=1$;
• with probability $1-p$ it survives, at which point the whole process restarts, one day older, so the remaining expected lifetime is again $\mathbb{E}[T]$.

Therefore

$$\mathbb{E}[T]=p\cdot1+(1-p)\cdot(1+\mathbb{E}[T]).$$

Rearranging,

$$\mathbb{E}[T]=1+(1-p)\mathbb{E}[T] \;\Longrightarrow\; p\,\mathbb{E}[T]=1 \;\Longrightarrow\; \mathbb{E}[T]=\frac1p .$$

This memorylessness is the heart of the geometric distribution: having survived so far does not make tomorrow more likely to fail. For the coin example $p=1/2$ gives $\mathbb{E}[f]=2$, matching the column sum.

Indicator functions and linearity

Turning events into random variables

For an event $A$, define the indicator

$$\mathbf{1}_A(x)= \begin{cases} 1, & x\in A,\\ 0, & x\notin A . \end{cases}$$

Its expectation is exactly the probability of $A$:

$$\mathbb{E}[\mathbf{1}_A]=\sum_{x\in\Omega}p(x)\mathbf{1}_A(x)=\sum_{x\in A}p(x)=P(A),$$

because only the terms with $x\in A$ survive. Events and random variables can therefore be translated back and forth at will.

Linearity

For any random variables $f,g$ and constants $a,b$,

$$\mathbb{E}[af+bg]=a\mathbb{E}[f]+b\mathbb{E}[g].$$

This holds for all random variables, with no independence required. The proof just splits and rearranges the sum:

$$\sum_x p(x)\bigl(af(x)+bg(x)\bigr) =a\sum_x p(x)f(x)+b\sum_x p(x)g(x).$$

A trap worth naming

Linearity gives

$$\mathbb{E}[\mathbf{1}_A+\mathbf{1}_B]=P(A)+P(B).$$

But this is not $P(A\cup B)$. The sum $\mathbf{1}_A+\mathbf{1}_B$ is no longer a $0$–$1$ indicator: on $A\cap B$ it equals $2$. The intersection is counted twice.

The correct relation is

$$P(A\cup B)=P(A)+P(B)-P(A\cap B),$$

which is the seed of the next lecture's inclusion–exclusion. In expectation language: linearity always holds, but adding indicators is not the same as taking the union of events.

Mossel's dice paradox

The question

Roll a fair die until the first $6$ appears, and let $T$ be the number of rolls. Given that every roll was even, what is $\mathbb{E}[T\mid\text{all even}]$?

The wrong intuition: $3$

A natural argument: since every roll is even, only $2,4,6$ are possible; each roll shows $6$ with probability $1/3$; so the expected number of rolls is $1/(1/3)=3$.

It sounds airtight. It is wrong.

The correct value: $3/2$

First compute the probability of the conditioning event. For the process to stop at roll $k$ with all rolls even, the first $k-1$ rolls must be $2$ or $4$ (probability $2/6=1/3$ each) and the last must be $6$ (probability $1/6$). Hence

$$P(T=k,\ \text{all even})=\left(\frac13\right)^{k-1}\frac16 .$$

Summing over $k$,

$$P(\text{all even})=\sum_{k=1}^{\infty}\left(\frac13\right)^{k-1}\frac16 =\frac16\cdot\frac{1}{1-\frac13} =\frac16\cdot\frac32=\frac14 .$$

The conditional distribution is therefore

$$P(T=k\mid \text{all even}) =\frac{\left(\frac13\right)^{k-1}\frac16}{\frac14} =\frac23\left(\frac13\right)^{k-1},$$

a geometric distribution with success probability $2/3$, whose expectation is

$$\mathbb{E}[T\mid \text{all even}]=\frac{1}{2/3}=\frac32 .$$

As a check,

$$\mathbb{E}[T\cdot \mathbf{1}_{\text{all even}}] =\sum_{k\ge1}k\left(\frac13\right)^{k-1}\frac16 =\frac16\cdot\frac{1}{(1-\frac13)^2} =\frac16\cdot\frac94=\frac38,$$

and dividing by $P=1/4$ gives $3/2$.

What went wrong

Conditioning is not "replace each roll's distribution by the uniform distribution on $\{2,4,6\}$". The event "all rolls are even" biases the length of the sequence: the probability of satisfying it decays like $(1/3)^{k-1}$, so long sequences are heavily down-weighted.

The conditional law makes this plain: $\mathbb{E}[T\mid\text{all even}]=3/2<3$, and

$$P(T=1\mid \text{all even})=\frac{1/6}{1/4}=\frac23 .$$

That is, given that all rolls were even, the first roll was a $6$ with probability $2/3$ — because a first roll of $6$ ends the experiment immediately and satisfies the condition automatically, whereas a first roll of $2$ or $4$ forces the process to continue and to stay even for longer and longer.

The lesson: correctness of conditional probability can only be guaranteed by computing. Here intuition is systematically wrong.

What this lecture establishes

• Conditional probability $P(A\mid B)=P(A\cap B)/P(B)$ renormalises onto $B$; what matters is the share of $B$ occupied by $A\cap B$.
• Independence has two equivalent forms: $P(A\cap B)=P(A)P(B)$ or $P(A\mid B)=P(A)$; it must be verified by definition, not by appearance.
• The law of total probability splits $P(A)$ as $\sum_i P(A\mid B_i)P(B_i)$ over a partition; the tree diagram is its picture.
• Bayes' rule $P(B\mid A)=P(A\mid B)P(B)/P(A)$ reverses the conditional; in the mammogram example a positive test implies disease with probability only about $8\%$, because the prior is $1\%$.
• A random variable is a function $\Omega\to\mathbb R$; its expectation is $\sum p(x)f(x)$.
• The geometric distribution has expectation $1/p$: $2$ flips on average to see a head, and $1/p$ days to failure.
• $\mathbb{E}[\mathbf{1}_A]=P(A)$; linearity is unconditional, but $\mathbf{1}_A+\mathbf{1}_B$ is not the indicator of a union — the intersection is double-counted.
• Conditional expectation can depart sharply from naive intuition: Mossel's paradox gives $3/2$, not $3$.

Comprehension check

1. Using the dartboard, explain why $P(A\mid B)$ does not depend on the share of the whole sample space occupied by $A$.

Reference answer: Conditioning shrinks the world from $\Omega$ to $B$: knowing the outcome lies in $B$, we renormalise so that the total probability of $B$ becomes $1$. The only place where $A$ can still occur is $A\cap B$, so

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)}=\frac{\text{share of }B\text{ covered by }A\cap B}{1}.$$

Any part of $A$ lying outside $B$ is excluded and contributes nothing; conversely, a small $A$ lying almost entirely inside $B$ can have a large conditional probability. The deciding quantity is the size of $A\cap B$ relative to $B$.

2. Roll a fair die twice. Let $A=\{x_1\text{ odd}\}$, $C=\{x_1-x_2=3\}$, $D=\{x_1-x_2=4\}$. Verify that $A$ and $C$ are dependent while $A$ and $D$ are independent.

Reference answer: $A$ contains $18$ outcomes, $P(A)=1/2$. $C=\{(4,1),(5,2),(6,3)\}$ with $P(C)=3/36$; $A\cap C=\{(5,2)\}$ with $P(A\cap C)=1/36$. But $P(A)P(C)=\frac12\cdot\frac{3}{36}=\frac1{24}\ne\frac1{36}$, so they are dependent. $D=\{(5,1),(6,2)\}$ with $P(D)=2/36$; $A\cap D=\{(5,1)\}$ with $P(A\cap D)=1/36$. And $P(A)P(D)=\frac12\cdot\frac{2}{36}=\frac1{36}$, equal, so they are independent. Both events depend on $x_1$, yet independence holds — a computational property, not a causal one.

3. State the general law of total probability and explain what each condition on the $B_i$ is needed for.

Reference answer: If $B_1,\dots,B_k$ are pairwise disjoint with $\bigcup_i B_i=\Omega$, then

$$P(A)=\sum_{i=1}^{k}P(A\mid B_i)P(B_i).$$

Pairwise disjointness ensures $A$ is cut into non-overlapping pieces $A\cap B_i$, so nothing is counted twice; covering $\Omega$ ensures nothing is missed, so every part of $A$ lies in some $B_i$. Without either condition the identity fails. The two-part version takes $B_1=B$, $B_2=B^c$.

4. In the mammogram example, if the prevalence were $10\%$ instead of $1\%$ (all else equal), what would $P(B\mid A)$ be, and what does that show?

Reference answer: $P(A)=0.9\times0.1+0.1\times0.9=0.09+0.09=0.18$, so

$$P(B\mid A)=\frac{0.9\times0.1}{0.18}=0.5 .$$

Raising prevalence from $1\%$ to $10\%$ raises the post-positive probability from about $8.3\%$ to $50\%$. The test's sensitivity and false positive rate are unchanged; only the composition of the tested population changed. Ignoring the base rate is the most common conditional-probability error in medicine.

5. Why is $\mathbb{E}[\mathbf{1}_A+\mathbf{1}_B]=P(A)+P(B)$ true, and why does it not imply $P(A\cup B)=P(A)+P(B)$?

Reference answer: Linearity of expectation holds for arbitrary random variables with no independence requirement, so the left side indeed equals $P(A)+P(B)$. But $\mathbf{1}_A+\mathbf{1}_B$ and $\mathbf{1}_{A\cup B}$ are different random variables: on $A\cap B$ the former equals $2$ while the latter equals $1$. Hence $P(A\cup B)=\mathbb{E}[\mathbf{1}_{A\cup B}]$ is not $\mathbb{E}[\mathbf{1}_A+\mathbf{1}_B]$. The correct identity is

$$P(A\cup B)=P(A)+P(B)-P(A\cap B),$$

the inclusion–exclusion principle; the subtracted term is exactly the double-counted intersection.

6. In Mossel's paradox, why is "only $2,4,6$ remain, so the expectation is $3$" wrong, and what is the intuitive reading of $3/2$?

Reference answer: The error is treating conditioning as "each roll is independently uniform on $\{2,4,6\}$". The event is actually "the entire sequence is even", whose probability decays like $(1/3)^{k-1}$, so the conditional law heavily down-weights long sequences. Precisely,

$$P(T=k\mid\text{all even})=\frac{(1/3)^{k-1}(1/6)}{1/4}=\frac23\left(\frac13\right)^{k-1},$$

a geometric law with success probability $2/3$, expectation $3/2$. Intuition: under the condition, the first roll is $6$ with probability $P(T=1)/P(\text{all even})=(1/6)/(1/4)=2/3$ — a first roll of $6$ ends the experiment at once and satisfies the condition for free, while a first roll of $2$ or $4$ forces a longer run that must stay even throughout. Conditioning systematically favours early stopping.

Relation to neighbouring lectures

This lecture pushes the language of events to its limit: conditioning, independence, partitions, trees, Bayes. The next lecture uses inclusion–exclusion to count "at least one event occurs", which answers the double-counting problem raised at the end here. Generating functions, tail bounds and coding theory later reuse these tools, especially the law of total probability and linearity of expectation.

One sentence to keep: the prior sets the order of magnitude of the posterior, and conditioning changes the whole distribution, not just a number.