Data Analytics / Chapter 05

The grammar of uncertainty

An alert changes what we know. It does not tell us everything. Move the prior rate and inspect the denominator before trusting the red light.

Private local developmental companion · constructed counts · no tracking or remote execution.

In this chapter
  1. What does a flagged component mean?
  2. At least one event
  3. A rate needs a duration
  4. Run the exact checks
  5. Code and captured output
Investigation 01 · reverse the conditional

What does a flagged component mean?

Set a prior defect rate and the instrument's conditional performance. The table shows expected counts among 10,000 components; fractional expected counts are allowed.

15.4% of alerts are defects

90 / (90 + 495) = 15.4%. Sensitivity is 90 / 100 = 90%; different denominators.

True stateAlertNo alertTotal
Defect9010100
Sound4959,4059,900
Fixed textbook reference tree for 10,000 components: 100 defects and 9,900 sound items branch into alerts and no-alerts. This image stays at the printed values when controls change.

The tree is fixed at the printed chapter values. The table and answer above respond to your inputs.

Investigation 02 · count opportunities

At least one event

For independent trials with a common event chance, no events has probability (1−p)n. A shared batch shock breaks independence.

65.1% at least one

1−(1−0.1)10 = 0.6513.

Investigation 03 · count exposure

A rate needs a duration

Under a constant-rate Poisson model, zero events in an interval has probability e−rt. Compare a half hour and two hours.

36.8% chance of none

Exposure mean: 1.0 event; P(0) = e−1.

Reproduce · question · extend

Run the exact checks

Download the standard-library Python lab. It reconstructs the four cells, verifies Bayes' rule, checks a binomial mass sum and enumerates the hypergeometric example.

python3 uncertainty_lab.py
Flagged: 585/10000; defective among flagged: 90/585 = 0.153846
Hypergeometric N=5,K=3,n=2: P(0,1,2) = 1/10, 3/5, 3/10
All exact and numerical checks passed.
Why can a 90% sensitive instrument have only 15.4% true defects among alerts?

It sees a rare defect. A 5% false-alert rate applies to 9,900 sound items, creating 495 false alerts beside 90 true alerts.

When does the binomial shortcut fail?

It needs a fixed number of independent trials with one common success probability. Shared shocks, changing conditions and sampling without replacement can break the model.

Fixed reference curve: with 90% sensitivity and 5% false-alert rate, true defects among alerts rise as prior defect prevalence rises.Fixed textbook comparison of binomial and Poisson probability masses and normal density; shaded area under the density represents probability.

Figures are fixed book illustrations and do not update with controls.

Chapter 5 · private developmental edition. No public QR route has been approved or released.

The reproducible notebook

Inspect the work.

Read the programs, run them in your own environment, and compare the results. The captured output records the chapter’s checked practical run; changing an input may change its answer.

Python uncertainty_lab.py
"""Executed, deterministic companion to Chapter 5. Python standard library only."""
from fractions import Fraction
from math import comb, exp, factorial, isclose

def posterior(prior, sensitivity, false_alert):
    numerator = sensitivity * prior
    denominator = numerator + false_alert * (1 - prior)
    if denominator == 0:
        raise ValueError("The alert event has zero probability under this model")
    return numerator / denominator

cells = {"defect_alert": 90, "defect_no_alert": 10,
         "sound_alert": 495, "sound_no_alert": 9405}
assert sum(cells.values()) == 10000
flagged = cells["defect_alert"] + cells["sound_alert"]
bayes = posterior(.01, .90, .05)
assert isclose(bayes, 90 / 585, abs_tol=1e-15)
print(f"Flagged: {flagged}/10000; defective among flagged: {cells['defect_alert']}/{flagged} = {bayes:.6f}")
print(f"Defective among unflagged: 10/9415 = {10/9415:.6f}")

for prior in (.01, .05, .1):
    print(f"Prior {prior:.0%} -> after alert {posterior(prior,.9,.05):.6f}")

binom = [comb(10,k)*(.1**k)*(.9**(10-k)) for k in range(11)]
assert isclose(sum(binom),1,abs_tol=1e-14)
print(f"Binomial n=10,p=.1: P(X=2)={binom[2]:.6f}; P(X>=1)={1-binom[0]:.6f}")

poisson0 = exp(-1)
poisson2 = exp(-3)*3**2/factorial(2)
print(f"Poisson: zero in 0.5 hour at 2/h = {poisson0:.6f}; two in 2h at 1.5/h = {poisson2:.6f}")

hyper = [Fraction(comb(3,x)*comb(2,2-x),comb(5,2)) for x in range(3)]
assert sum(hyper)==1
assert sum(x*hyper[x] for x in range(3))==Fraction(6,5)
print(f"Hypergeometric N=5,K=3,n=2: P(0,1,2) = {', '.join(map(str,hyper))}")
print("Common shock: P(both defects)=0.1, independent shortcut=0.01")
print("All exact and numerical checks passed.")
Output Checked chapter run
Flagged: 585/10000; defective among flagged: 90/585 = 0.153846
Defective among unflagged: 10/9415 = 0.001062
Prior 1% -> after alert 0.153846
Prior 5% -> after alert 0.486486
Prior 10% -> after alert 0.666667
Binomial n=10,p=.1: P(X=2)=0.193710; P(X>=1)=0.651322
Poisson: zero in 0.5 hour at 2/h = 0.367879; two in 2h at 1.5/h = 0.224042
Hypergeometric N=5,K=3,n=2: P(0,1,2) = 1/10, 3/5, 3/10
Common shock: P(both defects)=0.1, independent shortcut=0.01
All exact and numerical checks passed.