Data Analytics / Chapter 03

The people outside the table

A row can be accurate and still leave the wrong people invisible. Change the unseen outcomes; watch what the evidence does, and does not, permit us to say.

Local developmental companion. Constructed teaching cases, no real household or service records. No data leaves this page.

In this chapter
  1. How many successes are unseen?
  2. Two groups, two answers
  3. The 80% that becomes 65%
  4. Take the reasoning further
  5. Code and captured output
Investigation 01 · missing outcomes

How many successes are unseen?

There are 200 eligible cases. We know 120 succeeded and 40 failed. Outcomes for 40 remain unknown. Move the scenario slider; the wide logical interval remains until those outcomes are actually recovered.

Scenario rate
72%
Possible full-cohort rates by assumed success countThe rate runs from 60 to 80 percent as zero to forty unknown cases are assumed successful.Assumed successes among unknown casesFull-cohort success rate (%)
60–80%logical bounds before assumptions
24 of 40unknown successes needed to reach threshold
Threshold met in this scenarioscenario conclusion, not an observed result

(120 + 24) / 200 = 72% under this scenario.

The 120 known successes are observed in this constructed case. The slider supplies a hypothetical count. A threshold crossing never identifies the actual missing outcomes.

Investigation 02 · sampling design

Two groups, two answers

An audit inspects 100 routine cases and 100 complex cases. The frame contains 800 routine and 200 complex cases. Compare the audited share with the frame-weighted estimate.

75%of 200 inspected cases
84%estimated rate for the 1,000-case frame

0.8 × 90% + 0.2 × 60% = 84%.

Expansion weights are 8 per routine case and 2 per complex case. This estimate assumes valid random inspection within each complete frame stratum and observed outcomes. It does not fix missing records.

Investigation 03 · response adjustment

The 80% that becomes 65%

Of 1,000 delivered invitations, 180 urban and 20 remote households answer. Their positive counts are 153 and 7. The respondent share is 160/200 = 80%. Reweighting to the known 600:400 delivered frame gives 65%.

RespondentsDelivered frameTeal: urban · copper: remote. Bar lengths show composition, not approval.

0.6 × (153/180) + 0.4 × (7/20) = 65%

The 65% relies on respondents representing nonrespondents within each location group. Remote respondents have an expansion weight of 20, six times the urban weight of 3⅓. Do not label this district satisfaction: the undelivered and unregistered are still outside the estimate.

Run · question · explain

Take the reasoning further

Why is the 75% inspected rate not the frame estimate?

Complex cases are 50% of inspected cases but only 20% of the frame. When their outcome differs, the unweighted inspected mean answers a question about the inspection mix. The 80:20 weighted estimate answers a question about the framed cases, under the design assumptions.

If the 40 missing cases are recovered as failures, what changes?

There would be no unknown outcomes left. The full-cohort rate would be 120/200 = 60%, so the former 60–80% uncertainty interval collapses to a known 60% for this constructed cohort. Recovery can change uncertainty and the result; merely moving the slider changes neither observed fact.

What evidence should the investigator seek next?

Check source logs and re-contact missing cases. Compare delivery, response and outcomes by location, access channel and case difficulty. A focused remote-household follow-up is valuable because 380 remote outcomes are unobserved in the response example.

Reproduce the arithmetic

Download and run the deterministic Python file; it prints the audited rates, design-only standard error, bounds, threshold count, poststratified rate and exact sample enumeration.

Download sampling_lab.py

python3 sampling_lab.py
Inspected cases: 200
Unweighted inspected rate: 75.0%
Design-weighted frame rate: 84.0%
Design-only estimated standard error: 2.36%
Routine / complex expansion weights: 8 / 2
Full-cohort missing-outcome bounds: 60.0%–80.0%

Run this on your own computer. The local page uses no server, account, tracking, or remote execution.

Chapter 3 · private developmental edition. Printed QR and public route remain pending the final companion release.

The reproducible notebook

Inspect the work.

Read the programs, run them in your own environment, and compare the results. The captured output records the chapter’s checked practical run; changing an input may change its answer.

Python sampling_lab.py
"""Synthetic, deterministic sampling and missing-outcome investigation.

All examples are constructed for teaching. No external data or network calls.
"""

from dataclasses import dataclass
from fractions import Fraction
from itertools import combinations
from math import sqrt


@dataclass(frozen=True)
class Stratum:
    name: str
    frame_size: int
    inspected: int
    successes: int

    def validate(self) -> None:
        if not 0 <= self.successes <= self.inspected <= self.frame_size:
            raise ValueError(f"inconsistent stratum: {self.name}")

    @property
    def expansion_weight(self) -> Fraction:
        self.validate()
        if not self.inspected:
            raise ValueError(f"no inspected cases in {self.name}")
        return Fraction(self.frame_size, self.inspected)


def stratified_rate(strata: list[Stratum]) -> Fraction:
    if not strata:
        raise ValueError("at least one stratum is required")
    frame = sum(s.frame_size for s in strata)
    if not frame:
        raise ValueError("empty frame")
    return sum((Fraction(s.frame_size, frame) * Fraction(s.successes, s.inspected)
                for s in strata), Fraction(0))


def stratified_design_variance(strata: list[Stratum]) -> Fraction:
    """Estimated design variance for independent within-stratum SRS without replacement.

    Assumes every inspected binary outcome was observed. Does not include frame,
    response, measurement or model uncertainty.
    """
    if not strata:
        raise ValueError("at least one stratum is required")
    frame = sum(s.frame_size for s in strata)
    if not frame:
        raise ValueError("empty frame")
    total = Fraction(0)
    for s in strata:
        s.validate()
        if s.inspected < 2:
            raise ValueError("at least two inspected units per stratum required")
        p = Fraction(s.successes, s.inspected)
        sample_variance = Fraction(s.inspected, s.inspected - 1) * p * (1 - p)
        W = Fraction(s.frame_size, frame)
        f = Fraction(s.inspected, s.frame_size)
        total += W * W * (1 - f) * sample_variance / s.inspected
    return total


def outcome_bounds(known_successes: int, known_failures: int, unknown: int) -> tuple[Fraction, Fraction]:
    if min(known_successes, known_failures, unknown) < 0:
        raise ValueError("counts must be nonnegative")
    total = known_successes + known_failures + unknown
    if not total:
        raise ValueError("empty cohort")
    return Fraction(known_successes, total), Fraction(known_successes + unknown, total)


def unknown_needed(known_successes: int, known_failures: int, unknown: int, threshold: Fraction) -> int:
    if not 0 <= threshold <= 1:
        raise ValueError("threshold outside [0,1]")
    outcome_bounds(known_successes, known_failures, unknown)
    total = known_successes + known_failures + unknown
    required_total = -(-(threshold.numerator * total) // threshold.denominator)
    return max(0, required_total - known_successes)


def exact_two_unit_sample_means() -> list[Fraction]:
    frame = [0, 0, 1, 1, 1]
    return [Fraction(a + b, 2) for a, b in combinations(frame, 2)]


def main() -> None:
    routine = Stratum("routine", 800, 100, 90)
    complex_ = Stratum("complex", 200, 100, 60)
    sample_rate = Fraction(routine.successes + complex_.successes,
                           routine.inspected + complex_.inspected)
    weighted = stratified_rate([routine, complex_])
    design_se = sqrt(float(stratified_design_variance([routine, complex_])))
    lower, upper = outcome_bounds(120, 40, 40)
    needed = unknown_needed(120, 40, 40, Fraction(72, 100))
    print(f"Inspected cases: {routine.inspected + complex_.inspected}")
    print(f"Unweighted inspected rate: {float(sample_rate):.1%}")
    print(f"Design-weighted frame rate: {float(weighted):.1%}")
    print(f"Design-only estimated standard error: {design_se:.2%}")
    print(f"Routine / complex expansion weights: {routine.expansion_weight} / {complex_.expansion_weight}")
    print(f"Full-cohort missing-outcome bounds: {float(lower):.1%}–{float(upper):.1%}")
    print(f"Unknown successes needed for 72%: {needed} of 40 ({needed / 40:.1%})")
    print(f"Urban/remote poststratified respondent rate: {float(Fraction(600,1000)*Fraction(153,180)+Fraction(400,1000)*Fraction(7,20)):.1%}")
    means = exact_two_unit_sample_means()
    assert len(means) == 10
    assert sum(means) / len(means) == Fraction(3, 5)
    assert sum((x - Fraction(3, 5)) ** 2 for x in means) / len(means) == Fraction(9, 100)
    print("Exact two-unit samples: 1 mean 0, 6 means 0.5, 3 means 1; variance 0.09")


if __name__ == "__main__":
    main()
Output Checked chapter run
Inspected cases: 200
Unweighted inspected rate: 75.0%
Design-weighted frame rate: 84.0%
Design-only estimated standard error: 2.36%
Routine / complex expansion weights: 8 / 2
Full-cohort missing-outcome bounds: 60.0%–80.0%
Unknown successes needed for 72%: 24 of 40 (60.0%)
Urban/remote poststratified respondent rate: 65.0%
Exact two-unit samples: 1 mean 0, 6 means 0.5, 3 means 1; variance 0.09