Data Analytics / Chapter 02

The ruler and the world

A sensor can be precise and still answer the wrong question. First check the ruler; then ask what world it was meant to measure.

Local developmental companion. Six synthetic readings; no real classroom data. The calculation remains on this device.

In this chapter
  1. What does 28 °C mean?
  2. How the line was made
  3. A line and its domain
  4. Challenge the measurement
  5. Run and inspect
  6. Code and captured output
Investigation 01 · a conditional correction

What does 28 °C mean?

The reference chamber and a constructed sensor give three paired means. Move the observed sensor reading. The correction uses the fitted line only within the measured 20–30 °C reference range.

27.075 °C

Reference estimate = (observed reading + 0.7) / 1.06

Within the calibrated reference range under the constructed line.

Measured pairs

How the line was made

ReferenceMean readingError
20 °C20.5 °C+0.5 °C
25 °C25.8 °C+0.8 °C
30 °C31.1 °C+1.1 °C

The mean error grows with temperature. Subtracting one fixed offset would correct only one point. This exact linear pattern was constructed for teaching; real calibration requires a design and an uncertainty budget.

Figure · read before trusting the equation

A line and its domain

Three synthetic reference temperatures from 20 to 30 degrees Celsius are paired with sensor means above each reference; a fitted line rises slightly more steeply than the equal-reading line.

The line cannot check drift after a month, placement beside a warm wall, or whether air temperature represents comfort for occupied desks.

Investigations 02–04

Challenge the measurement

  1. Derive slope and intercept from the endpoint means. Check the middle mean without refitting.
  2. Add a later-session drift of +0.3 °C. What new reference observation would reveal it?
  3. Change one row’s unit label to Fahrenheit without converting its numeric value. Write a validation rule that rejects the mixed units.
  4. Explain why a 0.224 °C repeatability standard deviation is not the whole uncertainty of a classroom estimate.
Reproduce

Run and inspect

Synthetic CSV · Executed Python practical · Executed base-R practical · Captured R output · Worked solutions

Run from this folder. Both programs assert the line and corrected reading before reporting success. R preserves the CSV’s reserved-word repeat header with check.names=FALSE.

Calibration: reading = -0.7 + 1.06 × reference
Corrected 28.0 °C reading: 27.075 °C
Five-reading sample SD: 0.224 °C

Program outputs are from constructed fixtures. They do not validate a real sensor or an indoor-health decision.

The reproducible notebook

Inspect the work.

Read the programs, run them in your own environment, and compare the results. The captured output records the chapter’s checked practical run; changing an input may change its answer.

Python calibration.py
"""Synthetic Chapter 3 calibration; standard-library Python 3."""
from collections import defaultdict
import csv
from pathlib import Path
from statistics import mean, stdev

path = Path(__file__).with_name("calibration.csv")
groups = defaultdict(list)
with path.open(newline="", encoding="utf-8") as handle:
    for row in csv.DictReader(handle):
        groups[float(row["reference_c"])].append(float(row["reading_c"]))
means = {reference: mean(values) for reference, values in sorted(groups.items())}
for reference, value in means.items():
    print(f"Reference {reference:.1f} °C: mean {value:.1f} °C, error {value-reference:+.1f} °C")
refs = sorted(means)
slope = (means[refs[-1]] - means[refs[0]]) / (refs[-1] - refs[0])
intercept = means[refs[0]] - slope * refs[0]
assert all(abs(means[reference] - (intercept + slope * reference)) < 1e-12 for reference in refs)
corrected = (28.0 - intercept) / slope
print(f"Calibration: reading = {intercept:.1f} + {slope:.2f} × reference")
print(f"Corrected 28.0 °C reading: {corrected:.3f} °C")
repeat_reads = [20.2, 20.4, 20.5, 20.6, 20.8]
print(f"Five-reading mean: {mean(repeat_reads):.1f} °C")
print(f"Five-reading sample SD: {stdev(repeat_reads):.3f} °C")
assert abs(corrected - 28.7 / 1.06) < 1e-12
assert abs(stdev(repeat_reads) - 0.05 ** .5) < 1e-12
R calibration.R
# Constructed Chapter 2 calibration. Base R; run from this directory.
d <- read.csv('calibration.csv', stringsAsFactors = FALSE, check.names = FALSE)
stopifnot(identical(names(d), c('reference_c','reading_c','repeat')))
stopifnot(nrow(d)==6L, all(is.finite(d$reference_c)), all(is.finite(d$reading_c)))
m <- aggregate(reading_c ~ reference_c, data=d, FUN=mean)
m <- m[order(m$reference_c),]
stopifnot(isTRUE(all.equal(m$reading_c,c(20.5,25.8,31.1))))
slope <- (m$reading_c[3]-m$reading_c[1])/(m$reference_c[3]-m$reference_c[1])
intercept <- m$reading_c[1]-slope*m$reference_c[1]
corrected <- (28-intercept)/slope
repeat_reads <- c(20.2,20.4,20.5,20.6,20.8)
repeat_sd <- sd(repeat_reads)
stopifnot(isTRUE(all.equal(slope,1.06)),isTRUE(all.equal(intercept,-0.7)))
stopifnot(isTRUE(all.equal(corrected,28.7/1.06)))
stopifnot(isTRUE(all.equal(repeat_sd,sqrt(.05))))
cat(sprintf('CALIBRATION reading = %.1f + %.2f * reference\n',intercept,slope))
cat(sprintf('CORRECTED 28.0 C READING %.3f C\n',corrected))
cat(sprintf('FIVE-READING SAMPLE SD %.3f C\n',repeat_sd))
cat('BASE-R CALIBRATION CHECKS PASS\n')
Output Checked chapter run
Reference 20.0 °C: mean 20.5 °C, error +0.5 °C
Reference 25.0 °C: mean 25.8 °C, error +0.8 °C
Reference 30.0 °C: mean 31.1 °C, error +1.1 °C
Calibration: reading = -0.7 + 1.06 × reference
Corrected 28.0 °C reading: 27.075 °C
Five-reading mean: 20.5 °C
Five-reading sample SD: 0.224 °C