Data Analytics / Chapter 16

Tomorrow, decisions and a complete investigation

Build a decision from evidence. Preserve the limits of every number.

Constructed teaching data. Local review only; no public QR destination yet.

In this chapter
  1. Investigation · What does the reference mix change?
  2. Prediction and decision checks
  3. Reproduce the final packet
  4. Code and captured output

Investigation · What does the reference mix change?

Cedar's old period had 81 timely routine cases among 90, and two timely complex cases among ten. The new period had 19 among 20 routine and 24 among 80 complex. Change the routine share in a common reference population; this is descriptive standardisation, not a causal effect.

Observed old versus new
83% versus 43%
Common-reference old versus new
New minus old, percentage points
At a 50 percent routine reference share, constructed Cedar observed rates are 83 and 43 percent and equal-mix rates are 55 and 62.5 percent.
The static figure records the original 50% reference case. The interactive numbers above change when the weight changes. · Select the figure to enlarge.

Prediction and decision checks

Classification

At one threshold: 160 true positives, 80 false positives, 40 false negatives, 720 true negatives. Calculate precision, recall, accuracy and a referral workload for capacity 180.

Constructed two-by-two prediction table with 160 true positives, 80 false positives, 40 false negatives and 720 true negatives.

Forecasting

Four observed days are 100, 120, 80, 100. Compare forecasts A: 100, 100, 100, 100 and B: 110, 110, 90, 90. Both have MAE 10; their RMSEs differ.

Four actual daily demand values and two forecasts with equal mean absolute error.

Reproduce the final packet

Download the constructed four-cell CSV, executed Cedar calculation, saved evidence report, executed model-method checks and draft exercise solutions. The report states the absent reopening, workload, staffing and independent severity evidence. It does not claim an intervention effect.

The reproducible notebook

Inspect the work.

Read the programs, run them in your own environment, and compare the results. The captured output records the chapter’s checked practical run; changing an input may change its answer.

Python cedar_capstone.py
"""Constructed Cedar case-mix capstone. Run: python3 cedar_capstone.py"""
import csv,json
from pathlib import Path
root=Path(__file__).resolve().parent
rows=list(csv.DictReader((root/'cedar_cells.csv').open()))
assert len(rows)==4
cells={(r['Period'],r['Category']):(int(r['Cases']),int(r['Timely'])) for r in rows}
assert set(cells)=={('old','routine'),('old','complex'),('new','routine'),('new','complex')}
assert cells=={('old','routine'):(90,81),('old','complex'):(10,2),('new','routine'):(20,19),('new','complex'):(80,24)}
observed={}
for period in ('old','new'):
 n=sum(cells[period,c][0] for c in ('routine','complex'))
 y=sum(cells[period,c][1] for c in ('routine','complex'))
 observed[period]={'cases':n,'timely':y,'rate':y/n}
 assert n==100
rates={p:{c:cells[p,c][1]/cells[p,c][0] for c in ('routine','complex')} for p in ('old','new')}
assert observed['old']['rate']==.83 and observed['new']['rate']==.43
assert rates=={'old':{'routine':.9,'complex':.2},'new':{'routine':.95,'complex':.3}}
def standardised(period,routine_weight):
 return routine_weight*rates[period]['routine']+(1-routine_weight)*rates[period]['complex']
sensitivity=[]
for w in (.2,.5,.8):
 old=standardised('old',w);new=standardised('new',w)
 sensitivity.append({'routine_reference_weight':w,'old_rate':round(old,6),'new_rate':round(new,6),'difference_points':round(100*(new-old),4)})
assert sensitivity[1]=={'routine_reference_weight':.5,'old_rate':.55,'new_rate':.625,'difference_points':7.5}
report={'source':'constructed cedar_cells.csv','observed':observed,'category_rates':rates,'equal_mix':sensitivity[1],'reference_weight_sensitivity':sensitivity,'interpretation':'The aggregate fall and category-specific rises are descriptive. Periods were not randomised; no process effect is identified.','missing_evidence':['reopening','staff workload','assignment mechanism','staffing changes','independent severity review']}
(root/'cedar_capstone_report.json').write_text(json.dumps(report,indent=2)+'\n')
print('OBSERVED old',observed['old'],'new',observed['new'])
print('EQUAL MIX old',sensitivity[1]['old_rate'],'new',sensitivity[1]['new_rate'],'difference points',sensitivity[1]['difference_points'])
print('SENSITIVITY',sensitivity)
print('CEDAR CAPSTONE CHECKS PASS')
Python final_investigation_lab.py
"""Constructed teaching fixtures only. Run: python3 final_investigation_lab.py"""
import math
import numpy as np
# Classification, one declared threshold.
tp,fp,fn,tn=160,80,40,720
assert tp+fp+fn+tn==1000
precision=tp/(tp+fp);recall=tp/(tp+fn);accuracy=(tp+tn)/1000
assert abs(precision-2/3)<1e-12 and recall==.8 and accuracy==.88
# Four genuine forecast targets under two already-issued teaching forecasts.
y=np.array([100.,120.,80.,100.]);a=np.array([100.,100.,100.,100.]);b=np.array([110.,110.,90.,90.])
mae=lambda z:float(np.mean(abs(y-z)))
rmse=lambda z:float(math.sqrt(np.mean((y-z)**2)))
assert mae(a)==mae(b)==10
assert abs(rmse(a)-math.sqrt(200))<1e-12 and rmse(b)==10
# One-dimensional clustering objective.
values=np.array([1.,2.,8.,9.]);cluster1=values[:2];cluster2=values[2:]
centers=[float(cluster1.mean()),float(cluster2.mean())]
within=float(((cluster1-centers[0])**2).sum()+((cluster2-centers[1])**2).sum())
assert centers==[1.5,8.5] and within==1
# PCA uses the original scales of two constructed numeric features.
points=np.array([[1.,1.],[2.,1.],[3.,2.],[4.,3.]])
cov=np.cov(points,rowvar=False);eig=np.linalg.eigvalsh(cov)
assert np.allclose(cov,[[5/3,7/6],[7/6,11/12]])
assert abs(eig[-1]/eig.sum()-.9743690699952497)<1e-12
# A simplified decision rule, valid only for the stated costs and calibrated p.
review_cost=2;miss_cost=20;threshold=review_cost/miss_cost
assert threshold==.1
print('CLASSIFICATION precision',round(precision,4),'recall',round(recall,4),'accuracy',round(accuracy,4))
print('FORECAST A MAE',mae(a),'RMSE',round(rmse(a),4))
print('FORECAST B MAE',mae(b),'RMSE',round(rmse(b),4))
print('K-MEANS teaching centers',centers,'within sum of squares',within)
print('PCA covariance',np.round(cov,4).tolist(),'first variance fraction',round(float(eig[-1]/eig.sum()),6))
print('DECISION break-even probability',threshold)
print('FINAL INVESTIGATION ARITHMETIC CHECKS PASS')
Output Checked chapter run
OBSERVED old {'cases': 100, 'timely': 83, 'rate': 0.83} new {'cases': 100, 'timely': 43, 'rate': 0.43}
EQUAL MIX old 0.55 new 0.625 difference points 7.5
SENSITIVITY [{'routine_reference_weight': 0.2, 'old_rate': 0.34, 'new_rate': 0.43, 'difference_points': 9.0}, {'routine_reference_weight': 0.5, 'old_rate': 0.55, 'new_rate': 0.625, 'difference_points': 7.5}, {'routine_reference_weight': 0.8, 'old_rate': 0.76, 'new_rate': 0.82, 'difference_points': 6.0}]
CEDAR CAPSTONE CHECKS PASS