Uncertainty Quantification & Conformal Prediction
Intervals and prediction sets with guarantees — and what those guarantees do not cover
Turn a point prediction into a statement you can act on: aleatoric versus epistemic uncertainty and the decisions that need an interval (a newsvendor order worked by hand), the classical tools and their assumptions (OLS prediction intervals, bootstrap, quantile regression with pinball loss, quantile crossing), why calibration is not coverage, and conformal prediction from first principles: exchangeability, nonconformity scores, split conformal with the quantile index ceil((n+1)(1-alpha)), the finite-sample marginal guarantee and its Beta-distributed spread, conformalized quantile regression, LAC/APS/RAPS prediction sets, class-conditional variants, full versus split versus CV+/jackknife+, and why conditional coverage cannot be guaranteed. Then weighted conformal under covariate shift, adaptive conformal inference for drift, deep ensembles and MC dropout and where they fail out of distribution, evaluating UQ with coverage, width, PICP/MPIW, CRPS and group audits, and using uncertainty for abstention, routing and risk-aware thresholds. Every number comes from a heteroscedastic simulation and a 10-class classifier run in NumPy and scikit-learn, including 500-point split conformal at 90 percent, CQR, a coverage-by-group audit, and a 3,000-split check against the Beta law.
Practice questions (6)
-
View →
Implement Split Conformal Intervals for a Delivery-Time Model
Intermediate · Free -
View →
Explain Exchangeability and Diagnose a Violation After a Promotion Launch
Advanced · Free -
View →
Choose Between CQR and Plain Conformal for Heteroscedastic Demand
Advanced -
View →
Design an Abstention Policy for Automated Claims Triage
Advanced -
View →
Evaluate the Uncertainty of a SKU-Level Demand Model
Advanced