MLOps & Production ML

ML System Design Interview Framework

Learn how ML system design interviews are scored, a 7-step framework from requirements to monitoring, back-of-envelope estimation for QPS, embeddings and GPU cost, common problem framings, and the mistakes that sink strong candidates.

ML Data Pipelines & Feature Stores

Design the data side of an ML system: logging for train/serve parity, labelling strategies, point-in-time joins, negative sampling, batch vs streaming features, feature stores, training–serving skew, data validation, lineage and a worked notification-click pipeline.

Airflow & Workflow Orchestration

A practitioner's tour of Airflow as a data engineering interview topic: how DAGs, tasks, and operators fit together; why the scheduling model's catchup default silently reprocesses history; why classic sensors starve the worker pool and what deferrable operators fix; how retries, SLAs, and alerting are actually wired up; dynamic task mapping for runtime-sized fan-out; why large payloads must never flow through XComs; idempotent task design as the property that makes retries and backfills safe; and a short comparison of Airflow's task-centric model to Dagster and Prefect's asset-centric one.

Data Quality, Testing & Observability

A practitioner's tour of data quality as a data engineering interview topic: the five dimensions of data quality (accuracy, completeness, timeliness, consistency, uniqueness) illustrated with concrete failures, the tradeoffs of enforcing checks at the source, in-pipeline, or at the warehouse, schema contracts between producers and consumers and how to catch breaking changes before they ship, freshness/volume/distribution anomaly monitoring and why it catches what a schema check never will, dbt tests versus dedicated data-observability tooling and when each is the right layer, and a full incident-response walkthrough — triage, containment, root cause, prevention — for a pipeline that silently broke a downstream dashboard.

Containers & Reproducible ML Environments

Learn how Docker images and layers actually work, why ML images balloon to multiple gigabytes, how dependency pinning and lockfiles prevent 'works on my machine' failures that are worse in ML because of native and CUDA dependencies, how to reason about GPU base images and multi-stage builds, and what true reproducibility requires beyond a container: seeding across every layer of randomness and treating data as a versioned pointer, not a copy.

CI/CD for ML & Data Code

Learn what makes CI/CD for ML and data pipelines different from general software CI: unit tests for data transforms, schema and data-contract validation gates, model-quality gates that block a merge on offline metric regression, and continuous deployment of both code and models through staging to production — all built on the core mental model that a model trained on different data is a different artifact requiring the same review rigor as a code change.

Model Training & Experimentation at Scale

Learn how to answer the training half of an ML system design interview: baselines, model family choice, temporal offline evaluation, tuning budgets, experiment tracking, distributed training, GPU cost estimation, embedding tables, retraining cadence, and off-policy evaluation.

Experiment Tracking & Model Registries

Learn how experiment tracking captures runs, hyperparameters, metrics, and artifacts in a tool-agnostic pattern (the MLflow/Weights & Biases model), how a model registry becomes the single source of truth for what's actually deployed, how stage transitions (staging to production to archived) should be gated and by whom, and how to trace a production prediction back through the exact model version, training data snapshot, and code commit that produced it.

MLOps Model Lifecycle & Governance

Learn the complete model lifecycle as a single governed pipeline, why every stage transition needs an owner and a sign-off, how to version data/code/model/environment so any prediction is reproducible months later, and what a model card documents and why regulated teams cannot ship without one.

Model Serving & Deployment

Learn to design the serving side of an ML system: batch vs online vs streaming vs hybrid inference, latency budgets with numbers, model server patterns, compression, canary and shadow rollouts, feature-store consistency, autoscaling, and safe fallbacks.

Model Release Strategies: Canary, Shadow & Rollback

Learn why a model release is not a normal software deploy because 'correct' is statistical rather than pass/fail, and how shadow mode, canary rollouts, blue/green deployment, gradual ramps, and automatic vs. human-gated rollback triggers manage that risk. Includes a risk-profile-to-strategy decision table and the release metrics to watch during a rollout.

Production Observability & Monitoring

Learn the general observability stack that keeps any production service healthy: the three pillars (logs, metrics, traces), how to build dashboards that answer questions instead of just displaying numbers, SLOs and error budgets, paging policy that avoids alert fatigue, and where ML-specific signals plug into this general foundation.

ML Monitoring, Drift & Retraining

Learn why deployed models decay, how to monitor them in layers from system health to business KPIs, compute PSI and other drift statistics with worked numbers, handle delayed labels, and design retraining triggers with safe validation gates and rollback.

Reliability, Resilience & Observability Patterns

Learn availability math, SLOs and error budgets, timeouts, retries with backoff and jitter, circuit breakers, bulkheads, load shedding, safe deploys, and the three pillars of observability, then apply them to a checkout service with a flaky payment provider.

Cost and Latency Engineering for LLM Apps

The reusable toolkit behind every 'make it cheaper and faster' interview question: why input tokens dominate cost and output tokens dominate latency, how prompt-prefix caching and semantic caching work and where each breaks, model routing and cascades, streaming and parallel tool calls as latency levers, the TTFT-plus-decode model of p95 latency, and the cost-per-1,000-conversations math that ties every lever to a number leadership will ask for.

Case Study: Design a Batch Analytics Platform

Model interview answer for designing the batch analytics platform behind a mid-size company's BI and reporting layer, from OLTP sources through CDC/ELT ingestion, lakehouse/warehouse landing, dbt transformation layers, and Airflow orchestration to dashboards: concrete row-count and byte-volume math that drives every downstream decision, the CDC-vs-batch-extract call made per source rather than uniformly, warehouse-vs-lakehouse and file-format/partitioning choices, staging/intermediate/marts layering in dbt, an Airflow DAG shaped around data-aware dependencies rather than fixed clock time, an SLA built with deliberate slack instead of run at the theoretical minimum, an idempotent partition-scoped backfill strategy, cost control through partition pruning and materialization rather than bigger warehouses, data-quality gates that quarantine rather than silently drop or block, and a team/ownership model that treats the staging layer as a reviewed public interface.

Case Study: Churn Model, End-to-End MLOps

Model interview answer that threads a single churn-prediction model through the full MLOps lifecycle: notebook prototype, CI tests and data-validation gates, experiment tracking and registry promotion, a canary release, a production drift alert, a safely-gated automated retrain, and a governance review before the retrained model ships. Each stage names the specific tool category and the specific failure it prevents.

Case Study: Visual Defect Detection on a Production Line

Model interview answer for industrial visual inspection (electronics, automotive, pharma manufacturing lines, Landing-AI-class vendors): why this is the block's detection-not-generation and edge case, argued against every other case study in the series. The approach ladder from impossible supervised detection to one-class anomaly detection trained on normal parts only — reconstruction-based methods (AnoGAN, f-AnoGAN, grounded in this track's GAN vocabulary) versus pretrained-feature memory-bank methods (PaDiM, PatchCore) — and why memory-bank methods are the current pragmatic default on MVTec-AD-style benchmarks. Golden-sample capture, alignment and lighting normalization, synthetic defects for validation only (cross-referencing this track's synthetic-data-for-CV case), pixel-level anomaly maps pooled into an image-level score, and image-level AUROC plus pixel-level PRO/AUROC as the evaluation pair FID cannot replace. A decision-policy section mirroring the fraud-detection case's asymmetric-cost framing, but for escape cost versus scrap cost. A system design where inference happens on an edge box synchronised to a PLC reject signal, and a scalability deep-dive built entirely around this case's signature problem: one model (or memory bank) per SKU per line, a model-zoo registry, physical drift fixed by re-capturing golden samples, and onboarding a new SKU in hours with 50 images as the real scaling unit.