Data Engineering in Production

Every other subject in this track teaches you to build a correct pipeline: the right SQL, the right data model, the right file format, the right DAG. This subject asks a different question, and it is the one that separates a senior data engineer from someone who has only ever run dbt run on their laptop: what happens after the pipeline is correct? Correct pipelines still break — a source system changes its schema at 2am, a warehouse bill triples overnight because someone forgot a LIMIT, a credential rotates and nobody updated the connector, the one person who understood the customer-lifetime-value model leaves the company and the next person has no idea why a certain CTE excludes refunds. None of this is solved by SQL skill. It's solved by engineering process, and interviewers who have actually run a data platform ask about it specifically because it's where candidates who've only done take-home projects fall apart.

The tell for a strong answer in this space is concrete operational detail: not "we'd test the pipeline" but "we'd run dbt build --select state:modified+ against a Slim CI job that diffs the PR branch against the production manifest, so we only rebuild and test what actually changed." Not "we'd manage costs" but "we set AUTO_SUSPEND = 60 on interactive warehouses and tagged every scheduled query with a pipeline_owner so finance could pull a per-team cost report." Weak answers stay abstract because the candidate has never been paged at 3am for a stuck DAG or watched a data warehouse bill jump 4x. This subject gives you that operational vocabulary directly, organized around the seven areas interviewers actually probe: CI/CD, environments, secrets, cost, on-call, system composition, and documentation.