Intermediate
Open
Pro
A Two-Hour-Late Dashboard and No One Was Told
Your team's daily_exec_dashboard_refresh DAG is supposed to finish
by 7am so executives see fresh numbers each morning. One Monday, the
DAG doesn't finish until 9:15am — the upstream load_orders task hit
a slow warehouse load and took over three hours instead of its usual
20 minutes. No one on the data team knew until an executive asked why
the dashboard looked stale, over an hour after the SLA was blown.
Post-incident, you find this configuration:
@dag(
dag_id="daily_exec_dashboard_refresh",
schedule="0 5 * * *",
start_date=datetime(2025, 6, 1),
catchup=False,
)
def daily_exec_dashboard_refresh():
@task(sla=timedelta(hours=2), retries=2, retry_delay=timedelta(minutes=5))
def load_orders() -> None:
_load_orders_from_source()
@task
def refresh_dashboard() -> None:
_rebuild_dashboard_tables()
refresh_dashboard()
daily_exec_dashboard_refresh()
Note: refresh_dashboard() is called with no dependency on
load_orders() in this code as written.
- There are two separate bugs here — a structural DAG bug and a missing-alerting bug. Identify both precisely.
- Fix both, including a concrete alerting configuration, not just "add monitoring."
- Even after your fix, explain why an SLA alone is not sufficient to guarantee executives never see stale data, and what additional check would close that gap.
Share this question