Intermediate
Open
Pro
A Bulk Load Broke a Join's Plan
Every night, a batch job loads ~2 million new rows into an
event_log table (previously ~500K rows total) as part of an ETL
pipeline. The morning after a load, a downstream query joining
event_log to a small event_types lookup table (40 rows) started
taking 90+ seconds instead of its usual 200ms:
SELECT e.id, e.payload, t.display_name
FROM event_log e
JOIN event_types t ON t.code = e.event_type_code
WHERE e.event_type_code = 'PAYMENT_FAILED';
EXPLAIN ANALYZE shows:
Nested Loop (cost=0.42..847.10 rows=12 width=96)
(actual time=0.089..91442.331 rows=418902 loops=1)
-> Index Scan using idx_event_types_code on event_types t
(cost=0.14..8.16 rows=1 width=24)
(actual time=0.021..0.024 rows=1 loops=1)
Index Cond: (code = 'PAYMENT_FAILED'::text)
-> Index Scan using idx_event_log_type_code on event_log e
(cost=0.42..835.82 rows=12 width=80)
(actual time=0.031..0.201 rows=419...loops=1)
Index Cond: (event_type_code = t.code)
(event_log.event_type_code has an index; both tables have
up-to-date-looking row counts in the query planner's mind before
this incident — this is the first load-induced slowdown the team
has seen.)
- Diagnose what actually went wrong here — note that indexes exist on both sides of the join, so "add an index" is not the fix. Point to the specific evidence in the plan that supports your diagnosis.
- Propose the immediate fix, and explain precisely why it resolves the issue mechanistically (not just "it usually helps").
- Propose a process change to the ETL pipeline so this doesn't recur every time a large load happens. Be specific about what changes and when it runs relative to the load.
Share this question