Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

A Bulk Load Broke a Join's Plan

Every night, a batch job loads ~2 million new rows into an event_log table (previously ~500K rows total) as part of an ETL pipeline. The morning after a load, a downstream query joining event_log to a small event_types lookup table (40 rows) started taking 90+ seconds instead of its usual 200ms:

SELECT e.id, e.payload, t.display_name
FROM event_log e
JOIN event_types t ON t.code = e.event_type_code
WHERE e.event_type_code = 'PAYMENT_FAILED';

EXPLAIN ANALYZE shows:

Nested Loop  (cost=0.42..847.10 rows=12 width=96)
             (actual time=0.089..91442.331 rows=418902 loops=1)
  ->  Index Scan using idx_event_types_code on event_types t
        (cost=0.14..8.16 rows=1 width=24)
        (actual time=0.021..0.024 rows=1 loops=1)
        Index Cond: (code = 'PAYMENT_FAILED'::text)
  ->  Index Scan using idx_event_log_type_code on event_log e
        (cost=0.42..835.82 rows=12 width=80)
        (actual time=0.031..0.201 rows=419...loops=1)
        Index Cond: (event_type_code = t.code)

(event_log.event_type_code has an index; both tables have up-to-date-looking row counts in the query planner's mind before this incident — this is the first load-induced slowdown the team has seen.)

  1. Diagnose what actually went wrong here — note that indexes exist on both sides of the join, so "add an index" is not the fix. Point to the specific evidence in the plan that supports your diagnosis.
  2. Propose the immediate fix, and explain precisely why it resolves the issue mechanistically (not just "it usually helps").
  3. Propose a process change to the ETL pipeline so this doesn't recur every time a large load happens. Be specific about what changes and when it runs relative to the load.

Share this question

← Back to SQL Query Optimization & Indexing practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.