Paths Subjects Questions Quizzes Pricing Search
Intermediate Open Pro

Choosing Between Query-Based and Log-Based CDC

You're designing ingestion for two source tables into the same warehouse:

  • product_catalog: ~50,000 rows, updated a few dozen times a day by internal staff, no deletes (products are marked discontinued, never removed). Downstream consumers (search indexing, the product page) tolerate up to 15 minutes of staleness.
  • payment_events: tens of millions of rows/day, a small percentage are corrected or reversed within minutes of creation (and the correction involves deleting the original erroneous row, not just updating it), and a fraud-detection service downstream needs to see both new events and deletions within seconds.

A junior engineer proposes standing up Debezium log-based CDC for both tables, reasoning "it's strictly more capable, so let's just use it everywhere and be consistent."

  1. For each table, say whether you'd use query-based (timestamp) CDC or log-based CDC, and justify each choice against the table's actual requirements rather than "more capable is always better."
  2. What operational cost is the team taking on if they follow the junior engineer's "use it everywhere" proposal, specifically for product_catalog?
  3. Six months later, product_catalog requirements change: the business wants true hard deletes when a product line is discontinued and fully removed. Does this change your answer for that table? Explain the decision process, not just the new answer.

Share this question

← Back to ETL vs ELT & Pipeline Design practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.