Streaming Fundamentals & Kafka
Batch vs streaming tradeoffs, Kafka's partition/replication model, delivery semantics and how exactly-once is actually achieved, event time vs processing time, watermarking, windowing, and where Kafka Streams, Flink, and Spark Structured Streaming each fit
A practitioner's tour of streaming as a data engineering interview topic: when streaming is genuinely justified versus when it's over-engineering a batch problem, Kafka's core model (topics, partitions, offsets, consumer groups, replication, and in-sync replicas) with a concrete partition-assignment walkthrough, the mechanics of at-most-once, at-least-once, and exactly-once delivery (idempotent producers and transactions, not magic), event time versus processing time, watermarking and late-arriving data, tumbling/sliding/session windowing, and a survey of Kafka Streams, Flink, and Spark Structured Streaming and how each sits on top of Kafka.
Practice questions (5)
-
View →
Debugging Duplicate Events Despite an 'Exactly-Once' Pipeline
Intermediate · Free -
View →
A Windowed Revenue Aggregate Is Undercounting After a Network Blip
Intermediate -
View →
Consumer Lag Keeps Growing Even After Adding More Consumers
Intermediate -
View →
Should This Inventory Sync Actually Be a Streaming Pipeline?
Intermediate -
View →
Session Analytics Are Wrong for Users Who Go Offline
Intermediate