DEA-C01 Practice Question 2: Flink Event Time Tumbling Window
AWS Certified Data Engineer – Associate (DEA-C01) · Test 1 · question 2 of 10
Connected vehicles publish telemetry to an Amazon Kinesis Data Streams stream. Each record carries the timestamp the device generated it. Because vehicles pass through tunnels and dead zones, a record can reach the stream up to three minutes after the moment it describes, and records for the same vehicle can arrive out of order.
The team needs a per-vehicle aggregate over five-minute tumbling windows. A record delayed by two minutes must still be counted in the window that contains its device timestamp, not the window in which it happened to arrive. Each window's result must be written to the S3 data lake as soon as that window closes, and the team does not want to provision or operate a data warehouse for this pipeline.
Which approach meets these requirements?
Show answer and explanation
Answer:
Only Apache Flink gives true event-time semantics here. Amazon Managed Service for Apache Flink (the service formerly named Amazon Kinesis Data Analytics for Apache Flink) lets you attach a watermark strategy that derives time from a field in the record, the device timestamp, rather than from arrival. A bounded-out-of-orderness watermark holds each window open until the watermark advances past the window end plus the configured lag, so a record that arrives two minutes late is still assigned to the window its own timestamp falls in. The window then fires once and is written to S3, and Flink's checkpointing keeps the in-flight window state durable across restarts. This is a fully managed streaming application, so no warehouse is involved.
Option C is the closest trap. Lambda tumbling windows do maintain state across invocations, but the documentation is unambiguous that "Lambda determines tumbling window boundaries based on the time when records were inserted into the stream"; that is, arrival time. A record delayed two minutes is therefore attributed to the wrong window, which is precisely what the requirement forbids. The state is also capped at 1 MB per shard and the window is scoped per shard, so a vehicle whose records span a reshard would be split.
Option A has no windowing engine at all. Firehose dynamic partitioning can route a record into a prefix derived from the device timestamp, but Firehose does not know when a window is complete, so a query that runs at the top of each five-minute period will miss the records that are still up to three minutes in flight. Re-querying wide time ranges to catch stragglers scans far more data and still produces results that change after the fact.
Option D violates the stated constraint about not operating a data warehouse, and it is structurally wrong besides: a materialized view refresh recomputes over whatever rows have been ingested at refresh time, with no watermark and no concept of window completeness, so late arrivals silently change already-published aggregates.
Reference: Track events in Managed Service for Apache Flink using the DataStream API