‹ All DEA-C01 questions

DEA-C01 Practice Question 33: Kinesis Enhanced Fan Out

AWS Certified Data Engineer – Associate (DEA-C01) · Test 4 · question 3 of 10

Data Ingestion and Transformationmedium

Three applications already consume an Amazon Kinesis data stream using GetRecords. When a fourth consumer was added, all four began logging ReadProvisionedThroughputExceededException, and the GetRecords.IteratorAgeMilliseconds metric climbed steadily for every one of them. The producers' write throughput has not changed. The team needs every consumer to process records in near real time and does not want to increase the shard count.

What should the data engineer do?

Select one answer
Show answer and explanation

Answer:

  • D. Register each application as an enhanced fan-out consumer and have it read with SubscribeToShard

Default consumers share a shard's read capacity: read throughput is "fixed at a total of 2 MB/sec per shard. If there are multiple consumers reading from the same shard, they all share this throughput." Four consumers on the same shards therefore contend, which is why adding the fourth one pushed everybody into read throttling and rising iterator age. Enhanced fan-out gives each registered consumer "its own read throughput per shard, up to 2 MB/sec, independently of other consumers", and Kinesis pushes records over HTTP/2 with SubscribeToShard instead of being polled. Propagation delay drops too ("typically an average of 70 ms whether you have one consumer or five consumers") which is the near-real-time requirement. None of this changes the shard count.

Option A works around the symptom at the wrong layer and is explicitly excluded by the stem. It also does not scale with consumers: adding a fifth application would put the four back into contention and require yet another reshard.

Option B builds a fan-out tier by hand. It adds a hop of latency, an SQS bill proportional to the record count, ordering loss, and a new single point of failure, the relay consumer, which is still subject to the same shared 2 MB/sec limit.

Option C increases the pressure that is causing the exceptions. The 2 MB/sec ceiling is per shard, not per call; polling harder or in larger batches simply hits the limit more often and generates more throttling.

Reference: Develop enhanced fan-out consumers with dedicated throughput

Practice Test 4 ›

Study guides for DEA-C01