DEA-C01 Practice Question 8: Kinesis Lambda Poison Record
AWS Certified Data Engineer – Associate (DEA-C01) · Test 1 · question 8 of 10
An AWS Lambda function consumes an Amazon Kinesis Data Streams stream through an event source mapping with a batch size of 500. Overnight, GetRecords.IteratorAgeMilliseconds for one shard climbed steadily to several hours while every other shard stayed near zero. Function logs show the same batch being invoked repeatedly, each time throwing a deserialization error on a single malformed record. The stream's retention period is 24 hours and the blocked shard's oldest records are approaching expiry.
The team must get the shard moving again, must not silently lose the records that cannot be parsed, and cannot change the shard count. What should they configure?
Show answer and explanation
Answer:
The failure mode is a poison-pill record. By default, "Lambda checkpoints to the highest sequence number of a batch only when the batch is a complete success. Lambda treats all other results as a complete failure and retries processing the batch up to the retry limit." With the default unlimited retries, one unparseable record blocks its shard until the records age out of the stream, matching the single climbing IteratorAgeMilliseconds while the other shards are healthy.
Option A addresses every part of the requirement. ReportBatchItemFailures lets the function return the sequence number of the offending record so Lambda checkpoints past the records that did succeed instead of replaying the whole batch. BisectBatchOnFunctionError splits a failing batch in two on each retry, isolating the bad record from the 499 good ones. A finite MaximumRetryAttempts ends the retry loop instead of grinding until expiry. And the on-failure destination writes metadata for the discarded records to an SQS queue, which is what keeps the failure from being a silent data loss.
Option B misdiagnoses the problem as throughput. A parallelization factor spreads concurrent batches across a shard while preserving in-order processing per partition key, but the failing batch still blocks its own sub-stream and still retries forever. Adding concurrency to a stalled consumer does not unstall it.
Option C only buys time. Nothing about a longer retention window makes the malformed record parseable; the same batch fails on the same error, and the backlog keeps growing for another six days before hitting the same wall.
Option D degrades throughput dramatically and still does not solve the core problem. With a batch size of 1 the bad record occupies its own batch, but with no retry limit and no failure destination Lambda keeps retrying that one-record batch until it expires from the stream; the shard stays blocked and the record is then lost with no record of it. The timeout is irrelevant: the invocation is erroring, not running long.
Reference: Configuring partial batch response with Kinesis Data Streams and Lambda