‹ All DEA-C01 questions

DEA-C01 Practice Question 62: Schema Registry Compatibility Enforcement

AWS Certified Data Engineer – Associate (DEA-C01) · Test 7 · question 2 of 10

Data Ingestion and Transformationmedium

Several teams publish JSON and Avro events to shared topics on Amazon MSK. Producers occasionally add required fields, which breaks downstream consumers that were built for the previous format. The data engineering team wants producers to be blocked at publish time when a schema change would break existing consumers. Which solution meets this requirement?

Select one answer
Show answer and explanation

Answer:

  • D. Register the schemas in the AWS Glue Schema Registry and apply a compatibility mode such as BACKWARD.

The AWS Glue Schema Registry lets you *"centrally discover, control, and evolve data stream schemas"* and integrates with Apache Kafka, Amazon MSK, Amazon Kinesis Data Streams and AWS Lambda. Each schema has a compatibility rule, and *"requests to register new schema versions are checked against this rule by the Schema Registry before they can succeed."* With BACKWARD compatibility, a new version that adds a required field is rejected, because consumers on the new version could not read old messages. The serializers used by producers fail the publish, so the breaking change never reaches consumers (option D).

- A is wrong because files in S3 are documentation, not enforcement. Nothing stops a producer from publishing a message that differs from the stored schema. - B is wrong because a crawler infers a schema after data has arrived and can only alert afterward. The breaking change has already reached the consumers. - C is wrong because data quality rules on downstream tables detect bad data after it has been processed. They do not control what producers publish.

Reference: AWS Glue Schema registry

Practice Test 7 ›

Study guides for DEA-C01