‹ All DEA-C01 questions

DEA-C01 Practice Question 41: Glue ResolveChoice Ambiguous Type

AWS Certified Data Engineer – Associate (DEA-C01) · Test 5 · question 1 of 10

Data Ingestion and Transformationhard

An AWS Glue crawler has typed the order_total field of a JSON dataset as choice<double,string>, because a minority of upstream files emit the value quoted ("19.99") while the rest emit it unquoted. The Glue job that writes the curated Parquet table now fails, and the downstream Athena table must end up with exactly one order_total column of type double, with the quoted values converted.

Which transformation should the data engineer apply to the DynamicFrame?

Select one answer
Show answer and explanation

Answer:

  • C. resolveChoice(specs=[("order_total", "cast:double")])

resolveChoice "resolves a choice type within this DynamicFrame and returns the new DynamicFrame", and the action chosen decides the shape of the result. The cast:{type} action "attempts to cast all values to the specified type", so cast:double converts the quoted strings into doubles and leaves the already-numeric values alone. The result is a single order_total column typed double, exactly what the curated Parquet table and the Athena schema require.

Option B produces the wrong shape. make_cols "converts each distinct type to a column with the name {columnName}_{type}", so the output would carry order_total_double and order_total_string side by side. The ambiguity is flattened rather than resolved, and every downstream query would have to coalesce two columns.

Option D has the same problem in a different form. make_struct "resolves a potential ambiguity by using a struct to represent the data"; each row gets a structure containing both an int/double and a string member. That is one column, but it is a struct, not a double.

Option A changes only metadata. The Data Catalog would claim the column is double, while the underlying files still contain quoted values and the DynamicFrame still reads a choice type. The job keeps failing, and if it did not, Athena would return nulls or errors for the quoted rows. (resolveChoice does offer a match_catalog action that casts each ChoiceType to the catalog's type, but that is a transformation on the data, not a crawler setting.)

Reference: DynamicFrame class - resolveChoice

Practice Test 5 ›

Study guides for DEA-C01