DEA-C01 Practice Question 18: Glue Crawler Table Grouping
AWS Certified Data Engineer – Associate (DEA-C01) · Test 2 · question 8 of 10
An AWS Glue crawler catalogs clickstream data under s3://lake/clickstream/, which is laid out as dt=YYYY-MM-DD/ folders. Every file is GZIP-compressed JSON. For months the crawler kept a single table, clickstream, partitioned by dt. Over the past quarter, product teams have shipped new event types, and each day's files now carry a different set of optional fields. After the latest crawl, analysts no longer see one clickstream table. The Data Catalog database instead holds dozens of new tables named like dt_2026_08_14, and Athena queries against clickstream miss the recent days.
What is the most likely cause, and how should it be fixed?
Show answer and explanation
Answer:
By default a crawler decides what counts as one table using two tests. The first is data compatibility: same format, same compression, the S3 path structure. The second is schema similarity between the objects it finds. The docs give this exact failure: two year= folders of JSON whose fields differ, and "by default, the crawler creates two tables, named year_2017 and year_2018 because the schemas are not sufficiently similar." That is the symptom here. The data stayed compatible (all GZIP JSON under one path), but the optional fields diverged until the similarity test failed. The crawler then treated each dt= folder as its own table and named it after the folder. With Create a single schema for each S3 path (TableGroupingPolicy: CombineCompatibleSchemas), the crawler "still considers data compatibility, but ignores the similarity of the specific schemas". It builds one table whose schema is the union of the fields, with dt as the partition key. The stray per-day tables should be deleted so analysts aren't confused.
Option A concerns what happens to a table the crawler has *already matched*. The schema change policy decides whether a changed schema updates the existing table or is only logged. It does not change how folders are grouped into tables, so the crawler would keep creating per-day tables.
Option B would not help. Incremental crawls only limit which folders are scanned. A full crawl applies the same similarity test to the same divergent folders and splits them the same way. It might even split older days too.
Option D misdiagnoses the format. The built-in JSON classifier handles GZIP JSON; the crawler cataloged this data for months, and it successfully built tables from the new folders. The data is being read fine. Only the grouping into one table is failing.
Reference: Creating a single schema for each Amazon S3 include path