DEA-C01 Practice Question 79: Glue Executor Metrics Overprovisioning
AWS Certified Data Engineer – Associate (DEA-C01) · Test 8 · question 9 of 10
A data engineer suspects that an AWS Glue Spark job is over-provisioned: it is billed for many workers, but the workload seems small. Which comparison of CloudWatch metrics shows whether the job really needs the workers it is allocated?
Show answer and explanation
Answer:
AWS Glue publishes executor metrics for jobs, including glue.driver.ExecutorAllocationManager.executors.numberAllExecutors (the executors currently allocated) and glue.driver.ExecutorAllocationManager.executors.numberMaxNeededExecutors (the executors the workload could use at its peak). AWS describes these metrics as a way *"to monitor the demand and optimized usage of executors in their Spark applications."* If the maximum needed stays far below the number allocated, the job is over-provisioned, so the team can reduce workers or turn on Auto Scaling (option A).
- B is wrong because bytes read and written show data volume. They do not show whether the Spark executors are busy or idle. - C is wrong because driver heap usage helps diagnose memory pressure on the driver, and run time alone does not say how many executors the workload could use. - D is wrong because task success and failure counts describe reliability, not how well the allocated capacity matches the workload.
Reference: Using auto scaling for AWS Glue