‹ All DEA-C01 questions

DEA-C01 Practice Question 79: Glue Executor Metrics Overprovisioning

AWS Certified Data Engineer – Associate (DEA-C01) · Test 8 · question 9 of 10

Data Operations and Supporteasy

A data engineer suspects that an AWS Glue Spark job is over-provisioned: it is billed for many workers, but the workload seems small. Which comparison of CloudWatch metrics shows whether the job really needs the workers it is allocated?

Select one answer
Show answer and explanation

Answer:

  • A. The maximum executors needed against the executors allocated

AWS Glue publishes executor metrics for jobs, including glue.driver.ExecutorAllocationManager.executors.numberAllExecutors (the executors currently allocated) and glue.driver.ExecutorAllocationManager.executors.numberMaxNeededExecutors (the executors the workload could use at its peak). AWS describes these metrics as a way *"to monitor the demand and optimized usage of executors in their Spark applications."* If the maximum needed stays far below the number allocated, the job is over-provisioned, so the team can reduce workers or turn on Auto Scaling (option A).

- B is wrong because bytes read and written show data volume. They do not show whether the Spark executors are busy or idle. - C is wrong because driver heap usage helps diagnose memory pressure on the driver, and run time alone does not say how many executors the workload could use. - D is wrong because task success and failure counts describe reliability, not how well the allocated capacity matches the workload.

Reference: Using auto scaling for AWS Glue

Practice Test 8 ›

Study guides for DEA-C01