‹ All DEA-C01 questions

DEA-C01 Practice Question 53: EMR Serverless Spark Jobs

AWS Certified Data Engineer – Associate (DEA-C01) · Test 6 · question 3 of 10

Data Ingestion and Transformationmedium

A data team runs about a dozen Apache Spark jobs a day on a long-running Amazon EMR cluster. Job sizes vary widely, so the cluster is often either over-provisioned and idle or too small and queueing. The team wants to keep its existing Spark code and EMR runtime behaviour but stop right-sizing, launching, and terminating clusters, and stop paying for idle capacity.

Which option meets these requirements with the least change to the Spark code?

Select one answer
Show answer and explanation

Answer:

  • A. Submit the jobs to an Amazon EMR Serverless application

EMR Serverless "is a deployment option for Amazon EMR that provides a serverless runtime environment… With EMR Serverless, you don't have to configure, optimize, secure, or operate clusters to run applications with these frameworks." It targets exactly the over- and under-provisioning problem described: "EMR Serverless helps you avoid over- or under-provisioning resources for your data processing jobs. EMR Serverless automatically determines the resources that the application needs, gets these resources to process your jobs, and releases the resources when the jobs finish." Because it is an EMR deployment option, the team keeps "the benefits of Amazon EMR, such as open source compatibility… and optimized runtime performance", so the Spark code moves across essentially unchanged.

Option B is a migration, not a lift. Glue has its own job model, its own library environment and its own Spark versions; scripts that rely on EMR configuration classifications, bootstrap actions or specific EMR runtime behaviour would need rework.

Option C removes the idle cost but keeps the cluster management. Somebody still chooses instance types and counts for each job, and every run pays cluster bootstrap time before any work starts.

Option D discards the managed runtime altogether. The team would own Spark images, versioning, the cluster manager configuration, the shuffle service and the S3 committer tuning, far more operational work than they have now, not less.

Reference: What is Amazon EMR Serverless?

Practice Test 6 ›

Study guides for DEA-C01