Databricks ML Associate Practice Question 6: Data Summarize EDA
Databricks Certified Machine Learning Associate · Test 1 · question 6 of 10
Before feature engineering, a data scientist wants a first look at a new Spark DataFrame in a Databricks notebook. For every column they want the missing-value count, the number of distinct values, and a histogram or frequent-value chart. They want this with one command, not a chart built for each column.
What should they run?
Show answer and explanation
Answer:
dbutils.data.summarize computes and displays summary statistics for a Spark or pandas DataFrame as an interactive profile. For each column it shows statistics such as missing and distinct counts, plus histograms for numeric columns and frequent-value charts for categorical ones. By default some statistics are approximate for speed; precise=True makes them more exact. The data profile in the notebook's table output shows the same profile.
Option B is useful for numeric summaries. DataFrame.summary() returns a plain table of count, mean, stddev, min, percentiles, and max, but no missing-value counts and no charts.
Option C is a smaller version of summary(): count, mean, stddev, min, and max, still with no visualizations or distinct counts.
Option D prints only column names, types, and nullability. It computes no statistics at all.
Reference: Databricks Utilities (dbutils) reference