Databricks ML Professional Practice Question 12: Models from Code Serialization
Databricks Certified Machine Learning Professional · Test 2 · question 2 of 10
An engineer wraps a retrieval model in a custom mlflow.pyfunc.PythonModel. The class holds a handle to an external service client created in __init__. Calling mlflow.pyfunc.log_model with an instance of the class fails while serializing, reporting that an object cannot be pickled. The client object cannot be made picklable.
What is the correct way to log this model?
Show answer and explanation
Answer:
The default path serializes the *instance* with cloudpickle, so every attribute it holds must be picklable, which an open service client is not. Models from code takes a different route: MLflow stores the Python script that *defines* the model instead of a binary of the object. set_model() marks which object in the script is the model, and at load time MLflow executes the script to reconstruct it. The client is then created during loading, in the environment where it will actually be used, so it never has to survive serialization. This also makes the logged artifact readable and diffable rather than an opaque pickle.
Option B misdiagnoses the failure. The recursion limit governs deeply nested structures; this object is not unpicklable because it is deep, and raising the limit changes nothing about a handle that has no pickle support.
Option C hits the same wall one level down. artifacts is for files MLflow copies alongside the model, so the client must still be pickled to become a file, and it cannot be.
Option D treats a property of the object as a Python version issue. Cloudpickle version compatibility is a real concern when moving pickles between environments, but it is not why an unpicklable client fails to pickle.
Reference: Models from code (MLflow)