Software Engineering Institute

Overview

A vulnerability in MLflow’s dspy and statsmodels model flavors allows unauthorized pickle deserialization executions despite a safety control. Specifically, the dspy flavor conditionally applies the control based on the model path’s file extension, and the statsmodels flavor does not apply the control.

Description

MLflow is an open-source platform for managing machine learning lifecycles, including model packaging, versioning, and deployment. "Flavors" refer to the specialized frameworks through which supported models are stored and loaded. In response to previous vulnerability concerns, MLflow implemented the MLFLOW_ALLOW_PICKLE_DESERIALIZATION safety control to block and disable executing any pickle deserialization and subsequent loads per the user’s choice.

When loading models through mlflow.pyfunc.load_model(model), users must specify a model flavor and path in an MLmodel file. With the dspy flavor, MLflow checks the value of MLFLOW_ALLOW_PICKLE_DESERIALIZATION, and whether the specified model path ends in .pkl. A model path that does not end in .pkl (even if the file is actually a pickle file), will route to a separate branch for pickle deserialization, bypassing the safety control. However, when loading through the statsmodels flavor, there is no check for MLFLOW_ALLOW_PICKLE_DESERIALIZATION at all.

Impact

Exploitation of this vulnerability allows for arbitrary remote code execution through a malicious pickle-loaded payload, regardless of a user explicitly disallowing pickle serialization, through vulnerable flavor specifications in the MLmodel configuration file. The attack path requires write access to any location from which a user obtains MLflow models. This vulnerability was confirmed against MLflow 3.12.0.

Solution

MLFlow could not be reached to coordinate this vulnerability; however, the statsmodels flavor was patched in versions >= 3.15.0. Users should upgrade immediately. Until a further fix remedying the dspy flavor vulnerability is available, MLflow users who wish to block pickle deserialization and loads should avoid loading any models via the dspy flavor.

Acknowledgements

Thanks to Prasanna Dabi for reporting this vulnerability. This document was written by Alex Lewis.

Vendor Information

References

• https://github.com/mlflow/mlflow/blob/master/mlflow/dspy/load.py
• https://github.com/mlflow/mlflow/blob/master/mlflow/statsmodels/__init__.py
• https://github.com/advisories/GHSA-gqvg-gmmx-x4hm