Supply Chain Security for AI Models: How a Downloaded Model Becomes Malware

Downloading a model from Hugging Face feels like downloading a data file. Technically, it's often closer to running a program you never reviewed.

The root problem: pickle executes code, not just loads data

Python's pickle serialization format, the dominant storage mechanism for ML model weights, allows arbitrary code execution at load time — a class of vulnerability that has been repeatedly exploited in practice on Hugging Face since at least March 2024. PyTorch and TensorFlow models in the Hugging Face community carry the highest risk of malicious code execution, with roughly 95% of identified malicious models built on PyTorch.

A real 2026 incident: zero to 244,000 downloads in 18 hours

On May 7, 2026, a repository called "Open-OSS/privacy-filter" appeared on Hugging Face and, in just 18 hours, climbed to the #1 trending spot, with more than 244,000 downloads and 667 likes — an illustration of how fast a malicious model can go viral before anyone audits it.

When the training framework itself trusts too much

InstructLab, a widely used open-source ML training framework, had `trust_remote_code=True` hardcoded in its training script — meaning a specially crafted malicious model uploaded to Hugging Face could achieve remote code execution on any InstructLab user who loaded it. No end-user error was required: the risk was baked into the tool's own default configuration.

The scale of the problem, in numbers

According to JFrog's 2026 Software Supply Chain Security Report, malicious packages grew 451% year over year, with researchers identifying more than 495 malicious AI models in public registries. It isn't a handful of isolated cases — it's a rapidly growing attack category.

How to protect yourself in practice

The most direct mitigation is preferring the safetensors format over pickle whenever it's available — safetensors doesn't allow arbitrary code execution by design. When pickle is unavoidable, scan files with tools like picklescan, which catches most common backdoors in serialized files. And at the organizational level: treat every model downloaded from a public registry, regardless of popularity or download count, as unaudited third-party code until proven otherwise — a repository's popularity isn't a security signal, as the privacy-filter case demonstrated.