Welcome back! Today’s playbook: why the AI layer in your stack isn't a dependency — it's a trust boundary, and why adding more vendor reviews won't close it.

The Trigger

On April 8, a pre-auth RCE was disclosed in marimo — the Python notebook platform widely used for ML work. It was actively exploited within roughly ten hours. By April 14, Sysdig had observed 662 exploit events across 11 unique source IPs. Attackers used it to distribute a Go-based backdoor (a variant of NKAbuse, using the NKN blockchain for command-and-control) through a typosquatted Hugging Face Space named "kagent" — deliberately mimicking a legitimate Kubernetes AI agent tool.

Twelve days later, on April 20, Vercel disclosed that customer credentials, API keys, source code, and database data had been stolen. The entry point: a Vercel employee had granted "Allow All" OAuth permissions to Context.ai, a third-party AI SaaS. Context.ai had been breached in March. Vercel's Google Workspace was compromised through that path, and the stolen data was listed for sale at $2M on BreachForums.

Two different incidents. One underlying problem.

What's Actually Wrong ?

The AI layer of the developer and employee stack is now load-bearing — notebook platforms, model registries, AI assistants, OAuth-connected SaaS — and most teams are adopting it faster than they're applying supply-chain discipline.

Classical software spent a decade learning to pin dependencies, sign artifacts, scope OAuth tokens, and maintain internal registries. The AI stack has inherited none of that maturity by default. An engineer pulling a model from Hugging Face, or an employee connecting an AI assistant to their Google Workspace, is crossing a trust boundary. In most orgs, that boundary isn't on any list, isn't reviewed, and has no blast-radius plan.

The marimo-to-HuggingFace chain exploits this gap on the dev side. The Vercel-to-Context.ai chain exploits it on the employee/SaaS side. The specifics differ. The diagnosis is the same.

The Concept

Trust Boundary
The point where control of code, data, or identity crosses from something you run to something you don't. Each one requires explicit review, minimum scope, and a plan for what happens when the other side is compromised.

Executable Artifact
Anything that runs code when loaded, installed, or authorized. Model weights in pickle format, notebook platforms with RCE surface, OAuth tokens that grant delegated access — all of these execute something on your behalf. The "passive data" mental model is wrong for all of them.

OAuth Scope Creep
The cumulative permission surface your organization has granted to third parties. "Allow All" grants compound silently over years into keys-to- the-kingdom access — which is exactly what the Vercel breach rode in on.

The Playbook

  1. Map every AI trust boundary you can name.
    Model registries, notebook platforms, AI IDEs, OAuth-connected AI assistants, AI SaaS tools. Put them in one list with a named owner for each. If it's not on the list, it isn't reviewed — and it will be the next incident.

  1. Pin ML artifacts to immutable identifiers, not tags.
    On Hugging Face, pin to the commit SHA of a revision, not a branch name — tags and branches are mutable. Same rule applies to pip, npm, and container images used by AI workloads.

  1. Treat model weights as executable code.
    Prefer safetensors over pickle-based formats (.bin, .pt) wherever the publisher supports it. Where pickle is unavoidable, scan with picklescan or ModelScan before loading, and run the first load in a sandbox — separate container, no egress, no mounted secrets.

  1. Audit OAuth grants for AI tools specifically.
    In Google Workspace, Microsoft 365, and GitHub, pull the list of third-party apps with broad scopes. Look for "Allow All" grants to AI SaaS. Revoke anything not strictly necessary. This is the exact path the Vercel breach took — an employee granted Allow-All to Context.ai, and the attacker rode that token into a Google Workspace account.

  1. Mirror public AI registries to an internal one.
    Pull from Hugging Face, PyPI, and container registries once, verify, then host internally. Production environments should not reach public AI registries directly — that's how typosquatted "kagent"-style artifacts end up running inside your infrastructure.

  1. Mirror public AI registries to an internal one. Include: vendor compromise, typosquatted artifacts, RCE in dev tooling, and delegated- access pivot as named scenarios. If they aren't documented, they aren't reviewed, and they aren't mitigated.

  • Mirroring registries and pinning by hash means you'll always be a step behind the newest releases. When someone on your team wants to try Gemma 4 or GLM-5.1 the day it drops, the answer is "not yet" — that's a real cost in a week where six frontier models shipped.

  • Internal registries and sandbox runners are infrastructure you now own. They need patching, access controls, and on-call coverage. Unmaintained internal infra becomes its own supply chain risk — the worst outcome is a neglected mirror serving stale artifacts no one's watching.

  • OAuth scope review and AI tool inventory will feel like friction to employees who just want to try the new assistant. Push too hard and you get shadow IT, which is worse than the original problem — because now the integrations exist and you can't see them.

What did you think of today's email?
Your feedback helps me create better emails for you! comment down 👇
Loved It 😊
It was ok 🙂
Could be better 🤔

Until next time - Teja Derangula,
The gap between thinking and building has shrunk — take advantage.

Reply

Avatar

or to participate

Keep Reading