AI Tools Gone Wrong

Tool descriptions tend to get shipped the way code comments do — written for the next engineer who'll open the source. Useful, low-stakes, mostly ignorable.

The model treats them differently.

Every tool description in your Model Context Protocol (MCP) server's tool list is part of the model's context. The model reads them like prompt instructions, because that's what they are — typed text in the same context window as the system prompt and the user message. Same tokens, same attention, same effect on the decision.

In April and May 2026, OX Security published a disclosure they titled "The Mother of All AI Supply Chains" — a systemic design flaw in the MCP STDIO transport across Anthropic's official SDKs (Python, TypeScript, Java, Rust), in which commands are executed even when the launched process errors, and Anthropic confirmed the behavior and declined to modify the protocol. Inside the same window, Endor Labs published the analysis of 2,614 MCP implementations in their 2025 Dependency Management Report: 82 percent used file-operation APIs prone to path traversal; 67 percent touched code-injection-related APIs; 34 percent used APIs susceptible to command injection. GitGuardian's State of Secrets Sprawl 2026 added a different surface — 24,008 unique secrets sitting in publicly committed MCP configuration files on GitHub, 2,117 of them still valid at scan time.

Those numbers are the failure surface. The design surface — where the engineer's choices show up before any of those failures appear — is the tool description.

Today's issue walks through:

—Teja Derangula

Pull the registration for actions_get out of GitHub's official MCP server. The description is one line, verbatim: "Get details of GitHub Actions resources (workflows, workflow runs, jobs, and artifacts)"

That's the part the model reads first. Underneath sits the input schema — a structured object with named parameters, types, requirement flags. The model also reads the parameter-level descriptions: method ("The method to execute"), owner ("Repository owner"), repo ("Repository name"), resource_id ("The unique identifier of the resource. This will vary based on the 'method' provided, so ensure you provide the correct ID...").

Three things matter about that surface.

It travels with every prompt. The tool list, including every description for every registered tool, is loaded into the model's context on the first turn. A server with thirty tools puts thirty descriptions into the prompt. Every prompt.

It is text the model interprets, not configuration it executes. The model reads "Get details of GitHub Actions resources" and decides whether the user's request fits. Same text, same model, same user request — you can still get different selections. Interpretation lives where the tokens are.

It is mutable at runtime. GitHub's MCP server ships an explicit feature for this: any tool description can be overridden by an environment variable matching GITHUB_MCP_TOOL_<NAME>_DESCRIPTION or by a github-mcp-server-config.json file in the binary's directory. The description that lives in the source repository is not necessarily the description the model sees in production.The description is the prompt. The rubric is what to check before shipping that prompt

Seven questions. Each one is one question to answer yes before the description goes out. None of them is novel by itself; the discipline is running all seven on every tool before shipping it.

1. Imperative voice. Does the description name the action the tool performs — not the engineer's intent in writing it? "Returns the list of open pull requests" passes. "Useful for when you want to find PRs" fails. The model is choosing tools by what they do, not by why you built them.

2. Side-effect declaration. Does the description name what changes outside the agent's context when the tool runs? Writes to a database, network calls, money spent, mail sent — anything that doesn't unwind when the agent's reasoning unwinds. A read tool has no side effects; a write tool should announce them in the description, not bury them in the parameter docs.

3. Idempotency note. Does the description state whether calling the tool twice with the same input is safe? It's the second call — the agent's quiet retry after a transient error — where production surprises tend to live.

4. Failure-mode disclosure. Does the description name what the tool returns or raises when it fails, and what conditions trigger that? An agent that knows what failure looks like adapts. An agent that doesn't, retries with the same arguments until it runs out of budget.

5. Authority scope. Does the description name what the tool cannot do? Read-only, no PII, no destructive operations on production resources. The Endor Labs finding is the kind of thing this element is designed to catch: across 2,614 MCP implementations, 67 percent surfaced code-injection-related APIs and 34 percent surfaced command-injection-susceptible APIs. Authority-scope statements in the description are one of the places where that scope can be communicated to the model — or left to be inferred.

6. Input bounds. Does the description state valid ranges or formats per parameter, beyond what the JSON schema enforces? Schemas catch types. Descriptions catch semantics — the date is in ISO-8601, the page size caps at 100, the path is relative to the workspace root.

7. Source provenance. Is the description in source control, reviewed like code, and not mutable by network-fetched config at runtime? GitHub's MCP server lets descriptions be replaced via environment variable, as quoted above. The OX Security May 2026 disclosure turned on a different design decision — the STDIO transport's command-execution behavior — and the same engineering principle applies: anything in the protocol that the model treats as authoritative is worth reviewing like code, not configuring like environment.

These are real registrations from GitHub's official MCP server, examined here through the rubric. None of them is "wrong" — GitHub shipped a working server. The rubric surfaces what's been left to the model's interpretation.

Case 1 — actions_get. Description: "Get details of GitHub Actions resources (workflows, workflow runs, jobs, and artifacts)." One tool, five distinct resource types dispatched through a method parameter (get_workflow, get_workflow_run, get_workflow_run_usage, get_workflow_run_logs_url, get_workflow_run_artifact, get_workflow_job). Imperative voice passes. Authority scope is silent on which methods are read versus mutation (in this case all are reads — the description doesn't say so). Input bounds are pushed into the resource_id parameter description, which expands to a five-bullet conditional explaining what to pass per method.

Case 2 — actions_run_trigger. Description: "Trigger GitHub Actions workflow actions." The tool bundles five operations under one description: run_workflow, cancel_workflow_run, rerun_workflow_run, rerun_failed_jobs, delete_workflow_run_logs. All five mutate state — two of them destructively (cancel_workflow_run, delete_workflow_run_logs); the other three trigger new runs or re-runs. Side-effect declaration is implicit in the verb "trigger" but does not name destructive operations specifically; failure-mode disclosure and authority scope are both left for the model to infer per-method.

Case 3 — assign_copilot_to_issue.custom_instructions. Parameter description: "Optional custom instructions to guide the agent beyond the issue body. Use this to provide additional context, constraints, or guidance that is not captured in the issue description." This is a description whose value, at call time, becomes instructions for a downstream agent. Source provenance matters here precisely because the instructions are user-supplied — the receiving Copilot agent reads them as part of its own context.

What this looks like applied to your own MCP setup:

The rubric is the work. The discipline is running it before every release.

The description is the prompt. Review it like one.