You looked at an MCP server before you connected it. You read the tool list, decided the tools were harmless and approved it. That decision was about the server as it was on that day.
The tool list is not fixed. The server sends it fresh every time a client asks. What you approved last month and what your agent sees today can be two different things.
What the spec allows
The MCP tools spec says the set of tools may change over time. A server can announce changes with a notifications/tools/list_changed message. That notification is a should, it only reaches clients that subscribed to it and nothing in the spec requires a client to ask the user again when the list changes.
The spec does tell clients to treat tool annotations as untrusted unless they come from a trusted server. It recommends a human in the loop who can deny a tool call. It does not define how a client should notice that a tool it already knows has become something else. That part is left to the client and to you.
The changes that matter
- A new tool that can write. A server that only searched documents now also has a tool that sends mail or deletes records.
- A known tool gets a new parameter. A read only lookup gains a
command,pathorurlargument. The name is the same. What it can reach is not. - A description changes. The model reads tool descriptions as part of its instructions. New text in a description is new input to the model that no user reviewed.
None of these has to be an attack. Servers add features. The point is that you want to hear about it before your agent does.
Pin the manifest and diff it
patchbay audit stores the tool manifest of a server the first time it audits it. Every later run compares the current list with the stored one.
npx -y @patchbay-nebius/agent audit https://mcp.example.com/mcpIf something changed you get finding MANIFEST-001 with the list of differences. The severity follows the most serious change:
- Critical: a new tool that looks like it writes or executes, or an existing tool with a new parameter such as
command,path,urlorsql - Medium: the description of an existing tool changed
- Low: a new read only tool or another change to an input schema
- Info: a tool was removed
The pinned manifests live on your machine under ~/.patchbay/manifests. That is why this check runs in the CLI and not in the browser scanner. The comparison is between your own runs.
When a change is expected
If you reviewed the differences and they are fine, pin the new state:
npx -y @patchbay-nebius/agent audit https://mcp.example.com/mcp --trustIf a change surprises you, do not pin it. Read the release notes of the server first. For a server you host yourself, check what was deployed.
Run it on a schedule
A diff only helps if something runs it. Put the audit in a scheduled CI job for the servers your agents depend on and let it fail on critical findings. On a fresh machine such as a GitHub Actions runner you have to keep the manifest store between runs. Without the cache step every run looks like the first one and the diff never fires.
- uses: actions/cache@v4
with:
path: ~/.patchbay/manifests
key: patchbay-manifests-v1
- uses: GodSmasher/patchbay-audit@v1
with:
url: https://your-mcp.example.com/mcp
fail-on: criticalThe cache is written once under that key and then stays as it is. That is what you want for a pinned state. When you have reviewed a change and want to accept it, raise the key to v2. The next run starts without a pinned manifest and pins the current one.
One limit to keep in mind: the check sees the tool list a server returns to the scanner. A server that only answers after login needs the --auth flag so the audit can read the list at all.