MCP (Model Context Protocol) has an auth story on paper: OAuth 2.1 with PKCE, dynamic client registration, resource indicators per RFC 8707. In practice I kept running into servers that were missing most of it. So I spent a week building patchbay audit, a free CLI that pokes an MCP server and reports what it finds. Then I pointed it at 25 public endpoints I could find.
Here's what I learned.
The numbers
25 servers scanned on 2026-11-02. All are publicly listed in community directories or vendor docs. The URL list is in bench/mcp-sweep-targets.txt.
| Reachable | 20 / 25 |
| Enforce authentication | 12 / 25 |
| Reachable + public (no auth) | 8 / 20 |
| Of those public: expose a mutating tool | 2 / 8 |
| Advertise OAuth 2.1 discovery metadata | 2 / 20 |
| Advertise resource_indicators_supported (RFC 8707) | 2 / 20 |
| Advertise PKCE S256 in code_challenge_methods_supported | 2 / 20 |
Three servers surfaced findings I thought were critical. All three got a responsible-disclosure email with a 90-day embargo before any public write-up names them. One of them (Chainflip's broker-as-a-service MCP) came back within hours with a link to their public AI documentation: the broker tools are intentionally no-auth because the actual value transfer is signed on-chain by the user, not initiated server-side. Public-by-design, documented, correctly scoped. So that one is off the critical list.
Public-by-design servers like DeepWiki and Chainflip are now flagged as info, public read-only or user-signed pattern rather than critical. That distinction matters more than it sounds: the first version of my tool called DeepWiki "critical no auth required", which would have made the whole post indefensible on day one. The Chainflip response is the second lesson: when a server links a /.well-known/ai-catalog.json or an AI-documentation page explicitly saying "no auth required", the audit should read that first and downgrade.
What the tool actually does
patchbay audit <url> runs 13 checks grouped roughly like this:
Auth presence (AUTH-001, AUTH-002)
Does the server accept tools/list without credentials? If yes, do any tool names look mutating (create / delete / send / execute / start-...)? Critical only when both are true. Public read-only servers are explicitly recognised as a legitimate pattern.
OAuth metadata completeness (AUTH-003 / 004 / 005)
If the server serves /.well-known/oauth-authorization-server, we read it and check for PKCE S256 advertisement, resource indicators, and a dynamic client registration endpoint. These are "not advertised" findings with confidence medium. A server can enforce the behaviour without exposing the metadata, so we only flag the metadata gap.
Scopes (SCOPE-001, SCOPE-002)
If the server advertises OAuth, do its tools declare per-tool scopes? Do write and read tools share a scope (so revoking write access also revokes reads)?
Secrets and transport
The directory-mode checks for hardcoded sk-... / ghp_... / AKIA... patterns in source files (skipping test fixtures and example patterns). Remote-mode checks whether the endpoint accepts plain HTTP, accepts a spoofed Origin header, and treats localhost consistently with 127.0.0.1 on the token endpoint.
Honest probe failures (PROBE-001)
If the probe can't learn anything (server is behind auth, uses an unknown transport, or just doesn't respond), we say so explicitly instead of returning "0 findings, all clean". That silent-false-negative trap is a real risk when scanning diverse servers.
The architecture is deliberately boring: packages/audit-core has zero I/O inside the check functions. Probing collects a TargetSnapshot object (what tools the server advertises, what headers it accepts, what's in its OAuth metadata, what secret regexes matched in a directory). Checks are (snapshot) => Finding[]. That makes them trivially unit-testable with fixture snapshots and lets me run them against a cached scan without re-probing the network.
Run it like this:
npx -y @patchbay-nebius/agent audit https://mcp.example.com/mcp
npx -y @patchbay-nebius/agent audit ./my-mcp-server --json
npx -y @patchbay-nebius/agent audit ./my-mcp-server --sarif > out.sarif--ci makes it exit non-zero on critical / high so your GitHub Action fails the build.
What I think is actually going on
The MCP 2025-03-26 Streamable HTTP spec is new enough that most server implementations are catching up to it. The 2024-11-05 spec and the Legacy SSE transport are what most servers still ship. Discovery metadata is optional in both, so vendors that already have working OAuth (because they already run OAuth-protected APIs for other reasons: Linear, Notion, Supabase, Sentry) tend to require auth but not necessarily serve the MCP-specific discovery documents.
The servers I'd worry about are the public ones with mutating tools and no documented reason for being public. Those are startups and solo projects shipping MCP endpoints that look like read-only wiki queries but include create_... or update_... tools in the list, with no AI-catalog entry saying this is intentional. The developer who shipped them might not have realised there's no auth gate.
If you're building an MCP server right now, the fastest wins are:
- serve
/.well-known/oauth-protected-resourceso clients can discover your auth server, - advertise
code_challenge_methods_supported: ["S256"]andresource_indicators_supported: truein your authorization server metadata, - give each tool its own scope and reject calls whose token lacks it.
OAuth 2.1 with PKCE and resource indicators is the whole spec; if you implement those three, you're ahead of most of this list.
Limitations and things I'll fix
Can't audit auth-walled servers deeper than "auth-walled"Shipped in 0.1.4 the same day.patchbay audit <url> --auth=<token>(orPATCHBAY_AUDIT_TOKEN=...env var) sends the bearer on every probe request, so Linear, Notion, Supabase, Sentry and your own OAuth-protected MCP get the full audit instead of "auth-walled, not audited".- SCOPE-002 is name-regex heuristics on tool names. Will produce occasional false positives on names like
post_user(meaning "the user who posted" vs. "HTTP POST a user"). - The probe tries the given URL,
/mcp,/sse,/messages, and/. It won't find endpoints at unusual paths. - Stdio transport servers (local MCP) are out of scope; audit the HTTP side.
Doesn't yet haveShipped in 0.1.3 the same day.--pr-draftpatchbay audit <target> --pr-draft --out PATCHBAY_AUDIT.mdrenders a PR-body-ready markdown report (summary table, grouped by severity, reproducer per finding). The example GitHub Action now posts it as a PR comment automatically.Doesn't yet readShipped in 0.1.2 the same day as this post. When a server publishes an ai-catalog declaring no-auth intentionally (/.well-known/ai-catalog.jsonauthentication: "none",auth_required: false,public: true, or similar), AUTH-001 now auto-downgrades to info with evidence pointing at the declaring field.
Try it
npx -y @patchbay-nebius/agent audit <your-server-or-repo>In CI, drop examples/github-action/patchbay-audit.yml into your own repo as .github/workflows/patchbay-audit.yml. It runs on every push and PR, fails the build on critical or high findings and uploads SARIF to the GitHub Code Scanning tab.
If it flags something and you think it's wrong, open an issue with the output. False-positive reports are the most useful feedback right now. The severity recalibration between first and second version came entirely from running the tool on known-good servers and finding where it was too alarmist.
Repo: GodSmasher/patchbay. MIT. No tracking, no backend, no account. Just a CLI.