Skip to content

UriTemplate.match() replaces non-UTF-8 percent-escapes with U+FFFD, corrupting the value handed to resource handlers #3721

Open

Description

@sachinsharma3191

Initial Checks

Release line

2.x (current stable)

Description

A resource URI whose percent-escapes are not valid UTF-8 is matched successfully, but the
captured value reaches the handler with each undecodable octet replaced by U+FFFD. The
original byte is gone and cannot be recovered by the handler.

RFC 3986 §2.1 percent-encoding encodes octets, not UTF-8 text, so file://caf%E9
(latin-1 café) is a well-formed URI. match() accepts it, but returns 'caf�'.

The cause is at every capture site in uri_template.py: urllib.parse.unquote is called
with its default errors="replace" (lines 607, 625, 657, 1015, 1048, 1114). Decoding
silently substitutes U+FFFD rather than failing or preserving the octet.

This also breaks the round-trip that match()'s own docstring claims. The docstring
discusses the separator limitation for {.ext}, but says nothing about lossy decoding:

match('file://caf%E9')   -> {'name': 'caf�'}
expand(that)             -> 'file://caf%EF%BF%BD'   # a different URI

Every operator form is affected, so this is not specific to reserved expansion:

template input captured
file://{name} file://caf%E9 'caf�'
file://{+path} file://caf%E9 'caf�'
x{#frag} x#caf%E9 'caf�'
x{.ext} x.caf%E9 'caf�'
x{/seg} x/caf%E9 'caf�'
x{;p} x;p=caf%E9 'caf�'
x{?q} x?q=caf%E9 'caf�'
x{/list*} x/a/caf%E9 ['a', 'caf�']

I expected either the octet to be preserved, or the URI to be rejected — not a silent
substitution that the handler cannot detect or undo.

Example Code

import anyio
from mcp import Client
from mcp.server.mcpserver import MCPServer

mcp = MCPServer("demo")
seen = []

@mcp.resource("file://{name}")
def read(name: str) -> str:
    seen.append(name)
    return f"got {name!r}"

async def main() -> None:
    async with Client(mcp) as client:
        await client.read_resource("file://caf%E9")
        print("handler received:", repr(seen[-1]))

anyio.run(main)
$ uv run --frozen python repro.py
handler received: 'caf�'

Possible approaches

Two that restore the contract, both one-line changes at the six call sites:

  1. errors="surrogateescape" — preserves the octet and round-trips exactly:
    unquote('caf%E9', errors='surrogateescape') gives 'caf\udce9', and
    quote(..., errors='surrogateescape') gives back 'caf%E9'. Handlers that only
    ever see UTF-8 are unaffected; those that care can recover the bytes.
  2. errors="strict" and treat the UnicodeDecodeError as a non-match — the URI then
    gets an honest "Unknown resource" instead of corrupted data.

I'd lean towards (1), since it keeps match() a true inverse of expand() and does not
turn a currently-working (if lossy) read into a failure. Happy to go with (2) if you'd
rather not have surrogates reach handler code. I'd like to fix this if you'll take it.

Python & MCP Python SDK

Python 3.14.3
pydantic 2.12.5
mcp / mcp-types 2.3.1.dev10+9f66f376 (main at 9f66f376)
src/mcp/shared/uri_template.py is byte-identical at v2.3.0 and main
  (last changed 2026-07-10, 1216c536), so the released 2.3.0 behaves the same.

Found by widening the value alphabet in the SDK's own property test
(test_match_inverts_expand_for_every_parseable_template); its _VALUE_CHARS is
ascii_lowercase + digits, which cannot emit a percent sign or any reserved character.

Activity

  1. added
    bugSomething isn't working
    v2Affects the v2 line (2.x on main)
    on Oct 17, 2026
  2. nikolas-sapa commented on Oct 17, 2026

    @nikolas-sapa

    Confirmed the repro across all template operators: urllib.parse.unquote defaults to errors='replace', causing non-UTF-8 percent-encoded octets (e.g. file://caf%E9) to be replaced with U+FFFD (caf), destroying the raw octet before it reaches resource handlers and breaking bidirectional invertibility with expand().

    Fix approach: pass errors="surrogateescape" to unquote() at all capture sites and in _encode() for quote(). Verified all 9 operator forms round-trip cleanly and all 316 unit tests in tests/shared/test_uri_template.py pass.

    A working patch with full regression test coverage is ready on nikolas-sapa:fix/uritemplate-surrogateescape-3721 (prepared in #3724). Could a maintainer assign this issue to me so the PR reopens? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingv2Affects the v2 line (2.x on main)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions