GitHub MCP conformance testing gives QA teams a structured way to validate protocol compatibility before changing agent clients, SDKs, or server infrastructure. GitHub’s official July 23, 2026 announcement says the Model Context Protocol moves to a stateless core on July 28, with sessions and initialize removed, faster connection setup, parallel handshakes, and multi-round-trip requests for capabilities such as elicitation.

GitHub MCP Server already supports the new specification. GitHub also says tier 1 SDKs preserved backward compatibility and shipped beta support. That reduces migration risk, but it does not replace testing in your client, network, permission, and repository environment. This tutorial combines the official conformance suite with a practical old-client/new-client matrix and deterministic GitHub checks.

What the official release confirms

The GitHub changelog identifies three implementation changes: Redis-backed sessions were removed, required logging and secret-scanning values can be read from guaranteed HTTP headers instead of deep payload inspection, and elicitation was updated so each step can use a separate HTTP request. The official Go SDK wrapper supports old and new mechanisms.

The release also points to an official MCP conformance suite, the draft specification, and tier 1 SDK implementations. Treat those as the protocol authority. Avoid claiming every client is automatically compatible: GitHub’s backward-compatibility statement applies to the tier 1 SDK work it cites, not every custom client, proxy, or bespoke server.

Test objective and acceptance criteria

Prove that supported client-server combinations connect, exchange valid requests, handle multi-step interactions, and preserve expected GitHub behavior without relying on hidden session state. Use a synthetic repository with test issues and branches. Keep production repositories, deployment tools, customer data, and privileged organization settings out of scope.

  • Every supported matrix combination passes applicable official conformance cases.
  • Independent requests do not depend on a previous session or initialization write.
  • Old and new supported clients return equivalent business results for read-only tools.
  • Elicitation completes, cancels safely, and rejects malformed follow-up data.
  • Parallel connections do not cross-contaminate identity or results.
  • Permission failures remain failures and never trigger wider fallback access.
  • Logs support correlation without exposing tokens or sensitive payloads.
  • State-changing calls are reconciled before retry.

Step 1: freeze the compatibility matrix

Record exact client, SDK, server, transport, proxy, authentication, and operating-system builds. Create four primary lanes: current approved client to current GitHub MCP Server, candidate client to current server, current client to a disposable candidate-spec server, and candidate client to the disposable server. Add any internally maintained client or gateway as a separate lane.

Lane Purpose Release gate
Current client / GitHub server Production-like baseline No regression
Candidate client / GitHub server Client migration Conformance plus GitHub checks
Current client / test server Backward compatibility Documented support only
Candidate client / test server New protocol behavior Full applicable suite

Do not label an unsupported combination as a product defect. First confirm it falls within the vendor or SDK support statement.

Step 2: run the official conformance suite

Pin the conformance repository revision in your evidence. Follow its README for the selected client or server adapter rather than copying an unversioned command into permanent automation. Capture the suite revision, adapter configuration, start and finish times, pass/fail counts, skipped cases, and machine-readable output where available.

Investigate skips. A green summary with required cases excluded is not a pass. Map each failure to the relevant draft-spec section and rerun it alone before filing a defect. Preserve the smallest sanitized request and response required for reproduction.

Step 3: prove stateless request behavior

Use a disposable server that exposes a read-only qa_echo tool. Send request A, terminate the connection, then send request B through a fresh connection with no session artifact. Request B must contain everything needed for its own result.

Next, run identical requests through two parallel clients with different correlation IDs and synthetic identities. Verify that outputs, authorization decisions, and logs never mix. Restart one server instance between calls and repeat. This catches accidental dependence on memory, sticky routing, or a removed initialization write.

Step 4: test old and new connection paths

For each supported lane, capture the visible connection outcome and server evidence. Confirm that the candidate client uses the expected new path and that an approved older client works only where backward compatibility is documented. Test connection interruption, retry, proxy header normalization, missing required headers, and duplicated correlation IDs.

GitHub says the new design can read required logging and secret-scanning values from guaranteed HTTP headers. Verify that intermediaries preserve required headers while stripping unauthorized additions. Never print authorization values in screenshots or defect attachments.

Step 5: validate elicitation round trips

Create a synthetic flow that asks for one harmless choice, such as selecting a test label. Exercise accept, cancel, timeout, invalid value, repeated response, and a lost second request. Each round trip should correlate to the original operation without unsafe shared session state.

Cancellation must not create the label or silently choose a default. A malformed response must be rejected. If the action outcome is uncertain after a network failure, query the repository before retrying. Multi-round-trip support is not permission to duplicate state-changing work.

Step 6: add deterministic GitHub checks

Protocol conformance proves wire-level expectations, not business correctness. Run read-only GitHub tools against the synthetic repository: list a known issue, read a known file at a pinned commit, and query a deliberately missing object. Compare IDs, commit hashes, ordering, pagination, error classes, and permission behavior across lanes.

Then run one reversible write, such as adding a temporary label to a synthetic issue. Use an idempotency or reconciliation strategy, verify the change through GitHub’s API or UI, and remove it after evidence capture. Require human approval for writes even if the MCP client can execute them automatically.

Step 7: inspect security and observability

Collect correlation ID, timestamp, client and server build, protocol revision, tool name, outcome class, and latency. Redact tokens, cookies, repository secrets, request bodies with user content, absolute paths, and elicited values. Confirm monitoring does not depend on deep inspection of every payload if the new implementation expects required metadata in headers.

Run negative tests with missing credentials, insufficient repository access, a revoked token, an unapproved tool, and a repository outside the fixture scope. Expected denial must remain denial across retries and compatibility wrappers.

Step 8: make the rollout decision

Publish a result matrix showing applicable conformance cases, skips, deterministic GitHub outcomes, elicitation paths, parallel isolation, reconnect behavior, and permission negatives. Roll out first to a small QA group with read-only tools. Define rollback as restoring the previous approved client or SDK and rerunning the baseline lane.

Block rollout if a required conformance case fails, a supported older client regresses, parallel requests mix context, cancellation creates a side effect, headers leak secrets, permissions widen, or a GitHub result differs without an explained API change.

Common mistakes

  • Equating SDK compatibility with every client: test custom clients and gateways explicitly.
  • Stopping at the conformance suite: add deterministic GitHub business checks.
  • Ignoring skipped tests: document why every required case ran.
  • Using production repositories: isolate synthetic data and reversible actions.
  • Retrying uncertain writes: reconcile repository state first.
  • Logging complete payloads: retain correlation evidence, not secrets.

Official sources