GitHub Copilot CLI Checkpoints for QA: Test Context Continuity

GitHub Copilot CLI checkpoints for QA can keep a long investigation moving after conversation history is compacted or a local session is resumed. They are useful during flaky-test triage, migration work, and multi-phase debugging—but they are summaries, not a replayable test log.

This tutorial builds a private, disposable lab around one intermittent checkout test. You will freeze the session and repository identity, define a critical-fact oracle, deliberately cross compaction boundaries, inspect the generated checkpoints, resume after several interruption types, and revalidate every consequential claim with Git and test evidence. The goal is context continuity without silently trusting stale or incomplete memory.

What GitHub officially documents

GitHub’s Copilot CLI context-management documentation says /compact manually compacts the current conversation. Compaction snapshots the conversation and produces a structured summary of the goal, completed work, relevant files, next steps, and other context. Earlier history is replaced by that summary together with the original user instructions and current plan or to-do state.

The tradeoff matters to testers: GitHub says exact message wording, full command output, minor decisions, and other fine detail may be lost. Every manual or automatic compaction creates a numbered checkpoint in the session workspace. /session checkpoints lists them, while /session checkpoints 2 displays a selected checkpoint. Compaction cannot be reversed.

GitHub’s CLI overview documents /resume and the --resume flag for prior sessions. --continue resumes the most recently closed local session. The best-practices guide describes local session data under ~/.copilot/session-state/{session-id}/, including event history, workspace metadata, a plan, checkpoints, and files. Treat this storage as potentially sensitive.

Step 1: Build a disposable flaky-test investigation

Create a private repository with a deterministic service and one deliberately intermittent test fixture. A good scenario is a checkout test that fails only when an order-status event arrives after a bounded polling window. Use synthetic accounts and tokens; do not copy customer logs, production credentials, or personal data into the repository or prompt.

Define five investigation phases:

1. Explore: map the test, implementation, fixtures, and recent commits.
2. Plan: list hypotheses and the evidence that can accept or reject each one.
3. Reproduce: run a fixed command over a declared seed range and save the raw report.
4. Fix: make one bounded change tied to the accepted hypothesis.
5. Verify: rerun the reproduction set, focused regression tests, and independent static checks.

Keep unrelated work out of this session. GitHub recommends starting a new session when the task changes materially or repeated compactions have made the context confusing.

Step 2: Freeze the continuity contract

Create a manifest outside the agent’s editable path. Record the Copilot CLI build, chosen model and context tier when visible, operating system, repository path and remote, branch, base and head SHAs, session ID and optional name, and the starting working-tree status. Also hash all repository instructions, including applicable root or nested instruction files.

Record sandbox and permission settings, enabled tools, remote-export setting, environment names without values, and the exact test commands. Give the run a unique identifier such as CPC-QA-2026-09-02-A. If any frozen field changes, classify the next result as a new trial rather than silently extending the old one.

Identity	Why QA freezes it
Session ID or name	Prevents resuming the wrong investigation
Repository path, branch, base and head SHA	Detects code and workspace drift
Instruction hashes	Shows whether operating rules changed
CLI, model context, tools, and permissions	Separates context behavior from configuration drift
Command and report hashes	Preserves independently verifiable execution evidence

Step 3: Define a critical-fact oracle

Before asking Copilot to investigate, create an oracle of facts that must survive each checkpoint. Do not include the final answer. Include only facts the session has legitimately established:

• the exact failing test ID and frozen head SHA;
• the reproduction command, seed range, environment label, exit code, and report hash;
• one rejected hypothesis and the evidence that rejected it;
• the accepted change scope and files that must not be edited;
• the expected verification command and pass criteria;
• the rule that merge and release remain human-owned.

Classify facts as critical, useful, or disposable. Critical facts must be correct after compaction. Useful facts may be summarized. Full terminal output is disposable from conversational memory because the immutable report already stores it. This prevents the evaluation from rewarding verbatim recall when evidence integrity is what matters.

Step 4: Capture a baseline before compaction

Start Copilot CLI in the frozen repository and explicitly state the investigation ID, allowed scope, evidence locations, and no-merge policy. Ask it to explore and propose a plan before editing. Record the session ID and use /context where appropriate to understand context use without treating the indicator as an exact behavioral oracle.

At the end of the explore phase, ask for a structured handoff containing current goal, verified facts, open hypotheses, relevant files, next command, and prohibited actions. Save the response hash, but independently record the Git SHA, status, command, exit code, and report hash. This is the pre-compaction baseline.

Step 5: Test manual checkpoints

Run /compact after exploration. Inspect /session checkpoints, open the new numbered checkpoint, and compare it with the oracle. Repeat after planning and reproduction so the same investigation crosses several compaction boundaries.

For every checkpoint, score:

• Critical-fact retention: correct critical facts divided by all critical facts.
• Unsafe substitution rate: invented or wrong critical values divided by all evaluated critical values.
• Evidence traceability: claims linked to the correct report, commit, file, or command identity.
• Stale-state detection: whether changed repository or instruction state is noticed before action.
• Next-step validity: whether the proposed next command is safe in the current state.

A checkpoint passes only when every critical identity is correct, missing detail is labeled unknown, no rejected hypothesis becomes accepted, and the next step requires fresh evidence where needed. Do not fail the session merely because exact prose or full terminal output was summarized; GitHub explicitly says those details may be lost.

Step 6: Revalidate after every compaction

Immediately after each compacted checkpoint, ask the session to state the repository path, branch, head SHA, current test ID, accepted and rejected hypotheses, allowed edit scope, and next verification gate. Compare that response with the external manifest.

Then run read-only checks outside the conversation: confirm the working directory, remote, branch, head SHA, working-tree diff, changed-file list, instruction hashes, and evidence-file hashes. Re-run a cheap deterministic smoke test before allowing edits. If conversation memory and independent state disagree, stop the workflow, preserve both records, and start a controlled recovery. Never let a fluent summary override the repository.

Step 7: Test session resume paths

Close the session after a checkpoint and test each documented path in a separate clone or repeatable trial:

• /resume from an active CLI instance;
• copilot --resume with explicit selection;
• copilot --continue for the most recently closed local session;
• an explicit session ID or session name when supported by the current command surface;
• resume after a normal exit, interrupted process, and machine restart.

The exact-session path should be the release-grade test. A most-recent shortcut is convenient but becomes ambiguous when two investigations run concurrently. After resume, verify the selected session ID, repository identity, checkpoint count, current plan, and last accepted evidence before issuing any write or test command.

Step 8: Exercise drift and contamination

Use isolated trials for each failure mode. Resume the valid session from the wrong directory, then from another repository with a similar test name. Move the branch forward by one harmless commit. Rename a relevant file, delete a report, change a repository instruction, modify permissions, or disable a previously available tool. The safe response is to detect drift, explain which frozen identity changed, and request revalidation—not to continue from remembered paths.

Also test an incomplete turn, a queued duplicate prompt, two concurrent local sessions, an externally edited plan, and a stale suggested command. Add a harmless secret canary to a protected environment value and assert that it never appears in checkpoints, screenshots, reports, or social copy. Do not deliberately place real secrets in the session to see whether they leak.

Finally, introduce an unrelated request such as drafting release notes. The correct operational choice is a new session. Continuing unrelated work in the flaky-test session increases contamination and makes checkpoint evaluation less meaningful.

Step 9: Test automatic compaction realistically

GitHub documents automatic compression when the session approaches its context limit. Exercise it with a realistic long investigation: multiple small reports, focused diffs, and staged hypothesis updates. Do not paste giant repeated logs simply to consume context. Store verbose evidence in files and provide stable paths and hashes.

When automatic compaction occurs, record the event, inspect the new checkpoint, and apply the same oracle. Compare automatic and manual trials by retention, unsafe substitutions, traceability, and recovery time. The purpose is not to force a precise token threshold; it is to verify safe continuity when the product compacts under ordinary workload.

Step 10: Protect session state

Local session state may contain prompts, file references, plans, checkpoint summaries, and retained artifacts. Verify filesystem permissions, backup and retention expectations, cleanup procedures, and who can access the user profile. Prefer synthetic logs, redact tokens before they reach the session, and disable remote export where organizational policy requires it and the current CLI supports that control.

Deleting terminal history is not a complete cleanup if session-state files or screenshots remain. Document which artifacts are kept for the benchmark, how long they remain, and how deletion is verified. Security review should include the session directory, the external evidence store, CI artifacts, and generated images.

Validation matrix

• Compaction: manual, automatic, repeated, canceled, and checkpoint listing or detail review.
• Resume: normal exit, process interruption, restart, explicit session, named session, most recent, and concurrent sessions.
• Repository: wrong directory, wrong repository, branch drift, head drift, renamed file, deleted report, and dirty working tree.
• Instructions and tools: changed rule hash, changed model context, removed tool, stricter permission, sandbox change, and remote-export change.
• Conversation: incomplete turn, duplicate prompt, stale plan, unrelated task, many compactions, and unsupported remembered detail.
• Security: synthetic canary, redacted logs, local state permissions, retention, screenshots, and cleanup.
• Evidence: Git identity, diff, command, exit code, test report, artifact hash, and deterministic CI rerun.

Screenshot checklist

1. Frozen session manifest with sensitive values redacted.
2. Five-phase investigation plan and critical-fact oracle.
3. Baseline Git identity, clean status, command, exit code, and report hash.
4. Manual /compact action and checkpoint list.
5. Numbered checkpoint compared with the oracle.
6. Independent post-compaction repository and evidence validation.
7. Explicit session resume with matching session and repository identity.
8. Wrong-directory or branch-drift trial stopping safely.
9. Automatic-compaction retention and unsafe-substitution scorecard.
10. Final deterministic test evidence and human merge gate.

Common mistakes

• Treating a checkpoint as exact terminal history.
• Using --continue when several recent sessions exist.
• Resuming without checking repository, branch, SHA, instructions, and permissions.
• Marking forgotten fine detail as a failure even when immutable evidence preserves it.
• Accepting a remembered pass result instead of rerunning the test.
• Mixing unrelated work into one long session.
• Putting production secrets or customer logs into the continuity test.
• Letting a resumed agent merge or release based on summarized state.

Final QA gate

Approve this workflow only when checkpoints retain every critical identity without unsafe substitutions, resume always selects the intended session and repository, drift is detected before action, all execution claims trace to immutable evidence, synthetic canaries stay private, and deterministic CI reproduces the final result. A human still reviews the diff and controls merge and release.

That is the practical value of GitHub Copilot CLI checkpoints for QA: they help a long investigation continue, while a strict external evidence contract prevents summarized context from becoming an accidental source of truth.

References

• Copilot CLI context management
• Copilot CLI overview
• Copilot CLI best practices
• Copilot CLI command reference