GitHub Copilot Hooks for QA: Test Pre-Tool Guardrails Safely

GitHub Copilot hooks for QA let teams run deterministic code at important agent lifecycle points. A pre-tool hook can inspect a proposed command or edit before it runs; a post-tool hook can capture sanitized evidence afterward. That sounds like a clean guardrail, but the failure semantics matter as much as the happy path.

GitHub’s current reference makes an important distinction: a crashing command-based preToolUse hook denies the tool call, while a hook timeout falls through to the normal permission flow. HTTP hook errors also fall through. A QA plan that tests only explicit allow and deny decisions will miss the riskiest cases.

This tutorial builds a disposable repository, runs the same decision matrix in Copilot CLI and cloud agent where available, and verifies the result with independent process, filesystem, Git, network, and audit evidence.

What GitHub officially documents

GitHub documents hooks for Copilot CLI and Copilot cloud agent. Events include session start and end, prompt submission, preToolUse, postToolUse, failures, agent and subagent stop points, errors, permission requests, and notifications, with differences by surface.

A preToolUse hook receives the session, timestamp, working directory, tool name, and tool arguments. It can return allow, deny, or ask, plus a reason or modified arguments. When multiple pre-tool hooks run, any deny blocks the action. Under cloud agent, ask acts as deny because no user is present to answer.

Repository configuration uses version 1 JSON under .github/hooks/*.json. For cloud agent, the file must be on the default branch. Cloud jobs run non-interactively in an ephemeral Linux sandbox and load repository hook files, not the CLI user’s settings, installed plugins, or machine policy hooks. Only Bash entries, or the cross-platform command fallback, apply there.

1. Create a harmless guardrail lab

Use a private repository with no production access, customer data, package publishing credentials, deploy keys, or writable external integrations:

copilot-hook-lab/
  .github/hooks/qa-guardrails.json
  scripts/qa-pretool-gate.js
  scripts/qa-posttool-evidence.js
  fixtures/pretool/
  evidence/
  tests/smoke/

Create six proposed actions: read a fixture, edit a permitted test file, run one deterministic smoke test, attempt to read a synthetic secret-canary path, call a non-routable network canary, and request a denied write outside the lab directory. None should contact production or make an irreversible change.

Freeze the Copilot surface and build, operating system, account policy, repository and commit, sandbox and permission state, hook file hash, script hashes, configuration source and load order, matcher spelling and casing, timeout, fixture hash, expected decision, and expected side effects. Repeat the matrix after any of these inputs changes.

2. Start with a narrow configuration

Keep the hook file small and send complex logic to a reviewed script:

{
  "version": 1,
  "hooks": {
    "preToolUse": [
      {
        "type": "command",
        "matcher": "bash|edit|view",
        "bash": "node scripts/qa-pretool-gate.js",
        "powershell": "node scripts/qa-pretool-gate.js",
        "cwd": ".",
        "timeoutSec": 3
      }
    ],
    "postToolUse": [
      {
        "type": "command",
        "matcher": "bash|edit|view",
        "bash": "node scripts/qa-posttool-evidence.js",
        "powershell": "node scripts/qa-posttool-evidence.js"
      }
    ]
  }
}

The gate should parse standard input as JSON, reject unknown types, normalize the working directory, tokenize supported command payloads without shell interpolation, compare exact paths and argument arrays, and emit exactly one compact JSON decision on standard output. Send diagnostics to standard error and redact secrets before logging.

Do not copy the sample allow rules into a real repository unchanged. Build the allowlist from reviewed commands and paths in your own test project.

3. Unit-test the payload contract first

GitHub recommends testing hook scripts locally by piping representative JSON into them. Store fixtures for every supported tool name and payload shape. Each fixture needs a case ID, input hash, expected exit code, expected decision, expected reason category, expected modified arguments, and expected log fields.

Compare parsed JSON, not human wording. A response can be syntactically valid yet unsafe if the reason, arguments, or action scope differs from the fixture.

4. Verify allow, ask, deny, and modified arguments

Run explicit decisions one at a time. For allow, capture the hook decision and prove that only the intended tool ran. For ask, verify the CLI displays the normal permission step and that denial leaves no side effect. In cloud agent, expect ask to become deny rather than waiting for an unavailable user.

For explicit deny, require a stable rule ID and concise reason. The agent should receive enough information to select a safe alternative, but the hook must not leak the rejected arguments when they contain a canary.

Test modifiedArgs with a harmless example that removes an unsupported flag or narrows a path. Independently inspect the spawned process argument array. The original broader command must never execute, and a malformed replacement must not silently revert to the original.

5. Test command-hook failure semantics

This is the core regression suite. GitHub’s reference says a command preToolUse hook denies the call when it crashes, exits with code 2, or returns another non-timeout non-zero exit. That denial still applies if standard output claims allow.

Create separate fixture modes for:

• explicit deny with exit 0;
• exit 2 with allow-shaped output;
• another non-zero exit;
• uncaught exception before output;
• valid JSON followed by a crash;
• empty output with a non-zero exit;
• malformed JSON with exit 0;
• two final JSON objects concatenated on standard output.

For every non-timeout failure, prove the proposed tool never spawned and the filesystem, Git tree, network log, and external canaries remain unchanged. Save the surfaced diagnostic, but score the actual side effect as the authority.

The multiple-output case deserves its own test: GitHub warns that two non-progress JSON objects concatenate into invalid JSON. Ensure the hook prints exactly one final decision object. Debug messages belong on standard error.

6. Treat timeout as a different safety case

GitHub documents timeouts as fail-open for all events, including preToolUse and administrator policy hooks. The timed-out hook is killed, a warning is surfaced, and the action continues through the normal permission flow rather than being denied by the hook.

Use a harmless bounded-delay fixture that exceeds the configured timeout. Verify the hook process terminates and the ordinary permission or sandbox layer still evaluates the proposed action. Test three outcomes separately: the normal layer blocks, asks, or permits the synthetic action.

Never describe timeout as an automatic allow. The accurate assertion is that the hook does not supply the denial and processing falls back to the regular control path. Your baseline permissions must therefore remain safe even when the hook is unavailable.

7. Compare command and HTTP hooks

HTTP hooks send JSON to a configured endpoint. GitHub says preToolUse and permissionRequest HTTP hooks require HTTPS because their responses can grant permissions. Network failures, timeouts, and non-success responses fall through to normal permission handling.

If your team uses HTTP hooks, point the lab to a private synthetic endpoint and test valid allow, valid deny, TLS failure, DNS failure, timeout, non-success response, malformed response, stale response, duplicate request, and unauthorized endpoint substitution. Capture only fixture data; do not transmit source code, prompts, tokens, or real tool arguments.

Compare the result with the command-hook matrix. A command crash and an HTTP outage do not have the same pre-tool behavior. Make that difference visible in the release evidence and alerting.

8. Challenge matcher and configuration discovery

Native camelCase event matchers are full-string regular expressions over runtime tool names. PascalCase compatibility events use different mapped tool names and matcher semantics. Freeze the event name, matcher, and observed tool name together.

Test exact match, alternation, an attempted partial match, case mismatch, invalid regular expression, unexpected tool alias, and a newly observed tool. Verify unmatched actions reach the baseline permission system; they must not be reported as hook-approved.

Also exercise structural configuration failures. A malformed item in a directory-loaded file can be dropped while valid siblings continue, but invalid JSON, a bad version, or a non-array event list can reject the entire file. Test two files so one failing configuration cannot be mistaken for complete hook absence.

9. Verify multiple hooks and load order

GitHub combines hooks from several CLI sources and runs entries in order. Policy-level hooks load before user, project, and plugin sources, and policy hooks cannot be disabled by ordinary disableAllHooks. Cloud agent loads only repository hook files by default and does not support CLI machine policy hooks.

Build two deterministic pre-tool hooks: one allows the read fixture and one denies the same action under a stricter condition. Confirm any deny wins. Then reverse file names where ordering applies, add a third observer, and verify the decision is unchanged while audit order remains explainable.

Test disableAllHooks at file scope in the disposable repository and confirm which hooks stop. Then restore the reviewed configuration and prove its hashes and behavior. Never use disablement as a production workaround without a documented owner and rollback time.

10. Test cloud-agent differences explicitly

Merge the lab hook into the default branch because the cloud agent uses that source. Record the default-branch SHA and assigned task SHA. Confirm the cloud job runs the Bash entry in its Linux sandbox; a PowerShell-only hook is not sufficient.

Test repository hooks with no user hook directory, no installed plugin hooks, no local settings file, and no machine policy hook. The absence of those CLI sources in cloud agent is expected, not a defect. Verify ask denies, interactive prompts never appear, and hook files or evidence written under the ephemeral home directory are not treated as durable records.

Export only the minimum sanitized evidence needed for review. The sandbox can be destroyed at job end, so persistence must be deliberate and must not expose sensitive data.

11. Attack the hook input and evidence path

Tool names, arguments, paths, prompts, outputs, environment data, and errors are untrusted. Seed quotes, separators, Unicode look-alikes, path traversal, oversized JSON, missing fields, nested objects, nulls, duplicate keys, newline injection, and synthetic secret canaries. Do not construct a shell command by concatenating these values.

Verify logs omit tokens, passwords, full environment dumps, sensitive paths, and raw prompts. Restrict script and log permissions, use per-session evidence files or atomic appends, and prevent concurrent sessions from overwriting one another. Hash evidence after the run.

GitHub recommends keeping synchronous hooks under five seconds when possible. Measure p50, p95, and worst-case duration with the observer disabled and enabled. A slow guardrail can create timeouts that change its own enforcement behavior.

12. Build a release scorecard

For each case, record the source and script hashes, session ID, tool name, normalized argument fingerprint, hook order, exit code, elapsed time, parsed decision, fallback path, process observation, changed files, Git status, network connections, evidence hash, and reviewer result.

Block rollout when an expected hook did not load, a matcher skipped a protected tool, a deny still spawned a process, modified arguments widened scope, a timeout reached an unsafe baseline permission, a log exposed canary data, cloud and CLI behavior were conflated, concurrent evidence was corrupted, or rollback failed.

Run deterministic CI on the hook scripts and fixtures. Require code review for hook changes because they execute inside developer or agent environments. Branch protection, sandboxing, repository permissions, and human approval remain the final enforcement layers.

Screenshot-friendly walkthrough

1. Official Copilot hooks event list and preToolUse decision fields
2. Disposable repository tree with hook, scripts, fixtures, and evidence
3. Version 1 repository hook configuration with frozen matcher and timeout
4. Local payload fixture producing one compact allow decision
5. Explicit deny with no process or filesystem side effect
6. Exit 2 or crash producing a denied tool call despite allow-shaped output
7. Timeout warning followed by the normal permission checkpoint
8. Matcher matrix showing exact, mismatched, invalid, and compatibility names
9. CLI-versus-cloud configuration and behavior matrix
10. Final evidence scorecard, passing CI, restored configuration, and human approval

Official GitHub references

• About hooks for GitHub Copilot
• GitHub Copilot hooks reference
• Customize cloud-agent workflows with hooks

Final takeaway

GitHub Copilot hooks for QA are strongest when teams test the control’s failure behavior, not only its intended decision. Prove allow, ask, deny, modified arguments, command errors, timeouts, HTTP outages, matcher gaps, surface differences, privacy, concurrency, and rollback. Keep the baseline permission system safe when hooks fail, and keep CI and humans in charge of merge and release.