GitHub Copilot agentic autofix can take a code-scanning alert beyond a one-file suggestion. In its current public preview, Copilot can explore relevant repository files, propose a patch, rerun CodeQL, iterate, and open a draft pull request. That is useful automation, but it does not turn a generated fix into automatic approval.

This tutorial shows a practical workflow for QA engineers and SDETs: establish the alert baseline, assign the issue, inspect the pull request, reproduce the vulnerable behavior, run focused regression tests, and document what the automated validation did not cover.

What agentic autofix actually validates

GitHub’s official documentation says assigning a supported code-scanning alert starts a Copilot cloud-agent session. The agent explores the codebase, creates a fix, and can rerun CodeQL before opening a draft pull request. You can assign one alert or group multiple alerts into one fix session.

The limits matter. GitHub describes agentic autofix as best effort. Its rerun uses the configured code-scanning query suite and cannot confirm that a change fixes alerts raised only by custom queries or the security-extended suite. Fix quality for alerts from third-party scanners is not guaranteed. The feature also requires the appropriate GitHub Code Security or GitHub Advanced Security access, a Copilot license, and Copilot cloud agent availability. Sessions consume AI credits and GitHub Actions minutes.

For QA, the correct mental model is simple: the draft pull request and CodeQL rerun are evidence, not complete proof.

Example QA scenario

Assume CodeQL reports that a web endpoint builds a database query from an untrusted request parameter. The alert has a clear source-to-sink path. You assign it to Copilot, and agentic autofix opens a draft pull request that replaces string construction with a parameterized query.

The change looks plausible and the alert closes on the default CodeQL rerun. Before merge, QA still needs to answer three questions: can the unsafe behavior be reproduced on the original revision, does the patched revision block it without breaking valid requests, and do all required security checks still pass?

Step 1: Capture the alert baseline

Before assigning the alert, record its rule ID, severity, affected branch, commit, source, sink, data-flow path, and query suite. Link the CodeQL result and save a redacted screenshot. If the alert came from a third-party tool or a custom query, mark that clearly because the agent’s built-in validation may not cover it.

Create a safe reproduction plan using a non-production environment and synthetic data. Define the vulnerable request, the expected unsafe result on the original revision, and the safe result after the patch. Never paste production secrets, customer data, or working exploit material into a public issue.

Step 2: Assign a narrow unit of work

Start with one closely related alert unless multiple alerts share the same root cause. A narrow assignment makes the generated diff easier to inspect and the validation easier to attribute. From the code-scanning alert, assign Copilot and wait for the draft pull request.

Record the session link, pull-request link, files changed, and validation summary. If no pull request appears, capture the failure rather than repeatedly spending credits without understanding the cause.

Step 3: Review the proposed patch

Read the full diff, not just Copilot’s summary. Confirm that the change addresses the reported source-to-sink path and does not merely hide the alert. Look for disabled rules, broad sanitization helpers, weakened validation, suppressed exceptions, removed assertions, or unrelated refactoring.

Check boundary behavior too. A parameterized query may close injection risk while changing null handling, Unicode input, pagination, sorting, or authorization behavior. Ask the application owner to review security-sensitive design choices. AI-generated code should not approve itself.

Step 4: Reproduce before and after

Run the safe reproduction against the vulnerable baseline first. Record the request, environment, commit, expected result, actual result, and evidence. Then run the same input against the patched branch. The original should demonstrate the problem, while the patched build should reject or safely handle the input.

Also include at least two valid controls. Confirm that ordinary input still succeeds and that a boundary value follows the product requirement. This separates a real security fix from a blunt change that breaks the feature.

Step 5: Add focused regression tests

Add a test at the lowest useful layer. For a query-construction flaw, a service or integration test is usually stronger than a mocked unit test because it observes the database boundary. Include the malicious-shaped input as data without embedding destructive behavior. Assert the security outcome and the expected user-visible response.

Keep deterministic checks in CI. The regression must fail on the vulnerable revision and pass on the patch. Review test names, fixtures, cleanup, logging, and secret handling. Avoid an assertion that checks only for a successful status code while ignoring the protected data or side effect.

Step 6: Rerun the complete security gates

Confirm the CodeQL result in GitHub rather than trusting copied text in the pull request. Then run every security gate required by the repository: the default suite, security-extended queries when used, custom queries, dependency scanning, secret scanning, and relevant third-party tools.

If agentic autofix validated only the default suite, document that exact scope. A closed alert does not prove the absence of related weaknesses elsewhere. Search for sibling code paths and repeat the focused test where the same helper or pattern is used.

Step 7: Document residual risk

Finish with a compact evidence table containing the alert, vulnerable commit, fixed commit, reproduction result, regression-test result, CodeQL configuration, other scanners, reviewer, and decision. Note any untested platform, data path, custom query, or deployment condition.

Useful decisions are specific: approved for merge after all required checks pass, blocked because the original vulnerability was not reproduced, or blocked pending security review of a shared helper. Avoid writing only “Copilot fixed it.”

QA review checklist

  • The original alert and query suite are recorded.
  • The reproduction uses synthetic data in a safe environment.
  • The generated diff fixes the root cause without suppressing analysis.
  • The vulnerable revision fails and the patched revision passes.
  • Valid and boundary inputs still behave correctly.
  • A focused regression test fails before and passes after.
  • Required CodeQL suites, custom queries, and third-party scanners run.
  • Logs and screenshots contain no secrets or customer data.
  • Security-sensitive code receives human review.
  • Residual risk and approval scope are documented.

Common mistakes to avoid

Do not treat a closed default CodeQL alert as proof that custom or security-extended queries pass. Do not merge a large multi-alert change without isolating which patch fixed which result. Do not accept a regression test that passes on both the vulnerable and patched revisions. Finally, do not let a generated summary replace inspection of the actual diff, checks, and artifacts.

Final takeaway

GitHub Copilot agentic autofix can reduce the mechanical work between a code-scanning alert and a reviewable patch. QA adds the missing assurance layer: reproduce the risk, challenge the change, verify observable behavior, run the full security toolchain, and preserve a scoped human decision.

Official references