Codex becomes more useful for QA teams when it knows how your test suite should be reviewed. A prompt can explain the current task, but a repository-level AGENTS.md file can carry the recurring instructions that every AI-assisted test review should follow.

This tutorial shows how to use Codex AGENTS.md for QA work without treating AI output as automatically correct. The goal is simple: give Codex clear review standards for test automation, then make the QA engineer validate the diff, test evidence, and remaining risk before anything is merged.

Why QA Teams Should Use AGENTS.md

AI coding agents often produce technically valid test code that still misses a team standard. A generated Playwright test might use brittle CSS selectors. A Selenium refactor might hide an important wait condition. An API test might check only a status code while ignoring response shape and business rules.

AGENTS.md is useful because it turns repeated review expectations into shared project context. Instead of reminding Codex in every thread to prefer user-facing locators, avoid weak assertions, run the smallest relevant test command, and summarize risks, you can place those expectations near the codebase.

Official Codex documentation describes AGENTS.md as a way to provide custom instructions that Codex discovers from the Codex home and project path. The same documentation notes that Codex rebuilds the instruction chain at the start of a run or TUI session. For QA teams, that means AGENTS.md can act like a lightweight testing handbook for AI-assisted work.

What To Put In A QA-Focused AGENTS.md

A good QA instruction file is short, specific, and testable. It should not read like a general company policy. It should tell Codex exactly how to behave when it touches tests, selectors, fixtures, assertions, or CI scripts.

Start with the parts of your test process that frequently fail in review:

  • Which test framework commands should be run for local validation.
  • Which locator strategy is preferred for UI automation.
  • Which assertion quality bar must be met before a test is accepted.
  • How test data should be created and cleaned up.
  • Which files are sensitive or should not be changed casually.
  • What evidence Codex should report after making a change.

Keep it practical. Codex does not need every testing principle your team knows. It needs the rules that repeatedly affect implementation and review.

Example AGENTS.md For Test Automation Review

Use this as a starting point, then adapt the commands and standards to your own repository.

# QA instructions for Codex

## Test automation standards
- Prefer stable, user-facing locators such as role, label, text, and test id.
- Avoid brittle selectors tied to generated class names or deep DOM structure.
- Every test must include meaningful assertions, not only navigation or status checks.
- For API tests, validate status, schema, important fields, and at least one business rule.
- Keep setup data explicit and isolated. Do not depend on previous test order.

## Before editing tests
- Read the related page object, fixture, helper, and existing test pattern first.
- Explain the likely failure mode before proposing a fix.
- Keep the change small unless the user asks for a refactor.

## Validation commands
- For Playwright UI changes, suggest the smallest relevant Playwright test command.
- For unit or API changes, suggest the nearest targeted test command.
- If a command cannot be run, state exactly why and list the manual check needed.

## Done criteria
- Summarize files changed, tests run, and tests not run.
- Call out any remaining flake risk, data dependency, or coverage gap.
- Do not claim the issue is fixed unless validation evidence supports it.

Notice that this file does not ask Codex to ?write better tests? in a vague way. It defines reviewable behavior: read existing patterns, prefer stable locators, improve assertion quality, run targeted validation, and report what remains uncertain.

Step-By-Step Workflow For A QA Review

Follow this process when using Codex to review or change a test automation diff.

1. Add The Instruction File

Create AGENTS.md at the repository root when the rules apply to the whole project. If a subfolder has a different framework, such as tests/e2e for Playwright and tests/api for pytest, add a more specific instruction file in that folder with only the differences.

Keep the first version small. Five to ten strong instructions are better than a long file nobody trusts.

2. Ask Codex For A Review Before A Fix

For a failing test or AI-generated test, start with review mode instead of immediately asking for edits:

Review this changed Playwright test against the project AGENTS.md instructions.
Focus on selector stability, assertion strength, test data isolation, and validation evidence.
Do not edit yet. Return the top risks and the smallest safe fix plan.

This helps separate diagnosis from implementation. QA engineers can reject a weak plan before Codex modifies files.

3. Use /review For Local Diff Checks

The Codex CLI documentation describes /review as a workflow that reads a selected diff and reports prioritized findings without touching the working tree. That fits QA review well because you can inspect uncommitted automation changes before they become a pull request.

Use a custom review instruction such as:

Focus on QA automation risks: weak assertions, brittle selectors, missing negative paths, hidden waits, shared state, and CI flake risk.

The output should not replace human review. Treat it as a second pass that helps you find issues worth checking manually.

4. Let Codex Patch Only The Agreed Scope

After reviewing the risks, give Codex a narrow task:

Apply only the locator and assertion changes from the approved plan.
Do not refactor unrelated helpers.
After editing, tell me the exact test command to run and what evidence would prove the fix.

This keeps the review manageable. Broad prompts often create broad diffs, and broad diffs are harder for QA teams to validate quickly.

5. Validate With Tests And Evidence

Codex can suggest commands and may run them depending on the environment and permissions, but the QA decision should be based on evidence. For UI automation, capture the failing test before the fix when possible, the changed selector or assertion, and the passing targeted run after the fix. For API tests, capture the response fixture or schema expectation that proves the assertion is meaningful.

Screenshot Checklist

Capture these screens while following the workflow:

  • The repository root showing AGENTS.md beside the test folders.
  • The AGENTS.md section that defines locator, assertion, and validation rules.
  • The changed test diff before accepting the AI-suggested edit.
  • The Codex review output or inline comments showing QA-specific findings.
  • The targeted test command and result after the change.
  • A final summary showing files changed, tests run, and remaining risks.

Common Mistakes To Avoid

Writing generic instructions. ?Follow best practices? is not enough. Replace it with concrete rules such as ?prefer role and label locators? or ?assert response fields that prove the business rule.?

Putting secrets or private credentials in AGENTS.md. The file should guide behavior, not store sensitive values. Use environment variables, secret managers, or existing local setup instructions for credentials.

Letting the instruction file grow forever. If every incident adds five new rules, Codex will receive noisy guidance. Keep only rules that change behavior during real QA work.

Skipping deterministic validation. AGENTS.md can improve the first draft and review pass, but it cannot prove that the product works. Keep targeted test runs, CI, exploratory checks, and human judgment in the release path.

Best Practices For Maintaining The File

Review AGENTS.md whenever the test framework changes, a new test folder is added, or the team changes a major convention. For example, update it when a Playwright suite moves from CSS selectors to role-based locators, when API tests adopt schema validation, or when CI commands change.

Make the file easy to audit. Use headings for test standards, validation commands, and done criteria. Avoid long paragraphs. If a rule cannot be checked in a diff, consider whether it belongs in a human QA checklist instead.

When using the Codex app review pane, inline comments are useful for precise feedback on specific changed lines. If Codex missed an assertion or touched an unrelated helper, comment on that line and then ask it to address only those comments while keeping scope minimal.

FAQ

Does AGENTS.md replace a QA checklist?

No. It turns repeated QA expectations into reusable Codex instructions, but the QA engineer still needs to review coverage, risk, and test evidence.

Should every test folder have its own AGENTS.md?

Only when the rules differ. A root file is enough for shared standards. Add folder-level files when UI, API, mobile, or performance tests need different commands or review rules.

Can Codex run the tests listed in AGENTS.md?

It can suggest or run commands depending on the surface, permissions, and local environment. The important rule is that the final post-change summary should clearly say what was run and what was not run.

What is the best first rule to add?

Start with your most common review failure. For many QA teams, that is assertion quality: generated tests often perform an action but do not verify the outcome deeply enough.

References

Conclusion

Codex AGENTS.md is most valuable when it captures the QA rules your team already enforces: stable locators, meaningful assertions, isolated test data, targeted validation, and honest risk reporting. Use it to make AI-assisted test review more consistent, then keep the final decision grounded in deterministic test evidence and human QA judgment.