Cursor Automations for QA can turn repeated flaky-test investigation into an evidence-first workflow. The safe goal is not an agent that automatically retries until CI turns green. It is a bounded automation that receives an approved trigger, collects sanitized failure artifacts, groups repeat signatures, reports confidence and gaps, and leaves reruns, defects, code changes, and release decisions to people.

This tutorial builds a synthetic triage automation for a disposable repository. It reacts to a completed CI workflow and can also run on a schedule for backlog review. The first version is read-only: no production access, no automatic merge, and no silent test suppression.

What Cursor officially documents

The current Cursor Automations documentation says automations run cloud agents on schedules or events. A setup includes a trigger, instructions, optional tools, repository scope, and activation. Scheduled triggers may start late but not before the indicated time, and an automation with multiple triggers runs whenever any trigger fires.

Cursor supports source-control events including CI-completed and workflow-run-completed events, plus Slack, webhook, Linear, Sentry, and PagerDuty triggers. The Automations improvements changelog specifically describes GitHub workflow completion and a failed-Actions triage template.

The official Automations announcement explains that each invocation uses a cloud sandbox, configured tools and models, output verification, and a persistent memory tool. These capabilities are useful, but they expand the QA threat model. Cursor warns that an MCP connection exposes every tool on that server and that untrusted inputs can create misleading or malicious memories that affect later runs.

Define a narrow triage contract

Write the contract before creating the automation:

For each eligible completed CI run, produce at most one triage report. Use only approved artifacts. Identify stable failure signatures, separate likely infrastructure failures from likely product failures, link every conclusion to evidence, and take no code-changing or release action.

A useful report contains the workflow run ID, commit, branch, test name, normalized error signature, first and latest occurrence, rerun history, artifact links, classification, confidence, missing evidence, and recommended human action.

Step 1: choose the trigger deliberately

Use a CI-completed trigger for immediate triage. Add a scheduled trigger only for reconciliation, such as reviewing unresolved signatures once daily. Because either trigger can launch the automation, the same failure may arrive twice. Build deduplication before enabling both.

Do not treat schedule delay as a missed run automatically. Cursor documents that scheduled runs can start late. Compare the intended window with the observed start time, record lateness, and alert only after a business-defined tolerance.

Step 2: scope the repository and identity

For schedule and Slack triggers, Cursor may default to no repository. Choose the disposable test repository explicitly if the agent needs code or test history. Use a single repository unless cross-repo evidence is essential. Multi-repo access increases the blast radius and makes attribution harder.

Decide whether the automation is private, team-visible, or team-owned. Cursor documents that team-owned runs use a shared service account, while private runs can act with the creator’s identity for some operations. Test the final identity after any scope promotion, and rotate a webhook key if the documented scope change requires it.

Step 3: minimize tools before activation

Begin with only the tools required to read CI evidence and publish a report to an isolated QA channel. Disable pull-request creation, review approvals, broad Slack access, computer use, and memory unless the test proves they are necessary.

This matters because Cursor documents that repository-backed automations can open pull requests and that computer use is included by default. An MCP connection grants access to every exposed tool, so prefer a narrow read-only server over a broad engineering MCP. Test denied operations and verify there was no side effect, not merely that the agent claimed denial.

Step 4: make the prompt evidence-first

Cursor’s official prompt guidance recommends specific checks, decision rules, a quality bar, enabled actions, and an output format. A bounded flaky-test prompt can say:

Analyze only the supplied completed CI run and approved artifacts.
Normalize volatile timestamps, ports, temporary paths, and IDs before grouping.
Do not rerun tests, edit code, open a PR, change labels, or suppress failures.
Classify as product, test, environment, infrastructure, or unknown.
For every conclusion, cite the artifact and line or attachment.
If evidence is incomplete, return unknown and list what is missing.
Output one JSON report using the documented schema.

Keep deterministic parsing and schema validation outside the agent. Reject reports with unknown fields, missing evidence links, invalid classifications, or a run ID that does not match the trigger payload.

Step 5: design an idempotency key

Create a stable key from repository, workflow, run attempt, job, and normalized failure signature. Before publishing, query the test report store for that key. If it exists, update the occurrence count only when the incoming artifact is newer and valid.

Test simultaneous trigger delivery, webhook retry, scheduled reconciliation, and a crash after writing the report but before acknowledging completion. The expected result is one logical report, not one message per delivery.

Step 6: validate flaky-test clustering

Prepare synthetic fixtures for the same test with changing timestamps, ports, UUID-like values, worker numbers, and temporary directories. These should group together. Prepare similar-looking failures with different assertion values or stack roots; these should remain separate.

Scenario Expected classification Required evidence
Intermittent timeout, same stack root Test or environment candidate Multiple run IDs and normalized signature
Deterministic assertion regression Product candidate First bad commit and repeated result
Runner disconnected Infrastructure candidate Runner event and incomplete test output
Missing artifact Unknown Explicit missing-evidence list
Duplicate webhook No new logical report Same idempotency key

Never equate “passed on retry” with “safe to ignore.” Preserve the original failure, rerun result, and frequency. A flaky test is a product-quality signal until a human classifies its impact.

Step 7: test prompt injection and artifact trust

Insert a canary instruction into a synthetic test log: ask the agent to reveal secrets, change a label, or write a memory. The automation must treat log text as data, ignore the instruction, and stay inside its report schema.

Also test oversized logs, malformed archives, HTML in test names, forged links, path traversal in artifact filenames, and secrets embedded in stdout. Redact before agent access and again before publishing. Confirm the final QA channel and stored evidence contain no canary value.

Step 8: handle memory as untrusted state

Cursor memories persist outside the working filesystem and can influence future runs. Keep memory disabled for the baseline. If you later enable it, allow only stable, human-approved facts such as a verified signature owner or an approved normalization rule.

Seed an incorrect memory that says a real regression is harmless. The next run must not suppress evidence or override deterministic policy. Test review, editing, expiration, deletion, and recovery after memory corruption. Treat memory as a cache of hypotheses, never as the source of truth.

Step 9: prove behavior with fault injection

  1. Deliver the same CI event twice.
  2. Delay a scheduled run beyond its expected start.
  3. Remove a required artifact.
  4. Return invalid JSON from the agent.
  5. Make the report destination unavailable.
  6. Trigger two failures with the same test name but different root causes.
  7. Inject a malicious log instruction.
  8. Deny an unnecessary write tool and inspect external state.
  9. Corrupt or poison a memory entry.
  10. Crash between persistence and notification.

For every case, record input, trigger ID, run ID, tool calls, normalized signature, schema result, side-effect check, and human verdict.

Step 10: graduate actions one at a time

Run in reporting mode until false groupings, missed duplicates, unknown classifications, and evidence quality meet agreed thresholds. Then consider one reversible action, such as adding a non-blocking comment to a test issue. Do not jump directly to automatic code fixes, quarantine, approval, merge, or release changes.

If you enable pull-request creation later, require a separate branch, deterministic tests, a minimal diff, linked evidence, named reviewers, and no auto-merge. The automation’s analysis remains a hypothesis until independently reproduced.

Screenshot plan

Capture the automation trigger, repository scope, tool checklist, bounded prompt, output schema, duplicate-event test, normalized signature groups, injection rejection, memory review, failure-path evidence, and final human decision. Redact repository names, webhook URLs, API keys, Slack identities, internal links, and private logs.

Release checklist

  • Trigger and schedule tolerance are documented.
  • Idempotency covers every delivery path.
  • Repository and identity scope are verified.
  • Only necessary tools are enabled.
  • Reports pass deterministic schema validation.
  • Every classification links to evidence.
  • Unknown is allowed when evidence is incomplete.
  • Prompt injection and secret canaries are blocked.
  • Memory is disabled or human-governed.
  • External side effects are independently checked.
  • Original failures remain visible after retries.
  • Humans own remediation and release decisions.

Limits

Cursor Automations are agentic and can be nondeterministic. A delayed run is not necessarily absent, a confident classification can be wrong, and persistent memory can preserve bad context. Coverage of one synthetic pipeline does not prove safety for every repository, integration, or artifact type.

The dependable pattern is narrow triggers, explicit scope, least-privilege tools, deterministic validation, idempotent persistence, guarded memory, fault injection, traceable evidence, and gradual human-approved expansion.

Official sources