Gemini CLI OpenTelemetry for QA turns an agent session into logs, metrics, and traces that testers can inspect. That makes it easier to answer practical questions: Which tool ran? Was it accepted, rejected, modified, or auto-accepted? How long did it take? Did a fallback occur? Can the event be tied to the correct prompt and session?

Telemetry is useful evidence, but it is not a security boundary or an infallible audit ledger. An exporter can fail, an event can be suppressed, a collector can drop data, and sensitive values can appear outside the prompt field. This tutorial builds a disposable local lab that tests both the value and the gaps.

Privacy first: Gemini CLI telemetry is disabled by default. When telemetry is enabled, the documented default for telemetry.logPrompts is true. Explicitly set it to false for this lab and verify the effective configuration before using any private repository.

What Gemini CLI records

Google’s official telemetry reference documents OpenTelemetry logs, metrics, and traces. Common attributes include session.id, installation.id, active_approval_mode, and an authenticated user.email. The gemini_cli.tool_call event can include:

  • function name and arguments;
  • duration, success, error, and error type;
  • an accept, reject, auto-accept, or modify decision;
  • prompt ID and native-versus-MCP tool type;
  • MCP server, extension identity, and edit metadata where applicable.

The reference also lists configuration, user-prompt, approval, file-operation, API, fallback, hook, output-masking, agent-finish, tool-count, and latency signals. Detailed inference attributes may contain input messages, output messages, and system instructions. Turning off prompt logging reduces one exposure path; it does not prove that every other field is content-free.

1. Create a disposable evidence lab

Use a private repository with no production source, credentials, customer data, remote writes, or live MCP servers. Add:

  • fixtures/failing-test.txt with a synthetic assertion failure;
  • fixtures/forbidden.txt containing a harmless unique canary such as QA_CANARY_DO_NOT_EXPORT_4821;
  • tmp/agent-output/ for permitted agent artifacts;
  • evidence/ for redacted run summaries;
  • a deterministic test command that returns a known result.

Exclude telemetry files, temporary output, and local settings from Git. The canary must never be a real secret. Its only purpose is to reveal an unexpected read or export.

Freeze these run facts before testing: Gemini CLI version, operating system, authentication type, selected model or route, approval mode, sandbox state, enabled native and MCP tools, extensions, effective settings hash, telemetry target and protocol, output-file or collector identity, fixture hash, timezone, and retention policy.

2. Configure local file telemetry

The official guide recommends file-based output for local development. Start with a local file so the QA team can inspect raw records without depending on an external backend.

{
  "telemetry": {
    "enabled": true,
    "traces": true,
    "target": "local",
    "outfile": ".gemini/telemetry-qa.log",
    "logPrompts": false
  },
  "tools": {
    "sandbox": "docker"
  }
}

Use a lab-specific home and clean profile where practical. Gemini CLI settings can be overridden by environment variables, so saving this file is not enough. Capture the startup gemini_cli.config event and assert that telemetry is enabled, prompt logging is disabled, the expected sandbox and approval mode are active, and the permitted tool inventory is correct.

Also record whether the output file takes precedence over an endpoint. The documentation says outfile overrides otlpEndpoint. Test that deliberately instead of assuming both destinations receive identical data.

3. Define an independent truth table

Telemetry must be compared with evidence outside Gemini CLI. For each case, capture Git status, file hashes, a directory inventory, relevant process state, deterministic test output, exit status, and collector counters. That independent view is the authority for side effects.

Case Expected telemetry Independent evidence
Read approved fixture Accepted successful native tool call Accessed file is approved; no file hash changes
Write permitted output Accepted tool call plus file update Only the expected path and hash change
Reject forbidden read Reject decision; no successful operation Canary absent from all artifacts
Modify tool request Modify decision tied to the same prompt Observed arguments match the approved scope
Tool failure Failed call with error classification No partial output promoted
Collector unavailable Visible export degradation or missing records Side-effect checks still complete

A missing event never proves a tool did not run. It means the audit is incomplete until independent evidence resolves the case.

4. Run one controlled session

Use a narrow prompt whose hash is stored separately:

Read only the approved synthetic failure fixture. Explain the observed mismatch and write one JSON summary under tmp/agent-output/. Do not read forbidden.txt, change tests, use the network, or access an MCP server. Ask before any other action.

Before submitting it, take the baseline snapshot. After the session, export the telemetry file and create a redacted case manifest with:

  • run ID, session ID, and prompt ID;
  • prompt-fixture hash rather than prompt text;
  • ordered tool-call IDs, names, types, decisions, success values, and durations;
  • related file-operation, API, fallback, hook, masking, and finish events;
  • first and last timestamps plus clock source;
  • independent file, Git, process, and deterministic-test results.

Do not put user email, installation ID, raw function arguments, response text, or system instructions into a broadly accessible report. Hash, redact, or omit fields according to a reviewed data policy.

5. Test correlation and event order

Correlate every tool call with the expected session and prompt. Then inject parser-level faults into a copied telemetry fixture: duplicate one event, remove one event, reverse two timestamps, truncate the final record, add an unknown event type, corrupt a numeric duration, and reuse a prompt ID across two synthetic sessions.

The consumer should flag:

  • a tool result without a matching request context;
  • an agent finish before its final tool event;
  • duplicate terminal events;
  • negative or impossible durations;
  • mixed session identities in one case;
  • an accepted call with a recorded failure;
  • missing configuration evidence.

Quarantine malformed evidence. Do not silently reorder, deduplicate, or fill missing data, because those repairs can hide the observability defect the test is meant to expose.

6. Verify prompt and secret privacy

Run three canary checks. First, place a unique harmless token in the prompt, then confirm it does not appear when logPrompts is false. Second, place a different token in the excluded file and prove the agent never reads or exports it. Third, pass a harmless token as a synthetic tool argument and observe whether function_args records it.

The third result is important: disabling prompt logging is not a universal redaction switch. Search the raw telemetry for all canaries, email addresses, path names, URLs, query strings, environment-variable shapes, authorization-header shapes, raw tool responses, input and output message fields, error strings, and system instructions. Fail the privacy test if an unapproved value reaches the output file or collector.

Validate filesystem permissions on the local telemetry file, collector authentication, transport security for remote export, retention duration, deletion behavior, backup copies, and access logs. A secure setting with an insecure evidence directory still leaks data.

7. Exercise approvals, hooks, and masking

Create separate cases for accept, reject, auto-accept, and modify decisions. Verify the decision recorded in telemetry matches the prompt shown to the human and the action that actually occurred. A model’s narrative claim is not evidence of approval.

Gemini CLI hooks are another boundary. The official hook reference says hooks communicate with JSON, use exit code 2 to block in relevant cases, and can set suppressOutput to hide internal hook metadata from logs or telemetry. Test a harmless hook with a unique ID, then compare hook execution evidence with gemini_cli.hook_call. If metadata is intentionally suppressed, document that audit gap.

Also test gemini_cli.tool_output_masking. Confirm that masking counts are plausible and that masked content is absent from downstream evidence. Masking telemetry indicates that a transformation occurred; it does not independently prove every sensitive token was removed.

8. Test exporter and collector failures

Move from the output file to a local test collector only after the baseline passes. Exercise both documented OTLP protocols separately:

  • correct gRPC endpoint;
  • correct HTTP endpoint;
  • HTTP client pointed at a gRPC listener and the reverse;
  • refused connection, slow collector, and interrupted connection;
  • collector restart mid-session;
  • expired certificate in an isolated TLS lab;
  • full disk or denied output path;
  • rotated output file during active export.

Record whether the CLI continues, warns, retries, or loses events in the installed version. Do not invent a guaranteed delivery promise. Your release gate should surface observability degradation without interpreting the agent’s work as trustworthy merely because it completed.

9. Add concurrency and volume tests

Run two isolated sessions with different lab homes, output files, fixture hashes, and custom surface labels. Confirm that their session IDs, prompt IDs, tool calls, and artifacts never cross. Then test a burst of synthetic read-only calls and a deliberately large tool output that triggers truncation evidence.

Track tool-call count, failure rate, reject rate, modify rate, latency percentiles, missing-correlation count, duplicate-event count, exporter-error count, canary-leak count, and telemetry-to-independent-evidence mismatch count. Avoid high-cardinality production labels such as raw prompts, file paths, or user identities.

10. Define a QA release gate

Fail the lab when prompt logging is unexpectedly enabled, a canary leaks, configuration identity is missing, a successful side effect has no correlatable event without a documented suppression, a denied action changes state, sessions mix, malformed evidence is accepted, or retention deletion fails. Mark the result inconclusive when the collector outage prevents a complete comparison.

Deterministic tests, Git and filesystem evidence, process observations, and remote-system audits remain authoritative. Telemetry helps explain what the agent and platform reported; it does not replace those checks. A human should decide whether an incident, defect, merge, or release can proceed.

Screenshot-friendly QA checklist

  • Effective configuration with prompt logging explicitly disabled
  • Frozen CLI, sandbox, approval, tool, extension, and collector identities
  • Clean Git and filesystem baseline
  • Accepted, rejected, modified, failed, and masked tool cases
  • Session and prompt correlation across ordered events
  • Canary scan across every telemetry field and exported artifact
  • Malformed, missing, duplicate, and out-of-order event handling
  • HTTP, gRPC, collector-down, full-disk, and rotation failures
  • Concurrent-session isolation and volume metrics
  • Independent side-effect evidence plus human release gate

Official references

Final takeaway

Gemini CLI OpenTelemetry for QA is most valuable when it is tested like any other evidence pipeline. Minimize collected content, freeze the effective configuration, correlate every event, break the exporter on purpose, and compare the trace with independent state. You gain useful visibility into agent tool behavior without mistaking a convenient dashboard for complete proof.