Chrome Extension default pinning for QA needs a careful test strategy because the behavior is not a stable promise. Google says Chrome 153 is experimenting with pinning an extension’s action icon to the top-level toolbar by default, and that the behavior is live for only some Chrome Canary users.

That creates a classic QA trap: a team sees the icon pinned on one machine, assumes every new user will see it, and builds onboarding, analytics, or support guidance around an experiment. This tutorial shows how to test the pinned and unpinned journeys, correlate visible state with the Action API, protect user choice, and keep the extension useful even when the experiment is absent.

What Google officially documents

Chrome’s current What’s New page describes default pinning as a Chrome 153 experiment available to some Canary users. It does not announce general stable rollout.

The official chrome.action reference says action icons appear in the extensions menu after installation and users can pin them to the top-level toolbar. chrome.action.getUserSettings() returns user-specified action settings, including isOnToolbar. The onUserSettingsChanged event reports related changes from Chrome 130.

The same reference says toolbar presence and enabled state are different: disabling an action affects whether its popup or click handler runs, not whether its icon is present. It also documents icon sizing, tooltip accessibility, badges, popups, click events, and per-tab state. Chrome’s E2E guide recommends basing integration tests on user-visible behavior and using internal state only as supporting evidence.

Build a disposable action extension

Create a private Manifest V3 sample with one narrow action: opening a popup that displays a synthetic environment label and a button that changes a visible marker on a local test page. Use no production credentials or real browsing data.

Provide multiple PNG icon sizes and a meaningful title:

{
  "manifest_version": 3,
  "name": "Toolbar Discovery QA Lab",
  "version": "1.0.0",
  "action": {
    "default_title": "Open toolbar discovery test",
    "default_popup": "popup.html",
    "default_icon": {
      "16": "icons/action-16.png",
      "24": "icons/action-24.png",
      "32": "icons/action-32.png"
    }
  }
}

Chrome documents a 16-DIP action icon and recommends multiple sizes because scaling the nearest image can reduce quality on less-common display factors. The action title becomes the tooltip and contributes accessible text when a screen reader focuses the button.

Step 1: freeze the test identity

Before installation, record:

  • Chrome channel and full build;
  • operating system and display scale;
  • new or reused profile identity;
  • extension ID, package version, and package hash;
  • manifest hash and action configuration;
  • icon hashes for every size;
  • expected popup, click, badge, and per-tab behavior;
  • whether the default-pinning experiment was observed; and
  • the user’s initial pin state.

Do not reuse a profile that already installed the extension for the fresh-install case. Prior user choices can make a new package look like a new default.

Step 2: capture Action API evidence

Add a small diagnostics page or development-only log that records the current user setting and changes:

async function recordToolbarState(reason) {
  const settings = await chrome.action.getUserSettings();
  console.info(JSON.stringify({
    reason,
    isOnToolbar: settings.isOnToolbar,
    capturedAt: new Date().toISOString()
  }));
}

chrome.runtime.onInstalled.addListener(() => {
  recordToolbarState('installed');
});

chrome.action.onUserSettingsChanged.addListener((change) => {
  console.info(JSON.stringify({
    reason: 'user-setting-change',
    isOnToolbar: change.isOnToolbar,
    capturedAt: new Date().toISOString()
  }));
});

Keep this diagnostic local and free of personal data. It observes the setting; it does not change it. The user remains responsible for pinning or unpinning through Chrome’s UI.

Step 3: run the fresh-profile matrix

Use at least four disposable profiles:

Profile Expected starting path Evidence
Canary, experiment observed Record actual default state Toolbar screenshot + isOnToolbar
Canary, experiment absent Normal extensions-menu discovery Menu screenshot + isOnToolbar
Stable control Do not expect experimental behavior Full build + visible state
Reinstall control Treat prior choice as stateful Profile history + API state

Do not force a Canary flag or copy a profile simply to manufacture the expected result unless Google publishes a supported test mechanism. The goal is to identify the cohort and validate both product journeys.

Step 4: test user pin and unpin

Ask a tester to pin the extension from the extensions menu. Verify:

  1. the icon appears in the top-level toolbar;
  2. isOnToolbar becomes true;
  3. one settings-change event is recorded;
  4. the tooltip and accessible name are correct; and
  5. the popup or action behavior still works.

Then unpin it. Verify the icon leaves the top-level toolbar, isOnToolbar becomes false, and the extension remains discoverable in the extensions menu. Unpinning is a user preference, not an error to repair.

Step 5: separate presence, enablement, and success

Create three states:

  • Pinned and enabled: the icon is visible and the intended action runs.
  • Pinned and disabled: the icon remains visible, but the popup or click behavior should not run for the disabled scope.
  • Unpinned and enabled: the icon is accessible from the extensions menu and the feature still works.

This matrix catches analytics that label every pinned icon as an active user or every unpinned icon as churn. Toolbar visibility is not the same as feature availability, a click, a successful task, or user satisfaction.

Step 6: test popup and click-handler contracts

Chrome documents that action.onClicked does not fire when a popup is configured for the current tab. Add tests that prove the correct branch:

  • with default_popup, clicking the action opens the popup and does not also run an onClicked path;
  • without a popup, the click handler runs exactly once;
  • per-tab popup changes apply only to the intended tab;
  • a disabled action does not perform the protected operation; and
  • opening the popup from an automated extension page produces the same user-visible outcome as the supported toolbar journey.

Chrome’s E2E guide notes that a popup can be opened with action.openPopup() where supported or tested through its extension URL when direct toolbar automation is impractical. Keep at least one manual visible check for actual toolbar discovery.

Step 7: validate icon quality and accessibility

Test 100%, 120%, 150%, and 200% display scaling when available. Inspect light and dark themes, high contrast, normal and small windows, and crowded toolbars. The icon should remain recognizable without relying only on a color that disappears in one theme.

Focus the toolbar button with the keyboard and a screen reader. Confirm the accessible name matches the action title and describes the result, not a vague brand word. Check that badge text is short, legible, and not the only way an important state is communicated.

Step 8: test restarts, updates, and profile boundaries

Restart Chrome after explicit pin and unpin choices and record the resulting state. Update the extension from version 1.0.0 to a synthetic 1.0.1 package without changing its identity, then check the toolbar state, popup, icon, title, and settings event log.

Repeat with the Manifest V3 service worker terminated and restarted. The toolbar state belongs to Chrome’s user setting; the extension should query it when needed instead of relying on a stale in-memory variable. Verify multiple windows show a consistent top-level toolbar state while tab-specific action behavior remains scoped.

Step 9: test onboarding without assuming the pin

Run the complete first-use journey in both states. An unpinned user should still be able to find the extension through the extensions menu and understand how to invoke it. A pinned user should not receive instructions claiming they must pin before continuing.

Avoid nagging users to reverse an explicit unpin. If your product explains pinning, make it optional, accurate, dismissible, and tied to a real benefit. Do not simulate or automate clicks in Chrome’s own toolbar UI to override user choice.

Step 10: design reliable measurement

Keep these signals separate in test reports and analytics:

  • experiment availability;
  • isOnToolbar state;
  • explicit settings-change event;
  • action invocation;
  • popup loaded;
  • core task succeeded; and
  • user completed onboarding.

Use synthetic local evidence for the QA lab. If a production product collects any of these signals, privacy, consent, minimization, retention, and disclosure require separate review.

QA evidence matrix

Risk Test Independent check
Assumed rollout Canary and stable clean profiles Full build and visible state
User choice overwritten Pin, unpin, restart isOnToolbar and menu state
Presence mistaken for success Pinned-disabled and unpinned-enabled Visible outcome and event log
Wrong click path Popup versus onClicked One expected handler only
Unreadable action Theme, scale, keyboard, screen reader Icon capture and accessible name
Stale state Worker restart and package update Fresh getUserSettings result

Screenshot plan

Capture the official experimental announcement, full browser build, clean-profile install, pinned and unpinned states, extensions-menu fallback, isOnToolbar evidence, settings-change event, enabled versus disabled behavior, popup versus click-handler path, icon scale and theme checks, keyboard focus and accessible name, restart and update evidence, and the final human rollout gate. Redact profile names, machine paths, account details, extension store credentials, and internal URLs.

Final checklist

  • Treat default pinning as a limited Canary experiment, not stable behavior.
  • Record channel, full build, profile, extension ID, package, and hashes.
  • Test both experiment-present and experiment-absent journeys.
  • Correlate visible toolbar state with isOnToolbar.
  • Respect explicit user pin and unpin choices.
  • Separate toolbar presence, enablement, invocation, and success.
  • Test popup, click-handler, per-tab, disabled, restart, and update paths.
  • Validate icon scaling, themes, badges, keyboard access, and screen-reader text.
  • Keep extensions-menu discovery and onboarding useful when unpinned.
  • Require human review before stable-channel rollout or analytics conclusions.

Conclusion

Chrome’s default-pinning experiment may improve discovery for some users, but QA should not turn an experiment into an assumption. The durable strategy is to observe the actual user setting, test both toolbar states, protect user choice, separate visibility from success, keep the unpinned journey functional, and validate icons plus accessible names across real browser conditions. That gives teams useful evidence today without claiming a stable rollout Google has not announced.