Stateful First-Run Test Isolation via Dedicated Bridge Partition

Stateful First-Run Test Isolation via Dedicated Bridge Partition

Problem

Tests for onboarding tours, setup wizards, or any first-run experience fail in the full test suite because the “first run” state gets consumed by test fixture setup or prior tests, and there’s no way to reset it within a test run.

Symptoms:

• Tour/wizard tests pass when run alone but fail in the full suite
• is_first_launch returns false even on a supposedly fresh instance
• The overlay/modal that should appear on first launch is never visible
• All serial tour tests fail after the first one fails (cascade failure)

Investigation

Steps Tried

1. Separate Playwright project — routed tour tests to a dedicated bridge port, but the appPage fixture (used by settings tests in the same file) still called complete_first_launch on that port before tour tests ran
2. Conditional fixture — tried to skip complete_first_launch for tour tests, but the fixture is project-wide and can’t branch per test
3. File split + dedicated partition — moved tour tests to their own file with a freshAppPage fixture — this worked

Root Cause

Two issues compound:

1. Shared fixture poisons state: The standard appPage fixture calls complete_first_launch to dismiss the onboarding overlay. When tour tests share a Playwright project with non-tour tests, the fixture runs before tour tests get a chance to see the overlay.

2. One-way state transition: complete_first_launch is irreversible within a bridge lifecycle. Once called, is_first_launch returns false for all subsequent requests on that port. The bridge stores this in a settings file, so even restarting the bridge doesn’t reset it unless the settings file is deleted.

Solution

1. Dedicated bridge partition with separate settings directory

Create a 5th bridge instance on its own port with an isolated settings directory:

# In start-infrastructure.sh
cargo run -p http-bridge -- \
  --port 9994 \
  --workspace .data-fresh/workspaces/qa-agent-5 &

The key is .data-fresh/ — a completely separate storage directory from the main .data/. The bridge derives its settings path from the workspace path’s grandparent, so a different workspace root means a different settings.json.

2. Separate test file for tour tests

Move first-run tests to their own file so they don’t share a Playwright project with tests that use appPage:

tests/navigation-first-launch.spec.ts       # Settings tests (use appPage)
tests/navigation-first-launch-tour.spec.ts  # Tour tests (use freshAppPage)

3. freshAppPage fixture (no complete_first_launch)

freshAppPage: async ({ page }, use) => {
  const freshPort = 9994;
  await page.addInitScript(`window.__BRIDGE_BASE_URL = 'http://localhost:${freshPort}';`);
  if (bridgeShim) { await page.addInitScript(bridgeShim); }
  await page.goto('/');

  // Wait for EITHER the PersonaSelector OR the sidebar.
  // Supports both pre- and post-completion states within a serial sequence.
  await Promise.race([
    page.waitForSelector('text=Welcome to Inklings!', { timeout: 15_000 }),
    page.waitForSelector('aside', { timeout: 15_000 }),
  ]);
  await use(page);
},

The Promise.race() pattern is critical — later tests in a serial sequence will see the sidebar instead of the PersonaSelector (because an earlier test completed the tour), and the fixture needs to handle both states.

4. Playwright config routing

// Route tour tests to the fresh-launch partition
{
  name: 'first-launch',
  use: { ...devices['Desktop Chrome'], bridgePort: 9994 } as any,
  testMatch: '**/navigation-first-launch-tour.spec.ts',
},
// Exclude tour tests from the navigation partition
{
  name: 'navigation-search',
  testMatch: '**/navigation-*.spec.ts',
  testIgnore: '**/navigation-first-launch-tour.spec.ts',
},

5. Serial test ordering for stateful sequences

test.describe.serial('Tour (fresh-launch bridge)', () => {
  test('13. Tour starts automatically', async ({ freshAppPage }) => { ... });
  test('14. Persona selection enables Continue', async ({ freshAppPage }) => { ... });
  // ...
  test('16. Completing all steps finishes the tour', async ({ freshAppPage }) => { ... });
  test('20. Tour does not appear after completion', async ({ freshAppPage }) => { ... });
});

Tests run in order. Test #16 calls complete_first_launch via the app’s “Got it!” button. Test #20 verifies the overlay is gone — the Promise.race() in the fixture handles this by waiting for the sidebar instead.

6. Reset between full-suite runs

The fresh bridge remembers complete_first_launch was called. Reset before re-running:

rm -f .data-fresh/settings.json
# Restart the fresh bridge
lsof -ti :9994 | xargs kill
cargo run -p http-bridge -- --port 9994 --workspace .data-fresh/workspaces/qa-agent-5 &

Prevention

Best Practices

• Any test requiring “first time” state needs its own bridge partition with isolated settings
• Use test.describe.serial() for sequences where each step builds on prior state
• Design fixtures with Promise.race() for branching conditions (pre/post state)
• Document the reset procedure in the project README

Warning Signs

• Tests that pass in isolation but fail in the suite — check if a fixture consumes one-time state
• “Element not visible” for overlays that should appear on fresh instances
• Cascade failures in serial test blocks (first test fails → all skip)

References

• tests/e2e/seed.spec.ts — freshAppPage fixture implementation
• tests/e2e/tests/navigation-first-launch-tour.spec.ts — serial tour tests
• tests/e2e/playwright.config.ts — partition routing with testMatch/testIgnore
• tests/e2e/playwright.config.ts webServer config — automated bridge instance setup (supersedes start-infrastructure.sh)