Partitioned Test Execution with Backend Data Isolation

Partitioned Test Execution with Backend Data Isolation

Problem

A large E2E test suite with a stateful backend becomes unreliable as tests accumulate data. Tests in one feature area create entities (pages, workspaces, tags) that affect tests in other areas through shared state.

Symptoms:

• Move Page dialog lists 50+ pages from other test files, making target selection unreliable
• Sidebar “No pages yet” assertion fails because a prior test file created pages
• Tests pass in isolation (pnpm test workspace-crud) but fail in full suite (pnpm test)
• Flakiness increases over time as more tests are added

Root Cause

All tests share a single backend instance. Each test creates entities that persist for the lifetime of the test run. By the time later tests execute, the database has accumulated data from dozens of prior tests, changing the behavior of UI components that render dynamic lists (page trees, move dialogs, search results).

Solution

Partition tests by feature area, with each partition targeting a dedicated backend instance that has its own isolated database.

Architecture

Playwright Workers (parallel)
  ├── workspace-pages project → Bridge :9990 → .data/workspaces/qa-agent-1/
  ├── editor-content project  → Bridge :9991 → .data/workspaces/qa-agent-2/
  ├── navigation project      → Bridge :9992 → .data/workspaces/qa-agent-3/
  └── advanced project        → Bridge :9993 → .data/workspaces/qa-agent-4/

1. Infrastructure: One backend per partition

start-infrastructure.sh

declare -A BRIDGES=(
  [9990]=".data/workspaces/qa-agent-1"
  [9991]=".data/workspaces/qa-agent-2"
  [9992]=".data/workspaces/qa-agent-3"
  [9993]=".data/workspaces/qa-agent-4"
)

for port in "${!BRIDGES[@]}"; do
  workspace="${BRIDGES[$port]}"
  cargo run -p http-bridge -- --port "$port" --workspace "$workspace" &
done

Each backend gets its own workspace directory with its own SQLite database. No shared state between partitions.

2. Playwright config: Route tests to partitions

playwright.config.ts

projects: [
  {
    name: 'workspace-pages',
    use: { ...devices['Desktop Chrome'], bridgePort: 9990 } as any,
    testMatch: '**/workspace-*.spec.ts',
  },
  {
    name: 'editor-content',
    use: { ...devices['Desktop Chrome'], bridgePort: 9991 } as any,
    testMatch: '**/editor-*.spec.ts',
  },
  // ...
],

3. Naming convention drives routing

The testMatch glob pattern means file naming determines which partition a test lands in:

File prefix	Partition	Bridge port
workspace-*.spec.ts	workspace-pages	9990
editor-*.spec.ts	editor-content	9991
navigation-*.spec.ts	navigation-search	9992
advanced-*.spec.ts	advanced-features	9993
smoke-*.spec.ts	default	9990

New test files automatically route to the correct partition based on their name prefix.

4. Health checks before test execution

for port in 9990 9991 9992 9993; do
  curl -sf "http://localhost:$port/health" > /dev/null \
    || { echo "Bridge on $port not healthy"; exit 1; }
done

Partitioning Principles

Group by data domain: Tests that create similar entities go in the same partition. Workspace CRUD and page hierarchy tests share the workspace-pages partition because they both create pages — but they don’t contaminate editor tests that only need a single page to type into.

Accept intra-partition contamination: Tests within the same partition still share state. Use unique identifiers (e.g., Date.now().toString(36) suffix) in entity names to avoid collisions within a partition.

Separate truly isolated flows: First-launch/onboarding tests need a completely pristine state — give them their own partition with a separate settings directory (see related doc on stateful first-run isolation).

What This Doesn’t Solve

• Intra-partition pollution: Tests in the same partition still accumulate data. For tests that are sensitive to this (e.g., Move Page dialog with many pages), consider using unique name prefixes or dedicated test cleanup.
• Cross-run pollution: Partitions persist between test runs. For a clean slate, delete workspace directories before starting infrastructure.
• Order-dependent tests: Parallel execution means test order within a partition is non-deterministic. Use test.describe.serial() only when truly needed.

Prevention

Best Practices

• Name test files with the correct area prefix so routing is automatic
• Use unique identifiers in test entity names (const uid = Date.now().toString(36))
• Add port conflict checks to the infrastructure startup script
• Document the partition map in the project README

Warning Signs

• A new test file doesn’t match any testMatch pattern — falls to the default partition
• Tests that worked before become flaky after adding many tests to the same partition
• “Element not found” errors on elements that definitely exist — may be looking at the wrong partition’s data

References

• tests/e2e/playwright.config.ts — partition configuration
• tests/e2e/playwright.config.ts webServer config — automated multi-bridge startup (supersedes start-infrastructure.sh)
• tools/qa/stop-infrastructure.sh — emergency cleanup
• tests/e2e/README.md — partition table and troubleshooting