E2E Test Robustness Patterns - Condition-Based Waits and Error Simulation

E2E Test Robustness Patterns

Problem

The E2E test suite had three categories of issues:

1. Timing brittleness: 21 hardcoded waitForTimeout() calls caused flaky tests
2. No error path testing: Couldn’t simulate backend errors without real failures
3. Incomplete mock responses: PageView component expected blocks array that mock didn’t provide

Symptoms:

• Tests failing intermittently with timeout errors
• Race conditions where UI state wasn’t ready when assertions ran
• Skipped tests with comments like “requires complex mock setup”
• No coverage for error handling UI

Investigation

Steps Tried

1. Increased timeout values - Masked the problem, didn’t fix root cause
2. Added retry logic - Made tests slower without improving reliability
3. Condition-based assertions - Worked! Tests wait exactly as long as needed

Key Discovery

The Playwright expect() assertions with toBeVisible() and toBeHidden() already implement intelligent waiting with configurable timeouts. Replacing blind waits with these assertions eliminates race conditions.

Root Cause

Timing issues: waitForTimeout(300) assumes the UI will be ready in 300ms. If it takes 350ms (CPU spike, CI load), the test fails. If it only takes 50ms, we waste 250ms per call.

Mock gaps: The Tauri mock’s get_page response didn’t include the blocks array that PageView.tsx expects at line 31: loadedPage.blocks[0]?.content

Solution

Pattern 1: Replace Timeouts with Condition-Based Waits

Before (brittle):

await skipButton.click();
await page.waitForTimeout(300);
// Assumes tour is gone after 300ms

After (robust):

await skipButton.click();
// Wait for actual condition - tour overlay disappears
await expect(page.locator('text=Welcome to Inklings!')).toBeHidden({ timeout: 2000 });

When waiting for content to appear:

// Before
await input.fill("Create");
await page.waitForTimeout(300);
const results = page.locator(selectors.commandPaletteItem);

// After
await input.fill("Create");
const results = page.locator(selectors.commandPaletteItem);
await expect(results.first()).toBeVisible({ timeout: 2000 });

Pattern 2: Error Simulation in Mock

Add error simulation capability to test error handling paths:

ErrorSimulation interface:

export interface ErrorSimulation {
  /** Command name to simulate error for */
  command: string;
  /** Error message to throw */
  error: string;
  /** If true, only trigger once then clear */
  once?: boolean;
}

export interface TauriMockState {
  // ... existing fields
  errorSimulation?: ErrorSimulation;
}

Check before command processing in mockInvoke:

// Check for error simulation BEFORE processing commands
if (state.errorSimulation && state.errorSimulation.command === cmd) {
  const errorMsg = state.errorSimulation.error;
  console.log(`[Tauri Mock] Simulating error for ${cmd}: ${errorMsg}`);
  if (state.errorSimulation.once) {
    state.errorSimulation = undefined;
  }
  throw new Error(errorMsg);
}

Helper functions:

export async function simulateError(
  page: Page,
  command: string,
  error: string,
  once: boolean = true
): Promise<void> {
  await page.evaluate(
    ({ command, error, once }) => {
      const state = (window as any).__MOCK_STATE__;
      state.errorSimulation = { command, error, once };
    },
    { command, error, once }
  );
}

export async function clearErrorSimulation(page: Page): Promise<void> {
  await page.evaluate(() => {
    const state = (window as any).__MOCK_STATE__;
    state.errorSimulation = undefined;
  });
}

Test fixture usage:

test("should show error when save fails", async ({ page, simulateCommandError }) => {
  await simulateCommandError("update_page", "Network error: connection refused");

  // Trigger save
  await page.keyboard.press("Meta+s");

  // Verify error UI appears
  await expect(page.locator('text=Network error')).toBeVisible();
});

Pattern 3: Align Mock Responses with Component Props

Problem: PageView expects blocks array

// PageView.tsx line 31
setContent(loadedPage.blocks[0]?.content || "");

Solution: Include blocks in mock response

case "get_page": {
  const page = state.pages.find((p) => p.slug === slug);
  if (page) {
    return {
      slug: page.slug,
      title: page.title,
      content: page.content,
      created_at: page.created_at,
      updated_at: page.updated_at,
      // Include blocks array for PageView component
      blocks: [
        {
          id: `block-${page.id || "1"}`,
          content: page.content || "",
          block_type: "markdown",
        },
      ],
    };
  }
  return null;
}

Acceptable Exceptions

Some timeouts are legitimate and should remain:

Animation delays between steps (no testable condition):

while (await nextButton.isVisible({ timeout: 500 }).catch(() => false)) {
  await nextButton.click();
  // Animation between tour steps - no DOM change to wait for
  await page.waitForTimeout(300);
}

Rapid interaction simulation:

// Testing rapid open/close doesn't cause crashes
for (let i = 0; i < 5; i++) {
  await page.keyboard.press("Meta+k");
  await expect(palette).toBeVisible({ timeout: 500 });
  await page.keyboard.press("Escape");
  await expect(palette).toBeHidden({ timeout: 500 });
}

Prevention

Best Practices

1. Never use waitForTimeout() without justification - Always prefer condition-based waits
2. Match mock responses to TypeScript interfaces - Check component prop types when mocks fail
3. Test error paths explicitly - Use error simulation for comprehensive coverage
4. Use explicit timeouts on assertions - { timeout: 2000 } makes failure modes clear

Warning Signs

• Comment saying “wait for X to load” followed by waitForTimeout()
• Tests that pass locally but fail in CI
• Skipped tests citing “complex mock setup”
• Error handling code with no test coverage

Code Review Checklist

• No new waitForTimeout() calls without documented justification
• Mock responses include all fields components expect
• Error paths have test coverage via error simulation
• Assertions have explicit timeout values

Results

Metric	Before	After
Passing tests	112	124
Hardcoded timeouts	21	3 (justified)
Error simulation	None	All commands
Skipped editor tests	6	0

References

• Commit: dd32706 - test: improve E2E tests with error simulation and condition-based waits
• Related: Playwright E2E Selector Issues
• E2E Test Guide: apps/desktop/tests/README.md