/iblai-vibe-ops-test

Mirrored from iblai/vibe · skills/ship/iblai-vibe-ops-test/SKILL.md. This page is generated — edit it in the repository, not here.

Two jobs. Before you say "done" — every change, a feature or a whole app: pnpm build, pnpm test, then the touch test below, which drives the running app in a real browser. When the user asks for tests or coverage: Steps 0–5 set up Vitest and Playwright journeys and lift coverage above 95%.

Never ask for a password. The touch test lets the user sign in by hand in the browser window; pnpm test:e2e signs in as two test accounts that pnpm e2e:accounts creates. Use pnpm; fall back to npm if it is not installed.

Before you say "done"

1. pnpm build — zero errors.
2. pnpm test — green.
3. The touch test below, on every route the change touched.
4. pnpm test:e2e (Chrome desktop and a phone, against the server you already run on :3000 — it stops at once when nothing answers; pnpm test:e2e:all adds Firefox and WebKit), then pnpm e2e:coverage. The first creates the two test accounts itself when e2e/.env.development lacks them (pnpm e2e:accounts does the same ahead of time): an org admin and a member in the organization iblai.env names (PLATFORM, never main), made with the Platform API Token in .env.local after the platform confirms the token belongs to that organization, saved in that gitignored file. Never read or print the file: report the two usernames it prints. If the creation or a sign-in fails, report its message and stop — there is no other way in. One exception: when it stops on PLATFORM (a placeholder), run /iblai-vibe-connect without --org — the user picks the organization on the connect page — then run it again. The second checks e2e/coverage.json against the specs and the app and approximates e2e coverage from the journeys; under 95% the work is not done (Step 4 below). No e2e:accounts script (not vibe-starter)? Run pnpm test:e2e only when e2e/.env.development is filled in.

A change with no screen (a script, a server route nothing renders yet) reports "no screen to touch" in place of step 3. Never say "done", "ready" or "complete" without the touch-test report.

Touch test in the browser

Tests prove the code; this proves the screen. Drive the running app in a real browser, use what you built, read the console. It can fail — when it does, fix the code and run it again.

1. Pick the tool from your tool list (Claude Code may list MCP tools as deferred; load them with ToolSearch):

   • mcp__claude-in-chrome__* — Claude in Chrome, the user's own signed-in Chrome (they started claude --chrome, or turned it on under /chrome).
   • else mcp__chrome-devtools__* — the Chrome DevTools MCP: vibe-starter and the scaffold ship it in .mcp.json; its Chrome keeps its own profile, so the sign-in survives between runs.
   • else mcp__playwright__* — the Playwright MCP older scaffolds ship.
   • else stop and say: "I can't touch test without a browser tool. Restart with claude --chrome, or turn on the chrome-devtools server (/mcp; I can add it to .mcp.json)." Codex and Cursor entries: references/browser-touch-test.md. No screenshot script instead, no password, and never report the touch test as run.

   The tool names below are the DevTools MCP's; the others have a matching tool — /mcp lists Claude in Chrome's. Never guess a name.

2. One dev server. curl -s -o /dev/null -w '%{http_code}' http://localhost:3000 prints anything but 000 → a server is up; if it is this app, reuse it. Otherwise start pnpm dev in the background and wait for Ready. Never a second one: Next moves to port 3001, which the organization's allowed redirect origins may not list.

3. Open the route. list_pages, then navigate_page to http://localhost:3000/<route>. Landed on the sign-in page? Say "Sign in in the browser window I opened, then tell me", and wait. Never type the user's credentials. A sign-in that never comes back means localhost:3000 is not an allowed redirect origin — say so. Give the first load a few seconds (one re-auth round trip, one service-worker reload) before you navigate on.

4. Use what you built. take_snapshot — the page as an accessibility tree with uids — then every control the change added or touched: tabs, buttons, menus, toggles, forms, dialogs. click, fill, fill_form, press_key by uid, and take_snapshot again after each. Expect content (no stuck Loading…, no empty SDK panel), each control doing what its label says, dialogs that open and close, a saved value that survives a reload. Use harmless values; ask before deleting data or sending an invite, an email or a payment. A combobox's rows (cmdk) do not show in the snapshot — read them from a screenshot or with evaluate_script.

5. Read the console and the network. list_console_messages (errors, warnings) and list_network_requests (4xx, 5xx; get_network_request for a body). An error the change caused is a failure. Quote URLs and status codes only: headers and localStorage carry tokens.

6. Phone width. emulate with viewport 412x915x2,mobile,touch, reload, then evaluate_script with () => document.documentElement.scrollWidth <= document.documentElement.clientWidth — it must return true. take_snapshot to check the controls are still reachable, then clear the viewport (emulate with ""). resize_page stops near 850 px, so it is not a phone. No viewport tool? Report "phone: not checked".

7. Evidence, then the report. take_screenshot of the end state on desktop and phone — evidence for the user, not the test; never into the repository. Then:

   Touch test: /<route> (Chrome DevTools MCP)
   Touched: <control> → <what happened>; …
   Console: clean | <errors>. Network: clean | <url → status>
   Phone 412 px: no sideways scroll | <what overflowed>
   Not touched: <control>, <why>
   E2E: <n> passed, <n> skipped, as <admin> + <member>; coverage <pct>% of <n> app lines | not run: <why>

Something broke: fix it, run pnpm build and pnpm test again, touch test again. After a deploy, run the same on the live URL — /iblai-vibe-ops-deploy says when.

Step 0: Detect what's already there

# Is vitest installed?
grep -E '"vitest"' package.json && echo "vitest: present" || echo "vitest: missing"

# Is playwright installed?
grep -E '"@playwright/test"' package.json && echo "playwright: present" || echo "playwright: missing"

# Does the app already have a test config?
ls vitest.config.* e2e/playwright.config.* 2>/dev/null

Branch from here:

• Both present → skip to Step 3 (write tests, not infra).
• Missing infra → Step 1 + Step 2 first.

Step 1: Vitest infrastructure

Install

pnpm add -D vitest @vitejs/plugin-react vite-tsconfig-paths jsdom \
  @testing-library/react @testing-library/jest-dom @testing-library/user-event \
  @vitest/coverage-v8

@vitest/coverage-v8 is required for the --coverage flag (and for the 95% thresholds we set below).

vitest.config.ts

Drop this in at the project root. The 95% thresholds make CI fail when coverage regresses; the exclude list keeps Next.js-generated files, config files, and test fixtures out of the denominator.

import { configDefaults, defineConfig } from 'vitest/config'
import react from '@vitejs/plugin-react'
import tsconfigPaths from 'vite-tsconfig-paths'

export default defineConfig({
  plugins: [tsconfigPaths(), react()],
  resolve: {
    alias: [
      // Vitest + Node ESM resolution needs the explicit extension for SDK imports.
      { find: 'next/navigation', replacement: 'next/navigation.js' },
      // The SDK can resolve to a pnpm package-local absolute path that bypasses
      // bare-import aliasing. Normalize to the project-level Next entry.
      {
        find: /\/node_modules\/\.pnpm\/@iblai\+web-containers@[^/]+\/node_modules\/next\/navigation$/,
        replacement: new URL(
          './node_modules/next/navigation.js',
          import.meta.url,
        ).pathname,
      },
    ],
  },
  test: {
    globals: true,
    setupFiles: ['./__tests__/vitest.setup.ts'],
    environment: 'jsdom',
    exclude: [...configDefaults.exclude, 'e2e/**'],
    server: { deps: { inline: true } },
    coverage: {
      provider: 'v8',
      include: [
        'app/**/*.{ts,tsx}',
        'components/**/*.{ts,tsx}',
        'hooks/**/*.{ts,tsx}',
        'lib/**/*.{ts,tsx}',
        'providers/**/*.{ts,tsx}',
        'contexts/**/*.{ts,tsx}',
        'actions/**/*.{ts,tsx}',
      ],
      exclude: [
        'node_modules/**', '.next/**', 'dist/**', 'build/**',
        '**/*.d.ts', '**/*.config.*', '**/__tests__/**', '**/__mocks__/**',
        // Next.js plumbing (covered by e2e, not unit)
        'app/**/loading.tsx', 'app/**/error.tsx', 'app/**/not-found.tsx',
        'instrumentation.ts', 'middleware.ts',
        // Tauri shells, generated UI primitives
        'src-tauri/**', 'components/ui/**',
      ],
      thresholds: {
        lines: 95,
        functions: 95,
        branches: 95,
        statements: 95,
      },
    },
  },
})

__tests__/vitest.setup.ts

jsdom is missing several APIs that Radix UI, ibl.ai SDK, and next/image depend on. Polyfill them once globally so individual tests don't have to:

import '@testing-library/jest-dom/vitest'
import { vi } from 'vitest'

// Blob URL helpers (file upload tests)
if (typeof URL.createObjectURL === 'undefined') {
  URL.createObjectURL = vi.fn(() => 'blob:mock-url')
}
if (typeof URL.revokeObjectURL === 'undefined') {
  URL.revokeObjectURL = vi.fn()
}

// localStorage mock — class-backed so tests can spy on prototype methods.
class LocalStorageMock implements Storage {
  private store: Record<string, string> = {}
  get length() { return Object.keys(this.store).length }
  clear() { this.store = {} }
  getItem(k: string) { return this.store[k] ?? null }
  key(i: number) { return Object.keys(this.store)[i] ?? null }
  removeItem(k: string) { delete this.store[k] }
  setItem(k: string, v: string) { this.store[k] = v }
}
Object.defineProperty(window, 'localStorage', { value: new LocalStorageMock(), writable: true })

// Radix UI in jsdom requires pointer-capture polyfills.
if (typeof Element.prototype.hasPointerCapture === 'undefined') {
  Element.prototype.hasPointerCapture = () => false
  Element.prototype.setPointerCapture = () => {}
  Element.prototype.releasePointerCapture = () => {}
}

// matchMedia + ResizeObserver stubs (shadcn / Radix primitives).
if (typeof window.matchMedia === 'undefined') {
  window.matchMedia = vi.fn().mockImplementation((q: string) => ({
    matches: false, media: q, onchange: null,
    addListener: vi.fn(), removeListener: vi.fn(),
    addEventListener: vi.fn(), removeEventListener: vi.fn(), dispatchEvent: vi.fn(),
  }))
}
if (typeof window.ResizeObserver === 'undefined') {
  // @ts-expect-error
  window.ResizeObserver = class { observe(){} unobserve(){} disconnect(){} }
}

package.json scripts

{
  "scripts": {
    "test": "vitest run",
    "test:watch": "vitest",
    "test:coverage": "vitest run --coverage"
  }
}

Step 2: Playwright infrastructure

vibe-starter ships it: e2e/playwright.config.ts, the real sign-in in e2e/auth.setup.ts (an admin and a member, from the e2e/.env.development that pnpm e2e:accounts writes — pnpm test:e2e runs it itself when the accounts are missing), the auth, member, admin, pages, api and mobile journeys, e2e/COVERAGE.md + coverage.json, and e2e/coverage.mjs behind pnpm e2e:coverage; run pnpm test:e2e against a running pnpm dev. playwright/.auth/ holds live tokens: keep /playwright/ gitignored and never git add -A after a run.

Not from vibe-starter: pnpm add -D @playwright/test and pnpm exec playwright install chromium, then take the config, the seedAuth localStorage stub for UI-only journeys and the test:e2e script from references/playwright-setup.md; for journeys that need real data, references/real-sso-auth-setup.md. Copy e2e/coverage.mjs from vibe-starter as it is (no dependencies) and add "e2e:coverage": "node e2e/coverage.mjs" to package.json.

Step 3: Write unit tests

Coverage target: ≥95% lines/functions/branches/statements on the include paths from vitest.config.ts. To hit it, you typically need tests in five buckets. Examples below — see references/test-patterns.md for the expanded recipes.

3a. Lib utilities (highest leverage, fastest)

// lib/iblai/__tests__/tenant.test.ts
import { afterEach, describe, expect, it } from 'vitest'
import { resolveAppTenant } from '../tenant'

describe('resolveAppTenant', () => {
  afterEach(() => localStorage.clear())

  it('returns "" when localStorage.tenant is unset', () => {
    expect(resolveAppTenant()).toBe('')
  })

  it('returns the raw key string', () => {
    localStorage.setItem('tenant', 'gwu')
    expect(resolveAppTenant()).toBe('gwu')
  })
})

3b. Hooks (mock next/navigation + SDK queries)

// lib/iblai/__tests__/use-url-context.test.tsx
import { afterEach, describe, expect, it, vi } from 'vitest'
import { renderHook, waitFor } from '@testing-library/react'

const params = vi.hoisted(() => ({ current: {} as Record<string, string | undefined> }))
vi.mock('next/navigation', () => ({ useParams: () => params.current }))

import { useUrlContext } from '../use-url-context'

describe('useUrlContext', () => {
  afterEach(() => {
    localStorage.clear()
    params.current = {}
  })

  it('prefers URL params over localStorage', async () => {
    localStorage.setItem('tenant', 'fallback')
    params.current = { tenantId: 'from-url', mentorId: 'm1' }
    const { result } = renderHook(() => useUrlContext())
    await waitFor(() => expect(result.current.ready).toBe(true))
    expect(result.current.tenantKey).toBe('from-url')
    expect(result.current.mentorId).toBe('m1')
  })
})

3c. Components (mock SDK + provider, then assert rendered DOM)

// components/__tests__/account-dropdown.test.tsx
import { describe, expect, it, vi } from 'vitest'
import { render, screen } from '@testing-library/react'

vi.mock('@iblai/iblai-js/data-layer', () => ({
  useGetUserMetadataQuery: () => ({ data: { display_name: 'Jane' } }),
}))
vi.mock('@/lib/iblai/use-url-context', () => ({
  useUrlContext: () => ({ tenantKey: 'gwu', username: 'jane', ready: true }),
}))

import { AccountDropdown } from '@/components/account-dropdown'

describe('AccountDropdown', () => {
  it('shows the user display name', () => {
    render(<AccountDropdown />)
    expect(screen.getByText('Jane')).toBeInTheDocument()
  })
})

3d. Providers + contexts and 3e. Data definitions

• Providers/contexts: mock <AuthProvider> and <TenantProvider> (they hit network); assert the provider passes the right value to children.
• Pure data (e.g. tab-definition arrays): pin the exact shape so a silent reordering or removal trips a test. See references/test-patterns.md for both recipes.

Step 4: Write e2e journeys — exhaustively

Golden paths are where a suite starts, not where it stops. Enumerate from the code, never from memory, and give every item one checkpoint: one Playwright test titled '<ID> · <what the user sees>' (A1, M4, K3…).

Never sign out in a journey (it ends the shared session); never send an invite, an email or a payment. After a change, the OS rule: a new page or feature gets a new journey; a new control, tab or validation gets checkpoints in its journey; a changed label or flow updates the checkpoint; a refactor with no visible change needs nothing — and the count never drops.

The bar: the OS runs 834 e2e tests over 93k lines (nine per 1,000 lines, 98 journeys over 43 routes); sis and crm six to seven per 1,000 lines and per route. Under six per 1,000 lines of your own code, you stopped early.

Journey template

import { expect, test } from './helpers' // vibe-starter: settles the SDK's re-auth round trip first

const appHost = process.env.APP_HOST || 'http://localhost:3000'

test.describe('pages journey', () => {
  test('K1 · /account shows the organization settings', async ({ page }) => {
    await page.goto(`${appHost}/account`)
    await expect(page).toHaveURL((url) => url.pathname === '/account', { timeout: 15_000 })
    await expect(page.locator('nav[aria-label="Organization settings"]')).toBeVisible({ timeout: 30_000 })
  })
})

In vibe-starter the journeys inherit the signed-in state from auth.setup.ts (member journeys switch to the member's with test.use) and import test from journeys/helpers.ts: auth.setup.ts saves each session after the SDK's one-time re-auth round trip through the login host (a state saved earlier bounces every test home a few seconds in), and the fixture opens home and waits for the service-worker reload before the test runs. In Next 16 dev <nextjs-portal> hosts the dev tools badge on every page, so it proves nothing; assert the page's own markup. Elsewhere the seedAuth stub lives in references/playwright-setup.md.

The ledger — e2e/coverage.json + e2e/COVERAGE.md — and pnpm e2e:coverage

coverage.json lists each journey with its spec, the pages and routes it visits (sourceFiles) and its checkpoints; COVERAGE.md is the same table for people. pnpm e2e:coverage (e2e/coverage.mjs) checks that every spec is a journey, every checkpoint id heads one test title and back, the summary adds up, every sourceFiles entry is a page the spec names by path, and every page and route under app/ is visited by some journey; then it approximates coverage: an app file counts when a journey visits a page that imports it (layouts and middleware.ts ride along; components/ui/** is not counted). Reached lines over app lines must be ≥ 95%, and the suite must hold at least six checkpoints per 1,000 app lines; below either it names each unreached file and the page that would reach it, or the density. A page nobody visits is a gap, never an exclusion — add the journey. The shape, the checks, and how the OS, sis and crm differ: references/e2e-coverage-tracking.md.

Step 5: Run coverage + iterate to ≥95%

pnpm test:coverage

@vitest/coverage-v8 prints a per-file report. Sort low → high and add tests to whatever's red. Three patterns close coverage gaps fast:

1. Untested branch in a hook — add a test that triggers the opposite condition (e.g. username === null vs set).
2. Uncovered error handler — mock the mutation to reject and assert toast.error was called.
3. Uncovered conditional render — pass the prop that toggles it and assert the alternate output.

If you can't get past ~90% without testing trivial wrappers, lower the threshold to 90 temporarily — but flag it so it doesn't quietly stick.

Then the e2e side:

pnpm e2e:coverage

Every unreached: line is a page to visit, or a file only an unvisited page reaches: add the journey (Step 4) and rerun until it passes.

Common pitfalls

Brand guidelines: BRAND.md