# Permutation runner

Automated CI tool that cycles soma components through their declared state space and validates morfo compliance at every step.

> **Why this exists.** `morfo:check` validates the DOM at page-load time, in a single state. `smoke` catches `pageerror` during mount. Neither ejercices the **transitions** between states — and that's where most of the reactivity bugs we've caught this quarter actually live (toolbar A35, form A36, slider RTL transform). The permutation runner is the third validation layer that closes that gap.

---

## What it validates

For every demo page, for every permutation the page declares:

1. **No reactivity loops.** `effect_update_depth_exceeded` AND microtask-mediated loops (A35 / A36) both count as failures. A long microtask starvation (>2 s) is the proxy for A36.
2. **No runtime errors.** `pageerror` or `console.error` during a state transition fails the permutation.
3. **Morfo contract holds.** After the transition settles, the emitted DOM still conforms to the component's morfo (same rules as `morfo:check` — declared attrs present, values in enum, no undeclared namespaced attrs).
4. **ARIA relationships remain valid.** If a part emits `aria-labelledby="X"`, the element with id `X` must exist in the DOM at that state.

What it does **not** validate (future work):

- Visual correctness (that's eidos' problem).
- Animation timing or `data-last-action` → `data-state` ordering (dedicated MutationObserver test).
- Cross-component interactions (it runs one demo at a time).

---

## Convention: `data-perm-*` attributes

Demo pages opt in by tagging interactive controls with `data-perm-*` attrs. The runner discovers these, clicks them in sequence, and validates after each click.

### `data-perm-step="N"`

Declares the click order. Starts at `0`. The runner clicks elements in ascending order.

```svelte
<button data-perm-step="0" onclick={() => dialogOpen = true}>Open</button>
<button data-perm-step="1" onclick={() => dialogOpen = false}>Close</button>
```

### `data-perm-label="..."`

Human-readable label for the log. Defaults to the element's text content.

```svelte
<button data-perm-step="0" data-perm-label="open dialog via trigger">
  Open
</button>
```

### `data-perm-settle="ms"`

Override the default 500ms settle time before morfo re-validation. Use for animations, async validation, etc.

```svelte
<button data-perm-step="0" data-perm-settle="800">
  Submit (async)
</button>
```

### `data-perm-skip-validate`

Click the element but don't validate afterwards. For intermediate actions (e.g. typing a value before submit). The next step validates.

### `data-perm-mode`

Overrides the default click interaction:

- `click` (default) — `await page.click(el)`.
- `focus` — `await el.focus()`.
- `type="abc"` — type the string.
- `key="Escape"` — dispatch a `keydown` with the given key.

```svelte
<input data-perm-step="0" data-perm-mode='type="ada@example.com"' />
<div data-perm-step="1" data-perm-mode='key="Tab"' />
```

---

## Example: Dialog demo

Minimal instrumentation that cycles open → close → open via Escape:

```svelte
<Dialog.Trigger data-perm-step="0" data-perm-label="open via trigger">
  Open
</Dialog.Trigger>

{#if dialogOpen}
  <Dialog.Content data-perm-step="1" data-perm-mode='key="Escape"' data-perm-label="close via Escape">
    …
  </Dialog.Content>
{/if}

<button data-perm-step="2" data-perm-label="re-open">Open</button>
```

Runner output:

```
PASS  dialog   3 permutations
  ·  step 0 (open via trigger)          morfo OK, 0 errors
  ·  step 1 (close via Escape)          morfo OK, 0 errors
  ·  step 2 (re-open)                   morfo OK, 0 errors
```

---

## Failure format

```
FAIL  toolbar  3 permutations, 1 failed
  ·  step 0 (focus first button)        morfo OK
  ·  step 1 (type "b")                  PAGEERR: effect_update_depth_exceeded
  ·  step 2 (click toggle)              skipped (previous step failed)
```

---

## CLI

```bash
# Requires `npm run dev` running (auto-detects port 5173–5180).
npm run perm:check
```

The current docs-shell route map is `/uix/components/{kebab}`. Components
without a routed page are reported as `SKIP no route`; pages that exist but
cannot be loaded are reported as `SKIP no demo load`; routed pages without
`data-perm-step` or `perm-axes` opt-in are reported as `SKIP no perm
instrumentation`. Override the prefix with
`PERM_ROUTE_PREFIX=/some/path` when testing another shell.

Exit codes:

- `0` — every instrumented routed demo passes all permutations. If no routed
  demo is instrumented yet, the runner exits cleanly and says so.
- `1` — at least one permutation failed.
- `2` — no dev server / no morfos found.

Demos without `data-perm-step` / `perm-axes` are skipped with a `SKIP` line
(they haven't opted in yet).

---

## v2 — URL-driven state injection

v1 covers "click through a state sequence and validate after each step." v2 adds **URL-seeded initial state**: the demo reads `?perm.X=Y` params from the URL and uses them as initial `$state` values. Every instrumented demo runs at least once on the baseline URL. Extra URL variants run only when the page declares the axis with `<meta name="perm-axes">`; this keeps coverage honest while the new docs shell is being instrumented.

### Layout plumbing

The old `/test/soma` shell used a shared `_perms.svelte.ts` context. That
shell was retired; the current runner only provides the URL variants and
expects each routed page under `web/routes` to read `?perm.*` params directly
or through a new shell-level helper when we decide to add one. Until a page
does that and declares `perm-axes` / `data-perm-step`, it is skipped rather
than counted as coverage.

### Demo opt-in

Read the perms and use them as **initial** state (not live-bound — the user can still click through transitions after landing):

```svelte
<script lang="ts">
  import { page } from '$app/state';
  const params = page.url.searchParams;

  // Perm-runner v2 seed — `?perm.open=true` / `?perm.variant=alertdialog`
  // land directly in that state without walking the click sequence.
  let open = $state(params.get('perm.open') === 'true');
  let variant = $state((params.get('perm.variant') as 'default' | 'alertdialog') ?? 'default');
</script>
```

Type coercion is naive: `'true'` / `'false'` → boolean, bare digits → number, everything else stays a string. Cast to the component's actual union at the read site.

### URL variants (v2 baseline + v3 axis-at-a-time)

The runner composes each demo's variant list from two sources:

1. **Baseline** — every instrumented demo runs once on its plain route.
2. **Per-demo axes** (v3) — the demo declares which axes it honours via `<meta name="perm-axes" content="...">`. The runner adds ONE variant per declared axis, flipping that axis from baseline while keeping everything else at default (linear, not cartesian — the matrix stays bounded).

Supported axes (v3 initial set):

| Axis | Baseline | Flipped variant | Typical use |
|------|----------|-----------------|-------------|
| `dir` | route default | `rtl` | Components with direction-sensitive keyboard/layout behavior. |
| `orientation` | `horizontal` | `vertical` | Toolbars, Tabs, Splitters, RadioGroup, Slider. |
| `disabled` | unset | `true` | Any component with a `disabled` prop. |
| `readonly` | unset | `true` | Inputs, Editable, Calendar. |
| `invalid` | unset | `true` | Form-participating components. |
| `loading` | unset | `true` | Async components with pending states. |

Demos opt in by listing the axes they honour. Meta-less demos still run the
baseline transition cycle if they have `data-perm-step` annotations.

### v3 opt-in — 3 steps

1. Add the meta tag (list only the axes your demo actually wires):

   ```svelte
   <svelte:head>
     <title>Tabs · Soma</title>
     <meta name="perm-axes" content="dir,orientation,disabled" />
   </svelte:head>
   ```

2. Seed the `$state` from perms for each declared axis:

   ```svelte
   <script lang="ts">
     import { page } from '$app/state';
     const params = page.url.searchParams;

     let orientation = $state<'horizontal' | 'vertical'>(
       (params.get('perm.orientation') as 'horizontal' | 'vertical') ?? 'horizontal'
     );
     let disabled = $state(params.get('perm.disabled') === 'true');
   </script>
   ```

3. Wire the state to the Provider:

   ```svelte
   <Tabs.Provider bind:value {orientation} {disabled}>
     ...
   </Tabs.Provider>
   ```

The runner now visits `/tabs?perm.orientation=vertical` and `/tabs?perm.disabled=true` in addition to `ltr` / `rtl`, and re-runs the full `data-perm-step` click cycle at each URL.

Per-component value seeds (`?perm.value=tab-2`, `?perm.open=true`, `?perm.format=bold,italic`) are NOT enumerated by the runner — they're just defaults the demo honours when testing manually or via direct URL navigation. Adding them to the enumerated matrix is a v4 task (the runner would need to read the morfo's `states` + `data.values` to know which values are meaningful).

### `[initial seed]` step

At each URL variant, the runner validates morfo BEFORE running any click step. That validation appears in the log as `[initial seed]` and catches morfo drift that exists at the URL-seeded state — exactly the gap a click-only cycle can't reach.

---

## Roadmap

- **v1 (shipped 2026-04-22)** — click-based state transitions, morfo re-validation after each step.
- **v2 (shipped 2026-04-22)** — URL-seeded initial state, universal axes (`dir`, `locale`), `[initial seed]` validation step before clicks. Demo layout reads `?perm.*` params, demos seed initial `$state` from them.
- **v3 (shipped 2026-04-22)** — per-demo axis matrix via `<meta name="perm-axes">`. Runner adds one variant per declared axis (axis-at-a-time, not cartesian). Supported axes: `dir` (universal), `orientation`, `disabled`, `readonly`, `invalid`, `loading`.
- **v4** — morfo-inferred value cycling. Reads `states[]` + `data.values` from each morfo and enumerates meaningful value combinations automatically (would cover `?perm.value=tab-2` / `?perm.variant=alertdialog` / `?perm.open=true` without per-demo declaration).
- **v5** — integrate with `MutationObserver` to validate `data-last-action` / `data-state` ordering for Sema.

The convention (`data-perm-step` / `data-perm-mode` / `perm.X=Y` / `perm-axes` meta) is forward-compatible: v4+ infrastructure adds automation without invalidating earlier annotations.

---

## Related

- [morfo README](./README.md) — authoring reference for the morfos this runner validates against.
- [`scripts/permutation-check.ts`](../../../scripts/permutation-check.ts) — the runner itself.
- [`scripts/morfo-check.ts`](../../../scripts/morfo-check.ts) — single-state DOM validator that shares the per-morfo validation logic.