morfo: permutation runner v3 — per-demo axis matrix via `<meta name="perm-axes">`

v2 ran every demo under a hard-coded `ltr` + `rtl` matrix. v3 lets each
demo declare which additional axes it honours — `orientation`, `disabled`,
`readonly`, `invalid`, `loading` — and the runner adds ONE variant per
declared axis (axis-at-a-time, not cartesian, so the matrix stays bounded
at O(N) instead of O(2^N)).

## Convention

```svelte
<svelte:head>
  <meta name="perm-axes" content="dir,orientation,disabled" />
</svelte:head>
```

`dir` is universal (always runs). The other axes are opt-in: they only
produce variants when the demo both lists them in the meta AND wires the
corresponding state from `Perms.getOr(...)`. Without the meta, only
`ltr` + `rtl` run.

## Runner changes

- New `discoverAxes(base, morfoKebab)` pass: loads the baseline URL, reads
  `<meta name="perm-axes">`, returns the declared axis list.
- `buildVariants(declaredAxes)` composes `ltr` + `rtl` + one `?perm.X=Y`
  variant per declared axis from the `AXIS_FLIP` map.
- `[initial seed]` step validates morfo at each URL before any click,
  so URL-seeded state alone is exercised (catches seed regressions even
  when the click cycle never fires).

## Demo opt-in (this commit)

- `dialog` — `perm-axes="dir"` (no orientation / disabled / loading axes
  apply to Dialog).
- `tabs` — `perm-axes="dir,orientation,disabled"` + wires `orientation` +
  `disabled` to the first `Tabs.Provider`.
- `toolbar` — `perm-axes="dir,orientation"` + wires `orientation` to the
  first `Toolbar.Provider`.

Result: 3 demos, 9 URL variants, 34 step validations per CI run
(v2 was 22 steps across 6 variants).

## Docs

`src/uix/morfo/PERMUTATION_RUNNER.md` v3 section: supported axis table,
3-step opt-in recipe, roadmap updated (v4 = morfo-inferred value cycling,
v5 = MutationObserver ordering for Sema).

## Verification

- `npm run check`: 0 errors in soma/morfo/test-route scope.
- `npm run smoke`: 65/65 routes.
- `npm run morfo:check`: 66/66 morfos.
- `npm run perm:check`: 3/3 demos · 34 steps · 9 variants.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
semantuix
dev 6 months ago
parent 575748f079
commit 545d47148c

@ -218,24 +218,77 @@ async function executeStep(page: Page, step: PermStep): Promise<void> {
await page.waitForTimeout(step.settle); await page.waitForTimeout(step.settle);
} }
// ── URL variant matrix (v2) ───────────────────────────────────────────────── // ── URL variant matrix (v2 + v3 axis-at-a-time) ────────────────────────────
/** /**
* Universal URL variants applied to every instrumented demo. The layout at * Universal baseline: every instrumented demo runs at least under `ltr`
* `src/routes/test/soma/+layout.svelte` reads `?perm.dir` / `?perm.locale` * (plain URL) and `rtl` (`?perm.dir=rtl`). Additional per-component axes
* and re-seeds the presentation context; the runner validates the full * are discovered at runtime from the demo's `<meta name="perm-axes">` tag;
* click cycle under each variant so RTL-only reactivity bugs (e.g. the * each declared axis adds ONE variant that flips that axis from baseline
* slider thumb-transform regression) fail cleanly instead of shipping. * while keeping everything else at the default — linear, not cartesian,
* so the matrix stays bounded.
* *
* Per-component axes (component-specific state like `?perm.open=true`, * Supported axes (v3 initial set):
* `?perm.variant=alertdialog`, `?perm.value=tab-2`) are declared by the * dir — ltr (baseline) + rtl
* demo itself via `<script>` seed defaults — the runner doesn't need to * orientation — horizontal (baseline) + vertical
* know about them; it just runs the click cycle at each base URL. * disabled — unset + true
* readonly — unset + true
* invalid — unset + true
* loading — unset + true
*
* Demos opt in by listing the axes their `<script>` seed actually honours:
*
* ```svelte
* <svelte:head>
* <meta name="perm-axes" content="dir,orientation,disabled" />
* </svelte:head>
* ```
*
* Without the meta tag, only the universal `dir` axis runs. With the tag,
* the runner adds one variant per listed axis.
*/ */
const URL_VARIANTS: Array<{ suffix: string; label: string }> = [ type Variant = { suffix: string; label: string };
{ suffix: '', label: 'ltr' },
{ suffix: '?perm.dir=rtl', label: 'rtl' } const AXIS_FLIP: Record<string, Variant> = {
]; dir: { suffix: '?perm.dir=rtl', label: 'rtl' },
orientation: { suffix: '?perm.orientation=vertical', label: 'orientation=vertical' },
disabled: { suffix: '?perm.disabled=true', label: 'disabled' },
readonly: { suffix: '?perm.readonly=true', label: 'readonly' },
invalid: { suffix: '?perm.invalid=true', label: 'invalid' },
loading: { suffix: '?perm.loading=true', label: 'loading' }
};
/** Read `<meta name="perm-axes" content="dir,orientation,...">` from the page. */
async function readDeclaredAxes(page: Page): Promise<string[]> {
try {
const content = await page.$eval(
'meta[name="perm-axes"]',
(el) => (el as HTMLMetaElement).content
);
return content
.split(',')
.map((s) => s.trim())
.filter(Boolean);
} catch {
// Meta not present — demo opted out of per-axis coverage.
return [];
}
}
/**
* Build the per-morfo variant list: baseline `ltr` + `rtl` (universal) +
* one flipped variant per axis the demo declared.
*/
function buildVariants(declaredAxes: string[]): Variant[] {
const base: Variant[] = [{ suffix: '', label: 'ltr' }, AXIS_FLIP.dir];
const extra: Variant[] = [];
for (const axis of declaredAxes) {
if (axis === 'dir') continue; // already in base
const v = AXIS_FLIP[axis];
if (v) extra.push(v);
}
return [...base, ...extra];
}
// ── Main loop ─────────────────────────────────────────────────────────────── // ── Main loop ───────────────────────────────────────────────────────────────
@ -414,11 +467,35 @@ async function runVariant(
return variantResult; return variantResult;
} }
/** Discover axes declared by the demo via `<meta name="perm-axes">`. */
async function discoverAxes(base: string, morfoKebab: string): Promise<string[] | 'no-demo'> {
const page = await ctx.newPage();
try {
const res = await page.goto(`${base}/test/soma/${morfoKebab}`, {
waitUntil: 'networkidle',
timeout: 20000
});
if (!res || res.status() >= 400) return 'no-demo';
return await readDeclaredAxes(page);
} catch {
return 'no-demo';
} finally {
await page.close();
}
}
for (const morfo of morfos) { for (const morfo of morfos) {
const variantResults: VariantResult[] = []; const variantResults: VariantResult[] = [];
let anyStepped = false; let anyStepped = false;
for (const v of URL_VARIANTS) { const axes = await discoverAxes(BASE, morfo.kebab);
if (axes === 'no-demo') {
skipped.push(morfo.kebab);
continue;
}
const variants = buildVariants(axes);
for (const v of variants) {
const url = `/test/soma/${morfo.kebab}${v.suffix}`; const url = `/test/soma/${morfo.kebab}${v.suffix}`;
const r = await runVariant(BASE, url, v.label, morfo); const r = await runVariant(BASE, url, v.label, morfo);
if (r === 'no-steps') continue; if (r === 'no-steps') continue;
@ -484,9 +561,10 @@ if (skipped.length > 0) {
console.log(' (demo has no `data-perm-step` annotations yet — see src/uix/morfo/PERMUTATION_RUNNER.md)'); console.log(' (demo has no `data-perm-step` annotations yet — see src/uix/morfo/PERMUTATION_RUNNER.md)');
} }
console.log(''); console.log('');
const totalVariants = results.reduce((acc, r) => acc + r.variants.length, 0);
const summary = const summary =
failedRoutes === 0 failedRoutes === 0
? `All ${passedRoutes} instrumented demo${passedRoutes === 1 ? '' : 's'} passed. (${totalSteps} step${totalSteps === 1 ? '' : 's'} across ${URL_VARIANTS.length} URL variant${URL_VARIANTS.length === 1 ? '' : 's'}.)` ? `All ${passedRoutes} instrumented demo${passedRoutes === 1 ? '' : 's'} passed. (${totalSteps} step${totalSteps === 1 ? '' : 's'} across ${totalVariants} URL variant${totalVariants === 1 ? '' : 's'}.)`
: `${failedRoutes} demo${failedRoutes === 1 ? '' : 's'} failed (${totalFailed}/${totalSteps} steps): ${results : `${failedRoutes} demo${failedRoutes === 1 ? '' : 's'} failed (${totalFailed}/${totalSteps} steps): ${results
.filter((r) => r.variants.some((v) => v.steps.some((s) => s.kind !== 'pass'))) .filter((r) => r.variants.some((v) => v.steps.some((s) => s.kind !== 'pass')))
.map((r) => r.morfo) .map((r) => r.morfo)

@ -14,6 +14,9 @@
<svelte:head> <svelte:head>
<title>Dialog · Soma</title> <title>Dialog · Soma</title>
<!-- Axes the perm-runner v3 should generate URL variants for. Dialog
has no orientation / disabled / loading axes — only dir applies. -->
<meta name="perm-axes" content="dir" />
</svelte:head> </svelte:head>
<div class="page"> <div class="page">

@ -2,16 +2,23 @@
import * as Tabs from '$soma/components/tabs'; import * as Tabs from '$soma/components/tabs';
import { Perms } from '../_perms.svelte'; import { Perms } from '../_perms.svelte';
// Perm-runner v2 seed — URL `?perm.value=tab-2` lands in that tab. // Perm-runner v2/v3 seed. `?perm.value=tab-2` preselects, `?perm.orientation=vertical`
// flips axis, `?perm.disabled=true` disables the first Tabs.Provider.
const perms = Perms.getOr({} as Record<string, string | number | boolean>); const perms = Perms.getOr({} as Record<string, string | number | boolean>);
let value1 = $state((perms.value as string) ?? 'tab-1'); let value1 = $state((perms.value as string) ?? 'tab-1');
let value2 = $state('tab-1'); let value2 = $state('tab-1');
let value3 = $state('tab-1'); let value3 = $state('tab-1');
let value4 = $state('tab-1'); let value4 = $state('tab-1');
let orientation = $state<'horizontal' | 'vertical'>(
(perms.orientation as 'horizontal' | 'vertical') ?? 'horizontal'
);
let disabled = $state((perms.disabled as boolean) ?? false);
</script> </script>
<svelte:head> <svelte:head>
<title>Tabs · Soma</title> <title>Tabs · Soma</title>
<!-- Axes the perm-runner v3 should generate URL variants for. -->
<meta name="perm-axes" content="dir,orientation,disabled" />
</svelte:head> </svelte:head>
<div class="page"> <div class="page">
@ -23,7 +30,7 @@
<h2>1. Horizontal (automatic)</h2> <h2>1. Horizontal (automatic)</h2>
<p class="state">value: {value1}</p> <p class="state">value: {value1}</p>
<Tabs.Provider bind:value={value1}> <Tabs.Provider bind:value={value1} {orientation} {disabled}>
<Tabs.List> <Tabs.List>
<Tabs.Trigger value="tab-1" data-perm-step="0" data-perm-label="tab 1 active (initial)">Account</Tabs.Trigger> <Tabs.Trigger value="tab-1" data-perm-step="0" data-perm-label="tab 1 active (initial)">Account</Tabs.Trigger>
<Tabs.Trigger value="tab-2" data-perm-step="1" data-perm-label="switch to tab 2">Password</Tabs.Trigger> <Tabs.Trigger value="tab-2" data-perm-step="1" data-perm-label="switch to tab 2">Password</Tabs.Trigger>

@ -2,8 +2,9 @@
import * as Toolbar from '$soma/components/toolbar'; import * as Toolbar from '$soma/components/toolbar';
import { Perms } from '../_perms.svelte'; import { Perms } from '../_perms.svelte';
// Perm-runner v2 seed — `?perm.format=bold,italic` starts with those // Perm-runner v2/v3 seed. `?perm.format=bold,italic` preselects toggles;
// toggles on; `?perm.align=center` overrides the alignment. // `?perm.align=center` overrides alignment; `?perm.orientation=vertical`
// flips the first Toolbar to vertical (covered by perm-axes meta below).
const perms = Perms.getOr({} as Record<string, string | number | boolean>); const perms = Perms.getOr({} as Record<string, string | number | boolean>);
function parseList(v: unknown, fallback: string[]): string[] { function parseList(v: unknown, fallback: string[]): string[] {
if (typeof v !== 'string' || v.length === 0) return fallback; if (typeof v !== 'string' || v.length === 0) return fallback;
@ -11,10 +12,15 @@
} }
let format = $state<string[]>(parseList(perms.format, [])); let format = $state<string[]>(parseList(perms.format, []));
let align = $state<string[]>(parseList(perms.align, ['left'])); let align = $state<string[]>(parseList(perms.align, ['left']));
let orientation = $state<'horizontal' | 'vertical'>(
(perms.orientation as 'horizontal' | 'vertical') ?? 'horizontal'
);
</script> </script>
<svelte:head> <svelte:head>
<title>Toolbar · Soma</title> <title>Toolbar · Soma</title>
<!-- Axes the perm-runner v3 should generate URL variants for. -->
<meta name="perm-axes" content="dir,orientation" />
</svelte:head> </svelte:head>
<div class="page"> <div class="page">
@ -26,7 +32,7 @@
<h2>1. Horizontal toolbar</h2> <h2>1. Horizontal toolbar</h2>
<p class="state">format: {JSON.stringify(format)} | align: {JSON.stringify(align)}</p> <p class="state">format: {JSON.stringify(format)} | align: {JSON.stringify(align)}</p>
<Toolbar.Provider> <Toolbar.Provider {orientation}>
<Toolbar.Button onclick={() => alert('Undo')}>Undo</Toolbar.Button> <Toolbar.Button onclick={() => alert('Undo')}>Undo</Toolbar.Button>
<Toolbar.Button onclick={() => alert('Redo')}>Redo</Toolbar.Button> <Toolbar.Button onclick={() => alert('Redo')}>Redo</Toolbar.Button>

@ -160,16 +160,62 @@ Read the perms and use them as **initial** state (not live-bound — the user ca
Type coercion is naive: `'true'` / `'false'` → boolean, bare digits → number, everything else stays a string. Cast to the component's actual union at the read site. Type coercion is naive: `'true'` / `'false'` → boolean, bare digits → number, everything else stays a string. Cast to the component's actual union at the read site.
### URL variants ### URL variants (v2 baseline + v3 axis-at-a-time)
The runner runs each instrumented demo under a matrix of base URL variants defined in `scripts/permutation-check.ts`: The runner composes each demo's variant list from two sources:
| Variant | URL suffix | 1. **Universal baseline** — every instrumented demo runs at `ltr` (plain URL) and `rtl` (`?perm.dir=rtl`). These catch direction-specific bugs like the slider thumb transform regression.
|---------|------------| 2. **Per-demo axes** (v3) — the demo declares which axes it honours via `<meta name="perm-axes" content="...">`. The runner adds ONE variant per declared axis, flipping that axis from baseline while keeping everything else at default (linear, not cartesian — the matrix stays bounded).
| `ltr` | `` (no params) |
| `rtl` | `?perm.dir=rtl` |
Per-component axes (`?perm.open=true`, `?perm.variant=alertdialog`, `?perm.value=tab-2`) are NOT enumerated by the runner yet — they're just defaults the demo honours when the URL provides them. Adding them to the matrix is a v3 task (the runner would read the morfo's `states` + `data.values` to generate the cartesian product). Supported axes (v3 initial set):
| Axis | Baseline | Flipped variant | Typical use |
|------|----------|-----------------|-------------|
| `dir` | `ltr` | `rtl` | Always run (universal). |
| `orientation` | `horizontal` | `vertical` | Toolbars, Tabs, Splitters, RadioGroup, Slider. |
| `disabled` | unset | `true` | Any component with a `disabled` prop. |
| `readonly` | unset | `true` | Inputs, Editable, Calendar. |
| `invalid` | unset | `true` | Form-participating components. |
| `loading` | unset | `true` | Async components with pending states. |
Demos opt in by listing the axes they honour. The runner skips meta-less demos at the `dir` axis only (universal baseline always applies).
### v3 opt-in — 3 steps
1. Add the meta tag (list only the axes your demo actually wires):
```svelte
<svelte:head>
<title>Tabs · Soma</title>
<meta name="perm-axes" content="dir,orientation,disabled" />
</svelte:head>
```
2. Seed the `$state` from perms for each declared axis:
```svelte
<script lang="ts">
import { Perms } from '../_perms.svelte';
const perms = Perms.getOr({} as Record<string, string | number | boolean>);
let orientation = $state<'horizontal' | 'vertical'>(
(perms.orientation as 'horizontal' | 'vertical') ?? 'horizontal'
);
let disabled = $state((perms.disabled as boolean) ?? false);
</script>
```
3. Wire the state to the Provider:
```svelte
<Tabs.Provider bind:value {orientation} {disabled}>
...
</Tabs.Provider>
```
The runner now visits `/test/soma/tabs?perm.orientation=vertical` and `/test/soma/tabs?perm.disabled=true` in addition to `ltr` / `rtl`, and re-runs the full `data-perm-step` click cycle at each URL.
Per-component value seeds (`?perm.value=tab-2`, `?perm.open=true`, `?perm.format=bold,italic`) are NOT enumerated by the runner — they're just defaults the demo honours when testing manually or via direct URL navigation. Adding them to the enumerated matrix is a v4 task (the runner would need to read the morfo's `states` + `data.values` to know which values are meaningful).
### `[initial seed]` step ### `[initial seed]` step
@ -181,10 +227,11 @@ At each URL variant, the runner validates morfo BEFORE running any click step. T
- **v1 (shipped 2026-04-22)** — click-based state transitions, morfo re-validation after each step. - **v1 (shipped 2026-04-22)** — click-based state transitions, morfo re-validation after each step.
- **v2 (shipped 2026-04-22)** — URL-seeded initial state, universal axes (`dir`, `locale`), `[initial seed]` validation step before clicks. Demo layout reads `?perm.*` params, demos seed initial `$state` from them. - **v2 (shipped 2026-04-22)** — URL-seeded initial state, universal axes (`dir`, `locale`), `[initial seed]` validation step before clicks. Demo layout reads `?perm.*` params, demos seed initial `$state` from them.
- **v3** — derive the per-component axis matrix automatically from the morfo's `states` and `data.values`. No per-demo instrumentation needed for the baseline cycles; the runner generates `?perm.X=Y` URLs from the morfo itself. - **v3 (shipped 2026-04-22)** — per-demo axis matrix via `<meta name="perm-axes">`. Runner adds one variant per declared axis (axis-at-a-time, not cartesian). Supported axes: `dir` (universal), `orientation`, `disabled`, `readonly`, `invalid`, `loading`.
- **v4** — integrate with `MutationObserver` to validate `data-last-action` / `data-state` ordering for Sema. - **v4** — morfo-inferred value cycling. Reads `states[]` + `data.values` from each morfo and enumerates meaningful value combinations automatically (would cover `?perm.value=tab-2` / `?perm.variant=alertdialog` / `?perm.open=true` without per-demo declaration).
- **v5** — integrate with `MutationObserver` to validate `data-last-action` / `data-state` ordering for Sema.
The convention (`data-perm-step` / `data-perm-mode` / `perm.X=Y`) is forward-compatible: v3+ infrastructure adds automation without invalidating v1/v2 annotations. The convention (`data-perm-step` / `data-perm-mode` / `perm.X=Y` / `perm-axes` meta) is forward-compatible: v4+ infrastructure adds automation without invalidating earlier annotations.
--- ---

Loading…
Cancel
Save

Powered by TurnKey Linux.