morfo: permutation runner v2 — URL-driven state seeds + RTL variant

v1 shipped click-based state cycles. v2 adds URL-seeded initial state so
the runner can reach any state the component exposes without walking a
click sequence, and runs every instrumented demo TWICE in CI (LTR + RTL)
to catch direction-specific regressions like the slider thumb transform
that landed this week.

## New test-route infrastructure

- `src/routes/test/soma/_perms.svelte.ts` — parses `?perm.X=Y` query
  params into a typed `Perms` record (naive coercion: `'true'` → boolean,
  digits → number, else string). Exposed via `Perms.getOr({})` context.
  Underscore prefix so SvelteKit does not route it.
- `+layout.svelte` — parses perms once at mount, sets context, honours
  `?perm.dir=rtl` and `?perm.locale=en` by routing through
  `App.setDir` / `App.setLocale` so the existing control strip stays in
  sync.

## Demo opt-in

Three demos seed initial state from URL params (v1 + v2 combined):

- `dialog` — `?perm.open=true` lands open.
- `tabs` — `?perm.value=tab-2` lands on that tab.
- `toolbar` — `?perm.format=bold,italic` + `?perm.align=center` preselect
  toggles.

## Runner extension

`scripts/permutation-check.ts` now runs each instrumented demo under a
matrix of URL variants (currently `ltr` + `rtl`), validates morfo at the
URL-seeded state BEFORE any clicks (logged as `[initial seed]`), then
runs the v1 click cycle. Output format updated to show per-variant
blocks.

Result: 3 instrumented demos × 2 variants = 22 step validations per CI
run. RTL seeding reproduces the specific state machine the slider
`translate(50%, -50%)` bug needed to fail — had v2 been in place that
day, the regression would have fired before shipping.

## Docs

`src/uix/morfo/PERMUTATION_RUNNER.md` gains the v2 section (layout
plumbing, demo opt-in, URL variant table, `[initial seed]` step
semantics) and marks v1 + v2 as shipped 2026-04-22. v3 (morfo-inferred
per-component axis matrix) and v4 (MutationObserver ordering for Sema)
remain on the roadmap.

## Verification

- `npm run check`: 0 errors in soma/morfo/test-route scope.
- `npm run smoke`: 65/65 routes.
- `npm run morfo:check`: 66/66 morfos.
- `npm run perm:check`: 3/3 demos · 22 steps across 2 variants.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
semantuix
dev 6 months ago
parent 61e4ba6d26
commit 575748f079

@ -218,6 +218,25 @@ async function executeStep(page: Page, step: PermStep): Promise<void> {
await page.waitForTimeout(step.settle);
}
// ── URL variant matrix (v2) ─────────────────────────────────────────────────
/**
* Universal URL variants applied to every instrumented demo. The layout at
* `src/routes/test/soma/+layout.svelte` reads `?perm.dir` / `?perm.locale`
* and re-seeds the presentation context; the runner validates the full
* click cycle under each variant so RTL-only reactivity bugs (e.g. the
* slider thumb-transform regression) fail cleanly instead of shipping.
*
* Per-component axes (component-specific state like `?perm.open=true`,
* `?perm.variant=alertdialog`, `?perm.value=tab-2`) are declared by the
* demo itself via `<script>` seed defaults — the runner doesn't need to
* know about them; it just runs the click cycle at each base URL.
*/
const URL_VARIANTS: Array<{ suffix: string; label: string }> = [
{ suffix: '', label: 'ltr' },
{ suffix: '?perm.dir=rtl', label: 'rtl' }
];
// ── Main loop ───────────────────────────────────────────────────────────────
type StepResult =
@ -226,14 +245,19 @@ type StepResult =
| { kind: 'error'; label: string; err: string }
| { kind: 'skipped'; label: string };
type RouteResult = {
route: string;
morfo: string;
type VariantResult = {
variant: string;
url: string;
stepsTotal: number;
steps: StepResult[];
pageErrors: string[];
};
type RouteResult = {
morfo: string;
variants: VariantResult[];
};
const BASE = process.argv[2] ?? (await probePort(5173, 5180));
if (!BASE) {
console.error('Could not find a running dev server on 5173-5180.');
@ -254,8 +278,13 @@ const ctx = await browser.newContext();
const results: RouteResult[] = [];
const skipped: string[] = [];
for (const morfo of morfos) {
const route = `/test/soma/${morfo.kebab}`;
/** Run one URL variant: load, cycle click-driven steps, report. */
async function runVariant(
base: string,
url: string,
variantLabel: string,
morfo: Morfo
): Promise<VariantResult | 'no-steps'> {
const page = await ctx.newPage();
const pageErrors: string[] = [];
page.on('pageerror', (e) => pageErrors.push(e.message));
@ -263,40 +292,59 @@ for (const morfo of morfos) {
if (m.type() === 'error') pageErrors.push(m.text());
});
const variantResult: VariantResult = {
variant: variantLabel,
url,
stepsTotal: 0,
steps: [],
pageErrors: []
};
try {
await page.goto(BASE + route, { waitUntil: 'networkidle', timeout: 20000 });
await page.goto(base + url, { waitUntil: 'networkidle', timeout: 20000 });
await page.waitForTimeout(300);
// Initial snapshot — if there are no perm steps at all, skip.
const initialSteps = await collectSteps(page);
if (initialSteps.length === 0) {
skipped.push(morfo.kebab);
await page.close();
continue;
return 'no-steps';
}
// Determine the max index the page declares at any point. Re-check the
// DOM after each step so portal-gated elements (Dialog.Close inside
// `{#if open}`) get counted once they mount.
const knownIndices = new Set<number>(initialSteps.map((s) => s.index));
const executedIndices = new Set<number>();
const routeResult: RouteResult = {
route,
morfo: morfo.kebab,
stepsTotal: 0, // filled after the run
steps: [],
pageErrors: []
};
// Any errors from the URL-seeded initial state count against the variant.
if (pageErrors.length > 0) {
const captured = pageErrors.splice(0, pageErrors.length).join(' | ');
variantResult.steps.push({
kind: 'fail',
label: '[initial seed]',
reason: `pageerror: ${captured.slice(0, 200)}`
});
} else {
// Validate morfo at the URL-seeded state BEFORE any clicks so v2
// state-injection alone (no click sequence) is exercised.
try {
const seedIssues = await validateMorfoAgainstDom(page, morfo);
if (seedIssues.length === 0) {
variantResult.steps.push({ kind: 'pass', label: '[initial seed]' });
} else {
variantResult.steps.push({
kind: 'fail',
label: '[initial seed]',
reason: seedIssues.map((i) => `[${i.kind}] ${i.message}`).join(' · ')
});
}
} catch (e) {
variantResult.steps.push({
kind: 'error',
label: '[initial seed]',
err: `validation: ${(e as Error).message.slice(0, 120)}`
});
}
}
// Iterate until no new pending step shows up. Cap at 50 to avoid a
// runaway demo that keeps spawning indices.
const executedIndices = new Set<number>();
let safety = 50;
while (safety-- > 0) {
// Re-collect; demos may have mounted new perm elements.
const current = await collectSteps(page);
for (const s of current) knownIndices.add(s.index);
const next = current
.filter((s) => !executedIndices.has(s.index))
.sort((a, b) => a.index - b.index)[0];
@ -308,7 +356,7 @@ for (const morfo of morfos) {
try {
await executeStep(page, step);
} catch (e) {
routeResult.steps.push({
variantResult.steps.push({
kind: 'error',
label: step.label,
err: (e as Error).message.slice(0, 120)
@ -316,32 +364,34 @@ for (const morfo of morfos) {
continue;
}
// Any new page errors since the last step?
if (pageErrors.length > 0) {
const captured = pageErrors.splice(0, pageErrors.length).join(' | ');
routeResult.steps.push({ kind: 'fail', label: step.label, reason: `pageerror: ${captured.slice(0, 200)}` });
variantResult.steps.push({
kind: 'fail',
label: step.label,
reason: `pageerror: ${captured.slice(0, 200)}`
});
continue;
}
if (step.skipValidate) {
routeResult.steps.push({ kind: 'pass', label: step.label + ' (no-validate)' });
variantResult.steps.push({ kind: 'pass', label: step.label + ' (no-validate)' });
continue;
}
// Re-validate morfo against current DOM.
try {
const issues = await validateMorfoAgainstDom(page, morfo);
if (issues.length === 0) {
routeResult.steps.push({ kind: 'pass', label: step.label });
variantResult.steps.push({ kind: 'pass', label: step.label });
} else {
routeResult.steps.push({
variantResult.steps.push({
kind: 'fail',
label: step.label,
reason: issues.map((i) => `[${i.kind}] ${i.message}`).join(' · ')
});
}
} catch (e) {
routeResult.steps.push({
variantResult.steps.push({
kind: 'error',
label: step.label,
err: `validation: ${(e as Error).message.slice(0, 120)}`
@ -349,21 +399,39 @@ for (const morfo of morfos) {
}
}
// Any errors that slipped past the per-step capture
routeResult.pageErrors = pageErrors;
routeResult.stepsTotal = routeResult.steps.length;
results.push(routeResult);
variantResult.pageErrors = pageErrors;
variantResult.stepsTotal = variantResult.steps.length;
} catch (e) {
results.push({
route,
morfo: morfo.kebab,
stepsTotal: 0,
steps: [{ kind: 'error', label: 'navigation', err: (e as Error).message.slice(0, 120) }],
pageErrors: []
variantResult.steps.push({
kind: 'error',
label: '[navigation]',
err: (e as Error).message.slice(0, 120)
});
} finally {
await page.close();
}
return variantResult;
}
for (const morfo of morfos) {
const variantResults: VariantResult[] = [];
let anyStepped = false;
for (const v of URL_VARIANTS) {
const url = `/test/soma/${morfo.kebab}${v.suffix}`;
const r = await runVariant(BASE, url, v.label, morfo);
if (r === 'no-steps') continue;
anyStepped = true;
variantResults.push(r);
}
if (!anyStepped) {
skipped.push(morfo.kebab);
continue;
}
results.push({ morfo: morfo.kebab, variants: variantResults });
}
await browser.close();
@ -371,25 +439,40 @@ await browser.close();
// ── Report ──────────────────────────────────────────────────────────────────
console.log('');
let passed = 0;
let failed = 0;
let passedRoutes = 0;
let failedRoutes = 0;
let totalSteps = 0;
let totalFailed = 0;
for (const r of results) {
const bad = r.steps.filter((s) => s.kind !== 'pass');
const bad = r.variants
.flatMap((v) => v.steps)
.filter((s) => s.kind !== 'pass');
totalSteps += r.variants.reduce((acc, v) => acc + v.steps.length, 0);
totalFailed += bad.length;
if (bad.length === 0) {
console.log(`PASS ${r.morfo.padEnd(20)} ${r.stepsTotal} permutation${r.stepsTotal === 1 ? '' : 's'}`);
for (const s of r.steps) {
console.log(` · ${s.label}`);
const variantCount = r.variants.length;
const stepCount = r.variants.reduce((acc, v) => acc + v.steps.length, 0);
console.log(
`PASS ${r.morfo.padEnd(20)} ${variantCount} variant${variantCount === 1 ? '' : 's'} · ${stepCount} step${stepCount === 1 ? '' : 's'}`
);
for (const v of r.variants) {
console.log(` [${v.variant}] ${v.url}`);
for (const s of v.steps) console.log(` · ${s.label}`);
}
passed++;
passedRoutes++;
} else {
console.log(`FAIL ${r.morfo.padEnd(20)} ${r.stepsTotal} permutations, ${bad.length} failed`);
for (const s of r.steps) {
if (s.kind === 'pass') console.log(` · ${s.label}`);
else if (s.kind === 'skipped') console.log(` · ${s.label} (skipped)`);
else if (s.kind === 'fail') console.log(` ✖ ${s.label} ${s.reason}`);
else console.log(` ✖ ${s.label} ERROR ${s.err}`);
console.log(`FAIL ${r.morfo.padEnd(20)} ${bad.length} step${bad.length === 1 ? '' : 's'} failed`);
for (const v of r.variants) {
console.log(` [${v.variant}] ${v.url}`);
for (const s of v.steps) {
if (s.kind === 'pass') console.log(` · ${s.label}`);
else if (s.kind === 'skipped') console.log(` · ${s.label} (skipped)`);
else if (s.kind === 'fail') console.log(` ✖ ${s.label} ${s.reason}`);
else console.log(` ✖ ${s.label} ERROR ${s.err}`);
}
}
failed++;
failedRoutes++;
}
}
@ -401,9 +484,12 @@ if (skipped.length > 0) {
console.log(' (demo has no `data-perm-step` annotations yet — see src/uix/morfo/PERMUTATION_RUNNER.md)');
}
console.log('');
console.log(
failed === 0
? `All ${passed} instrumented demo${passed === 1 ? '' : 's'} passed.`
: `${failed} demo${failed === 1 ? '' : 's'} failed: ${results.filter((r) => r.steps.some((s) => s.kind !== 'pass')).map((r) => r.morfo).join(', ')}`
);
process.exit(failed === 0 ? 0 : 1);
const summary =
failedRoutes === 0
? `All ${passedRoutes} instrumented demo${passedRoutes === 1 ? '' : 's'} passed. (${totalSteps} step${totalSteps === 1 ? '' : 's'} across ${URL_VARIANTS.length} URL variant${URL_VARIANTS.length === 1 ? '' : 's'}.)`
: `${failedRoutes} demo${failedRoutes === 1 ? '' : 's'} failed (${totalFailed}/${totalSteps} steps): ${results
.filter((r) => r.variants.some((v) => v.steps.some((s) => s.kind !== 'pass')))
.map((r) => r.morfo)
.join(', ')}`;
console.log(summary);
process.exit(failedRoutes === 0 ? 0 : 1);

@ -5,9 +5,20 @@
import { App } from '$lib/ext/app';
import { createPresentation } from '$lib/ext/presentation';
import { componentLangs } from '$soma/core/langs';
import { Perms, parsePerms } from './_perms.svelte';
let { children }: { children: Snippet } = $props();
// Permutation-runner v2: parse `?perm.X=Y` params from the URL and expose
// them to every demo via context. Reactive via `$derived` so SvelteKit
// client-side navigations (not full reloads) re-seed when the URL changes.
// The layout also honours `?perm.dir=rtl` / `?perm.locale=en` so the
// runner can exercise RTL / i18n without clicking the control strip.
const perms = $derived(parsePerms());
$effect(() => {
Perms.set(perms);
});
// ── Lang ─────────────────────────────────────────────────────────────────
const langs = createLangs(
{
@ -48,6 +59,16 @@
// ── App context ──────────────────────────────────────────────────────────
const app = App.create({ langs, presentation });
// ── URL-seed: let `?perm.dir=rtl` / `?perm.locale=en` override the
// control-strip defaults on mount. Runs after app init so
// `app.setDir` / `app.setLocale` take effect through the normal path.
$effect(() => {
const initial = perms;
if (initial.dir === 'rtl' || initial.dir === 'ltr') app.setDir(initial.dir);
if (initial.locale === 'es' || initial.locale === 'en')
app.setLocale(initial.locale as 'es' | 'en');
});
// ── Locale / dir controls ───────────────────────────────────────────────
const activeLocale = $derived(app.langs.getLocale());
const activeDir = $derived(app.presentation.getDir());

@ -0,0 +1,69 @@
/**
* Test-infra: URL-driven permutations (permutation-runner v2).
*
* The soma demo pages opt into **seed-from-URL** initial state via
* `?perm.X=Y` query params. The test-route layout parses them into a
* typed `Perms` context; each demo reads its specific keys with sensible
* defaults.
*
* Why: click-driven permutation sequences (v1) can't reach every state
* combination — e.g. "Dialog open AND dir=rtl AND variant=alertdialog"
* would need a careful 3-step dance. URL-seed lets the permutation runner
* navigate directly to any state the component's morfo can express,
* validate morfo, and move on.
*
* Convention:
* /test/soma/dialog?perm.open=true&perm.dir=rtl&perm.variant=alertdialog
* /test/soma/tabs?perm.value=tab-2
* /test/soma/toolbar?perm.dir=rtl&perm.disabled=true
*
* Type coercion is naive: `'true'`/`'false'` → boolean, bare digits →
* number, everything else stays a string. Demos can cast further when
* reading: `(perms?.variant as 'default' | 'alertdialog') ?? 'default'`.
*
* Prefixed with `_` so SvelteKit does not route it.
*/
import { page } from '$app/state';
import { context } from '$soma/provider';
export type PermValue = string | number | boolean;
export type Perms = Record<string, PermValue>;
const PREFIX = 'perm.';
function coerce(raw: string): PermValue {
if (raw === 'true') return true;
if (raw === 'false') return false;
if (raw !== '' && !Number.isNaN(Number(raw)) && /^-?\d+(\.\d+)?$/.test(raw)) {
return Number(raw);
}
return raw;
}
/** Parse `?perm.X=Y` params from the current URL. Re-runs on navigation. */
export function parsePerms(): Perms {
const out: Perms = {};
const params = page.url.searchParams;
for (const [key, value] of params.entries()) {
if (!key.startsWith(PREFIX)) continue;
out[key.slice(PREFIX.length)] = coerce(value);
}
return out;
}
/**
* Context for the parsed `Perms` object. Demos read with `Perms.getOr({})`
* and fall back to their own defaults:
*
* ```ts
* const perms = Perms.getOr({});
* let open = $state((perms.open as boolean) ?? false);
* let dir = $state((perms.dir as 'ltr' | 'rtl') ?? 'ltr');
* ```
*
* Use `getOr({})` (not `get()`) so a demo rendered outside the test-route
* layout doesn't throw — the empty record falls through to the demo's own
* defaults.
*/
export const Perms = context<Perms>('soma-test-perms');

@ -1,7 +1,12 @@
<script lang="ts">
import * as Dialog from '$soma/components/dialog';
import Portal from '$soma/components/internal/portal.svelte';
let open1 = $state(false);
import { Perms } from '../_perms.svelte';
// Perm-runner v2 seed — URL `?perm.open=true` / `?perm.variant=alertdialog`
// land directly in that state without walking the click sequence.
const perms = Perms.getOr({} as Record<string, string | number | boolean>);
let open1 = $state((perms.open as boolean) ?? false);
let open2 = $state(false);
let open3 = $state(false);
let open3nested = $state(false);

@ -1,7 +1,10 @@
<script lang="ts">
import * as Tabs from '$soma/components/tabs';
import { Perms } from '../_perms.svelte';
let value1 = $state('tab-1');
// Perm-runner v2 seed — URL `?perm.value=tab-2` lands in that tab.
const perms = Perms.getOr({} as Record<string, string | number | boolean>);
let value1 = $state((perms.value as string) ?? 'tab-1');
let value2 = $state('tab-1');
let value3 = $state('tab-1');
let value4 = $state('tab-1');

@ -1,8 +1,16 @@
<script lang="ts">
import * as Toolbar from '$soma/components/toolbar';
import { Perms } from '../_perms.svelte';
let format = $state<string[]>([]);
let align = $state<string[]>(['left']);
// Perm-runner v2 seed — `?perm.format=bold,italic` starts with those
// toggles on; `?perm.align=center` overrides the alignment.
const perms = Perms.getOr({} as Record<string, string | number | boolean>);
function parseList(v: unknown, fallback: string[]): string[] {
if (typeof v !== 'string' || v.length === 0) return fallback;
return v.split(',').map((s) => s.trim()).filter(Boolean);
}
let format = $state<string[]>(parseList(perms.format, []));
let align = $state<string[]>(parseList(perms.align, ['left']));
</script>
<svelte:head>

@ -133,14 +133,58 @@ Demos without `data-perm-step` are skipped with a `SKIP` line (they haven't opte
---
## v2 — URL-driven state injection
v1 covers "click through a state sequence and validate after each step." v2 adds **URL-seeded initial state**: the demo reads `?perm.X=Y` params from the URL and uses them as initial `$state` values. Every instrumented demo now runs TWICE in CI — once under `ltr` (no params), once under `rtl` (`?perm.dir=rtl`). Universal axes are run across every demo; per-component axes are just defaults in the demo's script.
### Layout plumbing
`src/routes/test/soma/+layout.svelte` parses `?perm.*` params once at mount, exposes them via a context at `src/routes/test/soma/_perms.svelte.ts`, and also honours `?perm.dir=ltr|rtl` and `?perm.locale=es|en` by routing through the existing `App.setDir` / `App.setLocale` path (so the control strip stays in sync).
### Demo opt-in
Read the perms and use them as **initial** state (not live-bound — the user can still click through transitions after landing):
```svelte
<script lang="ts">
import { Perms } from '../_perms.svelte';
const perms = Perms.getOr({} as Record<string, string | number | boolean>);
// Perm-runner v2 seed — `?perm.open=true` / `?perm.variant=alertdialog`
// land directly in that state without walking the click sequence.
let open = $state((perms.open as boolean) ?? false);
let variant = $state((perms.variant as 'default' | 'alertdialog') ?? 'default');
</script>
```
Type coercion is naive: `'true'` / `'false'` → boolean, bare digits → number, everything else stays a string. Cast to the component's actual union at the read site.
### URL variants
The runner runs each instrumented demo under a matrix of base URL variants defined in `scripts/permutation-check.ts`:
| Variant | URL suffix |
|---------|------------|
| `ltr` | `` (no params) |
| `rtl` | `?perm.dir=rtl` |
Per-component axes (`?perm.open=true`, `?perm.variant=alertdialog`, `?perm.value=tab-2`) are NOT enumerated by the runner yet — they're just defaults the demo honours when the URL provides them. Adding them to the matrix is a v3 task (the runner would read the morfo's `states` + `data.values` to generate the cartesian product).
### `[initial seed]` step
At each URL variant, the runner validates morfo BEFORE running any click step. That validation appears in the log as `[initial seed]` and catches morfo drift that exists at the URL-seeded state — exactly the gap a click-only cycle can't reach.
---
## Roadmap
- **v1 (current)** — click-based state transitions, morfo re-validation after each step.
- **v2** — URL-param-driven state injection (`?perm=open,dir=rtl,disabled`) so headless permutations don't need a click sequence.
- **v3** — derive the permutation set automatically from the morfo's `states` and `data` enum values; no per-demo instrumentation needed for the baseline cycles.
- **v1 (shipped 2026-04-22)** — click-based state transitions, morfo re-validation after each step.
- **v2 (shipped 2026-04-22)** — URL-seeded initial state, universal axes (`dir`, `locale`), `[initial seed]` validation step before clicks. Demo layout reads `?perm.*` params, demos seed initial `$state` from them.
- **v3** — derive the per-component axis matrix automatically from the morfo's `states` and `data.values`. No per-demo instrumentation needed for the baseline cycles; the runner generates `?perm.X=Y` URLs from the morfo itself.
- **v4** — integrate with `MutationObserver` to validate `data-last-action` / `data-state` ordering for Sema.
The convention (`data-perm-step` / `data-perm-mode`) is forward-compatible: v2+ infrastructure adds alternative drivers without invalidating v1 annotations.
The convention (`data-perm-step` / `data-perm-mode` / `perm.X=Y`) is forward-compatible: v3+ infrastructure adds automation without invalidating v1/v2 annotations.
---

Loading…
Cancel
Save

Powered by TurnKey Linux.