sw-ml-study/demo-extensions

demonstrate sw-mlpl language extensions in dynamically loaded Rust

0

stars

149

commits

Rust

primary language

Sep 3, 2026

updated

README

demo-extensions

demo-extensions explores how independently built Rust libraries can add native capabilities to sw-MLPL without adding each domain to the language runtime. The repository now includes the original headless hello extension foundation plus native interactive line-rendered applications, bounded model/media/filesystem demonstrations, and a Rust/Yew ML microscope.

The wireframe cube remains the smallest visual teaching path. All interactive native demos now initialize one retained scene, use stable-ID patches for geometry/style changes, and use view updates for camera/help/status changes.

The repository proves the downstream extension boundary with a real .dylib/.so and a Rust host harness. sw-MLPL now exposes a separate static scalar registry plus a byte-compatible C-descriptor adapter: both its built-in hello:answer() and this repository's _hello:answer() provider are proven through the interpreter. Arrays, persistent handles, and nested records are now also proven through the real downstream descriptor. use hello, compilation, dynamic loading and compiled-provider startup remain tracked contracts.

What is here

crates/mlpl-extension-abi/      Versioned C-compatible ABI and validation
crates/mlpl-extension-loader/   Package resolver, dynamic loader, and registry
crates/mlpl-extension-sdk/      Safe author-facing SDK scaffold
crates/mlpl-native3d-scene/      Generic line/point scenes and headless planning
lib/native3d/                    Reusable MLPL camera, picking, geometry, app loop
demos/wireframe-cube/            MLPL-owned bulk-array cube scene
demos/tic-tac-toe/               MLPL rules, minimax, and generic line scene
demos/life-plane/                 MLPL finite-grid Life model and presets
extensions/hello/               Rust cdylib, package manifest, and MLPL facade
extensions/boundary-probe/      Public-SDK array/handle/record host probe
extensions/native3d/            Generic headless viewer and bulk line provider
tests/                          Native mlplunit and structural tests
docs/                           Architecture, contracts, plans, and evidence

The hello package demonstrates the intended separation:

  • Rust exports private _hello.answer, _hello.fail, and _hello.panic functions through one sw_mlpl_extension_v1 entry point.
  • extension.toml selects an exact macOS or Linux native artifact and declares the public hello package separately from the private _hello namespace.
  • module.mlpl is the public MLPL facade. It is tested as ordinary MLPL today and will bind to the native namespace once sw-MLPL provides the host hook.

Prerequisites

  • Rust 1.85 or newer.
  • just for repository task aliases.
  • The adjacent ../sw-mlpl checkout with target/release/mlpl-repl or target/debug/mlpl-repl, or an absolute MLPL override.
  • mlplunit on PATH, an absolute MLPLUNIT override, or the adjacent /Users/mike/github/softwarewrighter/mlplunit/bin/mlplunit checkout used by the project scripts.

The scripts only select existing tools; they never install or overwrite them. Environment overrides must be absolute paths.

Build and test

Build the workspace and the independently loadable hello library:

cargo build --workspace
cargo build -p mlpl-extension-hello

Open the interactive native cube:

just cube-3d

Open the playable native tic-tac-toe game:

just tic-tac-toe

Open the editable native Life plane:

just life-3d

Open Life on a native 3D torus with wrap-around in both grid axes:

just life-torus

A glider advancing across the wrapped native Life torus

The tracked eight-second loop is 720 pixels wide and about 31 KiB. See graphical demo captures to reproduce the still and animated WebP with macOS screencapture, ffmpeg, and WebP tools. Use playwright-cli for browser-rendered demos; it cannot capture native winit/wgpu windows. The same just capture-native NAME workflow works for all native demos. The audio-spectrum/MP3 visualizer is the recommended next animation, followed by point cloud and Model Atlas; a small representative gallery is preferable to capturing every demo. OBS is optional.

Visualize bounded chunks from a selected MP3 or Ogg file:

just audio-spectrum

A native spectrum view changing as bounded audio chunks play

This 20-frame loop shows the MLPL-owned spectrum geometry responding during native playback. The selected recording is demonstration media; the repository stores only the derived capture, not the source audio.

Explore deterministic MLPL-owned point arrays with native retained updates, stable-ID selection, and camera controls:

just point-cloud

A colored point cloud moving through the native retained renderer

The lightweight loop shows the same bulk point attributes and generic renderer used by the interactive teaching app; application meaning remains in MLPL.

Measure bounded Model Atlas range scanning against growing sparse files:

just model-atlas-memory-evidence

Validate the derived Safetensors/GGUF tensor-city handoff:

just model-atlas-contract

Open the interactive native Model Atlas tensor city:

just model-atlas

A native tensor city orbiting while one Model Atlas building remains selected

The compact loop shows the derived seven-tensor fixture, architecture labels, variable-height buildings, camera movement, and persistent selected-tensor detail. It demonstrates the checked-in interchange fixture rather than claiming that an arbitrary model was loaded.

Choose and inspect an actual local Safetensors model without reading its tensor payload:

just model-atlas-file

The picker searches ../demo-ml-utils/models by default. Use an absolute override such as MODEL_ROOT=/absolute/model/directory just model-atlas-file. Use Up/Down and Enter to choose a file, then M to return to the menu. See Model Atlas real files for bounds, ownership, format support, and the filesystem containment contract.

Explore a bounded sample of real Safetensors or GGUF tensor values:

just weight-distribution
just microscope-web

The default picker searches ../demo-ml-utils/models; set an absolute MODEL_ROOT to inspect a different confined tree. Choose a model and tensor with Up/Down and Enter. The histogram uses value units on X and logarithmic sample counts on Y; drag to orbit, Shift-drag or middle-drag to pan, use the wheel to zoom, Left/Backspace to return to tensors, and M to return to models. Only explicitly supported integer and Q8_0 decoders are enabled. See the weight-distribution guide and blocker matrix.

The bundled, derived fixture renders tensor metadata as stable-ID wireframe buildings grouped into Safetensors and GGUF districts. Building height uses a visible logarithmic stored-byte scale, so one large tensor cannot flatten the rest of the view. Click a building to inspect its name and heuristic role, drag to orbit, Shift-drag or middle-drag to pan, and use the wheel to zoom. A selects all tensors; S and G toggle their Safetensors or GGUF filter; L switches independently between footprint and height detail; R resets the view. The visible source panel and legend document provenance, architecture metadata, colors, scales, current filter/LOD state, and controls. See the Model Atlas native city for the data, ownership, bounded-detail, and platform contracts.

Click toggles a cell and Control-left-drag paints live cells. Plain left-drag orbits/tilts, Shift-left-drag or middle-drag pans, and the wheel zooms. Space runs/pauses, N steps, C clears, plus/minus changes speed, and B/H/I/T/G/U/R select block, beehive, blinker, toad, glider, Gosper gun, and seeded random. The same complete legend is visible inside the native window. The torus uses the same controls and presets; cells crossing either edge continue at the opposite edge, and clicking or painting follows the curved surface.

Click an empty square to move. Left-drag orbits/tilts, the wheel zooms, and Shift-left-drag or middle-drag pans; crossing the four-pixel drag threshold suppresses mark placement. X/O chooses the human mark, 1/2 chooses first or second, R restarts with those choices, and Escape closes. The board rules, perfect-play strategy, picking, choices, turns, hover, and scene arrays are MLPL-owned; the shared Rust host only normalizes input and renders generic lines.

The window keeps winit/wgpu on the main thread and runs sw-MLPL on a worker. controls.mlpl receives generic key/resize events and sends retained patches or view-only commands back to the renderer after initialization. Use W/S for width, arrows for height, A/D for length, +/- for signed speed, Space for pause, C for color, brackets for thickness, R for reset, and Escape to close. Left-drag orbits and tilts, the wheel zooms, and Shift-left-drag or middle-drag pans; these mappings and camera state are implemented in MLPL and shown in the window legend.

Run focused Rust or MLPL tests:

just rust-tests
just tests
just list-tests

The native demos include an editable Conway's Life plane. Its MLPL layer provides dead finite boundaries, whole-array B3/S23 evolution, owned cell updates, deterministic replacement presets, mouse cell editing, animation and speed controls, shared orbit/pan/zoom, and the complete visible control legend. connects to the existing native host through generic stable-ID bulk line arrays.

Run the mandatory pre-commit gate:

just check

The complete gate checks repository layout, .gitignore, tracked files, public/private namespaces, canonical MLPL formatting, MLPL module comments and function docstrings, Rust formatting, compilation, clippy, all Rust tests, native mlplunit tests, and whitespace. Set MLPL_FMT to an absolute formatter path when the adjacent ../sw-mlpl checkout is unavailable. The intentional panic test may print its panic-hook message; the test verifies that the panic is converted to ExtensionPanicked before it can unwind across the C ABI.

To run only the dynamic hello acceptance tests:

cargo build -p mlpl-extension-hello
cargo test -p mlpl-extension-loader --test hello_registration
cargo test -p mlpl-extension-loader --test manifest_resolution

Current status

The delivered repository proves:

  • fixed-layout ABI V1 values, errors, descriptors, and version negotiation;
  • bounded fail-closed metadata validation and host-owned metadata copies;
  • independent shared-library loading with library lifetime retention;
  • namespaced typed success, extension failure, and contained-panic calls;
  • deactivation that rejects later calls;
  • deterministic manifests, exact target selection, canonical path confinement, stable diagnostics, and duplicate/mismatch rejection;
  • typed function/default/return and native-type metadata with deterministic validation and stable help rendering;
  • bounded dense numeric arrays with validated dtype, rank, shape, byte strides, alignment, storage length, and one-call [N,3] acceptance;
  • extension-scoped, type-tagged generational handles with stale/cross-extension rejection and deterministic resource finalization;
  • macro-generated ABI descriptors/trampolines around safe Rust handlers, with hello containing no handwritten unsafe code;
  • a public MLPL facade kept separate from private native functions.
  • a deterministic MLPL wireframe-cube scene with independently adjustable dimensions, rotation speed, RGBA line color, and thickness;
  • a renderer-neutral Rust line-scene contract that validates bulk [N,3] positions and [M,2] edges before later GPU work.
  • a bounded renderer-neutral point-scene contract for [N,3] positions and parallel size/color/opacity/stable-ID arrays, with deterministic upload planning and explicit owned-byte accounting;
  • deterministic headless point projection, viewport/near-plane culling, far-to-near alpha ordering, stable-ID picking ties, and raster evidence;
  • a native wgpu/winit circular point-sprite pipeline with full stable IDs and an opt-in just point-cloud-smoke fixture on graphical macOS/Linux sessions;
  • atomic retained point replacements/patches and precision-safe stable-ID selection events, with application meaning reserved for MLPL;
  • a deterministic just point-cloud MLPL teaching app with selection, camera, patches, lifecycle, and a headless real-worker acceptance test;
  • a bounded just point-cloud-acceptance release probe and evidence matrix that separate CPU byte/timing observations from unmeasured GPU/Linux behavior;
  • a deterministic headless transform, perspective projection, clipping, and thick-line raster pipeline with portable PPM evidence.
  • a real headless _native3d provider with typed viewer lifecycle, bulk line arrays, state/size records, and explicit MLPL-supplied render state.
  • an MLPL-owned control reducer for dimensions, signed speed, pause/reset, palette, thickness, resize/close events, and deterministic bulk updates.
  • bounded/coalesced input, camera/picking, single-flight animation frames, and atomic retained-scene patches across every interactive native demo;
  • native tic-tac-toe, Life plane/torus, Model Atlas, disk usage, audio spectrum, and weight-distribution applications with MLPL-owned semantics;
  • an offline-first Rust/Yew microscope over schema-validated, hash-pinned MM01, LR01, and KM01 MLPL recordings with generic rank/shape rendering.

The opt-in wgpu/winit window is connected to the MLPL reducer through sw-MLPL's parked-main Port contract. Only owned event and scene values cross between the main-thread UI and worker interpreter. Dynamic loading by sw-MLPL, real unload/hot reload, facades, and compiled-provider startup remain future work.

Documentation

Repository workflow

Work is test-driven and tracked with AgentRail sagas. Changes are committed and pushed directly to main after just check; this repository does not use feature branches, pull requests, the gh CLI, or GitHub Actions for publication. See AGENTS.md for the complete process.

License

Copyright (c) 2026 Michael A Wright. Distributed under the MIT License.

Contributors

softwarewrighter

149 commits

sw-ml-study/demo-extensions

demonstrate sw-mlpl language extensions in dynamically loaded Rust

0

stars

149

commits

Rust

primary language

Sep 3, 2026

updated

README

demo-extensions

demo-extensions explores how independently built Rust libraries can add native capabilities to sw-MLPL without adding each domain to the language runtime. The repository now includes the original headless hello extension foundation plus native interactive line-rendered applications, bounded model/media/filesystem demonstrations, and a Rust/Yew ML microscope.

The wireframe cube remains the smallest visual teaching path. All interactive native demos now initialize one retained scene, use stable-ID patches for geometry/style changes, and use view updates for camera/help/status changes.

The repository proves the downstream extension boundary with a real .dylib/.so and a Rust host harness. sw-MLPL now exposes a separate static scalar registry plus a byte-compatible C-descriptor adapter: both its built-in hello:answer() and this repository's _hello:answer() provider are proven through the interpreter. Arrays, persistent handles, and nested records are now also proven through the real downstream descriptor. use hello, compilation, dynamic loading and compiled-provider startup remain tracked contracts.

What is here

crates/mlpl-extension-abi/      Versioned C-compatible ABI and validation
crates/mlpl-extension-loader/   Package resolver, dynamic loader, and registry
crates/mlpl-extension-sdk/      Safe author-facing SDK scaffold
crates/mlpl-native3d-scene/      Generic line/point scenes and headless planning
lib/native3d/                    Reusable MLPL camera, picking, geometry, app loop
demos/wireframe-cube/            MLPL-owned bulk-array cube scene
demos/tic-tac-toe/               MLPL rules, minimax, and generic line scene
demos/life-plane/                 MLPL finite-grid Life model and presets
extensions/hello/               Rust cdylib, package manifest, and MLPL facade
extensions/boundary-probe/      Public-SDK array/handle/record host probe
extensions/native3d/            Generic headless viewer and bulk line provider
tests/                          Native mlplunit and structural tests
docs/                           Architecture, contracts, plans, and evidence

The hello package demonstrates the intended separation:

  • Rust exports private _hello.answer, _hello.fail, and _hello.panic functions through one sw_mlpl_extension_v1 entry point.
  • extension.toml selects an exact macOS or Linux native artifact and declares the public hello package separately from the private _hello namespace.
  • module.mlpl is the public MLPL facade. It is tested as ordinary MLPL today and will bind to the native namespace once sw-MLPL provides the host hook.

Prerequisites

  • Rust 1.85 or newer.
  • just for repository task aliases.
  • The adjacent ../sw-mlpl checkout with target/release/mlpl-repl or target/debug/mlpl-repl, or an absolute MLPL override.
  • mlplunit on PATH, an absolute MLPLUNIT override, or the adjacent /Users/mike/github/softwarewrighter/mlplunit/bin/mlplunit checkout used by the project scripts.

The scripts only select existing tools; they never install or overwrite them. Environment overrides must be absolute paths.

Build and test

Build the workspace and the independently loadable hello library:

cargo build --workspace
cargo build -p mlpl-extension-hello

Open the interactive native cube:

just cube-3d

Open the playable native tic-tac-toe game:

just tic-tac-toe

Open the editable native Life plane:

just life-3d

Open Life on a native 3D torus with wrap-around in both grid axes:

just life-torus

A glider advancing across the wrapped native Life torus

The tracked eight-second loop is 720 pixels wide and about 31 KiB. See graphical demo captures to reproduce the still and animated WebP with macOS screencapture, ffmpeg, and WebP tools. Use playwright-cli for browser-rendered demos; it cannot capture native winit/wgpu windows. The same just capture-native NAME workflow works for all native demos. The audio-spectrum/MP3 visualizer is the recommended next animation, followed by point cloud and Model Atlas; a small representative gallery is preferable to capturing every demo. OBS is optional.

Visualize bounded chunks from a selected MP3 or Ogg file:

just audio-spectrum

A native spectrum view changing as bounded audio chunks play

This 20-frame loop shows the MLPL-owned spectrum geometry responding during native playback. The selected recording is demonstration media; the repository stores only the derived capture, not the source audio.

Explore deterministic MLPL-owned point arrays with native retained updates, stable-ID selection, and camera controls:

just point-cloud

A colored point cloud moving through the native retained renderer

The lightweight loop shows the same bulk point attributes and generic renderer used by the interactive teaching app; application meaning remains in MLPL.

Measure bounded Model Atlas range scanning against growing sparse files:

just model-atlas-memory-evidence

Validate the derived Safetensors/GGUF tensor-city handoff:

just model-atlas-contract

Open the interactive native Model Atlas tensor city:

just model-atlas

A native tensor city orbiting while one Model Atlas building remains selected

The compact loop shows the derived seven-tensor fixture, architecture labels, variable-height buildings, camera movement, and persistent selected-tensor detail. It demonstrates the checked-in interchange fixture rather than claiming that an arbitrary model was loaded.

Choose and inspect an actual local Safetensors model without reading its tensor payload:

just model-atlas-file

The picker searches ../demo-ml-utils/models by default. Use an absolute override such as MODEL_ROOT=/absolute/model/directory just model-atlas-file. Use Up/Down and Enter to choose a file, then M to return to the menu. See Model Atlas real files for bounds, ownership, format support, and the filesystem containment contract.

Explore a bounded sample of real Safetensors or GGUF tensor values:

just weight-distribution
just microscope-web

The default picker searches ../demo-ml-utils/models; set an absolute MODEL_ROOT to inspect a different confined tree. Choose a model and tensor with Up/Down and Enter. The histogram uses value units on X and logarithmic sample counts on Y; drag to orbit, Shift-drag or middle-drag to pan, use the wheel to zoom, Left/Backspace to return to tensors, and M to return to models. Only explicitly supported integer and Q8_0 decoders are enabled. See the weight-distribution guide and blocker matrix.

The bundled, derived fixture renders tensor metadata as stable-ID wireframe buildings grouped into Safetensors and GGUF districts. Building height uses a visible logarithmic stored-byte scale, so one large tensor cannot flatten the rest of the view. Click a building to inspect its name and heuristic role, drag to orbit, Shift-drag or middle-drag to pan, and use the wheel to zoom. A selects all tensors; S and G toggle their Safetensors or GGUF filter; L switches independently between footprint and height detail; R resets the view. The visible source panel and legend document provenance, architecture metadata, colors, scales, current filter/LOD state, and controls. See the Model Atlas native city for the data, ownership, bounded-detail, and platform contracts.

Click toggles a cell and Control-left-drag paints live cells. Plain left-drag orbits/tilts, Shift-left-drag or middle-drag pans, and the wheel zooms. Space runs/pauses, N steps, C clears, plus/minus changes speed, and B/H/I/T/G/U/R select block, beehive, blinker, toad, glider, Gosper gun, and seeded random. The same complete legend is visible inside the native window. The torus uses the same controls and presets; cells crossing either edge continue at the opposite edge, and clicking or painting follows the curved surface.

Click an empty square to move. Left-drag orbits/tilts, the wheel zooms, and Shift-left-drag or middle-drag pans; crossing the four-pixel drag threshold suppresses mark placement. X/O chooses the human mark, 1/2 chooses first or second, R restarts with those choices, and Escape closes. The board rules, perfect-play strategy, picking, choices, turns, hover, and scene arrays are MLPL-owned; the shared Rust host only normalizes input and renders generic lines.

The window keeps winit/wgpu on the main thread and runs sw-MLPL on a worker. controls.mlpl receives generic key/resize events and sends retained patches or view-only commands back to the renderer after initialization. Use W/S for width, arrows for height, A/D for length, +/- for signed speed, Space for pause, C for color, brackets for thickness, R for reset, and Escape to close. Left-drag orbits and tilts, the wheel zooms, and Shift-left-drag or middle-drag pans; these mappings and camera state are implemented in MLPL and shown in the window legend.

Run focused Rust or MLPL tests:

just rust-tests
just tests
just list-tests

The native demos include an editable Conway's Life plane. Its MLPL layer provides dead finite boundaries, whole-array B3/S23 evolution, owned cell updates, deterministic replacement presets, mouse cell editing, animation and speed controls, shared orbit/pan/zoom, and the complete visible control legend. connects to the existing native host through generic stable-ID bulk line arrays.

Run the mandatory pre-commit gate:

just check

The complete gate checks repository layout, .gitignore, tracked files, public/private namespaces, canonical MLPL formatting, MLPL module comments and function docstrings, Rust formatting, compilation, clippy, all Rust tests, native mlplunit tests, and whitespace. Set MLPL_FMT to an absolute formatter path when the adjacent ../sw-mlpl checkout is unavailable. The intentional panic test may print its panic-hook message; the test verifies that the panic is converted to ExtensionPanicked before it can unwind across the C ABI.

To run only the dynamic hello acceptance tests:

cargo build -p mlpl-extension-hello
cargo test -p mlpl-extension-loader --test hello_registration
cargo test -p mlpl-extension-loader --test manifest_resolution

Current status

The delivered repository proves:

  • fixed-layout ABI V1 values, errors, descriptors, and version negotiation;
  • bounded fail-closed metadata validation and host-owned metadata copies;
  • independent shared-library loading with library lifetime retention;
  • namespaced typed success, extension failure, and contained-panic calls;
  • deactivation that rejects later calls;
  • deterministic manifests, exact target selection, canonical path confinement, stable diagnostics, and duplicate/mismatch rejection;
  • typed function/default/return and native-type metadata with deterministic validation and stable help rendering;
  • bounded dense numeric arrays with validated dtype, rank, shape, byte strides, alignment, storage length, and one-call [N,3] acceptance;
  • extension-scoped, type-tagged generational handles with stale/cross-extension rejection and deterministic resource finalization;
  • macro-generated ABI descriptors/trampolines around safe Rust handlers, with hello containing no handwritten unsafe code;
  • a public MLPL facade kept separate from private native functions.
  • a deterministic MLPL wireframe-cube scene with independently adjustable dimensions, rotation speed, RGBA line color, and thickness;
  • a renderer-neutral Rust line-scene contract that validates bulk [N,3] positions and [M,2] edges before later GPU work.
  • a bounded renderer-neutral point-scene contract for [N,3] positions and parallel size/color/opacity/stable-ID arrays, with deterministic upload planning and explicit owned-byte accounting;
  • deterministic headless point projection, viewport/near-plane culling, far-to-near alpha ordering, stable-ID picking ties, and raster evidence;
  • a native wgpu/winit circular point-sprite pipeline with full stable IDs and an opt-in just point-cloud-smoke fixture on graphical macOS/Linux sessions;
  • atomic retained point replacements/patches and precision-safe stable-ID selection events, with application meaning reserved for MLPL;
  • a deterministic just point-cloud MLPL teaching app with selection, camera, patches, lifecycle, and a headless real-worker acceptance test;
  • a bounded just point-cloud-acceptance release probe and evidence matrix that separate CPU byte/timing observations from unmeasured GPU/Linux behavior;
  • a deterministic headless transform, perspective projection, clipping, and thick-line raster pipeline with portable PPM evidence.
  • a real headless _native3d provider with typed viewer lifecycle, bulk line arrays, state/size records, and explicit MLPL-supplied render state.
  • an MLPL-owned control reducer for dimensions, signed speed, pause/reset, palette, thickness, resize/close events, and deterministic bulk updates.
  • bounded/coalesced input, camera/picking, single-flight animation frames, and atomic retained-scene patches across every interactive native demo;
  • native tic-tac-toe, Life plane/torus, Model Atlas, disk usage, audio spectrum, and weight-distribution applications with MLPL-owned semantics;
  • an offline-first Rust/Yew microscope over schema-validated, hash-pinned MM01, LR01, and KM01 MLPL recordings with generic rank/shape rendering.

The opt-in wgpu/winit window is connected to the MLPL reducer through sw-MLPL's parked-main Port contract. Only owned event and scene values cross between the main-thread UI and worker interpreter. Dynamic loading by sw-MLPL, real unload/hot reload, facades, and compiled-provider startup remain future work.

Documentation

Repository workflow

Work is test-driven and tracked with AgentRail sagas. Changes are committed and pushed directly to main after just check; this repository does not use feature branches, pull requests, the gh CLI, or GitHub Actions for publication. See AGENTS.md for the complete process.

License

Copyright (c) 2026 Michael A Wright. Distributed under the MIT License.

Contributors

softwarewrighter

149 commits

Languages

Rust

93.9%

Shell

4.8%