go-joker

An optimized fork of go-joker (a Clojure-like Lisp interpreter) for inclusion in gi, a self-hosted coding agent.

Compatibility additions

This fork includes practical Babashka/let-go compatibility work beyond upstream Joker:

• pods and babashka.pods with bencode routing, subprocess lifecycle, dynamic vars, and JSON/EDN/Transit+JSON payloads.
• joker.edn plus edn alias namespace with read-string/write-string using the reader/printer without evaluation.
• Expanded joker.transit for pod-oriented Transit+JSON payloads.
• CLI entrypoint lives in cmd/joker; build with make cli (output: /workspace/tmp/go-joker/build/joker).
• Portable Babashka fixture suite via make bb-compat.
• Script-driven Babashka shim policy documented in docs/BABASHKA_SHIM_ASSESSMENT.md.
• Tracing/profile tooling documented in docs/TRACING.md; latest benchmark refresh in docs/BENCHMARK_RESULTS_2026-09-06.md, with profile audit context in docs/BENCHMARK_PROFILE_2026-05-12.md.

Performance

Performance — Joker vs Python vs Bun/JSC vs Goja vs let-go

vs. original Joker

Highlights

Fresh output-validated measurements from 2026-09-06, Intel Core i7-12700, Go 1.26.5. All four comparison runtimes were rerun; the previous snapshot is archived.

See benchmark results and methodology and the full sample summary. These microbenchmarks are not a general application-speed guarantee.

What's different from upstream Joker

Native integer codegen for recursive fns

Pure-integer recursive defn bodies (fib, tak) are compiled to fixed-arity native Go closures, eliminating all Object boxing and interface dispatch. fib(35) runs in 0.5s (53× faster than tree-walker).

IR bytecode interpreter (typed + boxed paths)

Hot loops and functions compile to a flat bytecode. Eligible primitive/string loops now run on a typed IR value stack, while collection-heavy or unsupported cases fall back to the boxed IR interpreter and then to the tree-walker.

WASM/wazero native compilation

Pure numeric loops compile further to WASM bytecode and execute via wazero's native code compiler. This achieves JIT-level performance (matching Bun/JSC) with zero CGo dependencies. The WASM bridge now supports value-producing if expressions inside numeric loops and fn-level loop bodies with init stores, so procedural raster kernels can stay in Joker code and still run through WASM.

Generic tail-call optimization

Self-recursive functions in tail position are automatically rewritten to recur at parse time, eliminating stack growth. A runtime trampoline handles cases the rewriter can't catch.

Transient vectors and maps

Loops that update non-escaping vectors or maps via assoc automatically use in-place mutation (Clojure-style transients), eliminating persistent copy/update overhead while preserving persistent results at loop return.

StringCursor native type

A zero-alloc O(1) string iterator with IR opcodes (irCursorChar, irCursorNext, irCursorDone). Cursor-based parsers run 3-3.5× faster than equivalent index-based code by eliminating per-character nth scanning and position arithmetic.

IntRange + seq-walking reduce

(range n) with integer arguments returns an IntRange that implements reduce directly without seq allocation. Reduce-over-range is 18× faster. Lazy seqs (LazySeq, ConsSeq, MappingSeq) also support fast reduce.

Full transducer semantics

map/filter/take transducer arities, transduce (3/4 arity), reduced/reduced?/ensure-reduced/unreduced, completing, eduction, and sequence 2-arity are implemented using a dedicated Reduced runtime type (not map-tag shims).

Evaluator fast paths

Numeric operations, binding resolution, and function dispatch all have type-specialized fast paths that avoid the generic Joker evaluation machinery.

Runtime introspection (joker.runtime)

Full IR/WASM/profiling introspection from Joker scripts: disassemble, analyze, wasm-diagnostic, escape-analysis, profile, benchmark, mem-stats, gc.

GIL-free concurrency

The Global Interpreter Lock has been removed. Goroutines run in true parallel on Go scheduler threads. Immutable data structures need no coordination. Atoms use per-atom mutexes. Concurrency primitives: alts!, timeout, future, promise, agent, pmap, pcalls, plus a clojure.core.async compatibility namespace with go-loop, put!/take!, pipe, merge, split, mult, and pub helpers. Channel close is idempotent and safe under concurrent callers; sends after close return false and takes from closed channels yield nil.

Additional namespaces / web runtime

• joker.ffi — no-cgo fixed C ABI calls to .so/.dylib/.dll libraries; contracts and SDL fluid sample
• joker.imaging — image processing (resize, crop, blur, overlay) via pure Go, with guarded image/color argument boundaries
• joker.term — raw terminal I/O, ANSI colors/styles, buffered frame rendering, key input (see docs/TERM.md)
• joker.svg — SVG generation + raster rendering, with guarded coordinate-vector handling
• joker.pdf — PDF document generation, with checked document-proc arities
• joker.random — random numbers (int, float, choice, shuffle, uuid, secure-bytes)
• joker.log — leveled logging (debug, info, warn, error)
• joker.http — persistent keep-alive HTTP client, Ring-style HTTP server, WebSocket and SSE/streaming response extensions
• joker.http.router — Bottle-style HTTP routing with path params, middleware, CORS (std/http/router/router.joke)
• joker.ai — experimental, dependency-free provider-neutral AI client example with SSE streaming, tools, structured output, and offline provider/auth fixtures (examples/ai/)

Clojure parity surface now implemented

• Protocols: public defprotocol, extend-type, extend-protocol, satisfies?, protocol method dispatch
• Records: public defrecord, generated ->Type/map->Type constructors, record?, protocol clauses
• Hierarchies: derive, underive, isa?, parents, ancestors, descendants, make-hierarchy
• Tagged literals/readers: #inst, #uuid, default-data-readers, *data-readers*, *default-data-reader-fn*
• Sorted collection API: sorted-map, sorted-set, sorted?, comparator, subseq, rsubseq
• Atom mutation parity: set-validator!, get-validator, add-watch, remove-watch, compare-and-set!
• Chunked seq API: chunk-buffer, chunk-append, chunk, chunk-cons, chunk-first, chunk-rest, chunk-next, chunked-seq?
• Unchecked arithmetic + primitive array helpers: unchecked-*, int-array, long-array, aget, aset, alength, aclone, make-array

SDL fluid simulation through FFI

An SDL2 example drives window, texture, render and event calls from Joker. A companion host supplies reusable simulation buffers. Build with make sdl-fluid; see instructions.

Architecture

Start with docs/START_HERE.md for the shortest contributor path through build, focused validation, API-stability, fuzz-smoke, and release-check expectations. The repository layout is being split along architectural boundaries in small, contract-tested slices rather than by broad rewrites. The module identity is github.com/rcarmo/go-joker/v42, the CLI lives in cmd/joker, tracing/IR/WASM/runtime leaf helpers are under core/{trace,ir,wasm,runtime}, data-only generated payloads and registries are under core/generated, and collection/reader construction is routed through guarded adapters before any package moves. docs/refactor/README.md and its linked notes describe the current boundary inventory and historical split context; active work should stay in issue/plan tooling unless it needs durable user-facing documentation. Standard validation now includes generated-file/bootstrap, import-identity, non-goal, layout, refactor-internal, native-int, error-handling, core object/protocol, runtime execution, std native-boundary, docs, Babashka fixture, test, and vet guardrails.

• WASM path: pure integer/float loops → wazero JIT → native code (~0.2ms)
• Typed IR path: primitive/string/cursor loops → irValue stack, zero-boxing (~2–8ms)
• Boxed IR path: collections, fn calls, transients → []Object interpreter (~10–40ms)
• Tree-walker: full Clojure semantics (macros, special forms, I/O)
• Fallback chain: WASM → Typed IR → Boxed IR → Tree-walker (automatic)
• WASM raster example: examples/graphics/fractal-flame.joke renders Mandelbrot, Tricorn, and cubic flame variants through joker.jit/compile-wasm plus joker.imaging/from-rgba32-domain-fn (about 1024×1024 in ~260ms per kernel on the current dev host).
• Wiki/static-site example: examples/wiki/static.joke ports the wiki/static subset of rcarmo/sushy from Hy to Joker: folder-per-page content, RFC2822-style front matter, Markdown/plain/HTML rendering, dynamic serving via joker.http/start-server, static build output, Atom feed, sitemap, aliases/interwiki mappings, and static asset copying.

The execution-tier audit documents numeric promotion, exact arithmetic, callback/fallback safety, WASM eligibility restrictions and remaining limitations. The September benchmark refresh records current measurements separately from historical charts.

Using from Go

Install the module with Go 1.25 or newer:

go get github.com/rcarmo/go-joker/v42@v42.12.1

Package imports include the major version, for example:

import "github.com/rcarmo/go-joker/v42/core"

To install the CLI:

go install github.com/rcarmo/go-joker/v42/cmd/joker@v42.12.1

v42.11.3 is the first v42 release with a valid Go module path. Existing Go consumers must add /v42 to their imports and run go mod tidy. Earlier v42 tags lacked this suffix and cannot be selected as module versions; those tags remain unchanged. Repository and source URLs still use github.com/rcarmo/go-joker.

Building & testing

go test ./core              # run all tests
go test ./benchmarks/core -bench .     # run all benchmarks
make core-contract-check    # focused object/protocol split guardrails
make runtime-contract-check # focused runtime/execution-envelope guardrails
make std-contract-check     # focused std native-boundary guardrails
make parity                 # run language parity suite + refresh divergence matrix
make jank-subset            # run imported jank-lang/clojure-test-suite smoke subset

Web notebooks

Joker includes a local EDN notebook runner/server for Mathematica-style exploratory documents with Observable-style manual dependency metadata:

joker notebook new example.edn --title Demo
joker notebook new example.edn --title Demo --serve --open -p 8080
joker notebook demo rich-demo.edn
joker notebook rich-demo.edn -p 8080 --open
joker notebook example.edn -p 8080       # local browser UI
joker notebook run example.edn           # headless execution, updates inline outputs
joker notebook run example.edn --no-save --summary --fail-on-error
joker notebook validate example.edn      # format/cycle validation
joker notebook status example.edn        # status/size JSON
joker notebook deps example.edn          # dependency graph/cycles JSON
joker notebook snapshots example.edn     # list recovery snapshots
joker notebook export example.edn -o report.md

Notebook files are regular EDN maps with :format :joker/notebook; outputs are stored inline by default for self-contained agent/debug reports. The browser UI uses local vendored CodeMirror/ECharts/Mermaid assets (no CDN), supports Joker/Markdown highlighting and Joker symbol autocomplete, and renders simple Mermaid flows with rounded boxes plus rounded orthogonal arrows. See docs/NOTEBOOKS.md.

Runtime documentation lookup

Joker includes a pydoc-style runtime documentation frontend backed by the live namespace metadata:

joker doc                         # list documented namespaces as Markdown
joker doc joker.string            # namespace documentation
joker doc joker.core/first        # qualified symbol documentation
joker doc search websocket        # full-text-ish namespace/var search
joker doc --format json first     # agent/tool-friendly JSON
joker doc serve -p 8080          # local browsable docs with search
joker doc serve --addr 127.0.0.1:8080

The CLI output is Markdown by default so agents and terminals can consume it directly. The local HTTP view uses the same runtime index and does not embed the generated static HTML/PNG documentation payload.

Benchmarks

Note: The CLBG programs were chosen as a starting point for optimizing the IR and WASM compilation pipeline, not because they represent realistic workloads. They stress specific interpreter bottlenecks (arithmetic loops, recursion, allocation, string processing) that guided the optimization work. Real-world gi scripts will have different profiles — the gains here prove the execution machinery works, not that every Joker program runs 500× faster.

# Full CLBG suite + micro benchmarks
go test ./core -run '^$' -bench 'BenchmarkCLBG|BenchmarkEval|BenchmarkWasm' -benchmem -benchtime=5x

# Cross-language comparison
python3 benchmarks/cross_lang_bench.py
bun benchmarks/cross_lang_bench.js

# Regenerate charts
go run ./tools/benchmarks/generate_svg.go ./benchmarks

# Check the same broad CI benchmark smoke ceilings used by GitHub Actions
go test ./benchmarks/core -bench 'BenchmarkCall|BenchmarkFib|BenchmarkTak|BenchmarkLoop|BenchmarkReduce|BenchmarkClosure|BenchmarkMap|BenchmarkVector|BenchmarkTransduce' -benchmem -benchtime=1s -count=3 > bench-results.txt
tests/benchmark_ci_check.sh bench-results.txt

Documentation

• docs/START_HERE.md — primary contributor entry point for clone/build/test orientation, focused validation, API-stability, fuzz-smoke, and release-check expectations
• examples/README.md — runnable examples grouped by agents/AI, graphics, terminal games, wiki/static serving, and notebooks
• examples/ai/README.md — experimental provider-neutral AI client, supported providers, security model, and offline checks
• docs/API_STABILITY.md — public namespace/API stability classifications
• docs/RELEASE_CHECKLIST.md — patch release checklist and validation commands
• docs/BENCHMARK_CI.md — CI benchmark smoke guard policy and local reproduction
• docs/RUNTIME_DOCS.md — joker doc Markdown/JSON lookup and local HTTP docs server
• docs/IMAGING.md — joker.imaging image processing API and Bun Image parity notes
• docs/FFI.md — optional trusted C ABI calls and SDL example
• docs/TERM.md — joker.term terminal I/O, ANSI rendering, buffered frames, key input
• docs/NOTEBOOKS.md — EDN notebooks, headless runs, Markdown export, and local notebook server
• docs/refactor/README.md — repository split plan and target folder structure
• docs/refactor/code-structure.md — package/module and coverage audit
• docs/refactor/module-structure-audit.md — current Go module/package layout and next structural improvements
• docs/refactor/ir-boundary.md — IR package boundary inventory
• docs/refactor/ir-program-split.md — next-step IR model/envelope split design
• docs/refactor/generated-bootstrap-contract.md — generated bootstrap data-only boundary design
• docs/refactor/runtime-execution-contract.md — runtime/executor metadata split prerequisites
• docs/refactor/reader-construction-contract.md — reader construction/tagged literal split prerequisites
• docs/refactor/core-split.md — collections/reader/runtime/WASM split candidates
• docs/refactor/object-protocol-contracts.md — object/protocol contracts blocking broad core moves
• docs/refactor/generated-boundary.md — generated-code boundary inventory and guardrails
• docs/OPTIMIZATION_REPORT.md — full technical report (phases, trade-offs, outcomes, suggested git history)
• docs/WEB_RUNTIME_AND_NAMESPACES.md — WebSocket/SSE usage + router + all new namespaces
• benchmarks/README.md — benchmark data and chart regeneration
• docs/PARITY_STATUS.md — let-go benchmark parity + language compliance status
• docs/DIVERGENCE_MATRIX.md — latest compliance matrix (271/271 pass)
• docs/PERFORMANCE_PLAN.md — optimization roadmap and milestones

Upstream

Based on the original Joker v1.7.2 codebase plus selected upstream feature ports. This fork is v42.12.1. Release notes: docs/RELEASE_NOTES_v42.12.1.md. Audit report: docs/AUDIT_REPORT_2026-07-09.md. Original README preserved as docs/archive/ORIGINAL_README.md.

Why v42?

Because 42 is the answer, and we didn't want to collide with upstream version numbers.

License

Same as upstream Joker (EPL-1.0).