jolt-openldk

[!WARNING] Alpha. This project is days old and still finding its shape. The API, the wire protocol between jolt and the bridge, the build layout and the supported platforms may all change without notice or a deprecation path. Don't build anything you can't easily rework on top of it yet.

Java libraries from jolt, with no JVM in the process.

OpenLDK is Anthony Green's Java runtime written in Common Lisp. It reads .class files, translates the bytecode into Lisp and lets SBCL compile that to machine code. This project loads SBCL as a shared library into a jolt process, starts it on a core with OpenLDK already warmed up, and gives Clojure a small API for calling into it.

(require '[net.b12n.jolt.openldk :as ldk])

(ldk/init! {:classpath ["classes" "lib/some.jar"]})

(ldk/call-static "java.lang.Math" "sqrt" "(D)D" 2.0)          ;=> 1.4142135623730951

(ldk/with-ref [xs (ldk/new-object "java.util.ArrayList" "()V")]
  (ldk/call xs "add" "(Ljava/lang/Object;)Z" "a")
  (ldk/call xs "add" "(Ljava/lang/Object;)Z" 2)
  (ldk/to-string xs))                                          ;=> "[a, 2]"

Verified on macOS arm64, Linux x86-64 and Linux arm64, with jolt 0.8.20, SBCL 2.6.9 and JDK 25, and checked on every push by CI. Platforms has the detail.

Is this the right tool?

Probably not, if what you want is a Java library at full speed. A JVM over JNI is faster, far more complete, and what everyone else uses. OpenLDK says so about itself: it is for when you are in Lisp and need that one Java library, and it isn't trying to compete with HotSpot.

The reasons to want it anyway are narrower. Nothing here starts a JVM, so there is no second runtime with its own heap tuning and startup cost, and the whole thing lives in one process that you control. The catch is that it still reads the JDK's class library at run time, so a JDK 25 has to be installed even though it never runs.

How it fits together

Everything runs in one process. jolt calls down through a small C shim into SBCL, and Java calls back up through one jolt callback.

flowchart TB
  subgraph jolt["jolt process"]
    api["net.b12n.jolt.openldk<br/>init!, call-static, new-object, call, implement, release!"]
    wire["net.b12n.jolt.openldk.wire<br/>request text out, reply text back"]
    tramp["upcall trampoline<br/>one :collect-safe ffi/callback"]
    api --> wire
  end
  subgraph shim["libjoltopenldk (bridge/ldk.c)"]
    shimcall["ldk_call / ldk_free"]
    shimup["ldk_upcall pointer<br/>set by ldk_set_upcall"]
  end
  subgraph sbcl["libsbcl + openldk.core"]
    bridge["bridge/bridge.lisp<br/>reads requests, prints replies, makes proxies"]
    ldk["OpenLDK<br/>JIT-translates each method on first call"]
  end
  jdk[("JDK 25 class library<br/>$JAVA_HOME/jmods or lib/modules")]

  wire -- "jolt.ffi, by name" --> shimcall
  shimcall -- "ldk_entry, filled in by SBCL at startup" --> bridge
  bridge --> ldk
  ldk -- reads --> jdk
  bridge -- "Java calls a proxy" --> shimup
  shimup --> tramp
  tramp -- "your fn" --> api

One call into Java, and one callback out of it, look like this:

sequenceDiagram
  participant C as Clojure
  participant S as C shim
  participant B as bridge.lisp
  participant J as OpenLDK (Java)
  C->>S: ldk_call with (:static java.util.Arrays sort ...)
  S->>B: ldk_entry
  B->>J: Arrays.sort(xs, comparator)
  J->>B: comparator.compare(a, b)
  B->>S: ldk_upcall with (:call 7 compare ...)
  S->>C: trampoline runs your fn
  C-->>B: reply (:ok (:i -1))
  B-->>J: -1
  J-->>B: sort returns
  B-->>C: reply (:ok (:void)), malloc'd, freed by ldk_free

Three things make the middle layer work, and each one cost a failed run to find.

1. SBCL can start from a library. save-lisp-and-die :callable-exports '(ldk_entry) saves a core with no toplevel. When initialize_lisp starts it, SBCL writes the address of the ldk_entry callable into the C global of that name and returns to the caller rather than starting a REPL.
2. SBCL finds that global with a process-wide dlsym. jolt.ffi/load-library loads RTLD_LOCAL, which hides it, and SBCL then dies at startup with UNDEFINED-ALIEN-VARIABLE-ERROR and takes jolt down with it. So ldk_init first re-opens itself and libsbcl with RTLD_NOLOAD | RTLD_GLOBAL.
3. jolt binds C functions by name and can't call through a pointer, so ldk_call is a named C function that calls the pointer.

Every request is one line of text. Strings, numbers and booleans travel as tagged Lisp forms, (:static "java.lang.Math" "sqrt" "(D)D" ((:d 2.0))). The reply is printed so that clojure.edn reads it unchanged, (:ok (:d 1.4142135623730951)). The bridge reads requests with *read-eval* off and interns nothing outside KEYWORD. wire.clj has the full grammar in its docstring.

Building

export JAVA_HOME=/path/to/jdk-25      # mise: ~/.local/share/mise/installs/java/25.0.2
bb build                              # runs bridge/build.sh

bridge/build.sh needs sbcl (any recent one, used only to compile SBCL itself), ocicl, git and cc on PATH. It writes nothing outside ~/.cache/jolt-openldk, or $JOLT_OPENLDK_HOME if that is set. It:

• clones SBCL 2.6.9 and builds it with its runtime as libsbcl. Homebrew's SBCL isn't built that way, so there is no shortcut.
• clones OpenLDK at commit 23da184e, installs its dependencies with ocicl, and dumps openldk.core with bridge/bridge.lisp loaded.
• compiles the shim, and writes manifest.edn naming what went in.

From a clean cache it took about three and a half minutes here, two of them for SBCL, which is built once and reused after that. Set OPENLDK_SRC=/path/to/checkout to clone OpenLDK from a local copy instead of GitHub. The checkout itself is never modified.

Behind an HTTPS proxy that can't tunnel to ghcr.io, ocicl install fails with Unable to establish HTTPS tunnel through proxy. Run the build with the proxy variables unset.

The result in dist/ is a 146 MB core, a 412 KB libsbcl.dylib and a 34 KB shim.

Using it

init! starts SBCL and sets the classpath, which OpenLDK only accepts once per process. Calling it again with the same classpath does nothing. Calling it with a different one throws. Before starting anything it checks that JAVA_HOME is a JDK 25 with its class library and that every classpath entry exists, because OpenLDK reacts to the first by exiting the process and to the second by failing on the first class load, after the classpath can no longer change. If init! fails after SBCL was started, later calls say so and the process has to be restarted, since SBCL can't be started twice.

Methods are named the way JNI names them, a name plus a descriptor, because Java overloads. The descriptor's return type also tells the bridge how to hand back the result:

A JavaRef keeps its object alive until release!. with-ref releases on the throwing path too. Every call that returns an object hands back a new handle, including one that returns the same object (StringBuilder.append), so release those as well.

Arrays go in as vectors and come back as handles, because an array is mutable and Java may keep it:

(ldk/call-static "java.util.Arrays" "toString" "([I)Ljava/lang/String;" [3 1 2])  ;=> "[3, 1, 2]"

(ldk/with-ref [xs (ldk/new-array "I" [3 1 2])]   ; an int[] you keep
  (ldk/call-static "java.util.Arrays" "sort" "([I)V" xs)
  (ldk/array->vec xs))                            ;=> [1 2 3]

array->vec converts by component type: int[] and long[] to integers, byte[] to signed integers as in Java, boolean[] to booleans, char[] to chars, String[] to strings. Object[] elements convert like returned values, so strings and boxed scalars become values and any other object, a nested array included, becomes a JavaRef to release. Only a vector is taken for an array, never a seq. A float[] reads back widened to double, so 0.1f arrives as 0.10000000149011612, the same as a returned float. A vector inside an Object[] is refused, since nothing says what array it should become: build it with new-array and put the handle in instead. Descriptors, for new-array and in method descriptors, are parsed strictly, so int, V or a dotted Ljava.lang.String; is an error rather than a strange array.

Java can call Clojure too. implement makes a Java object implementing one or more interfaces whose methods run Clojure fns:

(ldk/with-ref [desc (ldk/implement "java.util.Comparator" (fn [a b] (compare b a)))
               xs   (ldk/new-array "Ljava/lang/String;" ["b" "c" "a"])]
  (ldk/call-static "java.util.Arrays" "sort" "([Ljava/lang/Object;Ljava/util/Comparator;)V" xs desc)
  (ldk/array->vec xs))                            ;=> ["c" "b" "a"]

(ldk/implement "java.util.Iterator" {"hasNext" (fn [] ...) "next" (fn [] ...)})

A fn handles every method, which suits a functional interface. A map picks a fn by method name, or by name plus descriptor for an overload. The object passes checkcast for its interfaces, and their default methods work, so Comparator.reversed() on it calls your compare. It's built the way OpenLDK builds classes for Java lambdas: a class made at run time whose superclasses are Object and the interfaces, with one method per abstract method.

• Arguments arrive converted like returned values. Objects among them are JavaRefs borrowed for the call: they're released when the fn returns, so copy out what you need first.
• The result converts by the method's return type, the same way an argument would, and a void method ignores it. A result that doesn't fit is a RuntimeException in Java, which Java code can catch.
• Objects the fn gets from its own calls into Java are ordinary JavaRefs it can keep. Only the arguments are borrowed.
• If the fn throws, Java gets a RuntimeException with its message. If that exception comes back out through the call you made, you get your original throwable back.
• The fn may call into Java on the same thread. Java threads can call in too. Every call from Clojure into Java takes one lock, so a fn on another Java thread that calls into Java waits until your thread's current call finishes. That's a deadlock only when your call is itself waiting for that Java thread, through join, a latch or a monitor. In that case the fn must not call into Java.
• toString gives jolt-openldk proxy for java.util.Comparator #7, and hashCode and equals are Object's. None of the three reach Clojure. getClass() throws ClassNotFoundException, since no Java class stands behind the one made at run time, so Java code that reflects on the object will fail.
• Like any JavaRef, the object lives until release!, along with its fn. A proxy passed inline and never released stays for the life of the process.
• release! on the object drops its fn as well, so Java calling a stored copy later gets a RuntimeException rather than a call into a fn that's gone.

A Java exception arrives as an ex-info:

(ex-data e)
;=> #:java{:class "java.lang.IllegalArgumentException",
;          :message "negative: -1",
;          :string "java.lang.IllegalArgumentException: negative: -1"}

Mistakes on the Clojure side, such as a missing class, a wrong argument count, or 99999999999 passed to an int, carry :openldk/error instead.

bb tour runs examples/tour, which works through Math, StringBuilder, TreeMap, java.time and a class of its own. Its date arithmetic and hex output were cross-checked against date and printf, since OpenLDK's correctness is the thing on trial.

What was measured

On the machine above, 2026-10-09:

• Starting SBCL on the core (ldk_init) took 47 ms. The tour's whole init!, which adds OpenLDK's classpath setup, took 244 ms.
• A call that is already compiled: about 87 µs (2,000 Math.max calls in 175 ms). Nobody has profiled where that goes. The request and reply text is the obvious suspect, and a binary encoding would be the first thing to try if it ever matters.
• A method's first call pays for its JIT translation. That is OpenLDK's cost and runs from milliseconds to seconds depending on how much of the JDK the method pulls in.
• A callback from Java into Clojure: about 49 µs (a Comparator sort of 1,000 strings made 8,528 calls in 419 ms, sorting correctly).
• jolt keeps working around it. Its collector, its other threads, and Ctrl-C (exit 130, the same as without OpenLDK) all behaved normally. A call from a second jolt thread works too.

What doesn't work yet

• Bulk arrays. Every element crosses as text. Measured with 100,000 elements, a byte[] or an int[] took about 60 ms to build from a vector and 260 to 330 ms to read back across two runs, and came back equal. A million ints took 0.7 s in and 3.4 s out. Fine for configuration, slow for image data.
• Implementing a class. implement takes interfaces only. Subclassing an abstract class from Clojure isn't supported.
• Wrapped callback exceptions. The original Clojure throwable comes back only when the RuntimeException itself reaches your call. If Java wraps it in another exception first, you get the wrapper as an ex-info.
• Concurrency. Calls are serialised behind one lock, because OpenLDK's own thread safety hasn't been looked at.
• Some failures take the host down. Java's System.exit exits the jolt process, since it is the only process there is (measured; Runtime.halt presumably does too, untested). So does SBCL on heap exhaustion or a corrupt core. ldk_init checks the core can be opened, and init! checks JAVA_HOME, before either reaches SBCL. Nothing else is covered. The heap is a fixed 8 GB reservation.
• JavaRef arguments aren't class-checked. Scalars, strings and vectors are checked against the parameter type, but a JavaRef goes to any reference parameter, so passing the wrong class surfaces later as whatever error the Java code raises.
• OpenLDK's own gaps. It's a young runtime. Two string bugs found while building this are filed upstream as #13 and #12, with notes in docs/openldk-upstream-notes.md. One is worked around here. The other, ("é✓".toUpperCase() + "!") throwing a NullPointerException that escapes catch (Throwable), is not.
• stdout ordering. Java's System.out and jolt's *out* share file descriptor 1 but buffer separately. The bridge flushes after every call, but output written during a call can still land before output jolt had buffered before it.

Platforms

Don't use bb linux with JOLT_OPENLDK_LINUX_PLATFORM=linux/amd64 on Apple silicon. Under Docker's x86-64 emulation, compiling SBCL 2.6.9 stops in src/code/irrat with the cross-compiler's Unimplemented., while the same build passes on real x86-64 hardware in CI. The emulator is at fault, not Linux.

CI (.github/workflows/ci.yml) runs lint and the three platform jobs on every push to main and every pull request. Each runs bridge/build.sh, the tests with JOLT_OPENLDK_REQUIRE=1 and the tour, with versions pinned and the SBCL build cached.

bb linux builds linux/Dockerfile for the machine's own architecture and runs linux/check.sh in it, with this checkout mounted read-only and the SBCL build cached in a Docker volume. jolt and ocicl publish no Linux arm64 binaries, so on arm64 the image builds both from their release tags. jolt brings its own pinned Chez Scheme, because Ubuntu's is too old (10.0.0 lacks vector-copy!). The first build takes about 8 minutes and later ones reuse it. JOLT_OPENLDK_LINUX_PLATFORM=linux/amd64 forces a platform.

On a network that inspects TLS, Zscaler for instance, ocicl can't fetch from ghcr.io inside the container, because the proxy re-signs that site with its own root. bb linux copies the host's CA bundle (JOLT_OPENLDK_CA_FILE, or else SSL_CERT_FILE) into linux/extra-ca/, which git ignores, and the image trusts it. With neither set, nothing is added.

Tests

bb test     # wire tests always; the OpenLDK tests skip without a build
bb gates    # lint, then all tests with JOLT_OPENLDK_REQUIRE=1 so a missing build fails, then the tour
bb linux    # the same build, tests and tour on Linux in Docker; slow, so not part of gates

Last run: wire tests 11 tests and 65 assertions, OpenLDK tests 17 tests and 147 assertions, all passing.

Licence

Two licences, by file, with NOTICE saying which is which.

• bridge/bridge.lisp is GPL-3.0-or-later with the Classpath exception (COPYING, COPYING.CLASSPATH-EXCEPTION), the same as OpenLDK. It has to be, since it's loaded into OpenLDK's image and calls its internals.
• Everything else is EPL-2.0 (LICENSE), like jolt, with GPL-2.0-or-later plus the Classpath exception named as a Secondary License.

Every source file names its licence in an SPDX-License-Identifier line. bridge/build.sh downloads and builds SBCL and OpenLDK but copies neither into this repository, so its output in dist/ carries their licences as well. NOTICE has the details.