Inkwell

A handwritten-formula recognition workstation that runs entirely in your browser. Sketch a math expression or upload an image, and a local vision-language model transcribes it to LaTeX. There is no server round-trip, and no formula image leaves your machine.

Inkwell is built on WebGPU and ONNX Runtime Web, so the inference pipeline runs locally against LiquidAI LFM2.5-VL, FastVLM, and Gemma 4 models.

Live demo: https://ink.aungzm.com/

Stack

• React 19 + Vite 7 + TypeScript for the UI
• onnxruntime-web (WebGPU) for in-browser model execution
• @huggingface/transformers for tokenizer/processor utilities
• perfect-freehand for natural ink strokes
• KaTeX for LaTeX rendering
• A small built-in solver for arithmetic expressions and single-variable linear equations

Getting started

pnpm install
pnpm dev

Then open the dev server URL printed in the terminal.

Browser support

Use a Chromium-based browser (Chrome, Edge, Brave, Arc, Opera). WebGPU is enabled by default in Chromium since v113 and the ONNX Runtime Web pipeline is well exercised there.

• Safari — WebGPU shipped in 18.4 but coverage of the ops Transformers.js needs is still partial; expect crashes or fallback errors.
• Firefox — WebGPU is behind dom.webgpu.enabled and considered experimental. Inkwell may load the runtime but fail during inference.

Supported models

• LiquidAI LFM2.5-VL 450M ONNX — default model for quick local recognition
• LiquidAI LFM2.5-VL 1.6B ONNX — larger WebGPU model for higher-capacity recognition
• FastVLM 0.5B ONNX — compact alternate adapter
• Gemma 4 E2B ONNX — experimental adapter

System requirements

• RAM: 8 GB minimum, 16 GB recommended for compact models. Use a machine with more headroom for the 1.6B model.
• GPU: any discrete GPU or modern integrated GPU (Intel Iris Xe, Apple M-series, AMD Radeon 6xxx+, NVIDIA GTX 10-series+) with WebGPU support.
• Disk: ~600 MB free per compact model, and several GB for the 1.6B ONNX model cache.

First load

The first time you pick a model, the browser downloads its ONNX weights from the HuggingFace CDN. Compact models are roughly 400–600 MB, depending on quantisation. The LFM2.5-VL 1.6B ONNX repository is much larger, so expect a longer first load and more browser cache use. Once cached by the browser, subsequent loads are much faster.

Other scripts:

pnpm build      # type-check + production build
pnpm preview    # preview the production build
pnpm test       # run vitest

How it works

1. You draw on the canvas with the Ink or Erase tool.
2. After a brief pause, the strokes are rasterized into a tight crop.
3. The active VLM adapter runs locally on WebGPU and returns LaTeX.
4. The result is rendered with KaTeX and the raw LaTeX source is shown alongside.
5. If the expression is numeric, or a linear equation with one variable, Inkwell shows the computed result.

Model adapters live in src/vlm/ and implement a common VLMAdapter interface, so additional models can be slotted in.

Roadmap

Available now

• Handwritten formula recognition to LaTeX (LiquidAI LFM2.5-VL 450M)
• LiquidAI LFM2.5-VL 1.6B as a larger local adapter
• FastVLM 0.5B as an alternate adapter
• Gemma 4 E2B as an experimental adapter
• Fully local, in-browser inference via WebGPU + ONNX Runtime Web
• Pencil + eraser tools with adjustable stroke width and ink tone
• Auto-crop and pause-detection on the drawing canvas
• Upload a formula image and optionally annotate it before recognition
• Live KaTeX preview of the recognized formula
• Raw LaTeX source panel
• Numeric evaluation and simple linear equation solving

Planned

• History panel — scroll back through previous recognitions and restore any prior result to the canvas
• Copy LaTeX — one-click copy of the LaTeX source to the clipboard
• Richer symbolic math — simplify, differentiate, and integrate recognized expressions
• Larger canvas — resizable / multi-row workspace for longer derivations

Have an idea? Open an issue at https://github.com/aungzm/inkwell/issues.

License

MIT