unitas

Composing parsers into a unified whole. A lightweight, TypeScript-first parser combinator library — write small parsers, combine them into big ones, get typed results back.

npm install unitas

import { grammar, run } from 'unitas';
import { choice, inner, separatedBy } from 'unitas/combinators';
import { char, regex } from 'unitas/terminals';
import { letters } from 'unitas/primitives';

const csv = grammar({
    row: (p) => separatedBy(p.value, char(',')),
    value: (p) => choice(p.quoted, p.unquoted),
    quoted: () => inner(char('"'), regex(/^[^"]*/), char('"')),
    unquoted: () => letters,
});

run(csv.row, 'a,b,c'); // ['a', 'b', 'c']
run(csv.row, '"a,b",c'); // ['a,b', 'c']

• Zero dependencies, ESM-only, fully typed
• Tree-shakeable — five entry points, import only what you touch
• 134 composable exports — grouped by what they do, indexed below
• Mutual recursion out of the box via grammar, no forward declarations
• Errors that point at the problem — line, column and what was expected

Note: This library is in active development. The API may change before v1.0.0.

Contents

• Entry points
• Which function do I need?
• Core concepts
• Error reporting
• More examples
• API index
• Migrating from 0.2.x

Entry points

Nothing is re-exported across entry points, so your bundler only ever sees what you import. Code shared between them ships once, however many you import from.

Which function do I need?

The library is broad on purpose: each function does one small thing, so grammars read like the format they parse. This table is the fastest way in — start from the intent, then look the name up in the API index.

Matching a single thing

Most primitives come in a singular and a plural form: letter matches one, letters matches a run of them. The same holds for digit/digits, alphaNum/alphaNums, hexDigit/hexDigits, octDigit/octDigits, lowercase/lowercases, uppercase/uppercases, space/spaces and whitespace/whitespaces. Need to know how far into the input you are? position.

Repeating

Choosing and branching

Sequencing and pulling values out

Shaping the result

Grammars, recursion and errors

Core concepts

The Parser type

A Parser<T> reads input from an offset and returns a Result<T>. That is the whole abstraction — everything else in this library either produces one or wraps one. Both extra arguments are optional, so a parser can still be called with just an input.

type Parser<T> = (input: string, index?: number, ctx?: Context) => Result<T>;

Parsers never slice the input. They pass the same string down and move an integer instead, which is what lets an error know where in the original source it happened.

ctx is the parse context, created by run and parse. It is where error information accumulates — see Error reporting. If you write your own combinator, pass it on to every parser you call.

The Result type

type Success<T> = { ok: true; value: T; index: number };
type Failure = { ok: false; index: number; expected: readonly string[] };
type Result<T> = Success<T> | Failure;

{ ok: true, value: 'hello', index: 5 }
       │           │               │
       │           │               └── how far this parser got
       │           └── the parsed value
       └── always true for success

index is the important part: it is how input gets consumed and how parsers chain. A failure reports the same index, plus the set of things that would have matched there:

{ ok: false, index: 6, expected: ['digit'] }

Backtracking is free

Combinators hand every alternative the same offset they started with, so a branch that fails can never leave the cursor moved. choice(a, b) always offers b the same starting offset, however far a got.

If you are coming from Parsec, this is why there is no try/attempt here — backtracking is unconditional, so there is nothing to opt into. What a failed branch does leave behind is its expectation, recorded in the parse context at the offset it reached, so a later error can still mention it.

Writing one by hand

Most of the time you compose existing pieces, but nothing stops you from dropping down a level:

import { create, success, failure } from 'unitas';

const parser = create<string>((input, index = 0, ctx) => {
    if (input.startsWith('hello', index)) {
        return success('hello', index + 5);
    }

    return failure(ctx, index, '"hello"');
});

Passing ctx to failure is what lets this parser's expectation appear in an error message even when a choice backtracks past it. Do the same for any parser you call:

const twice = <T>(parser: Parser<T>) =>
    create<[T, T]>((input, index = 0, ctx) => {
        const first = parser(input, index, ctx);
        if (!first.ok) return first;

        const second = parser(input, first.index, ctx);
        if (!second.ok) return second;

        return success([first.value, second.value], second.index);
    });

Grammars

grammar hands every rule a proxy p, so rules can reference each other — including themselves — without forward declarations or lazy boilerplate.

import { grammar, run } from 'unitas';
import { chainLeft1, choice, map, sequence } from 'unitas/combinators';
import { char } from 'unitas/terminals';
import { digits } from 'unitas/primitives';

type Math = { expr: number; term: number; value: number };

const g = grammar<Math>({
    expr: (p) =>
        chainLeft1(
            p.term,
            map(char('+'), () => (l: number, r: number) => l + r),
        ),
    term: (p) =>
        choice(
            p.value,
            map(sequence(char('('), p.expr, char(')')), ([, v]) => v),
        ),
    value: () => digits,
});

run(g.expr, '1+2+3'); // 6
run(g.expr, '(1+2)'); // 3

Error reporting

When a parse fails, run throws a ParseError that says where and what was expected:

import { run } from 'unitas';
import { separatedBy, sequence } from 'unitas/combinators';
import { char } from 'unitas/terminals';
import { digits, letters } from 'unitas/primitives';

const pair = sequence(letters, char('='), digits);
const config = separatedBy(pair, char('\n'));

run(config, 'host=1\nport=8080\ndebug=yes');
// ParseError: 3:7 expected digit, found 'y'
//   3 | debug=yes
//     |       ^

The position is the furthest offset any branch reached, not wherever the parse happened to stop. That distinction is what makes the message useful: separatedBy succeeded with two pairs and left the rest unconsumed, but the error still points at the y that actually broke it.

Alternatives are reported together, since choice unions the expectations of every branch it tried at the same offset:

run(sequence(choice(string('let'), string('const')), char(' ')), 'lot x');
// ParseError: 1:1 expected 'const' or 'let', found 'l'
//   1 | lot x
//     | ^

Use parse when you would rather branch on the failure than catch it:

import { parse } from 'unitas';

const result = parse(config, 'host=1\nport=oops');

if (!result.ok) {
    result.line; // 2
    result.column; // 6
    result.expected; // ['digit']
    result.message; // the formatted excerpt above
}

label replaces the expectations of a whole parser with one description, so errors can talk about your grammar instead of its characters:

import { label } from 'unitas';

run(label(pair, 'a key=value pair'), '!!!');
// ParseError: 1:1 expected a key=value pair, found '!'

More examples

JSON value — a tagged union of scalars

import { grammar, run } from 'unitas';
import { choice, quoted, value } from 'unitas/combinators';
import { string } from 'unitas/terminals';
import { bool, digits, letters } from 'unitas/primitives';

const json = grammar({
    value: (p) => choice(p.string, p.number, p.bool, p.null),
    string: () => quoted(letters),
    number: () => digits,
    bool: () => bool,
    null: () => value(string('null'), null),
});

run(json.value, '"hello"'); // 'hello'
run(json.value, '42'); // 42
run(json.value, 'true'); // true
run(json.value, 'null'); // null

Query string — separated pairs folded into an object

import { grammar, run } from 'unitas';
import { map, outer, separatedBy } from 'unitas/combinators';
import { char } from 'unitas/terminals';
import { letters } from 'unitas/primitives';

type Query = {
    params: Record<string, string>;
    param: [string, string];
    key: string;
    value: string;
};

const query = grammar<Query>({
    params: (p) =>
        map(separatedBy(p.param, char('&')), (pairs) =>
            Object.fromEntries(pairs),
        ),
    param: (p) => outer(p.key, char('='), p.value),
    key: () => letters,
    value: () => letters,
});

run(query.params, 'foo=bar&baz=qux'); // { foo: 'bar', baz: 'qux' }

INI section — a header plus an entry, with the punctuation dropped

import { grammar, run } from 'unitas';
import { bracketed, map, outer, sequence } from 'unitas/combinators';
import { char, regex } from 'unitas/terminals';
import { letters, nl } from 'unitas/primitives';
import { pick } from 'unitas/utils';

const ini = grammar({
    section: (p) =>
        map(
            sequence(bracketed(p.name), nl, p.entry),
            pick(0, 2),
            ([name, entry]) => ({ name, entry }),
        ),
    name: () => letters,
    entry: (p) => outer(p.key, char('='), p.value),
    key: () => letters,
    value: () => regex(/^[^\n]+/),
});

run(ini.section, '[database]\nhost=localhost'); // { name: 'database', entry: ['host', 'localhost'] }

API index

One line per export. Every name links to its full description and runnable example in the reference pages under doc/api.

Core — unitas

Types, constructors and the grammar runner. 17 exports — full reference →

Terminals — unitas/terminals

Factories that match the input string directly. 12 exports — full reference →

Primitives — unitas/primitives

Ready-made parsers for the usual suspects. 33 exports — full reference →

Combinators — unitas/combinators

Take parsers, return a new parser. 64 exports — full reference →

Utils — unitas/utils

Plain helpers for map callbacks. 8 exports — full reference →

Migrating from 0.2.x

0.3.0 changes how a parser reports where it got to. Parsers now take an offset into the input instead of receiving a sliced suffix, which is what makes line/column error reporting possible.

If you only compose the built-in parsers, nothing changes. run, grammar, map, choice and everything built on them behave the same, and both new arguments are optional, so myParser('input') still works.

Two things do change for everyone:

// reading a result
result.remaining; // 0.2.x
input.slice(result.index); // 0.3.0 — or just use result.index

// a failed run
run(p, 'bad'); // 0.2.x: Error: Parsing failed: Unexpected error
run(p, 'bad'); // 0.3.0: ParseError, with .line, .column and .expected

If you write parsers by hand, the shape changed:

// 0.2.x
create<string>((input) => {
    if (input.startsWith('hello')) {
        return success('hello', input.slice(5));
    }

    return failure('expected "hello"');
});

// 0.3.0
create<string>((input, index = 0, ctx) => {
    if (input.startsWith('hello', index)) {
        return success('hello', index + 5);
    }

    return failure(ctx, index, '"hello"');
});

The rest, in full:

New in this release: parse (non-throwing), ParseError, locate, format, and context/record for hand-written parsers. memoize is now keyed by offset rather than by the remaining input, so its cache entries are cheap and are dropped when a different input is parsed. Left-recursive grammar rules now throw a named error instead of overflowing the stack.

Contributing

npm test                  # run the suite
npm run build             # compile to dist/
npm run generate:index    # regenerate barrel files after adding a source file
npm run generate:readme   # regenerate README.md and doc/api/ from JSDoc

Every export lives in its own file with a co-located test and a JSDoc @example. Those examples are the single source of truth: they are extracted into test/examples.test.ts and into the documentation, so they cannot drift.

License

MIT © Oleg Kamlowski