text-processing-rs

A Rust port of NVIDIA NeMo Text Processing supporting both Inverse Text Normalization (ITN) and Text Normalization (TN).

What it does

ITN: Spoken → Written

Converts spoken-form ASR output to written form:

TN: Written → Spoken

Converts written-form text to spoken form (useful for TTS preprocessing):

Usage

Rust

use text_processing_rs::{normalize, tn_normalize};

// ITN: spoken → written
let result = normalize("two hundred");
assert_eq!(result, "200");

let result = normalize("five dollars and fifty cents");
assert_eq!(result, "$5.50");

// TN: written → spoken
let result = tn_normalize("$5.50");
assert_eq!(result, "five dollars and fifty cents");

let result = tn_normalize("123");
assert_eq!(result, "one hundred twenty three");

JavaScript (WASM)

Build wasm artifacts:

npm run wasm:build:node
npm run wasm:build:web

Node usage:

import * as wasm from "./pkg-node/text_processing_rs.js";

console.log(wasm.normalize("two hundred")); // "200"
console.log(wasm.tnNormalize("$5.50")); // "five dollars and fifty cents"

wasm.addRule("gee pee tee", "GPT");
console.log(wasm.normalize("gee pee tee")); // "GPT"

The generated npm package name is @fluidinference/text-processing-rs.

Web project usage (Vite / Next.js / webpack):

npm install @fluidinference/text-processing-rs

import init, * as wasm from "@fluidinference/text-processing-rs";

async function run() {
  // Loads and initializes the .wasm module (required once at startup)
  await init();

  const itn = wasm.normalize("two hundred");
  const tn = wasm.tnNormalize("$5.50");

  console.log(itn); // "200"
  console.log(tn); // "five dollars and fifty cents"

  wasm.addRule("gee pee tee", "GPT");
  console.log(wasm.normalize("gee pee tee")); // "GPT"
}

run();

If your framework supports top-level await, you can initialize at module load time:

import init, * as wasm from "@fluidinference/text-processing-rs";
await init();

Sentence-level normalization scans for normalizable spans within a larger sentence:

use text_processing_rs::{normalize_sentence, normalize_sentence_lang, tn_normalize_sentence};

// ITN sentence mode
let result = normalize_sentence("I have twenty one apples");
assert_eq!(result, "I have 21 apples");

// ITN sentence mode, language-aware ("en", "fr", "es", "de", "zh", "hi", "ja")
let result = normalize_sentence_lang("j'ai vingt et un ans", "fr");
assert_eq!(result, "j'ai 21 ans");

// TN sentence mode
let result = tn_normalize_sentence("I paid $5 for 23 items");
assert_eq!(result, "I paid five dollars for twenty three items");

Swift

import NemoTextProcessing

// ITN: spoken → written
let result = NemoTextProcessing.normalize("two hundred")
// "200"

// TN: written → spoken
let spoken = NemoTextProcessing.tnNormalize("$5.50")
// "five dollars and fifty cents"

// Sentence modes
let itn = NemoTextProcessing.normalizeSentence("I have twenty one apples")
// "I have 21 apples"

// Language-aware ITN sentence mode ("en", "fr", "es", "de", "zh", "hi", "ja")
let itnFr = NemoTextProcessing.normalizeSentence("j'ai vingt et un ans", language: "fr")
// "j'ai 21 ans"

let tn = NemoTextProcessing.tnNormalizeSentence("I paid $5 for 23 items")
// "I paid five dollars for twenty three items"

CLI

# ITN
nemo-itn two hundred thirty two        # → 232
nemo-itn -s "I have twenty one apples" # → I have 21 apples

# TN
nemo-tn 123                            # → one hundred twenty three
nemo-tn '$5.50'                        # → five dollars and fifty cents
nemo-tn -s 'I paid $5 for 23 items'    # → I paid five dollars for twenty three items

# Pipe from stdin
echo "2:30 PM" | nemo-tn               # → two thirty p m

Compatibility

ITN (Spoken → Written)

98.6% compatible with NeMo text processing test suite (1200/1217 tests passing).

TN (Written → Spoken)

Features

• ITN (Inverse Text Normalization): spoken → written form for ASR post-processing
• TN (Text Normalization): written → spoken form for TTS preprocessing
• Cardinal and ordinal number conversion (both directions)
• Decimal numbers with scale words (million, billion)
• Currency formatting (USD, GBP, EUR, JPY, and more)
• Measurements including temperature (°C, °F, K) and data rates (gbps)
• Date parsing (multiple formats) and decade verbalization (1980s → nineteen eighties)
• Time parsing with AM/PM, 24-hour format, and timezone preservation
• Email and URL normalization
• Phone numbers, IP addresses, SSN
• Case preservation for proper nouns and abbreviations
• Sentence-level normalization with sliding window span matching
• Custom rules for domain-specific terms
• C FFI for integration with Swift, Python, and other languages

Building

Rust

cargo build
cargo test

WASM + JavaScript

# Build + smoke test (Node) + build browser artifact
npm run wasm:ci

# Create a tarball from the browser package
npm run wasm:pack

# Publish browser package to npm (requires npm auth)
npm run wasm:publish

CLI Tools

# Build the Rust library (release, with FFI)
cargo build --release --target aarch64-apple-darwin --features ffi

# Build Swift CLI tools
cd swift-test && swift build

Binaries are at swift-test/.build/debug/nemo-itn and swift-test/.build/debug/nemo-tn.

Swift (XCFramework)

# Install Rust targets
rustup target add aarch64-apple-darwin x86_64-apple-darwin
rustup target add aarch64-apple-ios aarch64-apple-ios-sim

# Build XCFramework
./build-xcframework.sh

Output:

• output/NemoTextProcessing.xcframework - Add to Xcode project
• output/NemoTextProcessing.swift - Swift wrapper

License

Apache 2.0

Acknowledgments

This project is a Rust implementation based on the inverse text normalization grammars from NVIDIA NeMo Text Processing. All credit for the original algorithms and test cases goes to the NVIDIA NeMo team.