@socprime/logtotal-sanitizer

Framework-agnostic log sanitizer for browsers and Node.js. It's designed to sanitize log files before they are sent to third-party platforms for processing, analysis, or troubleshooting. The sanitizer redacts secrets, identifiers, and PII and replaces each value with a stable HMAC token to reduce the risk of accidentally exposing sensitive or confidential information contained in logs.

This library is already integrated into LogTotal and runs directly in the browser, allowing logs to be sanitized locally before they leave the user's environment.

If you require a fully controlled and isolated data sanitization environment, you can deploy and run this library within your own infrastructure. The sanitizer can also be installed and used in air-gapped environments, ensuring that sensitive log data remains within an environment you fully control.

The library has no runtime dependencies. The same compiled rules run in a browser tab and in a Node.js CLI.

Install

npm install @socprime/logtotal-sanitizer

Requires Node.js 20 or newer when used from Node. Browsers need globalThis.crypto.getRandomValues.

Quick start

import { createSanitizer, generateKey } from '@socprime/logtotal-sanitizer';

const key = generateKey();
const sanitizer = createSanitizer({
  key,
  rules: ['secrets', 'ips', 'hosts', 'users'],
});

const { output, report } = sanitizer.sanitizeText(
  'user alice@example.test logged in from 10.0.0.1 with Bearer abcdefghijklmnop',
);

console.log(output);
console.log(report.counts);

Reuse key across files when the same original value must map to the same token. Persistence is your responsibility; the library never writes the key to disk or sessionStorage.

Options

createSanitizer(options) compiles rules once. sanitizeText(text, options) is a one-shot wrapper.

Priority

1. neverRedact (global values, global patterns, then byRule)
2. alwaysRedact
3. Selected rules, in the order you passed them (built-in registry order when you omit rules)

A rejected validate() result also skips the span. The engine does not retry a later overlapping rule from the same start offset.

alwaysRedact / neverRedact

const sanitizer = createSanitizer({
  key,
  alwaysRedact: {
    values: ['acme-internal-host'],
    patterns: [/CASE-\d{6}/],
  },
  neverRedact: {
    values: ['127.0.0.1'],
    byRule: [{ ruleId: 'hosts', values: ['localhost'] }],
  },
});

alwaysRedact matches appear in the report under rule id custom with token prefix CUSTOM.

Built-in rules

Enabled in this order when you omit rules:

mask mode (<R:…>) hides the kind of secret. pseudo mode keeps a type prefix so you can still read the timeline.

JSON lines: if a line parses as a JSON object or array, fields listed in a rule's jsonKeys are redacted by name (case- and -/_-insensitive). Other string values still go through the regex pass.

Custom rules

import { createSanitizer, defineRule } from '@socprime/logtotal-sanitizer';

const ticket = defineRule({
  id: 'ticket',
  label: 'Ticket ids',
  description: 'Internal ticket numbers',
  mode: 'pseudo',
  token: 'TICKET',
  patterns: ['(?:CASE-\\d{6})'],
});

const sanitizer = createSanitizer({ rules: ['secrets', ticket] });

id must be a JavaScript identifier. At most one capturing group per fragment; it marks the value to replace (the rest of the match is kept). Put expensive checks in validate, not in the regex.

Streaming and I/O

import { createSanitizer, fromString, toString } from '@socprime/logtotal-sanitizer';

const sink = toString();
const report = await createSanitizer({ key }).sanitizeStream(fromString(text), sink);
const output = sink.result();

Browser helpers: fromBlob, fromWebStream, toWebStream.

Node helpers (subpath @socprime/logtotal-sanitizer/node):

import { sanitizeFile } from '@socprime/logtotal-sanitizer/node';

await sanitizeFile('./app.log', './app.sanitized.log', { rules: ['secrets', 'ips'] });

Also: fromFile, toFile, fromNodeStream, toNodeStream.

The root entry does not import node:*.

Report

{
  counts: { ips: 2, secrets: 1 },
  totalMatches: 3,
  lineCount: 40,
  replacements: [{ ruleId, original, replacement, count, contextBefore?, contextAfter? }],
  preview: { before: [{ text, changed }], after: [{ text, changed }] }
}

preview covers the first 256 KiB of output.

CLI

npx @socprime/logtotal-sanitizer --help

# local
npx logtotal-sanitize ./app.log -o ./app.sanitized.log --report ./report.json

# stdin / stdout
cat app.log | npx logtotal-sanitize - --stdout > app.sanitized.log

# correlate tokens across files
npx logtotal-sanitize a.log -o a.out --print-key
npx logtotal-sanitize b.log -o b.out --key "$KEY"

Global install:

npm install -g @socprime/logtotal-sanitizer
logtotal-sanitize ./app.log -o ./app.sanitized.log

Exit codes: 0 ok, 1 runtime error or --fail-on-match, 2 usage error.

Token correlation

Tokens are HMAC-SHA-256 of ruleId || 0x00 || original, truncated to 16 hex chars. Same key + same value + same rule ⇒ same token. A different key produces different tokens. Generated keys use encoding hex; pasted keys default to utf8.