@napi-rs/webcodecs

WebCodecs API implementation for Node.js using FFmpeg, built with NAPI-RS.

Features

• W3C WebCodecs API compliant - Full implementation of the WebCodecs specification with native DOMException errors
• Video encoding/decoding - H.264, H.265 (with Alpha), VP8, VP9 (with Alpha), AV1
• Encoding Alpha channel - VP9 and HEVC alpha encoding/decoding with transparency support (See canvas-to-video.js example and video.html)
• Audio encoding/decoding - AAC, Opus, MP3, FLAC, Vorbis, PCM variants
• Container muxing/demuxing - MP4, WebM, MKV containers with seeking support
• Image decoding - JPEG, PNG, WebP, GIF, BMP, AVIF, JPEG XL
• Canvas integration - Create VideoFrames from @napi-rs/canvas for graphics and text rendering
• Hardware acceleration - Zero-copy GPU encoding with VideoToolbox (macOS), NVENC (NVIDIA), VAAPI (Linux), QSV (Intel)
• Cross-platform - macOS, Windows, Linux (glibc/musl, x64/arm64/armv7)
• Structured logging - FFmpeg logs redirected to Rust tracing crate for easy integration

Installation

bun add @napi-rs/webcodecs
# or
pnpm add @napi-rs/webcodecs
# or
yarn add @napi-rs/webcodecs

Optional: Canvas Integration

For creating VideoFrames from canvas content, install @napi-rs/canvas:

npm install @napi-rs/canvas

Quick Start

Video Encoding

import { VideoEncoder, VideoFrame } from '@napi-rs/webcodecs'

const encoder = new VideoEncoder({
  output: (chunk, metadata) => {
    console.log(`Encoded ${chunk.type} chunk: ${chunk.byteLength} bytes`)
  },
  error: (e) => console.error(e),
})

encoder.configure({
  codec: 'avc1.42001E', // H.264 Baseline
  width: 1920,
  height: 1080,
  bitrate: 5_000_000,
  hardwareAcceleration: 'prefer-hardware', // Use GPU when available
  latencyMode: 'realtime', // Optimize for low latency
})

// Create and encode frames
const frameData = new Uint8Array(1920 * 1080 * 4) // RGBA
const frame = new VideoFrame(frameData, {
  format: 'RGBA',
  codedWidth: 1920,
  codedHeight: 1080,
  timestamp: 0,
})

encoder.encode(frame)
frame.close()

// Force a keyframe for seeking/streaming
const frame2 = new VideoFrame(frameData, {
  format: 'RGBA',
  codedWidth: 1920,
  codedHeight: 1080,
  timestamp: 33333, // 30fps
})
encoder.encode(frame2, { keyFrame: true }) // Force I-frame
frame2.close()

await encoder.flush()
encoder.close()

Video Decoding

import { VideoDecoder, EncodedVideoChunk } from '@napi-rs/webcodecs'

const decoder = new VideoDecoder({
  output: (frame) => {
    console.log(`Decoded frame: ${frame.codedWidth}x${frame.codedHeight}`)
    frame.close()
  },
  error: (e) => console.error(e),
})

decoder.configure({
  codec: 'avc1.42001E',
  codedWidth: 1920,
  codedHeight: 1080,
})

// Decode chunks
const chunk = new EncodedVideoChunk({
  type: 'key',
  timestamp: 0,
  data: encodedData,
})

decoder.decode(chunk)
await decoder.flush()
decoder.close()

Audio Encoding

import { AudioEncoder, AudioData } from '@napi-rs/webcodecs'

const encoder = new AudioEncoder({
  output: (chunk, metadata) => {
    console.log(`Encoded audio: ${chunk.byteLength} bytes`)
  },
  error: (e) => console.error(e),
})

encoder.configure({
  codec: 'opus',
  sampleRate: 48000,
  numberOfChannels: 2,
  bitrate: 128000,
})

const audioData = new AudioData({
  format: 'f32-planar',
  sampleRate: 48000,
  numberOfFrames: 1024,
  numberOfChannels: 2,
  timestamp: 0,
  data: new Float32Array(1024 * 2),
})

encoder.encode(audioData)
audioData.close()

await encoder.flush()
encoder.close()

Image Decoding

import { ImageDecoder } from '@napi-rs/webcodecs'
import { readFileSync } from 'fs'

const imageData = readFileSync('image.png')
const decoder = new ImageDecoder({
  data: imageData,
  type: 'image/png',
})

const result = await decoder.decode()
console.log(`Image: ${result.image.codedWidth}x${result.image.codedHeight}`)
result.image.close()
decoder.close()

When data is a ReadableStream, ImageDecoder accumulates every chunk so it can re-probe the growing input. This buffer currently has no fixed limit, so applications should bound or reject unexpectedly large streams before passing them to the decoder.

Container Demuxing

Read encoded video/audio from MP4, WebM, or MKV containers:

import { Mp4Demuxer, VideoDecoder, AudioDecoder } from '@napi-rs/webcodecs'

// Create decoder instances
const videoDecoder = new VideoDecoder({
  output: (frame) => {
    console.log(`Decoded frame: ${frame.timestamp}`)
    frame.close()
  },
  error: (e) => console.error(e),
})

const audioDecoder = new AudioDecoder({
  output: (data) => {
    console.log(`Decoded audio: ${data.timestamp}`)
    data.close()
  },
  error: (e) => console.error(e),
})

// Create demuxer with callbacks
const demuxer = new Mp4Demuxer({
  videoOutput: (chunk) => videoDecoder.decode(chunk),
  audioOutput: (chunk) => audioDecoder.decode(chunk),
  error: (e) => console.error(e),
})

// Load from file or buffer
await demuxer.load('./video.mp4')
// or: await demuxer.loadBuffer(uint8Array)

// Configure decoders with extracted configs
videoDecoder.configure(demuxer.videoDecoderConfig)
audioDecoder.configure(demuxer.audioDecoderConfig)

// Get track info
console.log(demuxer.tracks) // Array of track info
console.log(demuxer.duration) // Duration in microseconds

// Demux all packets (calls callbacks)
demuxer.demux()

// Or demux in batches
demuxer.demux(100) // Demux up to 100 packets

// Seek to timestamp (microseconds)
demuxer.seek(5_000_000) // Seek to 5 seconds

demuxer.close()

Container Muxing

Write encoded video/audio to MP4, WebM, or MKV containers:

import { Mp4Muxer, VideoEncoder, AudioEncoder } from '@napi-rs/webcodecs'
import { writeFileSync } from 'fs'

// Create muxer
const muxer = new Mp4Muxer({ fastStart: true })

// Track description will be set from encoder metadata
let videoDescription: Uint8Array | undefined

// Create encoder that feeds into muxer
const videoEncoder = new VideoEncoder({
  output: (chunk, metadata) => {
    // Capture codec description from first keyframe
    if (metadata?.decoderConfig?.description) {
      videoDescription = metadata.decoderConfig.description
    }
    muxer.addVideoChunk(chunk, metadata)
  },
  error: (e) => console.error(e),
})

videoEncoder.configure({
  codec: 'avc1.42001E',
  width: 1920,
  height: 1080,
  bitrate: 5_000_000,
})

// Add tracks before adding chunks
muxer.addVideoTrack({
  codec: 'avc1.42001E',
  width: 1920,
  height: 1080,
})

// Encode frames...
// videoEncoder.encode(frame)

await videoEncoder.flush()

// Finalize and write output
const mp4Data = muxer.finalize()
writeFileSync('output.mp4', mp4Data)
muxer.close()

Streaming Muxer Mode

For live streaming or large files, use streaming mode:

import { Mp4Muxer } from '@napi-rs/webcodecs'

const muxer = new Mp4Muxer({
  fragmented: true, // Required for MP4 streaming
  // Maximum bytes returned by each read(); this is not a memory limit.
  streaming: { bufferCapacity: 256 * 1024 },
})

muxer.addVideoTrack({ codec: 'avc1.42001E', width: 1920, height: 1080 })

// Add chunks as they arrive
muxer.addVideoChunk(chunk, metadata)

// Read available data incrementally
const data = muxer.read()
if (data) {
  stream.write(data)
}

// Check when finished
if (muxer.isFinished) {
  stream.end()
}

Streaming writes are queued in a growable native buffer so addVideoChunk() and addAudioChunk() never wait for JavaScript to call read(). Applications must drain read() regularly; an indefinitely stalled consumer can retain the entire muxed output in memory. bufferCapacity controls only the maximum size of each returned read chunk.

VideoFrame from Canvas

Create VideoFrames from @napi-rs/canvas for graphics, text rendering, or image compositing:

import { VideoFrame } from '@napi-rs/webcodecs'
import { createCanvas } from '@napi-rs/canvas'

const canvas = createCanvas(1920, 1080)
const ctx = canvas.getContext('2d')

// Draw graphics
ctx.fillStyle = '#FF0000'
ctx.fillRect(0, 0, 1920, 1080)
ctx.fillStyle = '#FFFFFF'
ctx.font = '48px sans-serif'
ctx.fillText('Hello WebCodecs!', 100, 100)

// Create VideoFrame from canvas (timestamp required per W3C spec)
const frame = new VideoFrame(canvas, {
  timestamp: 0,
  duration: 33333, // optional: frame duration in microseconds
})

console.log(frame.format) // 'RGBA'
console.log(frame.codedWidth, frame.codedHeight) // 1920, 1080

// Use with VideoEncoder (see Video Encoding section)
frame.close()

Note: Canvas pixel data is copied as RGBA format with sRGB color space.

Native float canvas interop

Native canvas providers can retain RGBA16Float images with createVideoFrameFromRGBA16Float(data, init). data contains tightly packed RGBA half-float bit patterns in a Uint16Array, with four components per coded pixel. The native frame copies the input once; clones share immutable storage, and closing one frame leaves its clones usable. Supply the image color space in init.colorSpace; the default is full-range sRGB.

These frames have format === null. Public allocationSize() throws and copyTo() rejects with NotSupportedError, including requests for an explicit eight-bit format. copyVideoFrameRGBA16Float(frame) gives native image consumers an independent copy of the half-float bits without changing that public contract. It returns the whole coded image (codedWidth * codedHeight * 4 values) and ignores visibleRect. These functions are module exports rather than additions to VideoFrame.prototype. Native GPU consumers can use registerVideoFrameCloseObserver(frame, callback) to expire their retained textures synchronously when that frame is explicitly closed. Clones have independent observers. Collecting a frame wrapper releases its observers while independently retained GPU images remain usable.

The FFmpeg encoder cannot consume this opaque image resource. encode() queues the input normally, then reports a native OperationError, rejects pending flush() calls with the same error, and closes the encoder without output. This interop does not implement WebGPU external-texture import or cross-worker frame transport.

Supported Codecs

Video

Note: Short form codec strings (vp9, av01, av1) are accepted for compatibility with browser implementations.

¹ HEVC Alpha Encoding Limitations:

• Requires software encoder (libx265) - set hardwareAcceleration: 'prefer-software'
• Hardware encoders (VideoToolbox, NVENC, VAAPI, QSV) do not support alpha
• Only YUVA420P (8-bit) and YUVA420P10 (10-bit Main 10 profile) pixel formats supported

Legend:

• ✅ Feature supported
• 🙅🏻‍♀️ Feature not supported by codec format
• ❓ Feature supported by codec format but not yet implemented

Audio

Image

Containers

Platform Support

Pre-built binaries are available for:

W3C Web Platform Tests Compliance

This implementation is validated against the W3C Web Platform Tests for WebCodecs.

Ported Tests Status

Skipped tests are due to platform-specific features or edge cases.

Tests Not Ported (Browser-Only)

19 WPT test files require browser APIs unavailable in Node.js:

See __test__/wpt/README.md for detailed test status.

Hardware Acceleration

Hardware encoding is fully supported with automatic GPU selection and fallback:

Configuration

encoder.configure({
  codec: 'avc1.42001E',
  width: 1920,
  height: 1080,
  // Hardware acceleration preference
  hardwareAcceleration: 'prefer-hardware', // 'no-preference' | 'prefer-hardware' | 'prefer-software'
  // Latency mode affects encoder tuning
  latencyMode: 'realtime', // 'quality' | 'realtime'
  // Alpha channel preservation (VP9 and HEVC only)
  alpha: 'discard', // 'keep' | 'discard' (default: 'discard')
})

• latencyMode: 'realtime' - Enables low-latency encoder options (smaller GOP, no B-frames, fast presets)
• latencyMode: 'quality' - Enables quality-focused options (larger GOP, B-frames, lookahead)
• alpha: 'keep' - Preserves alpha channel (VP9 and HEVC only). For HEVC, requires hardwareAcceleration: 'prefer-software'

The encoder automatically applies optimal settings for each hardware encoder based on the latency mode.

Alpha Channel Encoding

Encode video with transparency using VP9 or HEVC:

import { VideoEncoder, VideoFrame } from '@napi-rs/webcodecs'

const encoder = new VideoEncoder({
  output: (chunk, metadata) => {
    console.log(`Alpha chunk: ${chunk.byteLength} bytes`)
  },
  error: (e) => console.error(e),
})

// VP9 alpha - works with hardware or software
encoder.configure({
  codec: 'vp09.00.10.08',
  width: 1920,
  height: 1080,
  alpha: 'keep',
})

// HEVC alpha - requires software encoder
encoder.configure({
  codec: 'hev1.1.6.L93.B0',
  width: 1920,
  height: 1080,
  alpha: 'keep',
  hardwareAcceleration: 'prefer-software', // Required for HEVC alpha
})

// Create frame with alpha channel (I420A format)
const width = 1920
const height = 1080
const frameData = new Uint8Array(width * height * 1.5 + width * height) // Y + U + V + A
const frame = new VideoFrame(frameData, {
  format: 'I420A',
  codedWidth: width,
  codedHeight: height,
  timestamp: 0,
})

encoder.encode(frame)
frame.close()

Limitations

Scalable Video Coding (SVC)

All scalability modes (L1Tx, L2Tx, L3Tx, S2Tx, S3Tx, and variants) are accepted and populate metadata.svc.temporalLayerId when temporal layers >= 2.

The W3C WebCodecs spec only defines temporalLayerId in SvcOutputMetadata - there is no spatialLayerId field in the spec. See W3C WebCodecs §6.7.

Note: This implementation computes temporal layer IDs algorithmically from frame index per W3C spec. FFmpeg is not configured for actual SVC encoding, so base layer frames are not independently decodable.

Error Handling

Synchronous errors (e.g., calling encode() on a closed encoder) throw native DOMException instances that pass instanceof DOMException checks per W3C spec:

try {
  encoder.encode(frame) // on closed encoder
} catch (e) {
  console.log(e instanceof DOMException) // true
  console.log(e.name) // "InvalidStateError"
}

Asynchronous error callbacks receive standard Error objects with the DOMException name in the message:

const encoder = new VideoEncoder({
  output: (chunk) => {},
  error: (e) => {
    console.log(e.message) // "EncodingError: ..."
  },
})

VideoFrame Format Conversion

VideoFrame.copyTo() and VideoFrame.allocationSize() support format conversion per W3C WebCodecs spec:

const frame = new VideoFrame(i420Data, {
  format: 'I420',
  codedWidth: 1920,
  codedHeight: 1080,
  timestamp: 0,
})

// Get allocation size for RGBA output
const rgbaSize = frame.allocationSize({ format: 'RGBA' })

// Copy with format conversion (I420 → RGBA)
const rgbaBuffer = new Uint8Array(rgbaSize)
const layout = await frame.copyTo(rgbaBuffer, { format: 'RGBA' })

frame.close()

Supported conversions:

Per WPT videoFrame-copyTo-rgb.any.js, RGB-to-YUV conversion throws NotSupportedError.

Custom layouts with overflow-inducing values (e.g., offset: 2³²-2) throw TypeError via checked arithmetic. The copy rect is validated against the output format. For YUV-to-RGB conversion, the read origin is then aligned down to the source chroma grid, as Chrome does.

ImageDecoder Options

ImageDecoder supports all W3C spec options:

Note: Per W3C spec, desiredWidth and desiredHeight must both be specified or both omitted.

Platform-Specific Notes

• ImageDecoder GIF animation: FFmpeg may return only the first frame. Use VideoDecoder with GIF codec for full animation.

Per-Frame Quantizer Control

Per-frame quantizer control is available for VP9 and AV1 when using bitrateMode: 'quantizer':

encoder.configure({
  codec: 'vp09.00.10.08',
  width: 1920,
  height: 1080,
  bitrateMode: 'quantizer',
})

// Per-frame QP control (0-255 range per W3C WebCodecs spec)
encoder.encode(frame, { vp9: { quantizer: 128 } })

Note: The 0-255 quantizer range for VP9/AV1 aligns with Chromium's WebCodecs implementation. Internally, values are converted to the 0-63 encoder range using q_index / 4.

Logging

This library uses Rust's tracing crate for structured logging. Enable logging via the WEBCODECS_LOG environment variable:

# Enable all logs at info level
WEBCODECS_LOG=info node your-app.js

# Enable FFmpeg logs at debug level
WEBCODECS_LOG=ffmpeg=debug node your-app.js

# Enable WebCodecs codec errors at warn, FFmpeg at info
WEBCODECS_LOG=webcodecs=warn,ffmpeg=info node your-app.js

# Enable trace-level logging for everything
WEBCODECS_LOG=trace node your-app.js

Log Targets

FFmpeg Log Level Mapping

Without WEBCODECS_LOG set, all logs are silently discarded.

API Reference

This package implements the W3C WebCodecs API. Key classes:

• VideoEncoder / VideoDecoder - Video encoding and decoding with EventTarget support
• AudioEncoder / AudioDecoder - Audio encoding and decoding with EventTarget support
• VideoFrame - Raw video frame data (supports buffer data, existing VideoFrame, or @napi-rs/canvas Canvas)
• AudioData - Raw audio sample data
• EncodedVideoChunk / EncodedAudioChunk - Encoded media data
• ImageDecoder - Static image decoding
• VideoColorSpace - Color space information
• Mp4Demuxer / WebMDemuxer / MkvDemuxer - Container demuxing with seeking
• Mp4Muxer / WebMMuxer / MkvMuxer - Container muxing with streaming support

All encoders and decoders implement the EventTarget interface with addEventListener(), removeEventListener(), and dispatchEvent().

For full API documentation, see the W3C WebCodecs specification.

Development

Requirements

• Rust (latest stable)
• Node.js 18+
• pnpm

The build downloads the pinned FFmpeg 9.0 release by default. If you opt out of that download or set FFMPEG_DIR, the supplied FFmpeg headers and libraries must be from FFmpeg 9.x; FFmpeg 8.x system/Homebrew installations are rejected by the ABI guard.

Build

pnpm install
pnpm build

Test

pnpm test

Lint

pnpm lint
cargo clippy

License

The original project source is MIT licensed. The prebuilt native addons statically link GPL-enabled FFmpeg, x264, x265, and other libraries; their distribution is subject to additional terms, including GPLv3.

Read LICENSES.md, THIRD_PARTY_NOTICES.md, and the outstanding Corresponding Source work before redistributing the prebuilt packages. The MIT source license alone does not cover the bundled native binaries.