xsAI 🤗 Transformers.js Provider

Playground in Browser

A special Transformers.js provider for xsai, the extra-small AI SDK. Capable of performing tasks of embedding, transcriptions, speech synthesis, and text generations right in the browser (Node.js supported too).

xsAI Transformers.js Provider aligned the API of xsAI:

import { createEmbedProvider } from '@xsai-transformers/embed'
import embedWorkerURL from '@xsai-transformers/embed/worker?worker&url'
import { embed } from '@xsai/embed'

const transformers = createEmbedProvider({ baseURL: `xsai-transformers:///?worker-url=${embedWorkerURL}` })

// [
//   -0.038177140057086945,
//   0.032910916954278946,
//   -0.005459371022880077,
//   // ...
// ]
const { embedding } = await embed({
  ...transformers.embed('Xenova/all-MiniLM-L6-v2'),
  input: 'sunny day at the beach'
})

Features

xsai-transformers is just a wrapper for 🤗 Transformers.js. Any HuggingFace available models are possible to inference with if ONNX format of model is prepared.

While enjoying the lightweight size and compositing of APIs from xsAI, we made every possible xsai-transformers sub-providers (embedding, transcription, speech) compatible to existing xsAI implementation, this means, for either xsai or @xsai/generate-transcription, @xsai/embed, there is no need to re-write everything to get on hands of xsai-transformers, the only needed thing is to install and go.

Runtime-agnostic

xsai-transformers doesn't depend on WebGPU or Browser Built-in Modules, it works well in Node.js and other runtimes, as long as Worker thread of WebGPU are ported and supported..

Usage

Install

Just like xsAI's atomic design of every feature, you can also install only some of the utils of xsai-transformers, such as @xsai-transformers/embed and @xsai-transformers/transcription.

# npm
npm install xsai-transformers

# yarn
yarn add xsai-transformers

# pnpm
pnpm install xsai-transformers

# bun
bun install xsai-transformers

# deno
deno install xsai-transformers

Examples

Embedding (see above)

Transcription

import { createTranscriptionProvider } from '@xsai-transformers/transcription'
import transcriptionWorkerURL from '@xsai-transformers/transcription/worker?worker&url'
import { generateTranscription } from '@xsai/generate-transcription'

const transformers = createTranscriptionProvider({ baseURL: `xsai-transformers:///?worker-url=${transcriptionWorkerURL}}` })
const file: File = undefined // Audio file
const { text } = await generateTranscription({ ...transformers.transcribe('onnx-community/whisper-large-v3-turbo'), file })

Status

xsAI Transformers.js is currently in an early stage of development and may introduce breaking changes at any time.

License

MIT