LTEngine

Free and Open Source Local AI Machine Translation API, written in Rust, entirely self-hosted and compatible with LibreTranslate. Its translation capabilities are powered by large language models (LLMs) that run locally on your machine via llama.cpp.

The LLMs in LTEngine are much larger than the lightweight transformer models in LibreTranslate. Thus memory usage and speed are traded off for quality of outputs, which for some languages has been reported as being on par or better than DeepL.

It is possible to run LTEngine entirely on the CPU, but an accelerator will greatly improve performance. Supported accelerators currently include:

• CUDA
• Metal (macOS)
• Vulkan

The largest model (gemma3-27b) can fit on a single consumer RTX 3090 with 24G of VRAM.

⚠️ LTEngine is in active development. Check the Roadmap for current limitations.

Requirements

• Rust
• clang
• CMake
• A C++ compiler (g++, MSVC) for building the llama.cpp bindings

Build

git clone https://github.com/LibreTranslate/LTEngine --recursive
cd LTEngine
cargo build [--features cuda,vulkan,metal] --release

Run

./target/release/ltengine

To run different LLM models:

./target/release/ltengine -m gemma3-12b [--model-file /path/to/model.gguf]

Models

LTEngine supports any GGUF language model supported by llama.cpp. You can pass a path to load a custom .gguf model using the --model-file parameter. Otherwise LTEngine will download one of the Gemma3 models based on the -m parameter:

Memory usage numbers are approximate.

Simple

Request:

const res = await fetch("http://0.0.0.0:5050/translate", {
  method: "POST",
  body: JSON.stringify({
    q: "Hello!",
    source: "en",
    target: "es",
  }),
  headers: { "Content-Type": "application/json" },
});

console.log(await res.json());

Response:

{
    "translatedText": "¡Hola!"
}

List of language codes: https://0.0.0.0:5000/languages

Auto Detect Language

Request:

const res = await fetch("http://0.0.0.0:5000/translate", {
  method: "POST",
  body: JSON.stringify({
    q: "Ciao!",
    source: "auto",
    target: "en",
  }),
  headers: { "Content-Type": "application/json" },
});

console.log(await res.json());

Response:

{
    "detectedLanguage": {
        "confidence": 83,
        "language": "it"
    },
    "translatedText": "Bye!"
}

Language Bindings

You can use the LTEngine API using the following bindings:

• Rust: https://github.com/DefunctLizard/libretranslate-rs
• Node.js: https://github.com/franciscop/translate
• TypeScript: https://github.com/tderflinger/libretranslate-ts
• .Net: https://github.com/sigaloid/LibreTranslate.Net
• Go: https://github.com/SnakeSel/libretranslate
• Python: https://github.com/argosopentech/LibreTranslate-py
• PHP: https://github.com/jefs42/libretranslate
• C++: https://github.com/argosopentech/LibreTranslate-cpp
• Swift: https://github.com/wacumov/libretranslate
• Unix: https://github.com/argosopentech/LibreTranslate-sh
• Shell: https://github.com/Hayao0819/Hayao-Tools/tree/master/libretranslate-sh
• Java: https://github.com/suuft/libretranslate-java
• Ruby: https://github.com/noesya/libretranslate
• R: https://github.com/myanesp/libretranslateR

Roadmap

• Remove mutex block that currently limits the software to process one single translation request at a time due to a possible bug in llama.cpp.
• Cancel inference (stop generating tokens) when HTTP connections are aborted by clients. I'm unsure how this could done with actix-web.
• Add support for /translate_file (ability to translate files).
• Add support for sentence splitting. Currently text is sent to the LLM as-is, but longer texts (like documents) should be split into chunks, translated and merged back.
• Better language detection for short texts (port LexiLang to Rust)
• Test/add more LLM models aside from Gemma3
• Create comparative benchmarks between LTEngine and proprietary software.
• Add support for command line inference (run ./ltengine translate as a command line app separate from ./ltengine server)
• Make ltengine available as a library, possibly creating bindings for other languages like Python.
• Automated builds / CI
• Your ideas? We welcome contributions.

Contributing

We welcome contributions! Just open a pull request.

Credits

This work is largely possible thanks llama-cpp-rs which provide the Rust bindings to llama.cpp.

License

GNU Affero General Public License v3

Trademark

See Trademark Guidelines