GLiNER2 Multi-v1 (ONNX Fragmented & IOBinding)

This repository contains the ONNX-exported weights for GLiNER2-Multi-v1. The model is specifically exported in a fragmented format (encoder, span_rep, count_pred, count_lstm, classifier) to be directly compatible with gliner2-rs, the official Zero-Python Native Rust Inference Engine for GLiNER2.

🆕 Update: V2 Zero-Copy IOBinding Models Available!

We have introduced V2 fused models (fp16_v2 and fp32_v2) that fuse Gather, ArgMax, and MatMul operations directly into the ONNX graph. By using ORT's IoBinding, these models ensure that tensors never leave the GPU/NPU VRAM, completely bypassing the PCIe bus and reducing inference latency by ~30% on discrete GPUs.

📂 Available Variants

• fp16_v2 (Recommended): Zero-Copy VRAM optimized models. Fused operations with Full IO Types (native FP16). Drastically reduces inference time on NVIDIA CUDA, AMD ROCm, and Apple CoreML. Requires gliner2-rs >= 0.4.1.
• fp32_v2: High precision V2 fusions for CPU execution (AVX, XNNPACK). Requires gliner2-rs >= 0.4.1.
• fp16 (Standard): Legacy Float16 ONNX models. Slower on discrete GPUs due to PCIe transfers, but completely stable and supported everywhere.
• fp32 (Standard): Legacy Float32 ONNX models.

🚀 Performance & Benchmarks

The ONNX conversion, combined with the Rust native engine (ort binding), allows this model to run extremely fast on both GPUs and edge devices like NPUs.

Benchmark Task: Tested on complex text extraction tasks spanning up to 62 classes (metrics normalized per extracted entity to allow cross-device comparison).

📦 Usage in Rust

You can dynamically download and execute these ONNX weights from Rust in 3 lines of code. With gliner2-rs >= 0.4.1, the engine automatically detects if you are using V1 or V2 models and routes the execution perfectly.

use gliner2_inference::{Gliner2Engine, ModelType, SchemaTask};

// Auto-downloads the V2 FP16 models from this HuggingFace repo
// and automatically switches to the high-performance IOBinding Engine!
let engine = Gliner2Engine::from_pretrained(
    "jugaadsrl/gliner2-multi-v1-onnx",
    Some("fp16_v2"),
    ModelType::HuggingFace
)?;

let text = "Mario Rossi works at Apple in Cupertino.";
let tasks = vec![
    SchemaTask::Entities(vec![
        "person".to_string(), "organization".to_string(), "location".to_string()
    ])
];

let (entities, _, _) = engine.extract(text, &tasks)?;

🔧 Model Fixes

• count_lstm has been successfully exported with dynamic sequence unrolling by replacing the training CompileSafeGRU with native nn.GRU, resolving Out-of-Bounds Gather ONNX errors for variable length texts.
• (V2) Scorer now uses a heavily optimized fused combination of Reshape + MatMul + Transpose instead of Einsum, ensuring compatibility with execution providers that don't support Einsum in FP16 (e.g. QNN, CoreML).