fast_gliner

Python bindings for the Rust inference engine gline-rs. They run fast CPU and GPU inference for:

• GLiNER
• GLiNER2
• GLiClass
• GLiFormer

fast_gliner keeps the Python API small. Rust does the inference. The runtime is ONNX Runtime.

---

✨ Features

• 🚀 High-performance inference using Rust
• 🧠 Supports GLiNER, GLiNER2, GLiClass, and GLiFormer models
• ⚡ ~4× faster CPU inference than the PyTorch implementation
• 🐍 Simple Python API
• 🖥 Optional CUDA execution through ONNX Runtime

---

⏳ Installation

Pre-built wheel (CPU)

$ pip install fast_gliner

Building from source

$ pip install --no-binary=:all: fast_gliner

Building with CUDA

$ pip install --no-binary=:all: fast_gliner[cuda]

---

🚀 Quickstart

Named Entity Recognition

GLiNER2 (recommended)

from fast_gliner import FastGLiNER2

model = FastGLiNER2.from_pretrained(
    "lion-ai/gliner2-multi-v1-onnx"
)

model.predict_entities(
    "I am James Bond",
    ["person"]
)

GLiNER

from fast_gliner import FastGLiNER

model = FastGLiNER.from_pretrained(
    model_id="onnx-community/gliner_multi-v2.1-onnx",
    execution_provider="cpu",
)

model.predict_entities("I am James Bond", ["person"])

Output:

[
    {
        'text': 'James Bond',
        'label': 'person',
        'score': 0.9012733697891235,
        'start': 5,
        'end': 15
    }
]

---

Classification

GLiNER2 classifies with the span-score head:

from fast_gliner import FastGLiNER2

model = FastGLiNER2.from_pretrained(
    "lion-ai/gliner2-multi-v1-onnx"
)

model.classify("Buy milk and eggs after work", ["shopping", "work", "personal"])

Output:

[
    ('shopping', 0.93), 
    ('personal', 0.61), 
    ('work', 0.44)
]

GLiClass scores one text against a label list. prompt_first comes from the checkpoint config.json. Results are (label, score) pairs, highest first.

from fast_gliner import FastGLiClass

model = FastGLiClass.from_pretrained(
    "knowledgator/gliclass-small-v1.0"
)

model.classify(
    "Rust is a systems programming language focused on safety, speed, and concurrency.",
    ["computing", "science", "programming", "travel", "food", "politics"],
)

Output:

[
    ('programming', 1.0),
    ('computing', 1.0),
    ('science', 0.9983),
    ('travel', 0.7708),
    ('politics', 0.4666),
    ('food', 0.3758)
]

Hierarchical labels are a dict. Scores use dotted names such as sentiment.positive.

model.classify(
    "The product quality is amazing but delivery was slow",
    {
        "sentiment": ["positive", "negative", "neutral"],
        "topic": ["product", "service", "shipping"],
    },
)

Output:

[
    ('topic.product', 1.0),
    ('topic.shipping', 1.0),
    ('topic.service', 1.0),
    ('sentiment.positive', 1.0),
    ('sentiment.neutral', 1.0),
    ('sentiment.negative', 1.0)
]

return_hierarchical=True returns a dict in the same shape as the labels.

model.classify(
    "The product quality is amazing but delivery was slow",
    {
        "sentiment": ["positive", "negative", "neutral"],
        "topic": ["product", "service", "shipping"],
    },
    return_hierarchical=True,
)

Output:

{
    'sentiment': {'positive': 1.0, 'negative': 1.0, 'neutral': 1.0},
    'topic': {'product': 1.0, 'service': 1.0, 'shipping': 1.0}
}

Few-shot examples are extra text in the prompt. They are not scored labels. The tokenizer must contain <<EXAMPLE>>.

model.classify(
    "Fast delivery and the item works perfectly!",
    ["positive", "negative", "product", "service", "shipping"],
    examples=[
        {"text": "Love this item, great quality!", "labels": ["positive", "product"]},
        {"text": "Customer support was unhelpful", "labels": ["negative", "service"]},
    ],
)

Output:

missing required GLiClass token in tokenizer vocabulary: <<EXAMPLE>>

A task prompt goes after the label separator.

model.classify(
    "The battery life on this phone is incredible",
    ["positive", "negative", "neutral"],
    prompt="Classify the sentiment of this product review:",
)

Output:

[
    ('positive', 1.0),
    ('neutral', 0.9999),
    ('negative', 0.9314)
]

---

Structured Extraction

from fast_gliner import FastGLiNER2

model = FastGLiNER2.from_pretrained(
    "lion-ai/gliner2-multi-v1-onnx"
)

text = """Contact: John Smith
Email: [email protected]
Phones: 555-1234, 555-5678
Address: 123 Main St, NYC"""

result = model.extract_json(
    text,
    {
        "contact": [
            "name::str",
            "email::str",
            "phone::list",
            "address"
        ]
    }
)

Output:

{
    'contact': [
        {
            'address': ['123 Main St, NYC'],
            'email': '[email protected]',
            'name': 'John Smith',
            'phone': ['555-1234', '555-5678']
        }
    ]
}

GLiFormer uses structure for nested records. Pass a dict or a Pydantic model. A list of field names is a flat record.

extract_json is only for GLiNER2.

from fast_gliner import FastGLiFormer

model = FastGLiFormer.from_pretrained(
    "talmago/gliformer-base-v1-onnx"
)

text = (
    "At Acme, Engineering includes Alice, a software engineer, and Bob, "
    "a designer. Sales includes Carol, an account manager."
)

records = model.structure(text, {
    "company": {
        "name": "str",
        "departments": [{
            "name": "str",
            "employees": [{"name": "str", "role": "str"}],
        }],
    }
})

Output:

{
    'company': [
        {
            'departments': [
                {
                    'employees': [
                        {'name': 'Alice', 'role': 'software engineer'},
                        {'name': 'Bob', 'role': 'designer'}
                    ],
                    'name': 'Engineering'
                },
                {
                    'employees': [
                        {'name': 'Carol', 'role': 'account manager'}
                    ],
                    'name': 'Sales'
                }
            ],
            'name': 'Acme'
        }
    ]
}

Install Pydantic first. It is not a dependency of this package. A Pydantic schema returns instances of that model. Nested models stay nested.

from pydantic import BaseModel

class Employee(BaseModel):
    name: str
    role: str

class Department(BaseModel):
    name: str
    employees: list[Employee]

class Company(BaseModel):
    name: str
    departments: list[Department]

records = model.structure(text, {"company": Company})

Output:

{
    'company': [
        Company(
            name='Acme',
            departments=[
                Department(
                    name='Engineering',
                    employees=[
                        Employee(name='Alice', role='software engineer'),
                        Employee(name='Bob', role='designer')
                    ]
                ),
                Department(
                    name='Sales',
                    employees=[
                        Employee(name='Carol', role='account manager')
                    ]
                )
            ]
        )
    ]
}

Each name maps to a list of records. Nested lists stay on their parent. Nested schemas need a multi-level structuring head.

---

Relation Extraction

GLiNER2

from fast_gliner import FastGLiNER2

model = FastGLiNER2.from_pretrained(
    "lion-ai/gliner2-multi-v1-onnx"
)

text = "Bill Gates founded Microsoft."

labels = ["person", "organization"]

schema = [
    {
        "relation": "founded",
        "subject_labels": ["person"],
        "object_labels": ["organization"]
    }
]

model.extract_relations(text, labels, schema)

GLiNER

from fast_gliner import FastGLiNER

model = FastGLiNER.from_pretrained(
    model_id="onnx-community/gliner-multitask-large-v0.5",
    onnx_path="onnx/model.onnx"
)

text = "Bill Gates is the founder of Microsoft."

labels = ["person", "organization"]

schema = [
    {
        "relation": "founder",
        "subject_labels": ["person"],
        "object_labels": ["organization"]
    }
]

model.extract_relations(text, labels, schema)

Output:

[{'relation': 'founder',
  'score': 0.9981993436813354,
  'subject': {'text': 'Bill Gates',
   'label': 'person',
   'score': 0.9981993436813354,
   'start': 85,
   'end': 94},
  'object': {'text': 'Microsoft',
   'label': 'organization',
   'score': 0.9981993436813354,
   'start': 85,
   'end': 94}}]

---

Multi-Task Pipeline

entities + classification + structured extraction

from fast_gliner import FastGLiNER2

model = FastGLiNER2.from_pretrained(
    "lion-ai/gliner2-multi-v1-onnx"
)

schema = (
    model.create_schema()
    # Extract entities
    .entities(["person", "company", "location"])
    
    # Classify sentiment
    .classification("sentiment", ["positive", "negative", "neutral"])
    
    # Extract structured product information
    .structure("product")
        .field("name", dtype="str")
        .field("price", dtype="str")
        .field("features", dtype="list")
        .field("category", dtype="str", choices=["electronics", "software", "service"])
)

text = """
Apple CEO Tim Cook announced the iPhone 15 for $999 with amazing new features.
This is exciting!
"""

result = model.extract(text, schema)

print(result)

Output:

{
    "classifications": {
        "sentiment": [
            {"label": "positive", "score": 0.9232913255691528},
            {"label": "neutral", "score": 0.19288331270217896},
            {"label": "negative", "score": 0.005759984254837036},
        ]
    },
    "entities": [
        {
            "text": "Apple",
            "label": "company",
            "score": 0.9991476535797119,
            "start": 1,
            "end": 6,
        },
        {
            "text": "Tim Cook",
            "label": "person",
            "score": 0.999701738357544,
            "start": 11,
            "end": 19,
        },
    ],
    "relations": [],
    "structures": {
        "product": {
            "name": ["iPhone 15"],
            "price": ["$999"],
            "features": ["amazing new features"],
            "category": [],
        }
    },
}

entities + relation extraction

schema = (
    model.create_schema()
    .entities(["person", "company"])
    .relation("founded", ["person"], ["company"])
    .relation("works_for", ["person"], ["company"])
)

text = """
Bill Gates founded Microsoft.
Satya Nadella works for Microsoft.
"""

model.extract(text, schema)

Output:

{
  "entities": [
    {"text": "Bill Gates", "label": "person"},
    {"text": "Microsoft", "label": "company"},
    {"text": "Satya Nadella", "label": "person"},
    {"text": "Microsoft", "label": "company"}
  ],
  "relations": [
    {
      "relation": "founded",
      "subject": {"text": "Bill Gates", "label": "person"},
      "object": {"text": "Microsoft", "label": "company"}
    },
    {
      "relation": "works_for",
      "subject": {"text": "Satya Nadella", "label": "person"},
      "object": {"text": "Microsoft", "label": "company"}
    }
  ]
}

---

GLiFormer

FastGLiFormer uses the same schema builder as FastGLiNER2. The return values match too.

Relations come from the joint head. Nested records use structure, shown above. This schema builder stays a flat field list.

from fast_gliner import FastGLiFormer

model = FastGLiFormer.from_pretrained("talmago/gliformer-base-v1-onnx")

schema = (
    model.create_schema()
    .entities(["person", "organization", "location"])
    .classification("sentiment", ["positive", "negative", "neutral"])
    .relation("works_at", ["person"], ["organization"])
    .relation("lives_in", ["person"], ["location"])
    .structure("employee")
        .field("name")
        .field("company")
)

result = model.extract("Alice works at Acme and lives in London.", schema)

Output:

{
    "classifications": {
        "sentiment": [
            {"label": "neutral", "score": 0.9998},
            {"label": "positive", "score": 0.0001},
            {"label": "negative", "score": 0.0},
        ]
    },
    "entities": [
        {"text": "Alice", "label": "person", "score": 0.998633, "start": 0, "end": 5},
        {"text": "Acme", "label": "organization", "score": 0.999901, "start": 15, "end": 19},
        {"text": "London", "label": "location", "score": 0.999399, "start": 33, "end": 39},
    ],
    "relations": [
        {
            "relation": "works_at",
            "score": 0.892890,
            "subject": {"text": "Alice", "label": "person", "score": 0.996878, "start": 0, "end": 5},
            "object": {"text": "Acme", "label": "organization", "score": 0.999975, "start": 15, "end": 19},
        },
        {
            "relation": "lives_in",
            "score": 0.897462,
            "subject": {"text": "Alice", "label": "person", "score": 0.996878, "start": 0, "end": 5},
            "object": {"text": "London", "label": "location", "score": 0.999963, "start": 33, "end": 39},
        },
    ],
    "structures": {
        "employee": {
            "name": ["Alice"],
            "company": ["Acme"],
        }
    },
}

---

Supported Models

---

Performance

fast_gliner runs on the Rust engine gline-rs and ONNX Runtime.

CPU inference is about 4× faster than the PyTorch implementation.

See the benchmark results in the gline-rs README.

---

Development

Set up environment

$ cd fast_gliner/bindings/python
$ make dev

Run code formatting

$ make style

Release package to PyPI

$ make build
$ make release

---

For Contributors

Start with these documents:

1. Inference: docs/MODELING.md — how each model family runs.

2. System design: ARCHITECTURE.md — Python API, Rust engine, and ONNX Runtime.

3. Development setup: docs/DEVELOPMENT.md — how to build and run the project.

Agents in this repository should also follow AGENTS.md.

---

Acknowledgements

This repository is a fork of gline-rs. That Rust engine runs the inference. Thanks to the authors of the original GLiNER paper [1].

References

[1] GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer.

@inproceedings{zaratiana-etal-2024-gliner,
  title   = "{GL}i{NER}: Generalist Model for Named Entity Recognition using Bidirectional Transformer",
  author  = "Zaratiana, Urchade and Tomeh, Nadi and Holat, Pierre and Charnois, Thierry",
  booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)",
  year    = "2024",
  url     = "https://aclanthology.org/2024.naacl-long.300"
}

[2] GLiNER2: Schema-Driven Multi-Task Learning for Structured Information Extraction

@inproceedings{zaratiana-etal-2025-gliner2,
    title = "{GL}i{NER}2: Schema-Driven Multi-Task Learning for Structured Information Extraction",
    author = "Zaratiana, Urchade and Pasternak, Gil and Boyd, Oliver and Hurn-Maloney, George and Lewis, Ash",
    booktitle = "EMNLP 2025 System Demonstrations",
    year = "2025"
}

[3] GLiClass: Generalist Lightweight Model for Sequence Classification Tasks.

@misc{stepanov2025gliclassgeneralistlightweightmodel,
    title = "{GL}i{C}lass: Generalist Lightweight Model for Sequence Classification Tasks",
    author = "Stepanov, Ihor and Shtopko, Mykhailo and Vodianytskyi, Dmytro and Lukashov, Oleksandr and Yavorskyi, Alexander and Yaroshenko, Mykyta",
    year = "2025",
    eprint = "2508.07662",
    archivePrefix = "arXiv",
    primaryClass = "cs.LG",
    url = "https://arxiv.org/abs/2508.07662"
}

[4] GLiFormer: A Generalist Multitask Transformer Encoder.

@misc{stepanov2026gliformer,
    title = "{GL}i{F}ormer: A Generalist Multitask Transformer Encoder",
    author = "Stepanov, Ihor and Shtopko, Mykhailo and Vodianytskyi, Dmytro and Lukashov, Oleksandr and Yaroshenko, Mykyta",
    year = "2026",
    url = "https://www.knowledgator.com/research/gliformer"
}