GLiNER Multi V2 for Entity Extraction from email signatures

A hybrid NER (Named Entity Recognition) approach is used for extracting PII entities from email signatures across multiple languages and writing systems.

Features

• Hybrid Extraction Approach: Combines GLiNER multi-v2.1, NuNER Zero-4k, and regex patterns for optimal precision and recall
• Multilingual Support: Tested across 6+ languages including:
  • Latin script: English, Spanish, French, German, Portuguese, Italian
  • Cyrillic script: Russian, Ukrainian
  • CJK: Chinese
  • Devanagari: Hindi
• Email Signature Detection: Intelligent separation of email body and signature blocks
• Entity Types: Extracts person names, job titles, organizations, phone numbers, addresses, emails, and URLs

Installation

pip install -r requirements.txt

Requirements (CVE-resolved versions)

• Python 3.8+
• gliner>=0.2.25 - GLiNER NER models (CVE-resolved)
• transformers>=4.51.3,<5.2.0 - Model backbone (CVE-resolved)
• email_reply_parser>=0.5.12 - Email parsing
• pytest>=7.0.0 - Testing framework
• urllib3>=2.2.1 - Secure HTTP (CVE-resolved)
• protobuf>=4.25.3 - Serialization (CVE-resolved)
• pillow>=10.3.0 - Imaging (CVE-resolved)
• pip>=24.0 - Package manager (CVE-resolved)
• filelock>=3.13.1 - File locking (CVE-resolved)
• wheel>=0.43.0 - Packaging (CVE-resolved)

Project Structure

GLiNER_Multi_V2/
├── src/
│   ├── hybrid_approach.py    # Main hybrid extraction (GLiNER + NuNER + regex)
│   └── email_sig.py          # Email signature detection and parsing
├── evals/
│   ├── eval_email_parts.py   # Evaluate body/signature separation
│   ├── eval_email_sig.py     # Evaluate entity extraction
│   └── eval_hybrid_on_signatures.py  # Combined evaluation pipeline
├── tests/
│   └── test_signatures.py    # Multilingual signature extraction tests
├── data/
│   ├── email_parts_evaluation_dataset.json
│   ├── multilingual_email_evaluation_dataset.json
│   └── predicted_output/     # Evaluation results
├── requirements.txt
└── README.md

Usage

Extract Entities from Email Signatures

from src.hybrid_approach import extract_signature_entities

signature = """
Best regards,
Sarah Jenkins
Senior Software Engineer
CloudScale Inc.
Tel: +1-555-0199
[email protected]
"""

entities = extract_signature_entities(signature)
for ent in entities:
    print(f"{ent['label'].upper()}: {ent['text']}")

Output:

PERSON: Sarah Jenkins
JOB TITLE: Senior Software Engineer
ORGANIZATION: CloudScale Inc.
PHONE NUMBER: +1-555-0199
EMAIL: [email protected]

Separate Email Body and Signature

from src.email_sig import identify_email_parts

email_text = """
Hi John,

Please find the report attached.

Best regards,
Maria López
Product Manager
"""

parts = identify_email_parts(email_text)
print("Body:", parts['body'])
print("Signature:", parts['signature_block'])

Models Used

Why NuNER Works with GLiNER

You might notice that we load NuNER using GLiNER.from_pretrained("numind/NuNER_Zero-4k"). This works because GLiNER is not a single fixed NER model — it's a framework and architecture that can load different pretrained checkpoints built on the same underlying design.

How it works:

1. Shared Architecture: Both GLiNER and NuNER use a span-based NER approach where entity labels are embedded alongside the input text, allowing zero-shot entity extraction without retraining.

2. Compatible Checkpoints: NuNER Zero was trained using the GLiNER architecture but with different training data and objectives (focused on higher precision). The model weights are stored in the same format, making them interchangeable.

3. Framework Flexibility: The GLiNER class acts as a loader that can instantiate any compatible checkpoint from HuggingFace Hub, similar to how transformers.AutoModel works.

from gliner import GLiNER

# Both use the same GLiNER class, different checkpoints
gliner_model = GLiNER.from_pretrained("urchade/gliner_multi-v2.1")  # Recall-focused
nuner_model = GLiNER.from_pretrained("numind/NuNER_Zero-4k")        # Precision-focused

This design allows us to ensemble multiple models with complementary strengths — GLiNER for broad multilingual coverage and NuNER for high-confidence predictions.

Testing

Run the test suite:

pytest tests/test_signatures.py -v

Test coverage includes:

• Russian, Chinese, Spanish, French, Hindi, German signatures
• Person names, job titles, organizations
• International phone number formats
• Academic titles (Dr., Prof., Dipl.-Ing.)
• Mixed-language signatures

Evaluation

The evals/ folder contains three evaluation scripts that measure different aspects of the pipeline.

Evaluation Scripts

Running Evaluations

# 1. Evaluate email body/signature separation
python evals/eval_email_parts.py

# 2. Evaluate entity extraction quality
python evals/eval_email_sig.py

# 3. Full pipeline: detect signature → extract entities
python evals/eval_hybrid_on_signatures.py

Metrics Calculated

eval_email_parts.py - Signature Detection:

• Exact match accuracy (body and signature)
• Jaccard similarity scores
• Results grouped by writing script (Latin, Cyrillic, CJK, etc.)

eval_email_sig.py - Entity Extraction:

• Precision: % of predicted entities that are correct
• Recall: % of ground truth entities that were found
• F1 Score: Harmonic mean of precision and recall
• Metrics broken down by:
  • Entity type (person, job title, organization, phone, email, etc.)
  • Writing script (Latin, Cyrillic, Chinese, Devanagari)
• Match types: exact, partial, label mismatch, false positive/negative

eval_hybrid_on_signatures.py - Pipeline Stats:

• Total entities extracted per email
• Entity type distribution
• Extraction success rate by script

Output Files

Results are saved to data/predicted_output/:

predicted_output/
├── email_parts_evaluation_report.json    # Detailed separation metrics
├── email_parts_evaluation_summary.csv    # Aggregated stats
├── email_parts_evaluation_details.csv    # Per-email results
├── email_signature_evaluation_report.json # Entity extraction metrics  
├── email_signature_evaluation_summary.csv # Precision/recall by type
├── email_signature_evaluation_details.csv # Per-entity matches
├── hybrid_extraction_on_signatures.json   # Full pipeline results
└── hybrid_extraction_on_signatures.csv    # Entity counts per email

Benchmark Results

Email Parts Detection (Body/Signature Separation)

By Writing Script:

Entity Extraction Performance

Overall Metrics (50 emails):

By Writing Script:

Pipeline Extraction Summary (60 emails)

Entity Type Distribution:

Configuration

Adjust extraction thresholds in hybrid_approach.py:

extract_signature_entities(
    text,
    gliner_threshold=0.4,  # Lower = more recall
    nuner_threshold=0.5,   # Higher = more precision
    min_length=2           # Minimum entity length
)