Cross-Lingual Information Extraction and Machine Translation System

NLP Applications (S2-25_AIMLCZG519) — Assignment 2, Part 1 BITS Pilani Work Integrated Learning Programmes — M.Tech. Artificial Intelligence and Machine Learning

An integrated NLP pipeline that translates text between English and four Indic languages (Hindi, Tamil, Marathi, Bengali) while extracting Named Entities, Relations, and Temporal Events, exposed through an interactive Streamlit web dashboard.

Repository: github.com/hamza-aziz-ai/CrossLingual-IE-MT

Group

Quick start

The project uses pipenv for dependency management:

pip install --user pipenv
pipenv install --dev
pipenv run streamlit run frontend/app.py

pipenv install --dev installs everything, including the spaCy en_core_web_sm model — no separate download step needed. See SETUP.md for full installation instructions (including CPU vs. GPU torch), troubleshooting, and how to generate the application screenshots.

Deliverables map

Architecture at a glance

Indic input  --[NLLB-200: Indic→English]--> English text --[NER + RE + Event]--> IE results
English input --[NLLB-200: English→Indic]--> Indic text (display only)
              \--[NER + RE + Event on the original English]--> IE results

Translation runs on facebook/nllb-200-distilled-600M; Information Extraction runs on the English side of the pair (Translate-then-Extract), using spaCy + IndicNER for NER, REBEL (with an automatic rule-based fallback) for Relation Extraction, and spaCy + dateparser for Event/Temporal Extraction. Full rationale, preprocessing steps, and challenges encountered are documented in docs/System_Report.docx. The comparison against a Direct Cross-Lingual Extraction architecture and a model-compression/deployment plan are documented in docs/Task_B_Efficiency_and_Optimization_Plan.pdf.