Indic Translate Server

Table of Contents

• Overview
• Prerequisites
• Setting Up the Development Environment
• Downloading Translation Models
• Running with FastAPI Server
• Evaluating Results
• Build Docker Image
• References
• Contributing
• License
• FAQ

Overview

This project sets up an Indic translation server, allowing translation between various languages including English, Kannada, Hindi, and others. It utilizes models from AI4Bharat to perform translations.

We suggest to use non-distilled models for better translation.

Live Server

We have hosted an Translation service for Indian languages.

• https://workshop.dwani.ai

Prerequisites

• Python 3.10
• Ubuntu 22.04
• Internet access to download translation models.

Setting Up the Development Environment

1. Create a virtual environment:

   python -m venv venv
   

2. Activate the virtual environment:

• For Mac/Linux

  source venv/bin/activate
  

• On Windows, use:

  venv\Scripts\activate
  

3. Install dependencies:

   pip install -r requirements.txt
   

Model Downloads for Translation

• Request Access permission for the model form HuggingFace
• Collection Models on HuggingFace - IndicTrans2

Below is a table summarizing the available models for different translation tasks:

Sample Code

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from IndicTransToolkit import IndicProcessor

DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

src_lang, tgt_lang = "hin_Deva", "eng_Latn"
model_name = "ai4bharat/indictrans2-indic-en-dist-200M"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

model = AutoModelForSeq2SeqLM.from_pretrained(
    model_name, 
    trust_remote_code=True, 
    torch_dtype=torch.float16, # performance might slightly vary for bfloat16
    attn_implementation="flash_attention_2"
).to(DEVICE)

ip = IndicProcessor(inference=True)

input_sentences = [
    "जब मैं छोटा था, मैं हर रोज़ पार्क जाता था।",
    "हमने पिछले सप्ताह एक नई फिल्म देखी जो कि बहुत प्रेरणादायक थी।",
]

batch = ip.preprocess_batch(
    input_sentences,
    src_lang=src_lang,
    tgt_lang=tgt_lang,
)

DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

# Tokenize the sentences and generate input encodings
inputs = tokenizer(
    batch,
    truncation=True,
    padding="longest",
    return_tensors="pt",
    return_attention_mask=True,
).to(DEVICE)

# Generate translations using the model
with torch.no_grad():
    generated_tokens = model.generate(
        **inputs,
        use_cache=True,
        min_length=0,
        max_length=256,
        num_beams=5,
        num_return_sequences=1,
    )

# Decode the generated tokens into text
with tokenizer.as_target_tokenizer():
    generated_tokens = tokenizer.batch_decode(
        generated_tokens.detach().cpu().tolist(),
        skip_special_tokens=True,
        clean_up_tokenization_spaces=True,
    )

# Postprocess the translations, including entity replacement
translations = ip.postprocess_batch(generated_tokens, lang=tgt_lang)

for input_sentence, translation in zip(input_sentences, translations):
    print(f"{src_lang}: {input_sentence}")
    print(f"{tgt_lang}: {translation}")

Run the sample code

python translate_code.py

Alternate forms of Development

Running with FastAPI Server

Install dependencies:

pip install -r server-requirements.txt

You can run the server using FastAPI:

1. with Distilled model

python src/server/translate_api.py --port 10802 --host 0.0.0.0 --device cuda --use_distilled

2. with non-distilled model

python src/server/translate_api.py --port 10802 --host 0.0.0.0 --device cuda 

Evaluating Results for FastAPI Server

You can evaluate the translation results using curl commands. Here are some examples:

English to Kannada

curl -X 'POST' \
  'http://localhost:10802/translate?tgt_lang=kan_Knda&src_lang=eng_Latn&device_type=cuda' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "sentences": [
    "Hello, how are you?", "Good morning!"
  ],
  "src_lang": "eng_Latn",
  "tgt_lang": "kan_Knda"
}'

Response:

{
  "translations": [
    "ಹಲೋ, ಹೇಗಿದ್ದೀರಿ? ",
    "ಶುಭೋದಯ! "
  ]
}

Kannada to English

curl -X 'POST' \
  'http://localhost:10802/translate?src_lang=kan_Knda&tgt_lang=eng_Latn&device_type=cuda' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "sentences": [
    "ನಮಸ್ಕಾರ, ಹೇಗಿದ್ದೀರಾ?", "ಶುಭೋದಯ!"
  ],
  "src_lang": "kan_Knda",
  "tgt_lang": "eng_Latn"
}'

Response:

{
  "translations": ["Hello, how are you?", "Good morning!"]
}

Kannada to Hindi

curl -X 'POST' \
  'http://localhost:10802/translate?src_lang=kan_Knda&tgt_lang=hin_Deva' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "sentences": [
    "ನಮಸ್ಕಾರ, ಹೇಗಿದ್ದೀರಾ?", "ಶುಭೋದಯ!"
  ],
  "src_lang": "kan_Knda",
  "tgt_lang": "hin_Deva"
}'

Response

{
  "translations": [
    "हैलो, कैसा लग रहा है? ",
    "गुड मॉर्निंग! "
  ]
}

---

Languages Supported

Here is the list of languages supported by the IndicTrans2 models:

References

• IndicTrans2 Paper
• AI4Bharat IndicTrans2 Model
• AI4Bharat IndicTrans2 GitHub Repository
• IndicTransToolkit
• Extra - pip install git+https://github.com/VarunGumma/IndicTransToolkit.git

Contributing

We welcome contributions! Please read the CONTRIBUTING.md file for guidelines on how to contribute to this project.

Also you can join the discord group to collaborate

License

This project is licensed under the MIT License - see the LICENSE file for details.

---

License

• IndicTrans License
• IndicTrans Data License

---

This README provides a comprehensive guide to setting up and running the Indic Translate Server. For more details, refer to the linked resources.

Citation

@article{gala2023indictrans,
title={IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages},
author={Jay Gala and Pranjal A Chitale and A K Raghavan and Varun Gumma and Sumanth Doddapaneni and Aswanth Kumar M and Janki Atul Nawale and Anupama Sujatha and Ratish Puduppully and Vivek Raghavan and Pratyush Kumar and Mitesh M Khapra and Raj Dabre and Anoop Kunchukuttan},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2023},
url={https://openreview.net/forum?id=vfT4YuzAYA},
note={}
}