Cross-Lingual Contrastive Learning

code for designing and experimenting with cross-lingual contrastive learning in MoE LLMs

Main gdoc: https://docs.google.com/document/d/1qT4KlRS1LK5fCZ9YlyJhD-hL49yjknHLnaazHi9SZDA/

This repo was split and copied over from https://github.com/mohsenfayyaz/moe on 4/2/2026. If any files are missing, please ask Lucas. If any code doesn't work because of misplaced files, please fix.

Code Organization

• contrastive_training/ is the main code containing implementation and scripts for contrastive training method
• language_evals/ contains the code for downstream task evaluation (and also SFT+eval)
• routing_analysis/

Conda envs

see contrastive_training/README.md and routing_analysis/README.md