helenqu/4M-MI

clone of https://github.com/apple/ml-4m/tree/main for mutual information benchmarking project

Python

23 commits

updated Apr 13, 2026

README

train tokenizer: scripts/train_tokenizer.sh save tokens: scripts/save_tokens.sh train 4m: scripts/train.sh regress theta from trained tokens: scripts/theta_prediction_from_raw.sh

Similar repos

0nutation/SLMTokBench

SLMTokBench for paper "SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models"

snimu/mixture-of-tokenizers

Mixture of Tokenizers

alexstein0/efficient_tokenization_for_inference

garyz712/GPS

Research on Token Reordering Effect in Multimodal Learning

liyunlongaaa/MiMo-Tokenizer-Trainer

Unofficial implementation of training pipeline in mimo-tokenizer about "MiMo-Audio: Audio Language…

MinishLab/tokenlearn

Pre-train Static Word Embeddings

111

will-singularity/Skywork-MM

Empirical Study Towards Building An Effective Multi-Modal Large Language Model

futureke/MTFSLaMM