Rethinking Tokenization for Clinical Time Series (ML4H Findings 2025)

Paper

Acknowledgements

This repository contains the code for the experiments in our paper, "Rethinking Tokenization for Clinical Time Series: When Less is More." The codebase is adapted from the meds-torch library. We thank the original authors for their foundational work. For the maintained version, please see the official repository.

---

Overview

This work presents a systematic evaluation of tokenization approaches for clinical time series modeling. We compare Triplet and TextCode strategies across four prediction tasks on MIMIC-IV to investigate the roles of time, value, and code representations. Our findings suggest that for transformer-based models, tokenization can often be simplified without sacrificing performance.

Key Findings Summary

Repository Structure

Research Code Variants

• triplet_encoder_time2vec.py - Time2Vec implementation for advanced time encoding
• triplet_encoder_lete.py - LeTE (Learnable Time Embeddings) implementation
• triplet_encoder_code_only.py - Code-only ablation (no time/value features)
• triplet_encoder_no_time.py - No-time ablation variant
• triplet_encoder_no_value.py - No-value ablation variant
• textcode_encoder_flexible.py - Flexible TextCode encoder with trainable/frozen modes

Experiment Scripts

• experiment_baseline_multiseed.sh - Baseline Triplet experiments
• experiment_time2vec_multiseed.sh - Time2Vec experiments
• experiment_lete.sh - LeTE experiments
• experiment_code_only.sh - Code-only ablation experiments
• experiment_no_time.sh - No-time ablation experiments
• experiment_no_value.sh - No-value ablation experiments
• experiment_flexible_textcode.sh - TextCode optimization experiments

Dataset and Framework

• Dataset: MIMIC-IV processed into MEDS format
• Tasks: In-hospital mortality, ICU mortality, post-discharge mortality, 30-day readmission
• Framework: MEDS-Torch with transformer encoders
• Evaluation: AUROC with 10 random seeds, statistical significance testing

---

This research suggests that simpler, more parameter-efficient tokenization approaches may achieve competitive performance in clinical time series modeling, raising questions about the necessity of complex temporal encodings and highlighting the task-dependent role of value features.

---

Citation

If you use this code or build on this work, please cite:

@misc{attrach2025ehrtokenization,
  title     = {Rethinking Tokenization for Clinical Time Series: When Less is More},
  author    = {Al Attrach, Rafi and Fani, Rajna and Restrepo, David and Jia, Yugang
               and Celi, Leo Anthony and Sch\"{u}ffler, Peter},
  year      = {2025},
  note      = {Machine Learning for Health (ML4H) 2025 - Findings Track}
}