PhyTS

PhyTS is a benchmark suite of precision scientific time series datasets for machine learning, spanning experiments in gravitational-wave detection, dark matter searches, neutrino mass determination, and stellar variability detection. Despite their diverse scientific goals, these domains share a common challenge: recovering weak, structured signals and estimating underlying physical parameters from noise-dominated measurements.

Unlike standard sequence modeling benchmarks such as audio and speech, these data exhibit non-Gaussian and nonstationary noise, long-range temporal correlations, detector-specific systematics, irregular sampling, and signals that are sparse, weak, or only partially modeled. As a result, they provide a challenging testbed for evaluating whether modern AI methods can support downstream scientific inference.

Existing time series benchmarks inadequately prepare models for scientific applications due to three critical limitations: (1) unrealistic noise assumptions — most benchmarks use Gaussian noise rather than the complex, frequency-dependent backgrounds found in real detectors; (2) missing physics constraints — standard metrics ignore that scientific applications require interpretable confidence estimates and respect for physical laws; and (3) simplified temporal structure — scientific signals often exhibit multi-scale dependencies and rare transient events that are poorly represented in current datasets.

PhyTS addresses these gaps across four dimensions:

• Realistic complexity: physics domains spanning 12 orders of magnitude in sampling rate with authentic detector noise, non-stationary backgrounds, and scientifically meaningful signal-to-noise ratios.
• Physics-informed evaluation: task formulations that reflect real experimental constraints, including parameter inference under uncertainty and noise reduction across orders of magnitude of signal frequency.
• Systematic baselines: comprehensive comparisons of supervised models and six zero-shot foundation models, revealing fundamental limitations of current architectures on scientific data.
• Grounding in scientific measurement: each predicted value propagates transparently into a downstream physics result, enabling evaluation by scientific impact.

Paper: PhyTS: A Benchmark for Scientific Time Series (NeurIPS 2026)
Data: PhyTS-team/PhyTS-bench on Hugging Face

---

Datasets

Detailed dataset descriptions and preprocessing steps are in the paper (Section 3).

---

Setup

Three environments are needed depending on which models you run. All main environments are managed by uv.

# PyTorch models (S4D, CNN, RNN, MLP, Conv-AE) and data download — covers all four datasets
make env
source .venv/bin/activate

# LinOSS (JAX/Equinox) — same environment with JAX + CUDA 12 added
make env-jax
source .venv/bin/activate

# Foundation models (MOMENT, Chronos, TimesFM, Time-MoE, MOIRAI, Granite TTM)
# Uses a separate Python 3.10 environment due to conflicting dependencies
make env-fm
source benchmarks/foundation/.venv/bin/activate

CUDA 13 (driver ≥ 580): replace env-jax with uv sync --extra jax --extra cu13.

---

Data

All datasets are on Hugging Face at PhyTS-team/PhyTS-bench.

For pipeline verification (NeurIPS reproducibility), sample files covering all four domains can be downloaded in one step:

python data/download.py --sample

To download a full dataset:

python data/download.py --domain tess        # 194 MB
python data/download.py --domain ligo        # ~157 GB
python data/download.py --domain project8    # ~44 GB
python data/download.py --domain tidmad      # ~163 GB; then run:
python data/TIDMAD/preprocess_tidmad.py \
    --data_dir data/TIDMAD/original --out_dir data/TIDMAD/preprocessed

See data/download.py --help for options.

---

Training

All training goes through main.py (LightningCLI). Pick any config from configs/:

python main.py fit --config <path/to/config.yaml>

One example per domain:

# LIGO — chirp-mass regression, S4D
python main.py fit --config configs/LIGO/train_ligo_s4d_gaussnll_regression.yaml

# ABRACADABRA — denoising, LinOSS  (requires env-jax)
python main.py fit --config configs/TIDMAD/train_tidmad_linoss_denoising.yaml

# TESS — variability classification, S4D
python main.py fit --config configs/TESS/train_tess_s4d_classification.yaml

# Project 8 — energy regression, S4D
python main.py fit --config configs/Project8/train_project8_s4d_regression_energy_gaussiannll.yaml

Any config value can be overridden on the command line:

python main.py fit --config configs/LIGO/train_ligo_conv1d_gaussnll_regression.yaml \
  --model.init_args.lr 1e-4 \
  --data.init_args.batch_size 64

The full set of configs is in configs/.

---

Benchmarks

Each domain has a pipeline script that trains all models and runs evaluation end-to-end:

bash benchmarks/LIGO/run.sh
bash benchmarks/TIDMAD/run.sh
bash benchmarks/TESS/run.sh
bash benchmarks/Project8/run.sh

For foundation models (zero-shot evaluation):

source benchmarks/foundation/.venv/bin/activate
python benchmarks/foundation/run_benchmark.py \
  --models moment chronos timesfm moirai granite_ttm \
  --tasks  forecasting denoising embedding \
  --mode   zero_shot

ABRACADABRA denoising score — after training, evaluate all variants:

PYTHONPATH=src python benchmarks/TIDMAD/evaluate_all.py

---

Results

Numbers from the paper (Table 2). Results are written to results/ after running the benchmark scripts. Foundation models evaluated zero-shot.

---

Repository layout

src/
  models/         # S4D, LinOSS, CNN, RNN, MLP, Conv-AE, classical filter
  tasks/          # LightningModules per domain (LIGO, TIDMAD, TESS, Project8)
  dataloader/     # PyTorch DataModules per domain
  functions/      # Loss functions, dropout, learning-rate schedules
configs/          # YAML training configs (one per model × domain × task)
benchmarks/
  LIGO/           # run.sh + evaluation pipeline
  TIDMAD/         # run.sh + evaluation pipeline
  TESS/           # run.sh + evaluation pipeline
  Project8/       # run.sh + evaluation pipeline
  foundation/     # Foundation-model wrappers and benchmark runner
data/
  LIGO/           # HDF5 strain files (gitignored; download from HuggingFace)
  TIDMAD/         # Raw HDF5 + preprocess_tidmad.py
  TESS/           # Parquet files (gitignored; bash benchmarks/TESS/setup_data.sh)
  Project8/       # HDF5 files (gitignored; download from HuggingFace)
plots/            # Generated by benchmark scripts (gitignored)
results/          # Generated by benchmark scripts (gitignored)
main.py           # LightningCLI entry point
pyproject.toml    # Dependencies (uv)
Makefile          # Environment setup targets

---

Models

Supervised (trained per domain)

Tasks are organized by domain; configs live in configs/<domain>/. Any model can be swapped into any compatible task by changing model.class_path in the YAML.

Foundation models (zero-shot and fine-tuned)

Foundation models are evaluated zero-shot and with lightweight fine-tuning via benchmarks/foundation/run_benchmark.py. Each wrapper exposes a uniform interface (forecast, denoise, embed) so the same evaluators run across all models.

---

Citation

@inproceedings{phyts2026,
  title     = {PhyTS: A Benchmark for Scientific Time Series},
  author    = {...},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026}
}

---

License

Apache 2.0. The S4D implementation is derived from state-spaces/s4 (Apache 2.0); see NOTICE for full attribution. LinOSS adapted from tk-rusch/linoss.