FETS Benchmark — Energy Benchmark TSFM

Code for the paper "FETS Benchmark: Foundation Models Enable Scalable and Generalizable Energy Time Series Forecasting" by Marco Obermeier, Marco Pruckner, Florian Haselbeck, and Andreas Zeiselmair.

A benchmark and comparison of timeseries foundation models (TSFM) on different prediction tasks for the energy industry.

Resources

Installation

Each model runs in its own isolated conda environment to avoid dependency conflicts.

1. Clone external repositories

# IBM Granite (FlowState)
git clone git@github.com:ibm-granite/granite-tsfm.git

# Google TimesFM
git clone git@github.com:google-research/timesfm.git

# TabPFN extensions
git clone https://github.com/priorlabs/tabpfn-extensions.git

The install scripts expect these repos to be cloned into the same parent directory as this repo.

2. Install model environments

Each script creates a dedicated conda env and installs all required dependencies:

bash conda_envs/install_chronos2.sh      # → fets_env_chronos2
bash conda_envs/install_timesfm.sh       # → fets_env_timesfm
bash conda_envs/install_tabpfn.sh        # → fets_env_tabpfn
bash conda_envs/install_tirex.sh         # → fets_env_tirex
bash conda_envs/install_tirex2.sh        # → fets_env_tirex2
bash conda_envs/install_xgboost.sh       # → fets_env_xgboost
bash conda_envs/install_randomforest.sh  # → fets_env_randomforest
bash conda_envs/install_flowstate.sh     # → fets_env_flowstate
bash conda_envs/install_toto.sh          # → fets_env_toto

Environment overview

Dataset setup

Download the FETS Dataset from Zenodo and point the benchmark to its location.

The benchmark configs in 3_config/2_multi_run_config/ reference only datasets redistributable under the Zenodo release. All paths are constructed from a single base_path variable at the top of each config — edit this one line to point at your local copy of the extracted Zenodo archive:

base_path: /path/to/your/3_published_data

Datasets that could not be redistributed (Netztransparenz, CAISO, 50Hertz, Bayernwerk) are kept in the file but commented out, with the restriction reason noted. To run the full benchmark as in the paper, obtain these sources directly from the original providers.

Weather covariates are not bundled per series. They are provided centrally in the Weather_OpenMeteo/ folder of the Zenodo archive (Open-Meteo, by GPS point: historical forecast, ERA5 reanalysis, ECMWF IFS single runs) and are injected at runtime from each dataset's gps_coordinates (defined in the configs). The per-series covariates/ folders contain only the holiday calendars.

Easy example (single run)

A single Hydra run on one dataset / one model / one configuration. Replace $DATA_ROOT with your local path to the extracted Zenodo archive (3_published_data/):

DATA_ROOT=/path/to/your/3_published_data

CUDA_VISIBLE_DEVICES=0 python energy_benchmark.py --multirun \
    model.active=chronos2 model.mode=covariate \
    evaluator.show_plots=true \
    dataset=0_covariate_default \
    dataset.data_path="$DATA_ROOT/Load/DE Power - Load DE/public_power.parquet" \
    dataset.target_column="Load" \
    dataset.covariate_data_paths=["$DATA_ROOT/Load/DE Power - Load DE/covariates/calendar_DE_2010_2030.parquet"] \
    results_path=./outputs/bench_de_load.csv \
    context_length=2000 horizon=192

covariate_data_paths holds only the calendar file; the GPS-based weather covariates for this dataset are added automatically at runtime from its gps_coordinates in config_energy_bench_pub.yaml (source selectable via weather_source=hist_fc|nwp|era5, default hist_fc).

Inference modes (model.mode): univariate forecasts the target column alone (default); covariate additionally uses the calendar and weather covariates as features.

Full benchmark

To reproduce the full benchmark (all models × all datasets × all context/horizon combinations, plus the sensitivity analyses), run the multi-GPU pipeline:

python pipeline_bench_multi_gpu.py

The pipeline dispatches jobs across GPUs/CPUs via a resource queue and merges per-worker CSVs into a single results file under ./outputs/worker/. It reads its dataset list from one of the configs in 3_config/2_multi_run_config/:

All these configs use the same base_path: line at the top — edit only this line to point at your local 3_published_data/ directory.

Individual stages are toggled via the if 0: / if 1: blocks at the bottom of pipeline_bench_multi_gpu.py:

• univariate (chronos2, timesfm, tirex, tirex2, flowstate, toto)
• covariate (tabpfn, chronos2, timesfm, tirex2)
• training (xgb_native_optuna, xgb_native_optuna_mo, rf_native_optuna, chronos2 fine-tuning)
• training with conformalized quantile regression (xgb_native_optuna_cqr, rf_native_optuna_cqr) — recalibrates the quantile forecasts of the tree-based baselines on a held-out calibration slice (CQR, Romano et al. 2019); used for the CQR rows in the PICP/nCRPS tables
• sensitivity analyses (context length, horizon, aggregation, weather source, full-year seasonality)

Resource limits per stage (max_parallel_jobs, cpus_per_job) are set in the same file and should be adjusted to your hardware.

Citation

If you use this code or data for your research, please cite both the paper and the dataset:

Paper

@article{obermeier_fets_benchmark,
  title   = {FETS Benchmark: Foundation Models Enable Scalable and Generalizable Energy Time Series Forecasting},
  author  = {Obermeier, Marco and Pruckner, Marco and Haselbeck, Florian and Zeiselmair, Andreas},
  year    = {2026},
  doi     = {https://arxiv.org/abs/2604.22328}
}

Dataset

@dataset{obermeier_fets_dataset,
  title     = {FETS Dataset: Foundation Models Enable Scalable and Generalizable Energy Time Series Forecasting},
  author    = {Obermeier, Marco and Pruckner, Marco and Haselbeck, Florian and Zeiselmair, Andreas},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19418721},
  url       = {https://zenodo.org/records/19418721}
}