Energy Scenario Evaluation Dataset and Benchmark (Energy-EVA)

Updates!!

• [2026-06-01]: Released Energy-EVA 2.0 — added a new day-ahead electricity price forecasting scene (3 sub-datasets); upgraded four third-party baselines (Chronos-2, Moirai-2.0-R-small, Toto-2.0-2.5B, TimesFM-2.5-200M); added EnergyTS V3.0.

Overview

The Energy Scenario Evaluation Dataset and Benchmark (Energy-EVA) serves as a dedicated evaluation standard for applications in the energy domain, focusing currently on zero-shot time series forecasting tasks, including renewable energy production and industrial usage. It provides a consistent framework and datasets for assessing model generalization within practical energy environments, and features a versatile structure to facilitate the future incorporation of multi-modal tasks. Looking ahead, Energy-EVA plans to broaden its scope to encompass various task types, including applications for energy-oriented large language models and visual tasks for contexts such as power grid inspections.

Key Features

• Time series datasets tailored for applications in energy and electricity
• Customized evaluation metrics for precise energy and electricity forecasting
• Performance comparison of leading open-source models (Moirai, Chronos, TiRex, Sundial, TOTO, TimesFM) alongside our proprietary model EnergyTS V3.0.

Scene Description

We provide evaluation benchmarks for four scenarios:

1. Univariate electricity load forecasting - Single variable power consumption prediction with datetime information
2. Photovoltaic power generation forecasting with meteorological covariates - Solar power prediction incorporating weather data and datetime information
3. Wind power generation forecast with meteorological covariates - Wind power prediction incorporating weather data and datetime information.
4. Day-ahead electricity price forecasting - Hourly day-ahead electricity price prediction with datetime information.

Every scenario encompasses several sub-datasets. All data is obtained from publicly accessible and traceable platforms. Using data pre-processing and processing methods, the initial raw data is converted to standardized evaluation data files.

Dataset Description

Download link for Solar/Wind/Load

Download link for Price

Leaderboard

Access the evaluation detail to examine comprehensive information.

Solar Power Generation Forecasting

Wind Power Generation Forecasting

Power Load Forecasting

Day-ahead Electricity Price Forecasting

Project Structure

├── Core # Modules that are commonly used or various utilities
│   ├── Models
│   ├── Utils
│   └── __init__.py
├── LEGAL.md
├── LICENSE
├── README.md # This document
├── pyproject.toml # Project requirements with `uv`
├── time_series_portal # Primary entrance for time series benchmark
│   ├── __init__.py
│   ├── benchmark_tasks.py  # Load difference scene benchmark tasks
│   ├── config.py # Several typical configurations, like the path for storing the model
│   ├── evaluation.py # Execute this for assessment
│   ├── evaluation_methods # Implemented evaluation methods
│   │   └── third_party_methods # Contains v1 baselines (chronos/, moirai/, timesfm/, toto/, tirex/, sundial/) and v2 baselines (chronos2/, moirai2/, timesfm25/, toto2/)
│   ├── evaluation_utils  # Utilities for evaluating time series
│   ├── leaderboard_generate.py # Generate customized leaderboard
│   └── visualize_multi_model_results.py  # Visualize predictions from multiple models
└── uv.lock # Generated by `uv`

Energy-EVA employs fev for time-series forecasting evaluation purposes. The system operates by interpreting datasets and transforming them into Context and Future segments to invoke models for inference.

Customized models must derive from either ArchAdapter or CallableAdapter and include the generate method. These callable algorithms are then registered within the registry using @registry.register("algorithm_name").

The benchmarking process comprises the following components:

• evaluation.py - Script for batch benchmarking
• visualize_multi_model_results.py - Visualization of results from multiple models
• leaderboard_generate.py - Leaderboard construction

Pipeline

Create virtual environment:

# cd into Energy-EVA
pip install uv
uv venv . # Substitute this with the path to your specific virtual environment
uv sync   # Install basic requirements. For third-party models, add necessary packages (e.g., chronos-forecasting).
source .venv/bin/activate

Run evaluation:

python time_series_portal/evaluation.py \ 
        --dataset_path PATH/TO/YOUR/DATASET/LOCATION \
        --target_path PATH/TO/YOUR/EVALUATION_RESULT/LOCATION \
        --scene wind load solar price \
        --model dummy_model \

Use python time_series_portal/evaluation.py --help to view more configuration options

Generate visualization of multi-models:

python time_series_portal/visualize_multi_model_results.py \
        --dataset_path PATH/TO/YOUR/DATASET/LOCATION \
        --target_path PATH/TO/YOUR/EVALUATION_RESULT/LOCATION \
        --scene load \
        --model dummy_model toto_2.0_2.5B

Use python time_series_portal/visualize_multi_model_results.py --help to view more configuration options

Generate leaderboard among multi-models:

python time_series_portal/leaderboard_generate.py \
        --source_path PATH/TO/YOUR/EVALUATION_RESULT/LOCATION \
        --target_path PATH/TO/YOUR/EVALUATION_RESULT/LOCATION/leaderboard \
        --select_column dataset_path

Evaluate proprietary algorithms

Examples are available in time_series_portal/evaluation_methods/third_party_methods. Implement your model in Core/Models/arch_adapter and invoke it via time_series_portal/evaluation_methods/adapter_methods.py.