README
PhyTS Dataset
Abstract
We introduce PhyTS, a benchmark suite of precision scientific time series datasets for machine learning, spanning experiments in gravitational-wave detection, dark matter searches, neutrino mass determination, and stellar variability detection. Despite their diverse scientific goals, these domains share a common challenge: recovering weak, structured signals and estimating underlying physical parameters from noise-dominated measurements. Unlike standard sequence modeling benchmarks such as audio and speech, these data exhibit non-Gaussian and nonstationary noise, long-range temporal correlations, detector-specific systematics, irregular sampling, and signals that are sparse, weak, or only partially modeled. As a result, they provide a challenging testbed for evaluating whether modern AI methods can support downstream scientific inference. We provide standardized tasks, data splits, and evaluation protocols for denoising, signal recovery, and parameter inference across physics domains, along with baseline results. By unifying diverse weak-signal inference problems under a common framework, this benchmark aims to enable reproducible evaluation and accelerate the development of more robust, interpretable, and physically grounded methods for scientific time series analysis.
[!NOTE] 📌 Github repo: kyoon-mit/PhyTS
LIGO
LIGO Dataset Field Description
The LIGO dataset consists of training, validation, and test samples. Each "sample" is defined as a set of two, 64s-long strain data from the gravitational wave detectors, plus a complete description of the source parameters of the binary neutron stars. For each strain data, two additional time series are provided, one containing only the injected signal and the other only the background. Both are whitened using the same whitening filter that is applied to the strain data.
For training and validation, the dataset is provided in signal-to-noise ratio (SNR) bins of 5, from 5 to 50, resulting in 9 bins total ([5, 10], [10, 15], ..., [45, 50]). Each SNR bin contains 50,000 gravitational wave samples, split into 40,000 training samples and 10,000 validation samples. The dataset is generated according to a uniform SNR prior distribution per bin.
For testing, the dataset is provided in a single SNR bin from 5 to 50, following an astrophysically motivated SNR prior that is proportional to SNR^(-3). A total of 100,000 test samples are provided.
For all training, validation, and test samples, the injected signal waveforms are generated from the parameters that are drawn from the priors below. Masses are in solar masses, distance in Mpc, and angles in radians.
| Parameter | Distribution | Range |
|---|---|---|
| Primary mass | Triangular | [1.0, 2.5], mode = 2.5 |
| Secondary mass | Uniform | [1.0, primary mass] |
| Primary spin magnitude | Uniform | [0, 0.4] |
| Secondary spin magnitude | Uniform | [0, 0.4] |
| Primary spin tilt | Sine | [-1, 1] |
| Secondary spin tilt | Sine | [-1, 1] |
| Spin azimuthal separation | Uniform | [0, 2π] |
| Precession cone azimuth | Uniform | [0, 2π] |
| Comoving distance | Power law (α = 2) | [100, 1000] |
| Coalescence phase | Uniform | [0, 2π] |
| Inclination | Sine | [-1, 1] |
The dataset is provided in hdf5 format. Training and validation dataset files contain 50 samples each (~ 40 MB/file). All train samples are contained in a single file (~ 80 GB). Each hdf5 file includes the following
| Parameter | Type / Size | Description |
|---|---|---|
whitened_bkg | size=(2, 16384), dtype=float64 | Whitened detector background (no injection) |
whitened_signal | size=(2, 16384), dtype=float64 | Whitened injected waveform alone |
whitened_injected | size=(2, 16384), dtype=float64 | Whitened detector background with injected signal |
mass_1 | dtype=float32 | Primary mass, in solar masses |
mass_2 | dtype=float32 | Secondary mass, in solar masses |
chirp_mass | dtype=float32 | Chirp mass M_c = (m_1 * m_2)^(3/5) / (m_1 + m_2)^(1/5), in solar masses |
mass_ratio | dtype=float32 | Mass ratio q = m_2 / m_1 |
a_1 | dtype=float32 | Primary spin magnitude (dimensionless) |
a_2 | dtype=float32 | Secondary spin magnitude (dimensionless) |
tilt_1 | dtype=float32 | Primary spin tilt angle, in radians |
tilt_2 | dtype=float32 | Secondary spin tilt angle, in radians |
phi_12 | dtype=float32 | Azimuthal angle between component spins, in radians |
phi_jl | dtype=float32 | Azimuthal precession-cone angle, in radians |
s1x, s1y, s1z | dtype=float32 | Cartesian components of the primary spin vector |
s2x, s2y, s2z | dtype=float32 | Cartesian components of the secondary spin vector |
distance | dtype=float32 | Comoving distance to the source, in Mpc |
phi | dtype=float32 | Right ascension of the source, in radians |
dec | dtype=float32 | Declination of the source, in radians |
inclination | dtype=float32 | Inclination angle of the binary, in radians |
psi | dtype=float32 | Polarization angle, in radians |
phic | dtype=float32 | Coalescence phase, in radians |
snr | dtype=float32 | Network signal-to-noise ratio of the injection |
TESS
The TESS dataset is broken into training, validation and test data, which are identically constructed. There is a set for classification '/tess/split/tess_classification_{train,val,test}.parquet' and one for regression '/tess/split/tess_regression_{train,val,test}.parquet. The dataset was split 80/10/10 into training, validation, and test sets. Since multiple TESS light curves, from different TESS sectors, are available per unique Gaia Data Release 3 (DR3) identifier (ID), the split ensures that no Gaia DR3 ID appears in more than one split to avoid label leakage, with the exception of when different light curves for the same Gaia DR3 ID have different labels due to variations in dominant variability detected in separate TESS sectors. The data includes the following
TESS Dataset Field Description
| Parameter | Type / Size | Description |
|---|---|---|
GaiaID | dtype=int | Unique Gaia DR3 Identifier |
TIC | dtype=int | TESS Input Catalog Identifier |
sector | dtype=int | TESS Observing Sector (https://tess.mit.edu/observations/) |
label | dtype=string | Class label [FOR THE CLASSIFICATION TASK] |
frot | dtype=double | Near-core rotation rate of the star [FOR THE REGRESSION TASK] |
frot_err | dtype=double | Near-core rotation rate error [FOR THE REGRESSION TASK] |
time | dtype=list | Time stamps in BTJD (Barycentric TESS Julian Date. Corrected for light travel time to solar system barycenter) |
flux | dtype=list | Time series measurements in normalized flux |
TIDMAD
The TIDMAD dataset is broken into training and validation data which are identically constructed differing only by the time at which the detector data was taken. This temporal difference results in variable detector noise conditions. There are 20 hdf5 files (~ 4 GB/file) in both training and validation datasets. The total dataset size is 163 GB. Each file contains ~ 200 examples and the associated meta data. Each example is a set of ch1 time series, ch2 time series, and injected frequency. This data in each hdf5 file includes the following
TIDMAD Dataset Field Description
| Parameter | Type / Size | Description |
|---|---|---|
chunk_length | dtype=int | Number of samples per example |
n_chunks | dtype=int | Number of examples per file |
sample_rate_hz | dtype=int | Data taking sample rate in Hz |
signal_freq_choices | size=(618,1), dtype=int | Array of signal frequencies injected across the full dataset, all files |
signal_frequency | size=(200,1), dtype=int | Injected frequency for the example in Hz (signal_frequency[i] is the injected signal frequency for example i) |
time_series_ch1 | size=(200,10M), dtype=int | Noisy time series for the example in DAQ units. Conversion factor from DAQ units to mV is 40mV/128. (time_series_ch1[i] is the ch1 time series for example i) |
time_series_ch2 | size=(200,10M), dtype=int | Signal time series for the example in DAQ units. Conversion factor from DAQ units to mV is 40mV/128. (time_series_ch2[i] is the ch2 time series for example i) |
Project 8
The Project 8 dataset is broken into training, validation, and test datasets which are identically constructed.
All contain a set of electron events with randomized energy, pitch angle, and radius parameters.
The data are divided into hdf5 files with ~5000 electrons per file; each file is about 5 GB, with the exception of one, which has fewer electrons and is smaller.
There are about 50,000 electrons total in the dataset. The total dataset size is about 50 GB.
The train dataset contains 8 hdf5 files, the validation dataset has 1 hdf5 file, and the test dataset has 2 (including the smaller file).
The experiments presented in the publication use output_ts_I + output_ts_I_cav_noise and output_ts_Q + output_ts_Q_cav_noise as inputs (noisy in-phase and quadrature time series).
The target for regression is energy_eV (truth value for energy of the electron, in eV).
This data in each hdf5 file includes the following
Project 8 Dataset Field Description
| Parameter | Type / Size | Description |
|---|---|---|
output_ts_I | size=(24576,1), dtype=float32 | In-phase signal time series |
output_ts_Q | size=(24576,1), dtype=float32 | Quadrature signal time series |
output_ts_I_cav_noise | size=(24576,1), dtype=float32 | In-phase cavity (frequency-dependent) noise time series |
output_ts_Q_cav_noise | size=(24576,1), dtype=float32 | Quadrature cavity (frequency-dependent) noise time series |
output_ts_I_gauss_noise | size=(24576,1), dtype=float32 | In-phase Gaussian noise time series |
output_ts_Q_gauss_noise | size=(24576,1), dtype=float32 | Quadrature Gaussian noise time series |
energy_eV | dtype=float32 | Truth value for energy of the electron (target for regression), in eV |
avg_carrier_frequency_Hz | dtype=float32 | Truth value for average cyclotron frequency of the electron, in Hz |
start_carrier_frequency_Hz | dtype=float32 | Truth value for starting cyclotron frequency of the electron, in Hz |
avg_axial_frequency_Hz | dtype=float32 | Truth value for average axial frequency of the electron, in Hz |
pitch_angle_deg | dtype=float32 | Truth value for pitch angle (direction of the electron’s momentum with respect to the magnetic field) of the electron, in degrees |
radius_input_m | dtype=float32 | Truth value for starting radius of the electron before one trajectory step (input value), in meters |
radius_m | dtype=float32 | Truth value for starting radius of the electron after one trajectory step, in meters |
radius_phase | dtype=float32 | Truth value for instantaneous polar angle in the X-Y plane of the electron’s starting position, in meters |
slope_Hz | dtype=float32 | Truth value for slope of the track in spectrogram space, in Hz/s |
Contributors
Similar repos
README
PhyTS Dataset
Abstract
We introduce PhyTS, a benchmark suite of precision scientific time series datasets for machine learning, spanning experiments in gravitational-wave detection, dark matter searches, neutrino mass determination, and stellar variability detection. Despite their diverse scientific goals, these domains share a common challenge: recovering weak, structured signals and estimating underlying physical parameters from noise-dominated measurements. Unlike standard sequence modeling benchmarks such as audio and speech, these data exhibit non-Gaussian and nonstationary noise, long-range temporal correlations, detector-specific systematics, irregular sampling, and signals that are sparse, weak, or only partially modeled. As a result, they provide a challenging testbed for evaluating whether modern AI methods can support downstream scientific inference. We provide standardized tasks, data splits, and evaluation protocols for denoising, signal recovery, and parameter inference across physics domains, along with baseline results. By unifying diverse weak-signal inference problems under a common framework, this benchmark aims to enable reproducible evaluation and accelerate the development of more robust, interpretable, and physically grounded methods for scientific time series analysis.
[!NOTE] 📌 Github repo: kyoon-mit/PhyTS
LIGO
LIGO Dataset Field Description
The LIGO dataset consists of training, validation, and test samples. Each "sample" is defined as a set of two, 64s-long strain data from the gravitational wave detectors, plus a complete description of the source parameters of the binary neutron stars. For each strain data, two additional time series are provided, one containing only the injected signal and the other only the background. Both are whitened using the same whitening filter that is applied to the strain data.
For training and validation, the dataset is provided in signal-to-noise ratio (SNR) bins of 5, from 5 to 50, resulting in 9 bins total ([5, 10], [10, 15], ..., [45, 50]). Each SNR bin contains 50,000 gravitational wave samples, split into 40,000 training samples and 10,000 validation samples. The dataset is generated according to a uniform SNR prior distribution per bin.
For testing, the dataset is provided in a single SNR bin from 5 to 50, following an astrophysically motivated SNR prior that is proportional to SNR^(-3). A total of 100,000 test samples are provided.
For all training, validation, and test samples, the injected signal waveforms are generated from the parameters that are drawn from the priors below. Masses are in solar masses, distance in Mpc, and angles in radians.
| Parameter | Distribution | Range |
|---|---|---|
| Primary mass | Triangular | [1.0, 2.5], mode = 2.5 |
| Secondary mass | Uniform | [1.0, primary mass] |
| Primary spin magnitude | Uniform | [0, 0.4] |
| Secondary spin magnitude | Uniform | [0, 0.4] |
| Primary spin tilt | Sine | [-1, 1] |
| Secondary spin tilt | Sine | [-1, 1] |
| Spin azimuthal separation | Uniform | [0, 2π] |
| Precession cone azimuth | Uniform | [0, 2π] |
| Comoving distance | Power law (α = 2) | [100, 1000] |
| Coalescence phase | Uniform | [0, 2π] |
| Inclination | Sine | [-1, 1] |
The dataset is provided in hdf5 format. Training and validation dataset files contain 50 samples each (~ 40 MB/file). All train samples are contained in a single file (~ 80 GB). Each hdf5 file includes the following
| Parameter | Type / Size | Description |
|---|---|---|
whitened_bkg | size=(2, 16384), dtype=float64 | Whitened detector background (no injection) |
whitened_signal | size=(2, 16384), dtype=float64 | Whitened injected waveform alone |
whitened_injected | size=(2, 16384), dtype=float64 | Whitened detector background with injected signal |
mass_1 | dtype=float32 | Primary mass, in solar masses |
mass_2 | dtype=float32 | Secondary mass, in solar masses |
chirp_mass | dtype=float32 | Chirp mass M_c = (m_1 * m_2)^(3/5) / (m_1 + m_2)^(1/5), in solar masses |
mass_ratio | dtype=float32 | Mass ratio q = m_2 / m_1 |
a_1 | dtype=float32 | Primary spin magnitude (dimensionless) |
a_2 | dtype=float32 | Secondary spin magnitude (dimensionless) |
tilt_1 | dtype=float32 | Primary spin tilt angle, in radians |
tilt_2 | dtype=float32 | Secondary spin tilt angle, in radians |
phi_12 | dtype=float32 | Azimuthal angle between component spins, in radians |
phi_jl | dtype=float32 | Azimuthal precession-cone angle, in radians |
s1x, s1y, s1z | dtype=float32 | Cartesian components of the primary spin vector |
s2x, s2y, s2z | dtype=float32 | Cartesian components of the secondary spin vector |
distance | dtype=float32 | Comoving distance to the source, in Mpc |
phi | dtype=float32 | Right ascension of the source, in radians |
dec | dtype=float32 | Declination of the source, in radians |
inclination | dtype=float32 | Inclination angle of the binary, in radians |
psi | dtype=float32 | Polarization angle, in radians |
phic | dtype=float32 | Coalescence phase, in radians |
snr | dtype=float32 | Network signal-to-noise ratio of the injection |
TESS
The TESS dataset is broken into training, validation and test data, which are identically constructed. There is a set for classification '/tess/split/tess_classification_{train,val,test}.parquet' and one for regression '/tess/split/tess_regression_{train,val,test}.parquet. The dataset was split 80/10/10 into training, validation, and test sets. Since multiple TESS light curves, from different TESS sectors, are available per unique Gaia Data Release 3 (DR3) identifier (ID), the split ensures that no Gaia DR3 ID appears in more than one split to avoid label leakage, with the exception of when different light curves for the same Gaia DR3 ID have different labels due to variations in dominant variability detected in separate TESS sectors. The data includes the following
TESS Dataset Field Description
| Parameter | Type / Size | Description |
|---|---|---|
GaiaID | dtype=int | Unique Gaia DR3 Identifier |
TIC | dtype=int | TESS Input Catalog Identifier |
sector | dtype=int | TESS Observing Sector (https://tess.mit.edu/observations/) |
label | dtype=string | Class label [FOR THE CLASSIFICATION TASK] |
frot | dtype=double | Near-core rotation rate of the star [FOR THE REGRESSION TASK] |
frot_err | dtype=double | Near-core rotation rate error [FOR THE REGRESSION TASK] |
time | dtype=list | Time stamps in BTJD (Barycentric TESS Julian Date. Corrected for light travel time to solar system barycenter) |
flux | dtype=list | Time series measurements in normalized flux |
TIDMAD
The TIDMAD dataset is broken into training and validation data which are identically constructed differing only by the time at which the detector data was taken. This temporal difference results in variable detector noise conditions. There are 20 hdf5 files (~ 4 GB/file) in both training and validation datasets. The total dataset size is 163 GB. Each file contains ~ 200 examples and the associated meta data. Each example is a set of ch1 time series, ch2 time series, and injected frequency. This data in each hdf5 file includes the following
TIDMAD Dataset Field Description
| Parameter | Type / Size | Description |
|---|---|---|
chunk_length | dtype=int | Number of samples per example |
n_chunks | dtype=int | Number of examples per file |
sample_rate_hz | dtype=int | Data taking sample rate in Hz |
signal_freq_choices | size=(618,1), dtype=int | Array of signal frequencies injected across the full dataset, all files |
signal_frequency | size=(200,1), dtype=int | Injected frequency for the example in Hz (signal_frequency[i] is the injected signal frequency for example i) |
time_series_ch1 | size=(200,10M), dtype=int | Noisy time series for the example in DAQ units. Conversion factor from DAQ units to mV is 40mV/128. (time_series_ch1[i] is the ch1 time series for example i) |
time_series_ch2 | size=(200,10M), dtype=int | Signal time series for the example in DAQ units. Conversion factor from DAQ units to mV is 40mV/128. (time_series_ch2[i] is the ch2 time series for example i) |
Project 8
The Project 8 dataset is broken into training, validation, and test datasets which are identically constructed.
All contain a set of electron events with randomized energy, pitch angle, and radius parameters.
The data are divided into hdf5 files with ~5000 electrons per file; each file is about 5 GB, with the exception of one, which has fewer electrons and is smaller.
There are about 50,000 electrons total in the dataset. The total dataset size is about 50 GB.
The train dataset contains 8 hdf5 files, the validation dataset has 1 hdf5 file, and the test dataset has 2 (including the smaller file).
The experiments presented in the publication use output_ts_I + output_ts_I_cav_noise and output_ts_Q + output_ts_Q_cav_noise as inputs (noisy in-phase and quadrature time series).
The target for regression is energy_eV (truth value for energy of the electron, in eV).
This data in each hdf5 file includes the following
Project 8 Dataset Field Description
| Parameter | Type / Size | Description |
|---|---|---|
output_ts_I | size=(24576,1), dtype=float32 | In-phase signal time series |
output_ts_Q | size=(24576,1), dtype=float32 | Quadrature signal time series |
output_ts_I_cav_noise | size=(24576,1), dtype=float32 | In-phase cavity (frequency-dependent) noise time series |
output_ts_Q_cav_noise | size=(24576,1), dtype=float32 | Quadrature cavity (frequency-dependent) noise time series |
output_ts_I_gauss_noise | size=(24576,1), dtype=float32 | In-phase Gaussian noise time series |
output_ts_Q_gauss_noise | size=(24576,1), dtype=float32 | Quadrature Gaussian noise time series |
energy_eV | dtype=float32 | Truth value for energy of the electron (target for regression), in eV |
avg_carrier_frequency_Hz | dtype=float32 | Truth value for average cyclotron frequency of the electron, in Hz |
start_carrier_frequency_Hz | dtype=float32 | Truth value for starting cyclotron frequency of the electron, in Hz |
avg_axial_frequency_Hz | dtype=float32 | Truth value for average axial frequency of the electron, in Hz |
pitch_angle_deg | dtype=float32 | Truth value for pitch angle (direction of the electron’s momentum with respect to the magnetic field) of the electron, in degrees |
radius_input_m | dtype=float32 | Truth value for starting radius of the electron before one trajectory step (input value), in meters |
radius_m | dtype=float32 | Truth value for starting radius of the electron after one trajectory step, in meters |
radius_phase | dtype=float32 | Truth value for instantaneous polar angle in the X-Y plane of the electron’s starting position, in meters |
slope_Hz | dtype=float32 | Truth value for slope of the track in spectrogram space, in Hz/s |