Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
C++
510
2,094 commits
updated Sep 30, 2026
Desbordante is a high-performance data profiler that is capable of discovering and validating many different patterns in data using various algorithms.
Why Desbordante? Why not just use an LLM or other black-box machine learning approaches?
Unlike these, Desbordante takes a symbolic, explainable approach to data analysis, producing explicit rules or rule-shaped artifacts that are verifiable, such as data dependencies, association rules, or other patterns. This is a distinct paradigm for data analysis that complements black-box learning. In contrast to free-form model outputs, discovered patterns have precisely defined semantics and can be algorithmically validated against the underlying data.
These patterns add data-grounded and rule-based structure, which provides useful properties for systems built around machine learning and large language models: consistency, provenance, explainability, and auditability. Further, the patterns can serve as inputs to downstream data-quality checks, validation procedures, integrity constraints, training-data analysis, and other rule-based safeguards. They can also be used to choose a direction of an ablation study, to generate examples that violate rules for use in machine learning model training and testing, to assist fact completion, and to facilitate acceptance checks and guardrails for pipelines involving retrieval-augmented generation. This is roughly the motivation used for the work in [1], but it applies more broadly to the patterns Desbordante works with.
[1] François Amat. Mining Rules on Tabular Data. Institut Polytechnique de Paris, 2025.
From other angles, the discovered patterns can be used in many ways:
For scientific data, especially those obtained experimentally, an interesting pattern allows to formulate a hypothesis that could lead to a scientific discovery. In some cases it even allows one to draw conclusions immediately, if there is enough data. At the very least, the found pattern can be used to provide a direction for further study.
For business data, it is also possible to obtain a hypothesis based on found patterns. However, there are more down-to-earth and more in-demand applications in this case: clearing errors in data, finding and removing inexact duplicates, performing schema matching, and many more.
For database data, found patterns can help with recovering/defining primary and foreign keys, checking/setting up all kinds of integrity constraints.
Desbordante supports three types of tasks.
The Discovery task is designed to identify all instances of a specified pattern type of a given dataset.
The Validation task is different: it is designed to check whether a specified pattern instance is present in a given dataset. This task not only returns True or False, but it also explains why the instance does not hold (e.g. it can list table rows with conflicting values).
For some patterns Desbordante supports a dynamic task variant. The distinguishing feature of dynamic algorithms compared to classic (static) algorithms is that after a result is obtained, the table can be changed and a dynamic algorithm will update the result based just on those changes instead of processing the whole table again. As a result, they can be up to several orders of magnitude faster than classic (static) ones in some situations.
Desbordante support four data types:
Desbordante can be used via two interfaces:
A brief introduction to the tool and its use cases can be found here (in English) and here (in Russian). Also, an extensive list of tutorial examples that cover each supported pattern is available here.
For information about the console interface check the repository.
Desbordante features can be accessed from within Python programs by employing the Desbordante Python library. The library is implemented in the form of Python bindings to the interface of the Desbordante C++ core library, using pybind11. Apart from discovery and validation of patterns, this interface is capable of providing valuable additional information which can, for example, describe why a given pattern does not hold.
We want to demonstrate the power of Desbordante through examples where some patterns are extracted from tabular data, providing non-trivial insights. The patterns are quite complex and require detailed explanations, as well as a significant amount of code. This takes up quite a bit of space. Therefore, we do not include the actual code here; instead, we provide a clear (albeit simplified) definition and a link to a Colab notebook with interactive examples. The examples themselves are very detailed and allow users to understand the pattern and how to extract it using Desbordante.
Desbordante offers examples for each supported pattern, sometimes several if the pattern is complex or needs to highlight its unique characteristics compared to others in the same family. We have mentioned only a small portion here, which is available in Colab. The rest can be found in our example folder.
Finally, Desbordante allows end users to solve various data quality problems by constructing ad-hoc Python programs, incorporating different Python libraries, and utilizing the search and validation of various patterns. To demonstrate the power of this approach, we have implemented several demo scenarios:
There is also an interactive demo for all of them , and all of these python scripts are here. The ideas behind them are briefly discussed in this preprint (Section 3).
No worries! Desbordante offers a novel type of data profiling, which may require that you first familiarize yourself with its concepts and usage. The most challenging part of Desbordante are the primitives: their definitions and applications in practice. To help you get started, here’s a step-by-step guide:
Here is a list of papers about patterns, organized in the recommended reading order in each item.
Desbordante is available at the Python Package Index (PyPI). Dependencies:
To install Desbordante type:
$ pip install desbordante
However, as Desbordante core uses C++, additional requirements on the machine are imposed. Therefore this installation option may not work for everyone. Currently, only manylinux_2_28 (Ubuntu 21.04+) and macOS 15.0+ (arm64, x86_64) is supported. If the above does not work for you consider building from sources.
The following instructions are expected to work for Ubuntu 21.04+ LTS and macOS Sequoia 15.0+ (Apple Silicon).
Prior to cloning the repository and attempting to build the project, ensure that you have the following software:
Instructions below are given for GCC (on Linux) and Apple Clang (on macOS). Instructions for other supported compilers can be found in Desbordante wiki.
For Ubuntu versions earlier than 24.04, you need to add the Kitware APT repository to your system by following their official guide to install the latest version of CMake.
Then run the following commands:
sudo apt update && sudo apt upgrade
sudo apt install g++ cmake ninja-build libboost-all-dev python3 python3-venv
export CXX=g++
The last line sets g++ as CMake compiler in your terminal session.
You can also set it by default in all sessions: echo 'export CXX=g++' >> ~/.profile
For Ubuntu 24.04 and above, you can skip to the build steps. For older versions the Ubuntu APT repository might not have a compatible version of Boost, so you'll need to install it manually:
wget https://archives.boost.io/release/1.89.0/source/boost_1_89_0.tar.gz
tar xzvf boost_1_89_0.tar.gz
cd boost_1_89_0 && ./bootstrap.sh
sudo ./b2 install --prefix=/usr/
Install Xcode Command Line Tools if you don't have them. Run:
xcode-select --install
Follow the prompts to continue.
To install the build dependencies on macOS we recommend to use Homebrew package manager. With Homebrew installed, run the following commands:
brew install cmake boost
After installation, check cmake --version. If command is not found, then you need to add to environment path to
homebrew installed packages. To do this open ~/.zprofile (for Zsh) or
~/.bash_profile (for Bash) and add to the end of the file the output of brew shellenv.
After that, restart the terminal and check the version of CMake again, now it should be displayed.
Run the following commands:
export CXX=clang++
export BOOST_ROOT=$(brew --prefix boost)
These commands set Apple Clang and Homebrew Boost as default in CMake in your terminal session.
You can also add them to the end of ~/.profile to set this by default in all sessions.
Clone the repository, change the current directory to the project directory and run the following commands:
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install .
Now it is possible to import desbordante as a module from within the created virtual environment.
Build the tests themselves:
./build.sh
The Python module can be built by providing the --pybind switch:
./build.sh --pybind
See ./build.sh --help for more available options.
The ./build.sh script generates the following file structure in /path/to/desbordante-core/build/target:
├───input_data
│ └───some-sample-csv\'s.csv
├───desbordante.cpython-*.so
The input_data directory contains several .csv files that are used by unit tests.
You can run tests with CTest from any directory in the Desbordante tree:
ctest --test-dir build --exclude-regex ".*HeavyDatasets.*" -j $JOBS
where $JOBS is the desired number of concurrent jobs.
desbordante.cpython-*.so is a Python module, packaging Python bindings for the Desbordante core library. In order to use it, simply import it:
cd build/target
python3
>>> import desbordante
The core library uses spdlog. All log messages are automatically bridged to Python's standard logging module.
Log level control:
import logging
import desbordante
# Get the logger provided by the library
log = logging.getLogger("desbordante")
# Set the desired log level using standard logging constants
log.setLevel(logging.INFO) # Or logging.DEBUG, logging.TRACE, etc.
If type hints don't work for you in Visual Studio Code, for example, then install stubs using the command:
pip install desbordante-stubs
NOTE: Stubs may not fully support current version of desbordante package, as they are updated independently.
If you use this software for research, please cite our core paper:
@inproceedings{10.1145/3703323.3703725,
author = {Chernishev, George and Polyntsov, Michael and Chizhov, Anton and Stupakov, Kirill and Shchuckin, Ilya and Smirnov, Alexander and Strutovsky, Maxim and Shlyonskikh, Alexey and Firsov, Mikhail and Manannikov, Stepan and Bobrov, Nikita and Goncharov, Daniil and Barutkin, Ilia and Yakshigulov, Vadim and Shalnev, Vladislav and Muraviev, Kirill and Rakhmukova, Anna and Shcheka, Dmitriy and Chernikov, Anton and Kuzin, Yakov and Sinelnikov, Michael and Abrosimov, Grigorii and Popov, Dmitriy and Demchenko, Artem and Belokonny, Sergey and Soloveva, Liana-Iuliia and Kurbatov, Yaroslav and Vyrodov, Mikhail and Saliou, Arthur and Gaisin, Eduard and Smirnov, Kirill},
title = {Desbordante: from benchmarking suite to high-performance science-intensive data profiler},
year = {2025},
isbn = {9798400711244},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3703323.3703725},
doi = {10.1145/3703323.3703725},
booktitle = {Proceedings of the 8th International Conference on Data Science and Management of Data (12th ACM IKDD CODS and 30th COMAD)},
pages = {234--243},
numpages = {10},
keywords = {Data Mining, Data Profiling, Pattern Extraction, Data Analysis, Knowledge Discovery, Data Exploration, Anomaly Detection, Data Wrangling},
location = {},
series = {CODS-COMAD '24}
}
or cite one of our papers, if you use a particular part:
If you have any questions regarding the tool usage you can ask it in our google group. To contact dev team email George Chernishev, Alexey Shlyonskikh or Michael Polyntsov.
2,112 followers · starred May 2024
10 followers · starred Sep 2026
14 followers · starred Feb 2025
C++
96.5%
CMake
2.4%
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
C++
510
2,094 commits
updated Sep 30, 2026
Desbordante is a high-performance data profiler that is capable of discovering and validating many different patterns in data using various algorithms.
Why Desbordante? Why not just use an LLM or other black-box machine learning approaches?
Unlike these, Desbordante takes a symbolic, explainable approach to data analysis, producing explicit rules or rule-shaped artifacts that are verifiable, such as data dependencies, association rules, or other patterns. This is a distinct paradigm for data analysis that complements black-box learning. In contrast to free-form model outputs, discovered patterns have precisely defined semantics and can be algorithmically validated against the underlying data.
These patterns add data-grounded and rule-based structure, which provides useful properties for systems built around machine learning and large language models: consistency, provenance, explainability, and auditability. Further, the patterns can serve as inputs to downstream data-quality checks, validation procedures, integrity constraints, training-data analysis, and other rule-based safeguards. They can also be used to choose a direction of an ablation study, to generate examples that violate rules for use in machine learning model training and testing, to assist fact completion, and to facilitate acceptance checks and guardrails for pipelines involving retrieval-augmented generation. This is roughly the motivation used for the work in [1], but it applies more broadly to the patterns Desbordante works with.
[1] François Amat. Mining Rules on Tabular Data. Institut Polytechnique de Paris, 2025.
From other angles, the discovered patterns can be used in many ways:
For scientific data, especially those obtained experimentally, an interesting pattern allows to formulate a hypothesis that could lead to a scientific discovery. In some cases it even allows one to draw conclusions immediately, if there is enough data. At the very least, the found pattern can be used to provide a direction for further study.
For business data, it is also possible to obtain a hypothesis based on found patterns. However, there are more down-to-earth and more in-demand applications in this case: clearing errors in data, finding and removing inexact duplicates, performing schema matching, and many more.
For database data, found patterns can help with recovering/defining primary and foreign keys, checking/setting up all kinds of integrity constraints.
Desbordante supports three types of tasks.
The Discovery task is designed to identify all instances of a specified pattern type of a given dataset.
The Validation task is different: it is designed to check whether a specified pattern instance is present in a given dataset. This task not only returns True or False, but it also explains why the instance does not hold (e.g. it can list table rows with conflicting values).
For some patterns Desbordante supports a dynamic task variant. The distinguishing feature of dynamic algorithms compared to classic (static) algorithms is that after a result is obtained, the table can be changed and a dynamic algorithm will update the result based just on those changes instead of processing the whole table again. As a result, they can be up to several orders of magnitude faster than classic (static) ones in some situations.
Desbordante support four data types:
Desbordante can be used via two interfaces:
A brief introduction to the tool and its use cases can be found here (in English) and here (in Russian). Also, an extensive list of tutorial examples that cover each supported pattern is available here.
For information about the console interface check the repository.
Desbordante features can be accessed from within Python programs by employing the Desbordante Python library. The library is implemented in the form of Python bindings to the interface of the Desbordante C++ core library, using pybind11. Apart from discovery and validation of patterns, this interface is capable of providing valuable additional information which can, for example, describe why a given pattern does not hold.
We want to demonstrate the power of Desbordante through examples where some patterns are extracted from tabular data, providing non-trivial insights. The patterns are quite complex and require detailed explanations, as well as a significant amount of code. This takes up quite a bit of space. Therefore, we do not include the actual code here; instead, we provide a clear (albeit simplified) definition and a link to a Colab notebook with interactive examples. The examples themselves are very detailed and allow users to understand the pattern and how to extract it using Desbordante.
Desbordante offers examples for each supported pattern, sometimes several if the pattern is complex or needs to highlight its unique characteristics compared to others in the same family. We have mentioned only a small portion here, which is available in Colab. The rest can be found in our example folder.
Finally, Desbordante allows end users to solve various data quality problems by constructing ad-hoc Python programs, incorporating different Python libraries, and utilizing the search and validation of various patterns. To demonstrate the power of this approach, we have implemented several demo scenarios:
There is also an interactive demo for all of them , and all of these python scripts are here. The ideas behind them are briefly discussed in this preprint (Section 3).
No worries! Desbordante offers a novel type of data profiling, which may require that you first familiarize yourself with its concepts and usage. The most challenging part of Desbordante are the primitives: their definitions and applications in practice. To help you get started, here’s a step-by-step guide:
Here is a list of papers about patterns, organized in the recommended reading order in each item.
Desbordante is available at the Python Package Index (PyPI). Dependencies:
To install Desbordante type:
$ pip install desbordante
However, as Desbordante core uses C++, additional requirements on the machine are imposed. Therefore this installation option may not work for everyone. Currently, only manylinux_2_28 (Ubuntu 21.04+) and macOS 15.0+ (arm64, x86_64) is supported. If the above does not work for you consider building from sources.
The following instructions are expected to work for Ubuntu 21.04+ LTS and macOS Sequoia 15.0+ (Apple Silicon).
Prior to cloning the repository and attempting to build the project, ensure that you have the following software:
Instructions below are given for GCC (on Linux) and Apple Clang (on macOS). Instructions for other supported compilers can be found in Desbordante wiki.
For Ubuntu versions earlier than 24.04, you need to add the Kitware APT repository to your system by following their official guide to install the latest version of CMake.
Then run the following commands:
sudo apt update && sudo apt upgrade
sudo apt install g++ cmake ninja-build libboost-all-dev python3 python3-venv
export CXX=g++
The last line sets g++ as CMake compiler in your terminal session.
You can also set it by default in all sessions: echo 'export CXX=g++' >> ~/.profile
For Ubuntu 24.04 and above, you can skip to the build steps. For older versions the Ubuntu APT repository might not have a compatible version of Boost, so you'll need to install it manually:
wget https://archives.boost.io/release/1.89.0/source/boost_1_89_0.tar.gz
tar xzvf boost_1_89_0.tar.gz
cd boost_1_89_0 && ./bootstrap.sh
sudo ./b2 install --prefix=/usr/
Install Xcode Command Line Tools if you don't have them. Run:
xcode-select --install
Follow the prompts to continue.
To install the build dependencies on macOS we recommend to use Homebrew package manager. With Homebrew installed, run the following commands:
brew install cmake boost
After installation, check cmake --version. If command is not found, then you need to add to environment path to
homebrew installed packages. To do this open ~/.zprofile (for Zsh) or
~/.bash_profile (for Bash) and add to the end of the file the output of brew shellenv.
After that, restart the terminal and check the version of CMake again, now it should be displayed.
Run the following commands:
export CXX=clang++
export BOOST_ROOT=$(brew --prefix boost)
These commands set Apple Clang and Homebrew Boost as default in CMake in your terminal session.
You can also add them to the end of ~/.profile to set this by default in all sessions.
Clone the repository, change the current directory to the project directory and run the following commands:
python3 -m venv .venv
source .venv/bin/activate
python3 -m pip install .
Now it is possible to import desbordante as a module from within the created virtual environment.
Build the tests themselves:
./build.sh
The Python module can be built by providing the --pybind switch:
./build.sh --pybind
See ./build.sh --help for more available options.
The ./build.sh script generates the following file structure in /path/to/desbordante-core/build/target:
├───input_data
│ └───some-sample-csv\'s.csv
├───desbordante.cpython-*.so
The input_data directory contains several .csv files that are used by unit tests.
You can run tests with CTest from any directory in the Desbordante tree:
ctest --test-dir build --exclude-regex ".*HeavyDatasets.*" -j $JOBS
where $JOBS is the desired number of concurrent jobs.
desbordante.cpython-*.so is a Python module, packaging Python bindings for the Desbordante core library. In order to use it, simply import it:
cd build/target
python3
>>> import desbordante
The core library uses spdlog. All log messages are automatically bridged to Python's standard logging module.
Log level control:
import logging
import desbordante
# Get the logger provided by the library
log = logging.getLogger("desbordante")
# Set the desired log level using standard logging constants
log.setLevel(logging.INFO) # Or logging.DEBUG, logging.TRACE, etc.
If type hints don't work for you in Visual Studio Code, for example, then install stubs using the command:
pip install desbordante-stubs
NOTE: Stubs may not fully support current version of desbordante package, as they are updated independently.
If you use this software for research, please cite our core paper:
@inproceedings{10.1145/3703323.3703725,
author = {Chernishev, George and Polyntsov, Michael and Chizhov, Anton and Stupakov, Kirill and Shchuckin, Ilya and Smirnov, Alexander and Strutovsky, Maxim and Shlyonskikh, Alexey and Firsov, Mikhail and Manannikov, Stepan and Bobrov, Nikita and Goncharov, Daniil and Barutkin, Ilia and Yakshigulov, Vadim and Shalnev, Vladislav and Muraviev, Kirill and Rakhmukova, Anna and Shcheka, Dmitriy and Chernikov, Anton and Kuzin, Yakov and Sinelnikov, Michael and Abrosimov, Grigorii and Popov, Dmitriy and Demchenko, Artem and Belokonny, Sergey and Soloveva, Liana-Iuliia and Kurbatov, Yaroslav and Vyrodov, Mikhail and Saliou, Arthur and Gaisin, Eduard and Smirnov, Kirill},
title = {Desbordante: from benchmarking suite to high-performance science-intensive data profiler},
year = {2025},
isbn = {9798400711244},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3703323.3703725},
doi = {10.1145/3703323.3703725},
booktitle = {Proceedings of the 8th International Conference on Data Science and Management of Data (12th ACM IKDD CODS and 30th COMAD)},
pages = {234--243},
numpages = {10},
keywords = {Data Mining, Data Profiling, Pattern Extraction, Data Analysis, Knowledge Discovery, Data Exploration, Anomaly Detection, Data Wrangling},
location = {},
series = {CODS-COMAD '24}
}
or cite one of our papers, if you use a particular part:
If you have any questions regarding the tool usage you can ask it in our google group. To contact dev team email George Chernishev, Alexey Shlyonskikh or Michael Polyntsov.
2,112 followers · starred May 2024
10 followers · starred Sep 2026
14 followers · starred Feb 2025
C++
96.5%
CMake
2.4%