lucidarium-systems/indexset

A pure-Rust two-level dynamic b-tree. This crate implements a compact set data structure that preserves its elements' sorted order and allows lookups of entries by value or sorted order position.

Rust

120

124 commits

updated Sep 9, 2026

See the code

README

indexset

crates.io docs

A pure-Rust two-level dynamic order-statistic b-tree.

This crate implements a compact set data structure that preserves its elements' sorted order and allows lookups of entries by value or sorted order position.

Under the feature concurrent you can find a version of the BTree that can be fearlessly shared between threads.

Both the concurrent and single-threaded versions are meant to be drop-in replacements for the stdlib BTree. This is mostly true for the latter but not for the former, yet.

The following table describes the variants of this data structure that are available:

Varianttl;drStability
crate::BTreeSetA single-threaded ordered setStable
crate::BTreeMapA single-threaded ordered mapStable
crate::concurrent::set::BTreeSetA concurrent ordered setBeta
crate::concurrent::map::BTreeMapA concurrent ordered mapBeta
crate::concurrent::multimap::BTreeMultiMapA concurrent ordered map where keys need not to be uniqueAlpha

Features

  • serde: implements serialization and deserialization traits for the single-threaded trees
  • concurrent: enables the three concurrent variants of BTreeSet referenced in the table above
  • cdc: provides helper methods to persist all concurrent trees
  • multimap: enables BTreeMultiMap

Background

This was heavily inspired by indexmap, and python's sortedcontainers.

It differs from both in that:

  • indexmap is a hashmap that provides numerical lookups, but does not maintain order in case of removals, while indexset's core data structure is a b-tree that irrespective of which mutating operation is run, always maintains order.
  • sortecontainers is similar in spirit, but utilizes a different routine for balancing the tree, and relies on a heap for numerical lookups.

indexset provides the following features:

  • As fast to iterate as a vec.
  • Zero indirection.
  • Lookups by position and range.
  • Minimal amount of allocations.
  • select(lookups by position) and rank operations in near constant time (not yet in the concurrent versions).

Performance

BTreeSet and BTreeMap derive their performance much from how they are constructed, which is:

A two-level B-Tree with a fenwick tree as a low-cost index for numerical lookups

Each node is a leaf, and each leaf is a vec with a fixed capacity of size B, with 1024 being the default.

The following hold:

  • Iteration is very fast since it is done by inheriting vec's iter struct.
  • Lookups only need two binary searches. One over n/B nodes and another over B elements: O(log(n/B) + log(B)) = O(log(n)).
  • Insertions are constant time O(B) in the best case and O(B^2) in the worst. Removals are O(log(n)).

Benchmarks

Running the suite

TargetCommandCoverage
Single-threaded set and mapcargo bench --bench single_threadedindexset and std::collections
Concurrent set and mapcargo bench --bench concurrent --features concurrentConcurrent trees and mutex-protected single-threaded baselines
Concurrent set, map, and multimapcargo bench --bench concurrent --features multimapAdds random- and ordered-discriminator multimap workloads
Concurrent map alternativescargo bench --bench comparison --features concurrentindexset, WorkTablesIndex, Arctic, and raw Congee maps

Method and units

The representative results below are Criterion median point estimates from commit 7e3ac23, measured on 2026-08-26 on a 12-core Apple M2 Pro with 16 GB RAM, macOS 26.5.2, and rustc 1.97.0-nightly (ad3a598ca 2026-05-03). Lower is better. The lowest comparable result for each workload or scenario is shown in bold.

Single-threaded results

The set cases use either an 8-byte u64 or a 64-byte record and start with 100,000 existing values:

WorkloadValuestd::collections::BTreeSetindexset, cap 256indexset, cap 1024
Insert a new valueu64 (8 B)144.9 ns134.6 ns168.1 ns
record_64b (64 B)230.0 ns264.3 ns564.3 ns
contains hitu64 (8 B)21.8 ns23.9 ns23.6 ns
record_64b (64 B)27.8 ns31.2 ns33.1 ns
Remove hitu64 (8 B)69.6 ns133.1 ns163.1 ns
record_64b (64 B)97.6 ns254.5 ns578.5 ns
get_indexu64 (8 B)N/A13.2 ns11.0 ns
record_64b (64 B)N/A13.2 ns11.1 ns
Full traversalu64 (8 B)92.5 µs37.5 µs37.3 µs
record_64b (64 B)108.2 µs63.1 µs61.4 µs
128-entry range traversalu64 (8 B)164.8 ns127.4 ns117.3 ns
record_64b (64 B)168.4 ns129.5 ns122.5 ns

The map cases use either a 16-byte u64 -> u64 entry or a 64-byte entry with a 56-byte value and start with 100,000 existing entries:

WorkloadEntrystd::collections::BTreeMapindexset, cap 256indexset, cap 1024
Insert a new entryentry_16b (16 B)178.5 ns152.5 ns223.7 ns
entry_64b (64 B)298.3 ns271.8 ns563.0 ns
Update an existing entryentry_16b (16 B)67.4 ns49.4 ns44.0 ns
entry_64b (64 B)77.2 ns75.3 ns66.0 ns
get hitentry_16b (16 B)24.7 ns46.5 ns38.7 ns
entry_64b (64 B)29.1 ns51.0 ns40.5 ns
Remove hitentry_16b (16 B)75.0 ns153.4 ns224.3 ns
entry_64b (64 B)117.0 ns262.8 ns587.5 ns
Full traversalentry_16b (16 B)89.9 µs59.7 µs59.6 µs
entry_64b (64 B)110.0 µs66.3 µs62.7 µs
128-entry range traversalentry_16b (16 B)163.8 ns155.6 ns139.0 ns
entry_64b (64 B)163.4 ns165.0 ns148.5 ns

Concurrent results

The following set results measure 10,000 operations over 12 worker threads and 100,000 existing values:

Implementation90/10, 8 B90/10, 64 B50/50, 8 B50/50, 64 B
Concurrent indexset, cap 2560.531 ms0.593 ms0.548 ms0.728 ms
Concurrent indexset, cap 10240.584 ms1.811 ms0.697 ms4.011 ms
Mutex-protected indexset, cap 2561.798 ms2.989 ms2.913 ms6.239 ms
Mutex-protected indexset, cap 10242.024 ms9.801 ms3.829 ms35.458 ms
Mutex-protected std::collections::BTreeSet1.496 ms1.913 ms1.886 ms2.545 ms

The map target uses the same operation count, thread count, and initial length:

Implementation90/10, 16 B90/10, 64 B50/50, 16 B50/50, 64 B
Concurrent indexset, cap 2560.562 ms0.593 ms0.581 ms0.722 ms
Concurrent indexset, cap 10240.659 ms1.829 ms0.790 ms3.945 ms
Mutex-protected indexset, cap 2562.341 ms3.216 ms4.150 ms6.272 ms
Mutex-protected indexset, cap 10242.637 ms10.461 ms5.292 ms35.900 ms
Mutex-protected std::collections::BTreeMap1.642 ms1.795 ms1.969 ms2.314 ms

Multimap results

These multimap results use 100,000 pairs, node capacity 1024 . v8b combines with an 8-byte key into a 16-byte pair; v56b produces a 64-byte pair. A hit query iterates and checksums every value for its key, so dense fanout is expected to cost more than sparse fanout.

Pair discriminatorPairInsert new, fanout 1-3Insert new, fanout 1,000-2,000Get hit, fanout 1-3Get hit, fanout 1,000-2,000
Randomv8b (16 B)322.9 ns295.8 ns648.9 ns7.146 µs
v56b (64 B)718.3 ns707.3 ns849.6 ns7.749 µs
Orderedv8b (16 B)248.6 ns209.6 ns736.1 ns6.035 µs
v56b (64 B)610.7 ns569.5 ns1.694 µs8.390 µs

Dense ordered mixed workloads are currently excluded pending issue #68; the remaining multimap cases retain their correctness checks during benchmark execution.

Limitations

  • BTreeMap is less polished than BTreeSet. This crate has been optimised for a leaner BTreeSet.
  • Concurrent BtreeSet, BTreeMap and BTreeMultiMap do not support serde serialization and deserialization nor are they order-statistic trees.

Naming

This library is called indexset because the base data structure is BTreeSet. BTreeMap is a BTreeSet with a Pair<K, V> item type, and BTreeMultiMap is one with a MultiPair<K, V> item.

Changelog

See CHANGELOG.md.

Mentions

Special thanks to Christopher Bergstrom from Pathscale for funding the development of this library.

Contributors

brurucy

80 commits

Handy-caT

26 commits

Cydhra

8 commits

michaelsutton

3 commits

lucidarium-systems/indexset

A pure-Rust two-level dynamic b-tree. This crate implements a compact set data structure that preserves its elements' sorted order and allows lookups of entries by value or sorted order position.

Rust

120

124 commits

updated Sep 9, 2026

See the code

README

indexset

crates.io docs

A pure-Rust two-level dynamic order-statistic b-tree.

This crate implements a compact set data structure that preserves its elements' sorted order and allows lookups of entries by value or sorted order position.

Under the feature concurrent you can find a version of the BTree that can be fearlessly shared between threads.

Both the concurrent and single-threaded versions are meant to be drop-in replacements for the stdlib BTree. This is mostly true for the latter but not for the former, yet.

The following table describes the variants of this data structure that are available:

Varianttl;drStability
crate::BTreeSetA single-threaded ordered setStable
crate::BTreeMapA single-threaded ordered mapStable
crate::concurrent::set::BTreeSetA concurrent ordered setBeta
crate::concurrent::map::BTreeMapA concurrent ordered mapBeta
crate::concurrent::multimap::BTreeMultiMapA concurrent ordered map where keys need not to be uniqueAlpha

Features

  • serde: implements serialization and deserialization traits for the single-threaded trees
  • concurrent: enables the three concurrent variants of BTreeSet referenced in the table above
  • cdc: provides helper methods to persist all concurrent trees
  • multimap: enables BTreeMultiMap

Background

This was heavily inspired by indexmap, and python's sortedcontainers.

It differs from both in that:

  • indexmap is a hashmap that provides numerical lookups, but does not maintain order in case of removals, while indexset's core data structure is a b-tree that irrespective of which mutating operation is run, always maintains order.
  • sortecontainers is similar in spirit, but utilizes a different routine for balancing the tree, and relies on a heap for numerical lookups.

indexset provides the following features:

  • As fast to iterate as a vec.
  • Zero indirection.
  • Lookups by position and range.
  • Minimal amount of allocations.
  • select(lookups by position) and rank operations in near constant time (not yet in the concurrent versions).

Performance

BTreeSet and BTreeMap derive their performance much from how they are constructed, which is:

A two-level B-Tree with a fenwick tree as a low-cost index for numerical lookups

Each node is a leaf, and each leaf is a vec with a fixed capacity of size B, with 1024 being the default.

The following hold:

  • Iteration is very fast since it is done by inheriting vec's iter struct.
  • Lookups only need two binary searches. One over n/B nodes and another over B elements: O(log(n/B) + log(B)) = O(log(n)).
  • Insertions are constant time O(B) in the best case and O(B^2) in the worst. Removals are O(log(n)).

Benchmarks

Running the suite

TargetCommandCoverage
Single-threaded set and mapcargo bench --bench single_threadedindexset and std::collections
Concurrent set and mapcargo bench --bench concurrent --features concurrentConcurrent trees and mutex-protected single-threaded baselines
Concurrent set, map, and multimapcargo bench --bench concurrent --features multimapAdds random- and ordered-discriminator multimap workloads
Concurrent map alternativescargo bench --bench comparison --features concurrentindexset, WorkTablesIndex, Arctic, and raw Congee maps

Method and units

The representative results below are Criterion median point estimates from commit 7e3ac23, measured on 2026-08-26 on a 12-core Apple M2 Pro with 16 GB RAM, macOS 26.5.2, and rustc 1.97.0-nightly (ad3a598ca 2026-05-03). Lower is better. The lowest comparable result for each workload or scenario is shown in bold.

Single-threaded results

The set cases use either an 8-byte u64 or a 64-byte record and start with 100,000 existing values:

WorkloadValuestd::collections::BTreeSetindexset, cap 256indexset, cap 1024
Insert a new valueu64 (8 B)144.9 ns134.6 ns168.1 ns
record_64b (64 B)230.0 ns264.3 ns564.3 ns
contains hitu64 (8 B)21.8 ns23.9 ns23.6 ns
record_64b (64 B)27.8 ns31.2 ns33.1 ns
Remove hitu64 (8 B)69.6 ns133.1 ns163.1 ns
record_64b (64 B)97.6 ns254.5 ns578.5 ns
get_indexu64 (8 B)N/A13.2 ns11.0 ns
record_64b (64 B)N/A13.2 ns11.1 ns
Full traversalu64 (8 B)92.5 µs37.5 µs37.3 µs
record_64b (64 B)108.2 µs63.1 µs61.4 µs
128-entry range traversalu64 (8 B)164.8 ns127.4 ns117.3 ns
record_64b (64 B)168.4 ns129.5 ns122.5 ns

The map cases use either a 16-byte u64 -> u64 entry or a 64-byte entry with a 56-byte value and start with 100,000 existing entries:

WorkloadEntrystd::collections::BTreeMapindexset, cap 256indexset, cap 1024
Insert a new entryentry_16b (16 B)178.5 ns152.5 ns223.7 ns
entry_64b (64 B)298.3 ns271.8 ns563.0 ns
Update an existing entryentry_16b (16 B)67.4 ns49.4 ns44.0 ns
entry_64b (64 B)77.2 ns75.3 ns66.0 ns
get hitentry_16b (16 B)24.7 ns46.5 ns38.7 ns
entry_64b (64 B)29.1 ns51.0 ns40.5 ns
Remove hitentry_16b (16 B)75.0 ns153.4 ns224.3 ns
entry_64b (64 B)117.0 ns262.8 ns587.5 ns
Full traversalentry_16b (16 B)89.9 µs59.7 µs59.6 µs
entry_64b (64 B)110.0 µs66.3 µs62.7 µs
128-entry range traversalentry_16b (16 B)163.8 ns155.6 ns139.0 ns
entry_64b (64 B)163.4 ns165.0 ns148.5 ns

Concurrent results

The following set results measure 10,000 operations over 12 worker threads and 100,000 existing values:

Implementation90/10, 8 B90/10, 64 B50/50, 8 B50/50, 64 B
Concurrent indexset, cap 2560.531 ms0.593 ms0.548 ms0.728 ms
Concurrent indexset, cap 10240.584 ms1.811 ms0.697 ms4.011 ms
Mutex-protected indexset, cap 2561.798 ms2.989 ms2.913 ms6.239 ms
Mutex-protected indexset, cap 10242.024 ms9.801 ms3.829 ms35.458 ms
Mutex-protected std::collections::BTreeSet1.496 ms1.913 ms1.886 ms2.545 ms

The map target uses the same operation count, thread count, and initial length:

Implementation90/10, 16 B90/10, 64 B50/50, 16 B50/50, 64 B
Concurrent indexset, cap 2560.562 ms0.593 ms0.581 ms0.722 ms
Concurrent indexset, cap 10240.659 ms1.829 ms0.790 ms3.945 ms
Mutex-protected indexset, cap 2562.341 ms3.216 ms4.150 ms6.272 ms
Mutex-protected indexset, cap 10242.637 ms10.461 ms5.292 ms35.900 ms
Mutex-protected std::collections::BTreeMap1.642 ms1.795 ms1.969 ms2.314 ms

Multimap results

These multimap results use 100,000 pairs, node capacity 1024 . v8b combines with an 8-byte key into a 16-byte pair; v56b produces a 64-byte pair. A hit query iterates and checksums every value for its key, so dense fanout is expected to cost more than sparse fanout.

Pair discriminatorPairInsert new, fanout 1-3Insert new, fanout 1,000-2,000Get hit, fanout 1-3Get hit, fanout 1,000-2,000
Randomv8b (16 B)322.9 ns295.8 ns648.9 ns7.146 µs
v56b (64 B)718.3 ns707.3 ns849.6 ns7.749 µs
Orderedv8b (16 B)248.6 ns209.6 ns736.1 ns6.035 µs
v56b (64 B)610.7 ns569.5 ns1.694 µs8.390 µs

Dense ordered mixed workloads are currently excluded pending issue #68; the remaining multimap cases retain their correctness checks during benchmark execution.

Limitations

  • BTreeMap is less polished than BTreeSet. This crate has been optimised for a leaner BTreeSet.
  • Concurrent BtreeSet, BTreeMap and BTreeMultiMap do not support serde serialization and deserialization nor are they order-statistic trees.

Naming

This library is called indexset because the base data structure is BTreeSet. BTreeMap is a BTreeSet with a Pair<K, V> item type, and BTreeMultiMap is one with a MultiPair<K, V> item.

Changelog

See CHANGELOG.md.

Mentions

Special thanks to Christopher Bergstrom from Pathscale for funding the development of this library.

Contributors

brurucy

80 commits

Handy-caT

26 commits

Cydhra

8 commits

michaelsutton

3 commits

Languages

Rust

100.0%