Awesome Dynamic Graphs

A collection of resources on dynamic/streaming/temporal/evolving graph processing systems, databases, data structures, datasets, and related academic and industrial work.

Contributing

To add, remove or change things on the list: please submit a pull request to the GitHub repository.

Open Source Projects

Data Structures, Systems and Frameworks

• Aspen [C++]
  • Streaming graph processing system based on compressed purely-functional trees
  • Paper
• Differential Dataflow [Rust]
  • General data-parallel programming framework suitable for processing large dynamic graphs
  • Paper
• GraphBolt [C++]
  • Streaming graph processing system that provides BSP guarantees
  • Paper
• GraphJet [Java]
  • Streaming graph processing system for generating content recommendations at Twitter
  • Paper
• GraphOne [C++]
  • Storage engine for streaming graph data
  • Paper
• GreyCat [Java]
  • Many-world temporal graph data store
  • Paper
• GoFFish[Java]
  • Multi-source streaming graph processing system
• KickStarter[C++]
  • Streaming graph processing system optimising for path-based/monotonic graph algorithms
  • Paper
• LLAMA [C++]
  • CSR-based storage engine and streaming graph processing system
  • Paper
• PCSR [C++]
  • CSR-inspired, packed memory array based data structure for streaming graphs
  • Paper
• PPCSR [C++]
  • Parallel and NUMA-aware implementation of PCSR data structure for streaming graphs
• Raphtory [Scala]
  • Snapshot-based distributed streaming graph processing system
  • Paper
• STINGER [C/C++]
  • Data structure for streaming graphs based on linked lists of blocks
  • Paper

Benchmarking

• GraphTides [Java/TypeScript]
  • Framework for evaluating streaming graph processing platforms
  • Paper

Papers

Check Julian Shun's list for a further reference and an extensive collection of related papers.

Data Structures and Storage

STINGER: Spatio-Temporal Interaction Networks and Graphs (STING) Extensible Representation (2009)

STINGER: High Performance Data Structure for Streaming Graphs (HPEC 2012)

Packed Compressed Sparse Row: A Dynamic Graph Representation (HPEC 2018)

GraphOne: A Data Store for Real-time Analytics on Evolving Graphs (FAST 2019)

GraphTinker: A High Performance Data Structure for Dynamic Graph Processing (IPDPS 2019)

CSR++: A Fast, Scalable, Update-Friendly Graph DataStructure (OPODIS 2020)

LiveGraph: A Transactional Graph Storage System With Purely Sequential Adjacency List Scans (VLDB 2020)

A Parallel Packed Memory Array to Store Dynamic Graphs (ALENEX 2021)

Systems and Frameworks

STINGER: Spatio-Temporal Interaction Networks and Graphs (STING) Extensible Representation (2009)

On Querying Historical Evolving Graph Sequences (VLDB 2011)

STINGER: High Performance Data Structure for Streaming Graphs (HPEC 2012)

Facilitating Real-Time Graph Mining (CloudDB 2012)

Kineograph: Taking the Pulse of a Fast-Changing and Connected World (EuroSys 2012)

Differential Dataflow (CIDR 2013)

Efficient Snapshot Retrieval over Historical Graph Data (ICDE 2013)

Naiad: A Timely Dataflow System (SOSP 2013)

Chronos: A Graph Engine for Temporal Graph Analysis (EuroSys 2014)

Towards Large-Scale Graph Stream Processing Platform (WWW 2014)

CellIQ : Real-Time Cellular Network Analytics at Scale (NSDI 2015)

DISTINGER: A Distributed Graph Data Structure for Massive Dynamic Graph Processing (Big Data 2015)

ImmortalGraph: A System for Storage and Analysis of Temporal Graphs (Transactions on Storage 2015)

LLAMA: Efficient Graph Analytics Using Large Multiversioned Arrays (ICDE 2015)

Real-time Analytics for Fast Evolving Social Graphs (CCGrid 2015)

GraphIn: An Online High Performance Incremental Graph Processing Framework (Euro-Par 2016)

GraphJet: Real-Time Content Recommendations at Twitter (VLDB 2016)

Storing and Analyzing Historical Graph Data at Scale (EDBT 2016)

Synergistic Analysis of Evolving Graphs (TACO 2016)

Time-Evolving Graph Processing at Scale (GRADES 2016)

Tornado: A System For Real-Time Iterative Analysis Over Evolving Data (SIGMOD 2016)

Towards a Distributed Large-Scale Dynamic Graph Data Store (IPDPSW 2016)

Automatic Algorithm Transformation for Efficient Multi-Snapshot Analytics on Temporal Graphs (VLDB 2017)

KickStarter: Fast and Accurate Computations on Streaming Graphs via Trimmed Approximations (ASPLOS 2017)

Auxo: A Temporal Graph Management System (BDMA 2018)

Distributed Evaluation of Subgraph Queries Using Worst-case Optimal Low-Memory Dataflows (VLDB 2018)

GraPU: Accelerate Streaming Graph Analysis through Preprocessing Buffered Updates (SOCC 2018)

GraphBolt: Dependency-Driven Synchronous Processing of Streaming Graphs (EuroSys 2019)

Low-Latency Graph Streaming Using Compressed Purely-Functional Trees (PLDI 2019)

GraphOne: A Data Store for Real-time Analytics on Evolving Graphs (FAST 2019)

LiveGraph: A Transactional Graph Storage System With Purely Sequential Adjacency List Scans (VLDB 2020)

Raphtory: Streaming analysis of distributed temporal graphs (FGCS 2020)

RisGraph: A Real-Time Streaming System for Evolving Graphs (2020)

Tegra: Efficient Ad-Hoc Analytics on Time-Evolving Graphs (NSDI 2021)

Surveys

The Ubiquity of Large Graphs and Surprising Challenges of Graph Processing (VLDB 2018)

Practice of Streaming and Dynamic Graphs: Concepts, Models, Systems, and Parallelism (2020)

Datasets

Note: Very small datasets are not included in the list.

About

This list was compiled by Domagoj Margan with help and resources from the systems community.