rajasekarv/vega

A new arguably faster implementation of Apache Spark from scratch in Rust

Rust

2,226

415 commits

updated Apr 18, 2022

See the code

README

vega

Previously known as native_spark.

Documentation

A new, arguably faster, implementation of Apache Spark from scratch in Rust. WIP

Framework tested only on Linux, requires nightly Rust. Read how to get started in the documentation.

Contributing

If you are interested in contributing please jump on Gitter and check out and head to the issue tracker where you can find open issues (anything labelled good first issue and help wanted is up for grabs).

Project health

Overall

3.1/10

Maintained

0/10

Code review

3/10

OpenSSF Scorecard

Significant stargazers

(top 24 of 33)

Anton Kochkov

704 followers · starred Dec 2019

siddontang

2,981 followers · starred Nov 2019

Graydon Hoare

7,937 followers · starred Feb 2021

Richard Dodd

67 followers · starred Jun 2021

Similar repos

apache/paimon-rust

Apache Paimon Rust The rust implementation of Apache Paimon.

200

mesos/spark

Lightning-fast cluster computing in Java, Scala and Python.

1,419

rewrite-bigdata-in-rust/RBIR

A collection of RBIR projects and posts for anyone interested in joining this journey.

332

LukeMathWalker/pavex

A backend framework for Rust professionals

2,069

apache/auron

Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to…

1,813

apache/spark

Apache Spark - A unified analytics engine for large-scale data processing

44,181

lakehq/sail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and…

3,431

vortex-data/vortex

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar…

3,252