Speculative Sampling

If you're using cs470 team server, check this. You don't need anything below.

Installation

Cargo & Rust

curl https://sh.rustup.rs -sSf | sh

If you don't have curl,

sudo apt install curl

CUDA

Install 12.1 or 12.4 from NVIDIA official website.

libtorch

Install from official website. Scroll down and select Stable > Linux > LibTorch > C++/Java. Select CUDA version that you installed and install libtorch.

How to build

cargo build --release

Usage

cargo run --release --bin cs470 -- --help

You can find input prompts here.

cargo run --release --bin cs470-batch -- --help

You can find batch test scenarios is here: