flaneur2020/llama2.rs

🦙🦙.🦀

stars

113

commits

Rust

primary language

Sep 24, 2023

updated

README

🦙🦙.🦀

An rust reimplementatin of https://github.com/karpathy/llama2.c.

Run it

This repo contains a tinystories15M in the testdata/ folder, you can run it with the following command:

cargo run --release
./target/release/llama2-rs "rust is a crab"

https://github.com/flaneur2020/llama2.rs/assets/129800/8754c25e-c016-47b0-bb97-99930624a097

Performance

model	tokens/s
tinystories15M	120~197

Contributors

flaneur2020

113 commits

Similar repos

gaxler/llama2.rs

Inference Llama 2 in one file of pure Rust 🦀

236

tairov/llama2.py

Inference Llama 2 in one file of pure Python

426

madroidmaq/llama2.kt

Port of Andrej Karpathy's llama2.c to Kotlin.

wizzard0/llama2.ts

Llama2 inference in one TypeScript file

nikolaydubina/llama2.go

LLaMA-2 in native Go

192

cgbur/llama2.zig

Inference Llama 2 in one file of pure Zig

220

epicure/llama2.js

Inference Llama 2 in one file of pure JavaScript(HTML)

trrahul/llama2.cs

Inference Llama 2 in one file of pure C#

103

Languages

Rust

99.7%

README

🦙🦙.🦀

An rust reimplementatin of https://github.com/karpathy/llama2.c.

Run it

This repo contains a tinystories15M in the testdata/ folder, you can run it with the following command:

cargo run --release
./target/release/llama2-rs "rust is a crab"

https://github.com/flaneur2020/llama2.rs/assets/129800/8754c25e-c016-47b0-bb97-99930624a097

Performance

model	tokens/s
tinystories15M	120~197