vagmi/llamaplay

My experments with rust and llama.cpp

Rust

0

1 commits

updated Apr 17, 2024

See the code

README

Llama Play

My experiments with Rust and Llama.cpp.

This repo is for me to learn the actual rust + cpp bindings used with llama.cpp.

  • Basic llama.cpp build
  • Load model
  • Construct new context
  • Tokenize prompt
  • Setup KV Cache
  • Setup a new batch with a single prompt
  • Decode batch to provide output

Contributors

vagmi

1 commits

vagmi/llamaplay

My experments with rust and llama.cpp

Rust

0

1 commits

updated Apr 17, 2024

See the code

README

Llama Play

My experiments with Rust and Llama.cpp.

This repo is for me to learn the actual rust + cpp bindings used with llama.cpp.

  • Basic llama.cpp build
  • Load model
  • Construct new context
  • Tokenize prompt
  • Setup KV Cache
  • Setup a new batch with a single prompt
  • Decode batch to provide output

Contributors

vagmi

1 commits

Languages

Rust

100.0%