abacaj/Replit-v2-CodeInstruct-3B-ggml

Model

stars

commits

repos using this model

linked in READMEs

Jun 29, 2023

updated

custom_code

mpt

text-generation

transformers

README

This is a ggml quantized version of Replit-v2-CodeInstruct-3B. Quantized to 4bit -> q4_1. To run inference you can use ggml directly or ctransformers.

Memory usage of model: 2GB~
Repo to run the model using ctransformers: https://github.com/abacaj/replit-3B-inference

Contributors

abacaj

10 commits

Similar repos

abacaj/replit-3B-inference

Run inference on replit-3B code instruct model using CPU

161

pixilated730/Replit_LLM

Run LLM on your CPU machine

teknium/Replit-v1-CodeInstruct-3B-fp16

Model

abacaj/mpt-30B-inference

Run inference on MPT-30B using CPU

574

TheBloke/CodeLlama-7B-GGUF

Model

134

TheBloke/CodeLlama-7B-Instruct-GPTQ

Model

TheBloke/speechless-codellama-34b-v2.0-GPTQ

Model

TheBloke/speechless-codellama-34b-v2.0-GGUF

Model

Used in code

taghash/ctransformers-inference-server

This creates a minimal, openai api compatible inference server to run models supported by…

1 file

Linked in READMEs

ggerganov/ggml

Tensor library for machine learning

15,294

README badge

marella/ctransformers

Python bindings for the Transformer models implemented in C/C++ using GGML library.

1,886

README badge

abacaj/replit-3B-inference

Run inference on replit-3B code instruct model using CPU

161

README badge

taghash/ctransformers-inference-server

This creates a minimal, openai api compatible inference server to run models supported by…

README badge

pixilated730/Replit_LLM

Run LLM on your CPU machine

README badge

abacaj/Replit-v2-CodeInstruct-3B-ggml

Model

stars

commits

repos using this model

linked in READMEs

Jun 29, 2023

updated

custom_code

mpt

text-generation

transformers

README

This is a ggml quantized version of Replit-v2-CodeInstruct-3B. Quantized to 4bit -> q4_1. To run inference you can use ggml directly or ctransformers.