chansung/alpaca-lora-30b

Model

This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The checkpoint is the output of instruction following fine-tuning process with the following settings on 8xA100(40G) DGX system.

5 commits

2 linked in READMEs

updated Apr 6, 2023

See the code

README

Dataset: cleaned-up Alpaca dataset up to 04/06/23
Training script: borrowed from the official Alpaca-LoRA implementation
Training script:

python finetune.py \
    --base_model='decapoda-research/llama-30b-hf' \
    --num_epochs=10 \
    --cutoff_len=512 \
    --group_by_length \
    --output_dir='./lora-alpaca' \
    --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
    --lora_r=16 \
    --batch_size=... \
    --micro_batch_size=...

alpaca

chat

llama

text2text-generation

Contributors

chansung

5 commits

Similar repos

chansung/alpaca-lora-13b

Model

This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The…

chansung/gpt4-alpaca-lora-30b

Model

This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The…

chansung/gpt4-alpaca-lora-7b

Model

This repository comes with LoRA checkpoint to make LLaMA into a chatbot like language model. The…

yahma/alpaca-7b-lora

Model

This repo contains a low-rank adapter for LLaMA-7b fit on the Cleaned Alpaca dataset (with the new…

project-baize/baize-lora-30B

Model

---

Angainor/alpaca-lora-13b

Model

This repo contains a low-rank adapter for LLaMA-13b fit on the Stanford Alpaca dataset.

aspctu/alpaca-lora

Instruct-tuning LLaMA on consumer hardware

tloen/alpaca-lora-7b

Model

This repo contains a low-rank adapter for LLaMA-7b

448

Linked in READMEs

tloen/alpaca-lora

Instruct-tune LLaMA on consumer hardware

18,903

README badge

gururise/AlpacaDataCleaned

Alpaca dataset from Stanford, cleaned and curated

1,609

README badge

chansung/alpaca-lora-30b

Model

5 commits

2 linked in READMEs

updated Apr 6, 2023

See the code

README

Dataset: cleaned-up Alpaca dataset up to 04/06/23
Training script: borrowed from the official Alpaca-LoRA implementation
Training script:

python finetune.py \
    --base_model='decapoda-research/llama-30b-hf' \
    --num_epochs=10 \
    --cutoff_len=512 \
    --group_by_length \
    --output_dir='./lora-alpaca' \
    --lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' \
    --lora_r=16 \
    --batch_size=... \
    --micro_batch_size=...

alpaca

chat

llama

text2text-generation