teknium/llama-deus-7b-v3-lora-merged

Model

17

stars

3

commits

May 17, 2023

updated

endpoints_compatible
llama
pytorch
text-generation
text-generation-inference
transformers

README


license: mit

Base Mode: Llama 7B LoRA is fully Merged with llama7b, so you do not need to merge it to load the model.

Llama DEUS v3 is the largest dataset I've trained on yet, including:

GPTeacher - General Instruct - Code Instruct - Roleplay Instruct
My unreleased Roleplay V2 Instruct
GPT4-LLM Uncensored + Unnatural Instructions
WizardLM Uncensored
CamelAI's 20k Biology, 20k Physics, 20k Chemistry, and 50k Math GPT4 Datasets
CodeAlpaca

This model was trained for 4 epochs over 1 day of training, it's a rank 128 LORA that targets attention heads, LM_Head, and MLP layers

Prompt format:

### Instruction:
<prompt>
  
### Response:
  

or

### Instruction:
<prompt>

### Input:
<input>
  
### Response:
  

Contributors

teknium

3 commits

teknium/llama-deus-7b-v3-lora-merged

Model

17

stars

3

commits

May 17, 2023

updated

endpoints_compatible
llama
pytorch
text-generation
text-generation-inference
transformers

README


license: mit

Base Mode: Llama 7B LoRA is fully Merged with llama7b, so you do not need to merge it to load the model.

Llama DEUS v3 is the largest dataset I've trained on yet, including:

GPTeacher - General Instruct - Code Instruct - Roleplay Instruct
My unreleased Roleplay V2 Instruct
GPT4-LLM Uncensored + Unnatural Instructions
WizardLM Uncensored
CamelAI's 20k Biology, 20k Physics, 20k Chemistry, and 50k Math GPT4 Datasets
CodeAlpaca

This model was trained for 4 epochs over 1 day of training, it's a rank 128 LORA that targets attention heads, LM_Head, and MLP layers

Prompt format:

### Instruction:
<prompt>
  
### Response:
  

or

### Instruction:
<prompt>

### Input:
<input>
  
### Response:
  

Contributors

teknium

3 commits