Shadowgraph
About
Sign in

beomi/KoAlpaca-Polyglot

Model

66

stars

13

commits

10

repos using this model

2

linked in READMEs

Sep 15, 2023

updated

endpoints_compatible
generated_from_trainer
gpt-neox
gpt_neox
KoAlpaca
polyglot-ko
pytorch
safetensors
text-generation
text-generation-inference
transformers

README

Update @ 2023.06.01

  • Add Safetensor sharded model weight (max shard = 1GB)

KoAlpaca-Polyglot-5.8B (v1.1b)

This model is a fine-tuned version of EleutherAI/polyglot-ko-5.8b on a KoAlpaca Dataset v1.1b

Detail Codes are available at KoAlpaca Github Repository

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 2.0
  • mixed_precision_training: Native AMP

Framework versions

  • Transformers 4.29.0.dev0
  • Pytorch 2.0.0+cu117
  • Datasets 2.10.1
  • Tokenizers 0.13.2

Contributors

beomi

7 commits

june42

5 commits

librarian-bot

1 commits

Similar repos

beomi/KoAlpaca-Polyglot-5.8B

Model

66

shared topics

beomi/KoAlpaca-Polyglot-12.8B

Model

63

shared topics

metterian/kullm-polyglot-12.8b

Model

4

shared topics

beomi/KoAlpaca-KoRWKV-6B

Model

8

shared topics

nlpai-lab/kullm-polyglot-5.8b-v2

Model

24

nlpai-lab/kullm-polyglot-12.8b-v2

Model

53

EleutherAI/polyglot-ko-5.8b

Model

66

EleutherAI/polyglot-ko-3.8b

Model

24

Used in code

beomi/koalpaca

KoAlpaca: 한국어 명령어를 이해하는 오픈소스 언어모델 (KoAlpaca: An open-source language model to understand Korean…

1,572

1 file

aws-samples/aws-ai-ml-workshop-kr

A collection of localized (Korean) AWS AI/ML workshop materials for hands-on labs.

308

1 file

VAIV-2023/RLHF-Korean-Friendly-LLM

Developing a Korean LLM model : Hate Speech Filtering, Improving conversational skills, Finetuning…

20

1 file

etri-edgeai/cloud-edge-aicontainers

14

1 file

jiwoochris/In-Memory-Vector-DB

Construct a vector database through sentence embedding. And make your LLM respond based on this…

10

1 file

yejining99/tag_classification

This code uses LLM and contrastive loss to classify the teg keywords for each news.

2

1 file

cockroach54/kogpt

0

1 file

rlawldud53/komo_coling

0

1 file

mnbv7581/document_summary

문서요약을 위한 챗봇

0

1 file

ksb12345/kwater

0

1 file

Linked in READMEs

beomi/koalpaca

KoAlpaca: 한국어 명령어를 이해하는 오픈소스 언어모델 (KoAlpaca: An open-source language model to understand Korean…

1,572

README badge

ksb12345/kwater

0

README badge

beomi/KoAlpaca-Polyglot

Model

66

stars

13

commits

10

repos using this model

2

linked in READMEs

Sep 15, 2023

updated

endpoints_compatible
generated_from_trainer
gpt-neox
gpt_neox
KoAlpaca
polyglot-ko
pytorch
safetensors
text-generation
text-generation-inference
transformers

README

Update @ 2023.06.01

  • Add Safetensor sharded model weight (max shard = 1GB)

KoAlpaca-Polyglot-5.8B (v1.1b)

This model is a fine-tuned version of EleutherAI/polyglot-ko-5.8b on a KoAlpaca Dataset v1.1b

Detail Codes are available at KoAlpaca Github Repository

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 2.0
  • mixed_precision_training: Native AMP

Framework versions

  • Transformers 4.29.0.dev0
  • Pytorch 2.0.0+cu117
  • Datasets 2.10.1
  • Tokenizers 0.13.2

Similar repos

beomi/KoAlpaca-Polyglot-5.8B

Model

66

shared topics

beomi/KoAlpaca-Polyglot-12.8B

Model

63

shared topics

metterian/kullm-polyglot-12.8b

Model

4

shared topics

beomi/KoAlpaca-KoRWKV-6B

Model

8

shared topics

nlpai-lab/kullm-polyglot-5.8b-v2

Model

24

nlpai-lab/kullm-polyglot-12.8b-v2

Model

53

EleutherAI/polyglot-ko-5.8b

Model

66

EleutherAI/polyglot-ko-3.8b

Model

24

Used in code

beomi/koalpaca

KoAlpaca: 한국어 명령어를 이해하는 오픈소스 언어모델 (KoAlpaca: An open-source language model to understand Korean…

1,572

1 file

aws-samples/aws-ai-ml-workshop-kr

A collection of localized (Korean) AWS AI/ML workshop materials for hands-on labs.

308

1 file

VAIV-2023/RLHF-Korean-Friendly-LLM

Developing a Korean LLM model : Hate Speech Filtering, Improving conversational skills, Finetuning…

20

1 file

etri-edgeai/cloud-edge-aicontainers

14

1 file

jiwoochris/In-Memory-Vector-DB

Construct a vector database through sentence embedding. And make your LLM respond based on this…

10

1 file

yejining99/tag_classification

This code uses LLM and contrastive loss to classify the teg keywords for each news.

2

1 file

cockroach54/kogpt

0

1 file

rlawldud53/komo_coling

0

1 file

mnbv7581/document_summary

문서요약을 위한 챗봇

0

1 file

ksb12345/kwater

0

1 file

Linked in READMEs

beomi/koalpaca

KoAlpaca: 한국어 명령어를 이해하는 오픈소스 언어모델 (KoAlpaca: An open-source language model to understand Korean…

1,572

README badge

ksb12345/kwater

0

README badge

Contributors

beomi

7 commits

june42

5 commits

librarian-bot

1 commits