Shadowgraph
TrendingAbout
Sign in

HiTZ/alpaca-lora-7b-en-pt-es-ca-eu-gl-at

Model

should probably proofread and complete it, then remove this comment. -->

1

49 commits

1 linked in READMEs

updated Mar 24, 2023

See the code

README

alpaca-lora-7b-en-pt-es-ca-eu-gl-at

This model is a fine-tuned version of decapoda-research/llama-7b-hf on the HiTZ/alpaca_mt ['en', 'pt', 'es', 'ca', 'eu', 'gl', 'at'] dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0667

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 26
  • eval_batch_size: 26
  • seed: 42
  • distributed_type: multi-GPU
  • gradient_accumulation_steps: 5
  • total_train_batch_size: 130
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 1
  • mixed_precision_training: Native AMP

Training results

Training LossEpochStepValidation Loss
1.37720.041001.3860
1.30430.072001.2904
1.23070.113001.2409
1.21320.154001.2086
1.19870.195001.1854
1.15510.226001.1660
1.16130.267001.1516
1.1440.38001.1407
1.14940.349001.1297
1.10720.3710001.1196
1.13020.4111001.1117
1.10740.4512001.1058
1.08460.4813001.0995
1.0860.5214001.0935
1.07930.5615001.0889
1.09310.616001.0847
1.09050.6317001.0804
1.07930.6718001.0775
1.07950.7119001.0748
1.08610.7420001.0725
1.08810.7821001.0705
1.06730.8222001.0691
1.06260.8623001.0681
1.06330.8924001.0674
1.06010.9325001.0669
1.08490.9726001.0667

Framework versions

  • Transformers 4.28.0.dev0
  • Pytorch 2.0.0+cu117
  • Datasets 2.10.1
  • Tokenizers 0.13.2
generated_from_trainer

Contributors

juletxara

49 commits

Similar repos

HiTZ/alpaca-lora-30b-en-pt-es-ca-eu-gl-at

Model

should probably proofread and complete it, then remove this comment. -->

0

HiTZ/alpaca-lora-13b-en-pt-es-ca-eu-gl-at

Model

should probably proofread and complete it, then remove this comment. -->

0

HiTZ/alpaca-lora-65b-en-pt-es-ca

Model

should probably proofread and complete it, then remove this comment. -->

2

wxjiao/ParroT-7b

Model

should probably proofread and complete it, then remove this comment. -->

2

yahma/alpaca-7b-lora

Model

This repo contains a low-rank adapter for LLaMA-7b fit on the Cleaned Alpaca dataset (with the new…

13

Tensoic/Tiny-Llama-openhermes-1.1B-step-715k-1.5T

Model

This model is a fine-tuned version of [PY007/TinyLlama-1.1B-intermediate-step-715k-1.5T](https://hu…

4

Pra-tham/results

Model

should probably proofread and complete it, then remove this comment. -->

0

tloen/alpaca-lora-7b

Model

This repo contains a low-rank adapter for LLaMA-7b

448

Linked in READMEs

tloen/alpaca-lora

Instruct-tune LLaMA on consumer hardware

18,903

README badge

HiTZ/alpaca-lora-7b-en-pt-es-ca-eu-gl-at

Model

should probably proofread and complete it, then remove this comment. -->

1

49 commits

1 linked in READMEs

updated Mar 24, 2023

See the code

README

alpaca-lora-7b-en-pt-es-ca-eu-gl-at

This model is a fine-tuned version of decapoda-research/llama-7b-hf on the HiTZ/alpaca_mt ['en', 'pt', 'es', 'ca', 'eu', 'gl', 'at'] dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0667

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 26
  • eval_batch_size: 26
  • seed: 42
  • distributed_type: multi-GPU
  • gradient_accumulation_steps: 5
  • total_train_batch_size: 130
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 1
  • mixed_precision_training: Native AMP

Training results

Training LossEpochStepValidation Loss
1.37720.041001.3860
1.30430.072001.2904
1.23070.113001.2409
1.21320.154001.2086
1.19870.195001.1854
1.15510.226001.1660
1.16130.267001.1516
1.1440.38001.1407
1.14940.349001.1297
1.10720.3710001.1196
1.13020.4111001.1117
1.10740.4512001.1058
1.08460.4813001.0995
1.0860.5214001.0935
1.07930.5615001.0889
1.09310.616001.0847
1.09050.6317001.0804
1.07930.6718001.0775
1.07950.7119001.0748
1.08610.7420001.0725
1.08810.7821001.0705
1.06730.8222001.0691
1.06260.8623001.0681
1.06330.8924001.0674
1.06010.9325001.0669
1.08490.9726001.0667

Framework versions

  • Transformers 4.28.0.dev0
  • Pytorch 2.0.0+cu117
  • Datasets 2.10.1
  • Tokenizers 0.13.2
generated_from_trainer

Similar repos

HiTZ/alpaca-lora-30b-en-pt-es-ca-eu-gl-at

Model

should probably proofread and complete it, then remove this comment. -->

0

HiTZ/alpaca-lora-13b-en-pt-es-ca-eu-gl-at

Model

should probably proofread and complete it, then remove this comment. -->

0

HiTZ/alpaca-lora-65b-en-pt-es-ca

Model

should probably proofread and complete it, then remove this comment. -->

2

wxjiao/ParroT-7b

Model

should probably proofread and complete it, then remove this comment. -->

2

yahma/alpaca-7b-lora

Model

This repo contains a low-rank adapter for LLaMA-7b fit on the Cleaned Alpaca dataset (with the new…

13

Tensoic/Tiny-Llama-openhermes-1.1B-step-715k-1.5T

Model

This model is a fine-tuned version of [PY007/TinyLlama-1.1B-intermediate-step-715k-1.5T](https://hu…

4

Pra-tham/results

Model

should probably proofread and complete it, then remove this comment. -->

0

tloen/alpaca-lora-7b

Model

This repo contains a low-rank adapter for LLaMA-7b

448

Linked in READMEs

tloen/alpaca-lora

Instruct-tune LLaMA on consumer hardware

18,903

README badge

Contributors

juletxara

49 commits