Multilingual-Medical-Corpus a 3 billion word multilingual corpus for training LLMs adapted to the medical domain. Multilingual-Medical-Corpus includes four languages, namely, English, Spanish, French, and Italian.
| Language | Source | Words |
|---|---|---|
| English | ClinicalTrials | 127.4M |
| EMEA | 12M | |
| PubMed | 968.4M | |
| Spanish | EMEA | 13.6M |
| PubMed | 8.4M | |
| Medical Crawler | 918M | |
| SPACC | 350K | |
| UFAL | 10.5M | |
| WikiMed | 5.2M | |
| French | PubMed | 1.4M |
| Science Direct | 15.2M | |
| Wikipedia - Médecine | 5M | |
| EDP | 48K | |
| Google Patents | 654M | |
| Italian | Medical Commoncrawl - IT | 67M |
| Drug instructions | 30.5M | |
| Wikipedia - Medicina | 13.3M | |
| E3C Corpus - IT | 11.6M | |
| Medicine descriptions | 6.3M | |
| Medical theses | 5.8M | |
| Medical websites | 4M | |
| PubMed | 2.3M | |
| Supplement description | 1.3M | |
| Medical notes | 975K | |
| Pathologies | 157K | |
| Medical test simulations | 26K | |
| Clinical cases | 20K |
| HiTZ/Medical-mT5-large | HiTZ/Medical-mT5-xl | HiTZ/Medical-mT5-large-multitask | HiTZ/Medical-mT5-xl-multitask | |
|---|---|---|---|---|
| Param. no. | 738M | 3B | 738M | 3B |
| Task | Language Modeling | Language Modeling | Multitask Sequence Labeling | Multitask Sequence Labeling |
@misc{garcíaferrero2024medical,
title={Medical mT5: An Open-Source Multilingual Text-to-Text LLM for The Medical Domain},
author={Iker García-Ferrero and Rodrigo Agerri and Aitziber Atutxa Salazar and Elena Cabrio and Iker de la Iglesia and Alberto Lavelli and Bernardo Magnini and Benjamin Molinet and Johana Ramirez-Romero and German Rigau and Jose Maria Villa-Gonzalez and Serena Villata and Andrea Zaninello},
year={2024},
eprint={2404.07613},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
6 commits
Multilingual-Medical-Corpus a 3 billion word multilingual corpus for training LLMs adapted to the medical domain. Multilingual-Medical-Corpus includes four languages, namely, English, Spanish, French, and Italian.
| Language | Source | Words |
|---|---|---|
| English | ClinicalTrials | 127.4M |
| EMEA | 12M | |
| PubMed | 968.4M | |
| Spanish | EMEA | 13.6M |
| PubMed | 8.4M | |
| Medical Crawler | 918M | |
| SPACC | 350K | |
| UFAL | 10.5M | |
| WikiMed | 5.2M | |
| French | PubMed | 1.4M |
| Science Direct | 15.2M | |
| Wikipedia - Médecine | 5M | |
| EDP | 48K | |
| Google Patents | 654M | |
| Italian | Medical Commoncrawl - IT | 67M |
| Drug instructions | 30.5M | |
| Wikipedia - Medicina | 13.3M | |
| E3C Corpus - IT | 11.6M | |
| Medicine descriptions | 6.3M | |
| Medical theses | 5.8M | |
| Medical websites | 4M | |
| PubMed | 2.3M | |
| Supplement description | 1.3M | |
| Medical notes | 975K | |
| Pathologies | 157K | |
| Medical test simulations | 26K | |
| Clinical cases | 20K |
| HiTZ/Medical-mT5-large | HiTZ/Medical-mT5-xl | HiTZ/Medical-mT5-large-multitask | HiTZ/Medical-mT5-xl-multitask | |
|---|---|---|---|---|
| Param. no. | 738M | 3B | 738M | 3B |
| Task | Language Modeling | Language Modeling | Multitask Sequence Labeling | Multitask Sequence Labeling |
@misc{garcíaferrero2024medical,
title={Medical mT5: An Open-Source Multilingual Text-to-Text LLM for The Medical Domain},
author={Iker García-Ferrero and Rodrigo Agerri and Aitziber Atutxa Salazar and Elena Cabrio and Iker de la Iglesia and Alberto Lavelli and Bernardo Magnini and Benjamin Molinet and Johana Ramirez-Romero and German Rigau and Jose Maria Villa-Gonzalez and Serena Villata and Andrea Zaninello},
year={2024},
eprint={2404.07613},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
6 commits