tarekeldeeb/ArabicCorpus2B

Dataset

BUILDING VOCABULARY

2

5 commits

4 linked in READMEs

updated Dec 14, 2022

README

BUILDING VOCABULARY
Processed 1754541204 tokens.
Counted 5329509 unique words.
Truncating vocabulary at min count 5.
Using vocabulary of size 1539115.

Build the Arabic Corpus

Dowload Resources

The arabic corpus {1.9B word} consists of the following resources:

ShamelaLibrary348.7z link {1.15B}
UN arabic corpus mirror1 mirror2 {0.37B}
AraCorpus.tar.gz link {0.14B}
Arabic Wikipedia Latest Articles Dump link {0.11B}
Tashkeela-arabic-diacritized-text-utf8-0.3.zip link {0.07B}
Arabic Tweets link {0.03B}
watan-2004.7z link {0.01B}

Build Script: https://github.com/tarekeldeeb/GloVe-Arabic/tree/master/arabic_corpus

Download the dataset

Mirror : https://archive.org/details/arabic_corpus

license: Waqf v2 (https://github.com/ojuba-org/waqf/tree/master/2.0)

Similar repos

ClusterlabAi/101_billion_arabic_words_dataset

101 Billion Arabic Words Dataset

73

Jr23xd23/ArabicText-Large

ArabicText-Large: High-Quality Arabic Corpus for LLM Training

69

MohamedRashad/arabic-billion-words

Arabic Billion Words Dataset 🌕

12

pain/Arabic-Tweets

Dataset Card for Dataset Arabic-Tweets

23

M-A-D/Mixed-Arabic-Datasets-Repo

Dataset Card for "Mixed Arabic Datasets (MAD) Corpus"

38

aubmindlab/bert-large-arabertv2

AraBERT v1 & v2 : Pre-training BERT for Arabic Language Understanding

12

aubmindlab/bert-base-arabertv02

AraBERT v1 & v2 : Pre-training BERT for Arabic Language Understanding

51

aubmindlab/bert-base-arabertv2

AraBERT v1 & v2 : Pre-training BERT for Arabic Language Understanding

46

Linked in READMEs

h9-tec/Awesome_Arabic_NLP

257

tarekeldeeb/GloVe-Arabic

GloVe model for distributed arabic word representation

38

bakrianoo/Datasets

25

ojuba-org/waqf

Waqf License

24

tarekeldeeb/ArabicCorpus2B

Dataset

BUILDING VOCABULARY

2

5 commits

4 linked in READMEs

updated Dec 14, 2022

README

BUILDING VOCABULARY
Processed 1754541204 tokens.
Counted 5329509 unique words.
Truncating vocabulary at min count 5.
Using vocabulary of size 1539115.

Build the Arabic Corpus

Dowload Resources

The arabic corpus {1.9B word} consists of the following resources:

ShamelaLibrary348.7z link {1.15B}
UN arabic corpus mirror1 mirror2 {0.37B}
AraCorpus.tar.gz link {0.14B}
Arabic Wikipedia Latest Articles Dump link {0.11B}
Tashkeela-arabic-diacritized-text-utf8-0.3.zip link {0.07B}
Arabic Tweets link {0.03B}
watan-2004.7z link {0.01B}

Build Script: https://github.com/tarekeldeeb/GloVe-Arabic/tree/master/arabic_corpus

Download the dataset

Mirror : https://archive.org/details/arabic_corpus

license: Waqf v2 (https://github.com/ojuba-org/waqf/tree/master/2.0)

Similar repos

ClusterlabAi/101_billion_arabic_words_dataset

101 Billion Arabic Words Dataset

73

Jr23xd23/ArabicText-Large

ArabicText-Large: High-Quality Arabic Corpus for LLM Training

69

MohamedRashad/arabic-billion-words

Arabic Billion Words Dataset 🌕

12

pain/Arabic-Tweets

Dataset Card for Dataset Arabic-Tweets

23

M-A-D/Mixed-Arabic-Datasets-Repo

Dataset Card for "Mixed Arabic Datasets (MAD) Corpus"

38

aubmindlab/bert-large-arabertv2

AraBERT v1 & v2 : Pre-training BERT for Arabic Language Understanding

12

aubmindlab/bert-base-arabertv02

AraBERT v1 & v2 : Pre-training BERT for Arabic Language Understanding

51

aubmindlab/bert-base-arabertv2

AraBERT v1 & v2 : Pre-training BERT for Arabic Language Understanding

46

Linked in READMEs

h9-tec/Awesome_Arabic_NLP

257

tarekeldeeb/GloVe-Arabic

GloVe model for distributed arabic word representation

38

bakrianoo/Datasets

25

ojuba-org/waqf

Waqf License

24