ArabicText-Large: High-Quality Arabic Corpus for LLM Training

Dataset Summary

ArabicText-Large is a comprehensive, high-quality Arabic text corpus comprising 743,288 articles with over 244 million words, specifically curated for Large Language Model (LLM) training and fine-tuning. This dataset represents one of the largest publicly available Arabic text collections for machine learning research.

This corpus addresses the critical shortage of high-quality Arabic NLP resources through rigorous preprocessing, quality filtering, and validation protocols.

Built by RightNow AI, the first GPU-native AI code editor.

Dataset DOI: https://doi.org/10.57967/hf/6685

Key Features

• Massive Scale: 743,288 articles with 244 million words
• High Quality: Multi-stage cleaning and quality filtering (average quality score: 58.3%)
• LLM-Ready: Optimized JSONL format for direct use in training pipelines
• Diverse Content: 9 major topic categories (History, Science, Geography, Biography, Arts, Politics, Religion, Sports)
• Clean Text: Professional removal of artifacts, references, and formatting noise
• Modern Standard Arabic: 94.2% Arabic content purity
• Rich Vocabulary: 1.5 million unique words
• Open License: Apache 2.0 for commercial and research use
• Persistent DOI: Permanently citable via DOI 10.57967/hf/6685

Dataset Statistics

Content Distribution

Quality Assessment

Average Quality Score: 58.3% High-Quality Articles (≥60%): 58.7%

Usage

Loading with Hugging Face Datasets

from datasets import load_dataset

# Load the dataset
dataset = load_dataset("Jr23xd23/ArabicText-Large")

# Access the training split
train_data = dataset["train"]

print(f"Total articles: {len(train_data)}")

# Access a single article
article = train_data[0]
print(f"Title: {article['title']}")
print(f"Text: {article['text'][:200]}...")

Loading with Python

import json

articles = []
with open('data.jsonl', 'r', encoding='utf-8') as f:
    for line in f:
        article = json.loads(line)
        articles.append(article)

print(f"Loaded {len(articles)} articles")

Data Format

Each entry in the dataset follows this structure:

{
  "id": "unique_article_identifier",
  "title": "Article Title in Arabic",
  "text": "Full cleaned Arabic text content...",
  "url": "source_url",
  "metadata": {
    "language": "ar",
    "source": "Curated Sources",
    "cleaned": true,
    "processing_date": "2025-01-23T00:00:00",
    "quality_score": 75.5
  }
}

Use Cases

Language Model Pre-training

• BERT-style models: Masked language modeling, text understanding
• GPT-style models: Causal language modeling, text generation
• T5-style models: Encoder-decoder architectures, sequence-to-sequence tasks
• Fine-tuning: Domain adaptation for Arabic-specific applications

Downstream NLP Tasks

• Text Classification: Sentiment analysis, topic classification, intent detection
• Named Entity Recognition: Entity extraction and tagging
• Question Answering: Reading comprehension, information retrieval
• Text Summarization: Abstractive and extractive summarization
• Machine Translation: Arabic-English, Arabic-French, multilingual translation
• Information Extraction: Relationship extraction, knowledge graph construction

Research Applications

• Arabic linguistics and computational morphology
• Cross-lingual transfer learning
• Multilingual model development
• Low-resource language processing research
• Comparative studies of Semitic languages

Data Processing Pipeline

Our multi-stage processing ensures the highest quality:

1. Source Collection: Curated from reliable, peer-reviewed sources
2. Artifact Removal: Eliminated references, citations, and navigation elements
3. Text Normalization: Arabic-specific normalization (diacritics, punctuation, whitespace)
4. Quality Filtering: Minimum 70% Arabic content, length constraints
5. Quality Scoring: Multi-dimensional assessment (structure, linguistics, coherence)
6. Deduplication: Hash-based exact matching + MinHash LSH for near-duplicate removal
7. Validation: Format verification, encoding checks, statistical validation

Quality Criteria

Articles are retained only if they meet all criteria:

• Minimum 100 characters, maximum 50,000 characters
• At least 70% Arabic characters
• Minimum 3 sentences for substantive content
• Quality score ≥40% on multi-dimensional assessment
• No stub indicators (e.g., "بحاجة للتوسيع")

Dataset Metrics

Length Distributions

Article Lengths:

• Minimum: 50 words
• Maximum: 20,757 words
• Median: 106 words
• Mean: 328.5 words
• Standard Deviation: 584.2 words

Sentence Lengths:

• Minimum: 1 word
• Maximum: 247 words
• Median: 16 words
• Mean: 19.7 words
• Standard Deviation: 12.3 words

Word Lengths:

• Minimum: 1 character
• Maximum: 42 characters
• Median: 4 characters
• Mean: 4.9 characters
• Standard Deviation: 2.8 characters

Vocabulary Statistics

• Total Unique Words: 1,529,064
• Vocabulary Richness: 0.0063
• Follows Zipf's Law: Yes (natural language distribution)

Most Frequent Words:

Technical Specifications

• Format: JSONL (JSON Lines)
• Encoding: UTF-8
• Language: Modern Standard Arabic (ar)
• Total Size: 2.8 GB (compressed)
• Processing Date: January 2025
• License: Apache 2.0
• Python Compatibility: 3.7+
• DOI: 10.57967/hf/6685

Comparison with Other Arabic Datasets

Limitations

• Dialectal Coverage: Primarily Modern Standard Arabic (MSA); limited dialectal variations
• Domain Bias: Encyclopedic content may not represent colloquial or conversational Arabic
• Temporal Coverage: Content reflects knowledge up to dataset collection date (January 2025)
• Size Trade-off: Smaller than billion-word web corpora but prioritizes quality over quantity

Future Enhancements

Planned improvements include:

• Dialectal Arabic expansion (Egyptian, Levantine, Gulf, Maghrebi)
• Domain diversification (literature, technical documents, news, social media)
• Parallel corpus creation (Arabic-English alignments)
• Linguistic annotations (POS tags, NER, dependency parsing)
• Regular updates with new content and quality improvements

License

This dataset is released under the Apache License 2.0.

Copyright 2025 Jaber Jaber

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Citation

If you use this dataset in your research, please cite:

@misc{jaber_2025,
  author       = {Jaber, Jaber},
  title        = {ArabicText-Large: A High-Quality 244-Million-Word Corpus for Arabic Language Model Training},
  year         = 2025,
  url          = {https://huggingface.co/datasets/Jr23xd23/ArabicText-Large},
  doi          = {10.57967/hf/6685},
  publisher    = {Hugging Face}
}

Research Paper:

@article{jaber2025arabictext,
  title={ArabicText-Large: A High-Quality 244-Million-Word Corpus for Arabic Language Model Training},
  author={Jaber, Jaber},
  journal={Journal of Open Humanities Data},
  year={2025},
  doi={10.57967/hf/6685},
  url={https://huggingface.co/datasets/Jr23xd23/ArabicText-Large}
}

Contributing

We welcome community contributions:

• Bug Reports: Report data quality issues or inconsistencies
• Feature Requests: Suggest dataset improvements or extensions
• Pull Requests: Contribute preprocessing enhancements or tools
• Feedback: Share your usage experience and research outcomes

Contact

For questions, collaborations, or research inquiries:

Author: Jaber Jaber Organization: RightNow AI Email: jaber@rightnowai.co Website: https://www.rightnowai.co

Acknowledgments

We extend our gratitude to:

• The Arabic NLP research community for valuable feedback and insights
• Open-source contributors for tools and frameworks that made this work possible
• Researchers and practitioners using this dataset to advance Arabic language technologies

---

Dataset Homepage: ArabicText-Large on Hugging Face DOI: https://doi.org/10.57967/hf/6685 License: Apache 2.0 Author: Jaber Jaber Year: 2025

Advancing Arabic NLP research and development