Synthetic Data Generated from IETF mailing lists.
The generated synthetic data was used to train retrieval models.
The synthetic data was generated using https://github.com/cheop-byeon/synthetic-data-kit, a tool-kit derived from https://github.com/meta-llama/synthetic-data-kit.
The dataset could be downloaded using the provided script download_RFCAlign.py. The topic is the corresponding WG (working group) name.
# Download entire RFCAlign dataset
python download_RFCAlign.py
huggingface-cli# Install CLI (if needed)
pip install -U "huggingface_hub[cli]"
# Download full RFCAlign repository to local folder
huggingface-cli download jiebi/RFCAlign --repo-type dataset --local-dir ./dataset/RFCAlign
python download_RFCAlign.py [--split <value>] [--topic <value>] [--no-download]
--splitTop-level folder to download.
Allowed values:
llama_non-verbosellama_verboseqwen_non-verboseqwen_verbose--topicOptional topic file name without .jsonl.
Rules:
--split<split>/<topic>.jsonlExamples:
--topic ace--topic quic--topic tls--no-downloadInspect remote repository structure and local folder tree only. No files are downloaded.
# 1) Inspect only (no download)
python download_RFCAlign.py --no-download
# 2) Download full RFCAlign dataset
python download_RFCAlign.py
# 3) Download one split folder only
python download_RFCAlign.py --split qwen_verbose
# 4) Download one specific file only
python download_RFCAlign.py --split qwen_verbose --topic ace
# 5) Another one-file download example
python download_RFCAlign.py --split llama_non-verbose --topic tls
./dataset/RFCAlign/.jsonl fileThe data was used for retrieval tasks, for training (https://github.com/cheop-byeon/FlagEmbedding) and for evaluation (https://github.com/cheop-byeon/mteb-R2Gen).
11 commits
Synthetic Data Generated from IETF mailing lists.
The generated synthetic data was used to train retrieval models.
The synthetic data was generated using https://github.com/cheop-byeon/synthetic-data-kit, a tool-kit derived from https://github.com/meta-llama/synthetic-data-kit.
The dataset could be downloaded using the provided script download_RFCAlign.py. The topic is the corresponding WG (working group) name.
# Download entire RFCAlign dataset
python download_RFCAlign.py
huggingface-cli# Install CLI (if needed)
pip install -U "huggingface_hub[cli]"
# Download full RFCAlign repository to local folder
huggingface-cli download jiebi/RFCAlign --repo-type dataset --local-dir ./dataset/RFCAlign
python download_RFCAlign.py [--split <value>] [--topic <value>] [--no-download]
--splitTop-level folder to download.
Allowed values:
llama_non-verbosellama_verboseqwen_non-verboseqwen_verbose--topicOptional topic file name without .jsonl.
Rules:
--split<split>/<topic>.jsonlExamples:
--topic ace--topic quic--topic tls--no-downloadInspect remote repository structure and local folder tree only. No files are downloaded.
# 1) Inspect only (no download)
python download_RFCAlign.py --no-download
# 2) Download full RFCAlign dataset
python download_RFCAlign.py
# 3) Download one split folder only
python download_RFCAlign.py --split qwen_verbose
# 4) Download one specific file only
python download_RFCAlign.py --split qwen_verbose --topic ace
# 5) Another one-file download example
python download_RFCAlign.py --split llama_non-verbose --topic tls
./dataset/RFCAlign/.jsonl fileThe data was used for retrieval tasks, for training (https://github.com/cheop-byeon/FlagEmbedding) and for evaluation (https://github.com/cheop-byeon/mteb-R2Gen).
11 commits