Dataset Card for "sampling-distill-train-data-aar-k4"
0
5 commits
1 linked in READMEs
updated May 23, 2024
README
Dataset Card for "sampling-distill-train-data-aar-k4"
Training data for sampling-based watermark distillation using the Aar \(k=4\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models. Llama 2 7B with decoding-based watermarking was used to generate 640,000 watermarked samples, each 256 tokens long. Each sample is prompted with 50-token prefixes from OpenWebText (prompts not included in the samples).
Dataset Card for "sampling-distill-train-data-aar-k4"
0
5 commits
1 linked in READMEs
updated May 23, 2024
README
Dataset Card for "sampling-distill-train-data-aar-k4"
Training data for sampling-based watermark distillation using the Aar \(k=4\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models. Llama 2 7B with decoding-based watermarking was used to generate 640,000 watermarked samples, each 256 tokens long. Each sample is prompted with 50-token prefixes from OpenWebText (prompts not included in the samples).