Dataset Card for "sampling-distill-train-data-kth-shift1"
0
5 commits
1 linked in READMEs
updated May 23, 2024
README
Dataset Card for "sampling-distill-train-data-kth-shift1"
Training data for sampling-based watermark distillation using the KTH \(s=1\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models. Llama 2 7B with decoding-based watermarking was used to generate 640,000 watermarked samples, each 256 tokens long. Each sample is prompted with 50-token prefixes from OpenWebText (prompts not included in the samples).
Similar repos
Dataset Card for "sampling-distill-train-data-kth-shift1"
0
5 commits
1 linked in READMEs
updated May 23, 2024
README
Dataset Card for "sampling-distill-train-data-kth-shift1"
Training data for sampling-based watermark distillation using the KTH \(s=1\) watermarking strategy in the paper On the Learnability of Watermarks for Language Models. Llama 2 7B with decoding-based watermarking was used to generate 640,000 watermarked samples, each 256 tokens long. Each sample is prompted with 50-token prefixes from OpenWebText (prompts not included in the samples).