💡GENIUS – generating text using sketches!
0
6 commits
1 linked in READMEs
updated Dec 8, 2022
💡GENIUS is a powerful conditional text generation model using sketches as input, which can fill in the missing contexts for a given sketch (key information consisting of textual spans, phrases, or words, concatenated by mask tokens). GENIUS is pre-trained on a large- scale textual corpus with a novel reconstruction from sketch objective using an extreme and selective masking strategy, enabling it to generate diverse and high-quality texts given sketches.
Model variations:
| Model | #params | Language | comment |
|---|---|---|---|
genius-large | 406M | English | The version used in paper (recommend) |
genius-large-k2t | 406M | English | keywords-to-text |
genius-base | 139M | English | smaller version |
genius-base-ps | 139M | English | pre-trained both in paragraphs and short sentences |
genius-base-chinese | 116M | 中文 | 在一千万纯净中文段落上预训练 |
💡GENIUS – generating text using sketches!
0
6 commits
1 linked in READMEs
updated Dec 8, 2022
💡GENIUS is a powerful conditional text generation model using sketches as input, which can fill in the missing contexts for a given sketch (key information consisting of textual spans, phrases, or words, concatenated by mask tokens). GENIUS is pre-trained on a large- scale textual corpus with a novel reconstruction from sketch objective using an extreme and selective masking strategy, enabling it to generate diverse and high-quality texts given sketches.
Model variations:
| Model | #params | Language | comment |
|---|---|---|---|
genius-large | 406M | English | The version used in paper (recommend) |
genius-large-k2t | 406M | English | keywords-to-text |
genius-base | 139M | English | smaller version |
genius-base-ps | 139M | English | pre-trained both in paragraphs and short sentences |
genius-base-chinese | 116M | 中文 | 在一千万纯净中文段落上预训练 |
