Risks and targets for assessing LLMs & LLM vulnerabilities
36
stars
38
commits
Python
primary language
May 27, 2024
updated
README
Language Model Risk Cards: Starter Set
A set of Language Model Risk cards for assessing a language model use case. To use these:
- Choose what use-case, model, and interface is to be assessed
- Select which of these risk cards is relevant in the given use-case scenario
- Recruit people to do the assessment
- For each risk card,
- Design how one will probe the model, and for how long
- Try to provoke the described behaviour from the language model, using your own prompts
- Record all inputs and outputs
- Compile an assessment report
Full details are given in the paper: Assessing Language Model Deployment with Risk Cards (2023), Leon Derczynski, Hannah Rose Kirk, Vidhisha Balachandran, Sachin Kumar, Yulia Tsvetkov, M. R. Leiser, Saif Mohammad
Contributors
Similar repos
Languages
Python
100.0%
Risks and targets for assessing LLMs & LLM vulnerabilities
36
stars
38
commits
Python
primary language
May 27, 2024
updated
README
Language Model Risk Cards: Starter Set
A set of Language Model Risk cards for assessing a language model use case. To use these:
- Choose what use-case, model, and interface is to be assessed
- Select which of these risk cards is relevant in the given use-case scenario
- Recruit people to do the assessment
- For each risk card,
- Design how one will probe the model, and for how long
- Try to provoke the described behaviour from the language model, using your own prompts
- Record all inputs and outputs
- Compile an assessment report
Full details are given in the paper: Assessing Language Model Deployment with Risk Cards (2023), Leon Derczynski, Hannah Rose Kirk, Vidhisha Balachandran, Sachin Kumar, Yulia Tsvetkov, M. R. Leiser, Saif Mohammad
Similar repos
Contributors
Languages
Python
100.0%