Can LLMs Lie? Investigation beyond Hallucination

This is the source code for the paper "Can LLMs Lie? Investigation beyond Hallucination".

Get a quick overview of the project at: https://llm-liar.github.io/.

Setup

Install with pip install -e ".[all]".

Then all env vars needed to be set are:

• OPENAI_API_KEY to convert counterfact dataset from completion to QA format. data/counterfact-qa.
• HF_TOKEN to download LLM weights from huggingface.

Control lying by steering vectors

[Optional] Reproduce data/combination_results.pt by running python steer_scripts/multi.py.

Evaluate how the steering vectors controls different types of lying, by running python steer_scripts/type_eval.py.

Evaluate derived steering vectors on the multiround salesperson setting: python steer_scripts/pareto.py.

Mechanistic Interpretation

Find attention heads that help lying

You can test the model with 0/1/2/4/8/16/20 lying heads disabled on free generation: cd scripts2 && python head_demo.py.

If you want to search for heads yourself follow these steps:

1. Unzip to get the dataset: unzip data/counterfact-qa.zip
2. Find top k heads related to lying, by cd scripts2 && python head_search_v3.py. This shall generate results.yaml in the specified output path
3. Copy the list of heads from the generated results.yaml, and overwrite the hardcoded HEADS_TO_DISABLE in scripts2/head_demo.py

In case of insufficient VRAM, run python head_search_v3.py search.batch_size=1.

Prerequisites and Setup

You need to set up API keys for evaluation. There are two options:

Option 1: Using OpenRouter API

1. Get an OpenRouter API key
2. Modify the API key in two files:
   • In mech_interp_scripts/gpt_rater.py: Change API_KEY = "FAKE_API_KEY" to your OpenRouter API key
   • In mech_interp_scripts/openrouter_eval.py: Set your OpenRouter API key

Option 2: Using OpenAI API

Alternatively, you can use OpenAI's API by uncommenting the OpenAI client configuration in mech_interp_scripts/gpt_rater.py and setting:

• Your OpenAI organization ID
• Your OpenAI API key

Liar Score Comparison

This section contains two main scripts for evaluation and visualization:

Running Evaluation

Use openrouter_eval.py to run the evaluation with the following parameters:

• --model_name: Name of the model to evaluate (default: "meta-llama--Llama-3.1-8B-Instruct")
• --output_root: Root directory for results (default: "results")
• --output_dir: Output directory name (default: "openrouter_eval")
• --n_data: Number of data points to evaluate (default: 100)
• --lie_suffix_path: Path to lie suffixes file (default: "lie_suffixes.yaml")
• --use_cot: Enable/disable chain of thought reasoning (optional flag)

Visualization

Run plot_liar_eval.py to visualize the evaluation results. The script reads data from the results directory by default and generates visualization plots for the liar score comparison.

Data Generation for Intervention Analysis

This section contains scripts for generating baseline data using the counterfact dataset:

Available Scripts

1. eval_llama_8b.py: Generates baseline data for LLaMA models
2. eval_qwen.py: Generates baseline data for Qwen models
3. eval_data_analysis.py: Analyzes the deception scores for each lie suffix

These scripts:

1. Filter samples from the counterfact dataset where the model knows the correct answer
2. Generate the model's original lies using different prompts
3. Calculate baseline deception scores for comparison with intervention results
4. Filter out the samples where the model is able to tell good lies at beginning

Common Parameters

--model_name          # Model to evaluate (e.g., "meta-llama--Llama-3.1-8B-Instruct")
--n_data             # Number of examples to process
--output_root        # Root directory for results (default: "results")
--output_dir         # Output directory name
--eval_model         # Model for evaluation ("gpt-4o" or "gpt-4o-mini")
--lie_suffix_path    # Path to the file containing lie-inducing prompts

The scripts will generate a YAML file containing:

• Original prompts and responses
• Questions and correct answers
• Keywords and subjects from the prompts
• Deception scores for each response

Neuron Intervention Experiments

This section contains various experiments for analyzing model behavior by zeroing out different components:

Available Scripts

1. intent_all_zero_gpt.py: Zeros out attention scores from intent tokens to the end of the intent sentence in the last k tokens (by default, masks 4 dummy tokens [-5:-1], excluding the final prediction token)

2. intent_subject_zero_gpt.py: Zeros out attention scores or key/value from intent/subject/both tokens in the last k tokens (by default, masks 4 dummy tokens [-5:-1], excluding the final prediction token)

3. intent2all_last_zero_gpt.py: Zeros out attention scores from intent/subject tokens to the last k tokens in the last token's query (by default, masks 4 dummy tokens [-5:-1], excluding the final prediction token)

4. intent2all_zero_gpt.py: Similar to above but for last k tokens' query (by default, masks 4 dummy tokens [-5:-1], excluding the final prediction token)

5. last_token_zero_gpt.py: Zeros out residual/attention/MLP outputs for the last token

6. last_token_zero_head_gpt.py: Zeros out specific attention heads in a given layer for the last token

7. token_attn_zero_gpt.py: Zeros out attention scores or key/value for the last k tokens in the last token's query (by default, masks 4 dummy tokens [-5:-1], excluding the final prediction token)

8. token_zero_gpt.py: Zeros out various components (residual/attention/MLP) for the last k tokens (by default, masks 4 dummy tokens [-5:-1], excluding the final prediction token)

Common Parameters

Most scripts share these common parameters:

--model_name          # Model to evaluate (e.g., "meta-llama--Llama-3.1-8B-Instruct")
--n_data             # Number of examples to process
--output_root        # Root directory for results (default: "results")
--output_dir         # Output directory name
--eval_model         # Model for evaluation ("gpt-4o" or "gpt-4o-mini")
--window_size        # Window size for zeroing (default: 1)
--recorded_data_dir  # Directory containing recorded data

Example Usage

python scripts/token_zero_gpt.py \
    --model_name meta-llama--Llama-3.1-8B-Instruct \
    --n_data 200 \
    --output_dir experiment_results \
    --eval_model gpt-4o \
    --layer_type mlp \
    --window_size 5 \
    --recorded_data_dir eval_data

Each script will generate:

1. A YAML file containing detailed results
2. Two visualization plots:
   • Average deception scores across layers
   • Count of correct answers across layers

Acknowledgements

Special thanks to https://github.com/kmeng01/rome that inspired the earliest versions of this code.