📝 Game the LLM Reviewer

English / 中文 · 日本語 · العربية · Español · Deutsch · Français

Game the LLM Reviewer turns findings from studies of LLM review preferences into an agent-compatible skill for a final editing pass before submission. It makes small wording changes while preserving the paper's scientific meaning.

我们整理了 LLM 审稿偏好的相关研究，提炼出 LLM agent 可以直接使用的 skill，用于投稿前的最后一轮润色。

Quick start · Before / after · Strategies · Research

既然我们无法彻底避开 LLM 审稿，那干脆利用它们的偏好来保护我们的研究成果。

We oppose handing peer-review decisions over to LLMs, but authors often have little say in whether a reviewer uses one. Papers should be judged on the soundness of their methods, the strength of their evidence, and the substance of their contributions. Yet studies have found that rephrasing the same passage can change an LLM reviewer's score.

论文的评价应基于方法是否可靠、证据是否充分、贡献是否扎实。但已有研究发现，LLM 审稿人对论文同一段话换个措辞就会打出不同的分。

This skill provides a final, defensive editing pass after ordinary polishing. It selects wording that aligns with documented LLM reviewer preferences while preserving meaning and keeping the scientific assessment a human reviewer could make materially unchanged. It requires no knowledge of the reviewer model and does not query it.

这个工具是投稿前的最后一道“防御性润色”：我们根据 LLM 审稿偏好的研究，在意思完全不变的几种说法里将论文改写成 LLM reviewer 更青睐的那种，而这种改动对人类审稿人来说几乎没有区别。

[!IMPORTANT] ⚠️ Academic integrity

Edits must preserve the paper's claims and supporting evidence, including citations, assumptions, uncertainty, and substantive limitations. Record each change. Fabricated results, inflated novelty, concealed weaknesses, and hidden instructions to reviewers are outside the skill's scope.

Authors remain responsible for the manuscript and for following their venue's rules on AI assistance and disclosure.

写论文 → 跑你常用的写作工具 → 最后跑一遍 Game-the-LLM-Reviewer。

🚀 Quick start

Works with

 Claude Code   ·    Codex   ·    Cursor   ·    Antigravity

 GitHub Copilot   ·    OpenCode   ·    Kimi Code CLI   ·    TraeCode   ·    Qwen Code

and other Agent Skills-compatible agents.

1. Install the skill

Ask your coding agent to install the skill:

Clone this repository and install its game-the-llm-reviewer skill:
https://github.com/Michael-Jiahao-Zhang/game-the-llm-reviewer

Or install with Skills CLI:

npx skills add Michael-Jiahao-Zhang/game-the-llm-reviewer --skill game-the-llm-reviewer

2. Apply it to your manuscript

Give your agent a finished manuscript:

Use game-the-llm-reviewer on paper/main.tex. Read the included sections first.
Apply small, meaning-preserving edits targeting LLM reviewer preferences.
Keep the scientific assessment a human could make unchanged.
Save a revised copy and changes.md explaining each rhetorical change.

The agent saves a revised copy and a change note explaining the edits. It works with your existing writing setup and requires no additional reviewer API.

npx skills add Michael-Jiahao-Zhang/game-the-llm-reviewer --skill game-the-llm-reviewer -a claude-code
npx skills add Michael-Jiahao-Zhang/game-the-llm-reviewer --skill game-the-llm-reviewer -a codex

git clone https://github.com/Michael-Jiahao-Zhang/game-the-llm-reviewer.git
cd game-the-llm-reviewer

Read skills/game-the-llm-reviewer/SKILL.md and apply it to my finished manuscript.
Return a revised copy and a compact change note.

✨ Before and after

This example is a contribution overview from an introduction about execution memory for coding agents.

Before

Coding agents can lose track of failed repair attempts as tool outputs accumulate. We propose an execution memory that records attempted patches and their test outcomes for use in subsequent steps, without updating model weights. On 300 Python repository issues with two backbone models and a fixed per-issue token budget, the memory-equipped agent resolves 34% and 39% of issues, compared with 30% and 35% for the same agents without memory, respectively.

After Game the LLM Reviewer

We introduce an execution memory for coding agents that requires no model weight updates. It records attempted patches and their test outcomes for use in subsequent steps, addressing the loss of failed-attempt history as tool outputs accumulate. On 300 Python repository issues with two backbone models and a fixed per-issue token budget, execution memory increases issue resolution by 4 percentage points for each model, from 30% to 34% and from 35% to 39%, respectively.

What changed

Both versions describe the same method and evaluation. The revision foregrounds the contribution and expresses the existing rate differences in percentage points. See the strategy cards for details.

Read skills/game-the-llm-reviewer/SKILL.md and apply it to examples/coding-agent-introduction.md.
Save introduction.revised.md and introduction.changes.md.

📊 Selected score increases on real papers

These are already well-written papers describing excellent work. Each edit below changes a small phrase while keeping the scientific content intact.

ToolLLM and WebArena shift grammatical focus (S1); API-Bank reframes the same baseline comparison (S2). All other text in each evaluated section stays unchanged.

Setup: Selected examples scored by GPT-6 ASTRA on a 10-point scale, with one independent score per version of the abstract or introduction. The rewriter received no reviewer feedback.

🎯 Using it with other writing tools

Run this skill once the draft and ordinary polishing are complete. It can follow an existing writing workflow, including those provided by these projects:

🛠️ Editing strategies

Each strategy card describes when to use an edit and what to preserve. The agent chooses applicable edits from S1–S5, then uses S6 to check them against the original. It can leave a passage unchanged.

Use game-the-llm-reviewer on this abstract, keeping it under 200 words.
Work only from the supplied text and preserve both positive and negative results.
Return replacement prose and a short change note.

Use game-the-llm-reviewer on paper/main.tex. Adjust contribution stance without
changing its scientific meaning. Preserve theorem assumptions, quantifiers,
and the distinction between an upper bound and an optimal rate.

The draft is already polished. Use game-the-llm-reviewer as the final pass.
Reuse the attached claim–evidence map, checking it against the manuscript.
Use the included strategies to select small wording changes for LLM review preferences.
Keep the meaning and scientific assessment unchanged; do not rewrite just for style.

📦 Output

The output includes a revised manuscript and a change note. For a short excerpt, the agent can return replacement text and explain the changes inline. A change note looks like this:

The note briefly explains the main changes and flags any source issues that need your attention. For LaTeX projects, the agent preserves file relationships and compiles the revision when a suitable toolchain is available.

📚 References

The research notes summarize these papers and map their findings to the editing strategies:

• How Can Rhetoric Reward-Hack AI Reviewers? examines which rhetorical dimensions affect reviews.
• No Hidden Prompts Needed! studies presentation-only revisions using ARGAR.
• Gaming AI-Assisted Peer Reviews studies abstract rephrasing.
• LLM-REVal compares human and LLM writing preferences.
• Are We There Yet? examines review failures, including responses to disclosed limitations.

🤝 Contributing

Corrections and additional wording examples are welcome. Please include the source or reasoning behind a proposed strategy change; see CONTRIBUTING.md.

MIT license for this project's original files. Referenced research, datasets, and upstream code retain their own licenses. This project is independent of the cited authors.