TL;DR
If you ask an AI to grade a text — a paper, a résumé, a business plan — the score depends not only on the content but on how confidently the text talks about itself. Researchers took the same scientific papers and rewrote them so the facts, data and methods stayed identical and only the delivery changed — and the AI reviewers' scores drifted noticeably in different directions.
Two rhetorical moves hit hardest: confident novelty claims ("we are the first to show X" instead of "this may be related to X") and emphasis on comparisons and patterns in the data ("the result outperforms by 15%" instead of a neutral recitation of numbers). What's more, if the AI was already skeptical (a low starting score), confident delivery lifts the score more. And if the AI is already impressed, cautious wording drops the score more than usual.
A simple fix partly works: if you explicitly tell the AI to demand concrete evidence and not count a confident tone as a plus, the average score falls — but the sensitivity to rhetoric doesn't disappear entirely. So delivery remains a lever on AI grading even when you explicitly ask the model to ignore it.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
IF YOU ARE THE AUTHOR of a text an AI will grade:
STEP 1: State novelty directly → "we are the first to show X", not "X may be related to Y"
STEP 2: Highlight comparisons and numbers → "the result is 15% higher", not a dry list of data
IF YOU ARE ASKING AN AI TO GRADE SOMEONE ELSE'S TEXT:
STEP 1: Add an evidence requirement to the prompt → "a high score only with concrete facts"
STEP 2: Explicitly forbid rewarding delivery → "don't reward a confident tone if the substance doesn't change"
Both steps happen in a single prompt — you just add the wording to the request.
Example in Action
Task: You're a product manager asking ChatGPT to grade your cover letter before sending it to a recruiter for a Senior PM role.
Prompt (cautious version):
Rate my cover letter on a scale of 1 to 10 for the
Senior PM role at [company]. Letter: "Perhaps my experience in product
analytics could be useful to your team. I think I'm fairly
comfortable with A/B tests and working with metrics."
Prompt (confident version, same experience):
Rate my cover letter on a scale of 1 to 10 for the
Senior PM role at [company]. Letter: "My experience in product analytics
applies directly to your team. I've launched more than 40 A/B tests
that raised conversion by 15%."
Result: The AI will most likely give the second version a higher score — even though the candidate's experience is the same. If you add the phrase "grade strictly: count only concrete facts and achievements, don't award points for confident wording" to the prompt, the gap between versions shrinks but doesn't vanish entirely.
Why It Works
An AI has no internal "truth detector" — it can't separately verify whether a confident claim matches reality. The model judges a text by how it sounds, because that's how high-quality texts in its training data are written: confidently, with numbers, with a clear focus on results.
That's also the model's strength — it reads rhetorical signals (confidence, specifics, comparisons) very well as markers of quality. But that very sensitivity becomes a vulnerability: the delivery changes, the content doesn't, and the score still creeps.
Control knobs: - A "strict" prompt (demand evidence, don't reward delivery) → lowers the average score but doesn't fully remove sensitivity to rhetoric - The AI's starting score — if a document is already rated low, polishing the rhetoric lifts the score more; if it's rated high, hedging drops it more - Repeatedly polishing the text doesn't produce a predictable gain — after the second pass the effect fades
Prompt Template
Grade {document} on these criteria: {criterion_1}, {criterion_2}, {criterion_3}.
Grading rules:
1. A high score only with concrete evidence: numbers,
facts, results. General confident wording does not count as evidence.
2. Do not reward confident delivery, flashy words or text
structure if it doesn't change the substance.
3. If a claim sounds convincing but isn't backed by
specifics, flag it separately and don't raise the score.
4. At the end, list separately: (a) what is confirmed by facts,
(b) what sounds good but isn't confirmed by facts.
Document:
{text}
Plug in your document (résumé, sales proposal, paper draft) and the grading criteria that matter for your task.
🚀 Quick start — paste into the chat:
Here is a strict prompt template for grading a document. Adapt it to my
task: {your task}. Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask which criteria matter and what kind of document is being graded — because without that it can't decide what counts as "evidence" in your specific case.
Limitations
⚠️ Doesn't remove the effect completely: even a strict prompt lowers the average score, but rhetorical sensitivity remains — the model still reacts a little to delivery.
⚠️ Works differently depending on the starting score: if the AI has already rated a document highly, trying to "sound more confident" may not work or may even backfire (the text comes across as overwrought).
⚠️ Repeated polishing doesn't guarantee improvement: several rounds of "punching up" a text give diminishing and unpredictable returns — after the second pass there's almost no benefit.
⚠️ Tested on grading tasks: the method covers situations where an AI assigns a score or writes a review. The findings weren't tested for generative tasks (writing a text, answering a question).
How They Studied It
The researchers took 120 papers submitted to ICLR 2026, covering the full quality range — from weak to strong as judged by human reviewers. Each paper was rewritten by two different AIs (GPT-5.5 and Claude Opus 4.8) along six rhetorical dimensions in opposite directions — for example, "confident about novelty" versus "cautious about novelty" — while the facts, methods and numbers stayed untouched (verified with dedicated technical checks).
That produced 4,200 manuscript versions, which five different AI reviewers graded under a standard and a strict protocol — 42,396 grades in total, costing nearly $30,000 in API calls.
The main surprise: a more elaborate rewrite doesn't necessarily mean a better result. Several rounds of "polishing" or incorporating feedback from the reviewer itself didn't produce a stable score gain — sometimes quite the opposite. That means one confident pass over the text often works as well as long rework.
Resources
How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review — Ming Li, Chenguang Wang, Xirui Li, Xinyue Zeng, Dianqi Li, Peng Shi, Dawei Zhou, Tianyi Zhou (University of Maryland, Virginia Tech, MBZUAI, University of Waterloo). Project: github.com/MingLiiii/Dissecting_AI_Reviews