Apply to my task
arXiv:2609.33287 ★ 86 Sep 27, 2026

Backward-Translation Check: Verify the Result Through a Retelling, Not by Trusting the Translation

Key result

Top LLMs hit an 89% accuracy ceiling when translating meaning into a formal result: code, a contract, a spec, a formula. Newer model versions bring almost no gains. The reason isn't model weakness: people themselves leave thresholds, deadlines and conditions unsaid, and guessing the unsaid is impossible. The Backward-Translation Check method lets you see the gap between your intent and what the AI understood before the result is put to use: the model translates the formal result back into a plain retelling, and you check it against your original idea.

TL;DR

Instead of asking the model to turn your intent straight into a complex formal result (code, a contract, a formula, a spec) and trusting that it understood correctly, you ask it to do the translation and then retell in its own words what came out. You compare the retelling with what you originally wanted. If it doesn't match, you correct it in plain language, and the loop repeats.

Models hit an accuracy ceiling on "meaning → formal result" translation tasks: even the top ones get stuck around 89%, and newer model versions bring no gains. The reason isn't model weakness; it's that the task is inherently unfair: what a person wrote doesn't always match what they meant. People leave out thresholds, conditions and time frames, and the model can't guess what was never said.

The method turns a one-shot translation into a loop: Translate → Retell → Check → Correct, which keeps spinning until the person confirms the result. It runs on an asymmetry: translating meaning into formal form is hard and error-prone, while translating an already finished formal result back into readable text is easy and accurate, because the source is already unambiguous.


Method Blueprint

STEP 1: The model translates the user's request into a formal result → code/contract/formula/spec
STEP 2: The model translates that result BACK into natural language → a retelling of "what this means"
STEP 3: The user compares the retelling with their intent → accepts or asks for a correction
STEP 4 (if correcting): the user gives a clarification in plain language → back to step 1, taking the history into account

All steps happen in one chat, as consecutive messages. The user never reads the formal result directly, only its retelling.


Example in Action

Task: You're writing a project brief for a freelancer on Upwork or Fiverr, but you're worried they'll understand the task differently from what you intended.

Prompt:

Write a project brief for a freelancer based on my description:

{task description in your own words}

First, write the brief itself.

Then, in a second block, retell this brief in your own words as if you were the contractor 
who read only the brief (not my original description) and are explaining to the client 
exactly what you understood and what you're going to do. Don't copy wording from the brief; retell the meaning.

Result: the model will return two blocks: the formal brief and, separately, a retelling of "how I understood it." You compare the retelling with your original idea. If the contractor (the model) got it wrong, the mismatch is immediately visible, and you write a clarification in plain language: "no, the deadline isn't 3 days, it's 3 business days, and revisions should be free for a week." The loop repeats until the retelling matches your intent.


Why It Works

The LLM's weakness is that when translating intent directly into formal text it has to guess the unstated details, and the error is invisible: the final text sounds convincing even if it means something other than what you wanted.

The LLM's strength: give it an already finished, fixed source (text, formula, code) and it's excellent at retelling its meaning in simple words. There's no guessing here; everything is already written, it just has to read and explain.

The method exploits this difference: it makes the model first do the translation and then, in the same conversation, explain what came out. The gap between "what the model understood" and "what you wanted" becomes visible before the result is put to use.

An important nuance from the study: the accuracy gain comes specifically from substantive feedback: a concrete pointer to what's wrong. If you just write "try again" without explaining the error, almost the entire effect disappears. So corrections need to be specific: not "that's not it," but "you wrote X, but I meant Y, because Z."

Control knobs: - Number of correction rounds → for simple tasks 1-2 are enough; don't artificially drag out the loop - Form of the retelling → you can ask for a retelling "as for a beginner" or "as for a lawyer"; different levels of detail expose different problems - The role the retelling comes from → "as a contractor who didn't see the original text" gives a more honest check than "explain what you wrote"


Prompt Template

I want to get a {formal result: contract/code/spec/formula} based on my description:

{task description}

First, produce the {formal result}.

Then retell in your own words, as if you were a person who sees only this result, 
without my original description, what it means and what conditions and constraints are built into it.

I'll compare your retelling with what I meant and tell you what needs fixing.

🚀 Quick start: paste this into the chat:

Here is a template for checking through a backward retelling. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask which formal result you need and which details of the task are critical, because without that the retelling can't expose the real gap between your intent and the result.


Limitations

⚠️ Feedback won't rescue a weak model: if the model is bad at the task to begin with, repeated corrections barely help; it just repeats the same mistake round after round. In that case the loop is a waste of time; you need a more capable model.

⚠️ Several corrections at once is a trap: if one piece of feedback asks for many unrelated details to be fixed at the same time, the model may fix one and get stuck, ignoring the rest even after repeated reminders. Fix one detail at a time.

⚠️ Empty feedback is useless: a bare "try again" gives almost no accuracy gain. Only a concrete pointer to the mismatch works.


How They Tested It

The researchers collected 500 requirements written by hand by experts, to be translated into the formal logic STL (a language for describing the behavior of technical systems: autonomous cars, drones, robots, smart homes). They ran them through seven models, from a tiny 0.5B one to top flagship models, and compared one-shot translation (output the result, trust it) with the "translate → retell → check → correct" loop.

The role of the "user" checking the retelling was mostly played by an AI judge, but they also checked it separately against real people: agreement came out at 99.5%, meaning the automated check can be trusted.

The size of the result was surprising: weak models gained 30+ percentage points (from 17.6% to 48%, from 45% to 76%), and top models that scored around 89% even without the loop reached 98-99%. In a separate test they replaced substantive feedback with an empty "try again," and the gain nearly vanished, proving it's not about the number of attempts but about the model hearing specifically what's wrong. They also found a capability threshold: the smallest model barely benefited from feedback; it was too weak to understand and apply the correction.


Resources

Bowen Ye, Xiang Yin, Shanghai Jiao Tong University. "Feedback Makes Perfect: A Closed-Loop Framework for NL-to-STL Translation"

Research digest

Key result

Top LLMs hit an 89% accuracy ceiling when translating meaning into a formal result: code, a contract, a spec, a formula. Newer model versions bring almost no gains. The reason isn't model weakness: people themselves leave thresholds, deadlines and conditions unsaid, and guessing the unsaid is impossible. The Backward-Translation Check method lets you see the gap between your intent and what the AI understood before the result is put to use: the model translates the formal result back into a plain retelling, and you check it against your original idea.

Principle

A simple asymmetry in the model's abilities is at work here. Translating meaning into formal text means guessing the unsaid, and it's easy to make a mistake without even noticing. Translating formal text back into plain words is just retelling a finished, unambiguous source, and it's almost impossible to slip up. The model is like a translator from plain English into legalese: it translates back into plain English with no trouble, but it can't guess what you meant when you wrote in plain English.

Why it works

Translating intent directly into a formal result is guesswork. The model fills in thresholds, deadlines and conditions you never said out loud. And the error is invisible: the text sounds convincing even if it means something other than what you wanted. A backward retelling removes the guessing: the source is already fixed, and the model only has to explain what's written there. An important nuance: only substantive corrections work. A plain "try again" barely improves accuracy; you need a concrete "you wrote X, but I meant Y."

When to apply

Work with texts where precise wording is critical → writing briefs for freelancers, contracts, calculation formulas, code from a description, especially when the person executing it might understand the task differently than you intended. Not suitable if the model is bad at the translation task to begin with; then corrections don't help, they just send the same mistake around in circles.

Mini recipe

1. Describe the task: in your own words, without trying to phrase it formally; the model will translate it into the needed format itself 2. Ask for two blocks at once: the formal result and, in the same message, a plain-language retelling of that result from the perspective of someone who never saw the original description 3. Compare the retelling with your intent: the mismatch is usually obvious right away; whatever isn't what you wanted sounds clearly different 4. Correct one detail at a time: don't dump five corrections at once; the model will fix one and ignore the rest 5. Repeat the loop: until the retelling matches what you originally meant

Examples

[BAD]: `Write a freelancer brief from this description: {text}` [GOOD]: `Write the brief. Then, in a second block, retell it in your own words as the contractor who read only the brief and never saw my original description. Don't copy the wording; retell the meaning of what you're going to do.`

In plain words

Large language models love to look confident while they're screwing up. When you ask a model to translate an ordinary thought into strict code, a formula or a contract, it starts filling in hidden details on its own. What comes out is a hallucination in nice wrapping: it looks solid but barely works. The core fix is simple: a closed feedback loop where the model has to retell its result in plain words.

It's like describing a complicated haircut to your barber and then silently waiting for the end result. He'll nod confidently and give you a buzz cut. But if you make him first repeat in his own words what exactly he's about to shave off, the misunderstanding surfaces before the first snip of the scissors. Same principle: let the AI prove first that it understood the task without improvising.

The method is built on a combo: forward translation into code, backward retelling into human language and step-by-step calibration. The model takes your sentence, generates a complex construct and then explains what it means in plain terms. You spot a mismatch, write a short "no, the second condition isn't needed", and the loop repeats until it's right. No digging through formulas: you steer the process in plain text.

The study ran on hardcore mathematical logic, Signal Temporal Logic (STL), but the mechanics are completely universal. This approach saves the day when generating complex SQL queries, drafting legal contracts, writing API integrations and briefs for freelancers. Anywhere the cost of a hidden model error is a blown budget or broken production.

The main takeaway: blind trust in generations leads to failure. Stop accepting complex code and formulas on faith just because they look smart. Make the model explain itself in plain language until your pictures of the world match one hundred percent.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on Backward-Translation Check: Verify the Result Through a Retelling, Not by Trusting the Translation arXiv:2609.33287

Sign in to apply this study New accounts get 50 free credits.