Reflective Translation: Two-Pass Generation With Self-Reflection

STEP 1: Generate a draft
→ first version of the text

STEP 2: Structured critique (in a separate request)
→ (1) list of errors
→ (2) instructions for fixing them
→ (3) key elements that must be kept

STEP 3: Generate a revision based on the critique
→ improved version

Three separate requests to the model.

Task: Write a LinkedIn post about a new feature in your SaaS product. You need to hit the right tone (not corporate, but professional) and not miss the key benefits.

Prompt 1 (draft):

Write a LinkedIn post for the SaaS company SubDesk.io (subscription management).

New feature: automatic renewal with a 15% discount for customers who have been paying for more than a year.

Tone: friendly pro, not corporate, no exclamation marks and no "we're thrilled".

Length: 3-4 sentences.

<START_POST>
[post text]
<END_POST>

Prompt 2 (critique):

Original task: [copy prompt 1]

Draft:
[paste the result from prompt 1]

Analyze the draft using this structure:

1. ERRORS: what's missing, distorted, or unnecessary
2. FIXES: specific instructions on what to correct
3. CRITICAL CONTENT: key elements that must be kept

Output only the analysis, no new text.

Prompt 3 (revision):

Original task: [copy prompt 1]

Based on this critique:
[paste the result from prompt 2]

Write an improved version of the post.

<START_POST>
[improved text]
<END_POST>

Result:

In prompt 2 the model will show what it spotted: maybe the draft is too formal, doesn't highlight the benefit ("you pay less" matters more than "auto-renewal"), or leaves out that the feature is live starting today. In prompt 3 it produces a version with those issues fixed: the tone is sharper, the emphasis is in the right place, the details are kept.

An LLM generates text sequentially, token by token, moving forward. The model doesn't "see the whole text at once" the way a human editor does. It moves linearly and doesn't go back to check how the parts fit together. Hence missed details, tone drifting partway through the text, and mismatches between the beginning and the end.

When the model analyzes a finished text, it switches into a different mode. The whole draft is in context, so it can compare it with the original task, check coherence and notice what fell out. Critiquing is easier than generating: finding problems in a finished text is easier than producing a perfect text on the first try.

The structure of the critique (errors → fixes → critical content) gives the model clear rails for its analysis. Without structure, the model may output a vague "not bad, but could be better". With structure, it has to name specific errors, spell out specific fixes and flag what can't be lost. That turns fuzzy self-reflection into instructions for the revision.

Control levers:

• Criteria in the "Errors" block: add your own: "check the facts", "find corporate jargon", "assess the emotional tone". The more specific the criterion, the more precisely problems get found.
• Keyword masking: if the model copies the draft almost verbatim, replace specific words/phrases in the critique with <MASK>. This forces the model to apply the critique semantically instead of copying the text.
• Number of iterations: for simple tasks one pass (draft → revision) is enough. For complex ones you can do a second round: revision → critique of the revision → third version.

Prompt 1: Draft:

{task_description}

{quality_criteria}

Output the result in this format:
<START_OUTPUT>
[text]
<END_OUTPUT>

Prompt 2: Critique:

Original task: {task_description}

Draft:
{result_from_prompt_1}

Analyze the draft using this structure:

1. ERRORS: what's missing, distorted, unnecessary, or doesn't match the task
2. FIXES: specific instructions on what to correct and how
3. CRITICAL CONTENT: key elements that must be kept

Output only the structured analysis. Don't write a new version of the text.

Prompt 3: Revision:

Original task: {task_description}

Based on this critique:
{result_from_prompt_2}

Write an improved version.

<START_OUTPUT>
[improved text]
<END_OUTPUT>

What to fill in: - {task_description}: what needs to be done + context - {quality_criteria}: tone, length, required elements, constraints - {result_from_prompt_1} and {result_from_prompt_2}: copy the model's answers from the previous steps

Optional: - Add your own criteria to the "ERRORS" block (fact-checking, emotional tone, structure) - If the model copies the draft almost word for word, replace key words in the draft with <MASK> before prompt 2

⚠️ Three requests instead of one: The method needs three separate calls to the model. For quick tasks ("translate this phrase") it's overkill. It pays off where quality matters more than speed.

⚠️ Not for every task: If the task is simple and clear-cut ("make a list of 5 items"), the model will nail it on the first try. Reflection helps where there are subjective criteria (tone, style, balance of detail) or complex constraints (keep the facts + simplify the language + cut it in half).

⚠️ The model only sees what you specified: If you didn't spell out quality criteria in prompt 1, the model won't guess them during the critique. The structure helps, but the basic task description has to be complete.

The team tested the method on machine translation from English into African languages (isiZulu and isiXhosa). These are low-resource languages with few parallel texts for training, and models often get them wrong. They tested on GPT-3.5 and Claude Haiku 3.5.

The logic is simple: they took sentence pairs (English → target language) from the OPUS-100 and NTREX-African datasets. The model translates twice: first directly, then a second time after self-reflection (draft → critique → revision). They compared the quality of the first and second translations on two metrics: BLEU (word overlap with the reference) and COMET (semantic accuracy, which judges meaning, not just words).

The result: the second translation is consistently better than the first. And COMET (semantic accuracy) grew more than BLEU (exact word match), which means the method improves grasp of meaning rather than just nudging words toward the reference. Statistically the improvement is very strong: the p-value for BLEU = 1.45×10⁻⁴⁴, for COMET = 1.10×10⁻⁶⁵ (the probability that it's chance is practically zero). The effect size of ~0.95 by rank-biserial correlation means it's a practically meaningful improvement, not just a statistical artifact.

They tested three prompting strategies (zero-shot, chain-of-thought, few-shot): reflection works with all of them, but few-shot + reflection gives the most stable results. In-context examples help the model calibrate its critique better.

An interesting detail: they tested a confidence threshold. If you select for revision only the translations where the model is unsure, the average quality gain is higher (but coverage is lower). That confirms reflection works as targeted correction, not a blanket improvement of everything.

🔧 Technique: Drop the critique structure → free-form self-reflection

If the task is creative (storytelling, creative copy), a rigid structure (errors → fixes → content) can be limiting. Try free-form self-reflection:

Prompt 2 (free-form critique):

Read the draft as an editor. What works? What doesn't? What should be strengthened, what should be cut?

Draft:
{text}

Give an honest editorial assessment.

The model will give a less structured but more holistic analysis, and it may catch things that don't fit into the "errors/fixes" frame.

🔧 Technique: Keyword masking → forced rephrasing

The study used RAKE to automatically extract keywords and replace them with <MASK>. By hand you can do it more simply:

Prompt 2 (with masking):

Draft:
"Our new <MASK> lets you <MASK> save up to <MASK> a month"

[Critique]

Important: reconstruct each <MASK> from the meaning; don't copy from the draft.

This forces the model to apply the critique semantically instead of just tweaking a couple of words in the finished text. Useful if the model is too conservative in its revision.

🔧 Combo: Reflection + A/B variants

Prompt 3 (multiple revisions):

Critique:
{analysis_from_prompt_2}

Based on the critique, create TWO variants of the improved text:

Variant A: minimal edits, keep the draft's structure
Variant B: a radical rework, prioritizing the critique

<VARIANT_A>
[text]
</VARIANT_A>

<VARIANT_B>
[text]
</VARIANT_B>

You get two approaches to revision: a conservative one and a bold one. You can pick one or combine the best of both.

Reflective Translation: Improving Low-Resource Machine Translation via Structured Self-Reflection
Nicholas Cheng (Independent Researcher), 2026
Code and data: github.com/Nickcheng123/reflective-translation-mt

The study builds on: - Reflexion (Shinn et al., 2023): verbal reinforcement learning through reflection - Self-Refine (Madaan et al., 2023): iterative refinement through internal feedback - Chain-of-Verification (Creswell & Shanahan, 2023): improving factuality through verification

Datasets: OPUS-100 (Tiedemann, 2012), NTREX-African (Nekoto et al., 2023)
Metrics: BLEU (Papineni et al., 2002), COMET (Rei et al., 2020)