Reflect: Post-Generation Checks for Principle Compliance

Task: Write an Instagram post for your food delivery startup. You have tone-of-voice guidelines: (1) no hard selling, (2) use Gen Z humor, (3) don't promise what we can't guarantee, (4) mention competitors only neutrally.

Prompt (step 1):

Here are our brand communication principles:
1. Avoid hard selling and manipulation
2. Use Gen Z humor (memes, self-irony, pop culture references)
3. Don't promise what we can't guarantee (delivery "always in 15 minutes")
4. Mention competitors only neutrally, never aggressively

Write an Instagram post announcing that we now deliver to a new neighborhood (Wicker Park, Chicago).

Prompt (step 2, after you get the first draft):

Score your post on each principle from 1 (fully violates) to 5 (fully complies).
Give a score and a short justification for each principle.

Prompt (step 3, if any scores are < 3):

Low-scoring principles: Gen Z humor, realistic promises.

Critique the text against these principles. Then rewrite the post based on the critique.

Format:
CRITIQUE: [what's wrong]
REVISION: [new text]

Result:

The model will produce a critique (for example: "the humor is too formal; the phrase 'guaranteed in 30 minutes' reads like a promise") and a rewritten post with meme-style delivery and careful wording ("we usually get there in 30-40 minutes, but traffic is traffic 🤷").

The LLM's weakness: A model is bad at following several principles at once while generating. When you list 4-5 rules at the top of a prompt, it can drop one of them, especially if the principles are complex or conflict with each other (for example, "be brief" and "explain in depth"). While generating, the model only sees the beginning of the prompt and has to "keep in mind" every principle as it writes.

The LLM's strength: A model is excellent at analyzing finished text. Give it a text and ask "is principle X violated?" and it will reliably find the problem. This works thanks to causal attention: when checking, the model sees both the principle and the entire text in context at the same time. It can "scan" the finished text and match it against each principle separately.

How the method uses this: Reflect splits the process into two stages: generation and checking. First the model tries to follow the principles (as best it can), then it switches into analyst mode: it sees the whole finished text and pinpoints the violations. Critique and revision in one prompt give the model context: it understands exactly what was violated and can fix those spots in a targeted way, leaving everything else alone.

Control levers:

• Score threshold (default 3 out of 5): Drop it to 2 if you only want revisions for blatant violations, which saves tokens. Raise it to 4 if your standards are strict and you want to polish even small deviations.

• Number of cycles: One cycle is enough for 5-12 principles. You can add a second cycle for especially hard tasks (rerun steps 2 and 3 on the revision), but returns diminish.

• Self-evaluation structure: Instead of "score from 1 to 5" you can ask it to "list the violated principles". Easier for the model, but you lose the gradation of severity.

• Critique detail: Add "point to the specific phrases that violate" or "explain HOW this hurts the goal of the text", and the model will give you a more practical revision instead of an abstract one.

Step 1: Generate with the principles in mind

Here are the principles you must follow:
{list_of_principles}

{your_task}

Step 2: Self-evaluation

Score your answer on each principle from 1 (fully violates) to 5 (fully complies).

Format:
Principle 1: [score]/5 — [short justification]
Principle 2: [score]/5 — [short justification]
...

Step 3: Critique and revision (if any scores are below 3)

Low-scoring principles: {list_of_violated_principles}

Critique your answer against these principles. 
Then rewrite the answer based on the critique.

Format:
CRITIQUE: [what exactly was violated and why it's a problem]
REVISION: [improved answer]

What to fill in: - {list_of_principles}: the rules the text must follow (tone of voice, brand guidelines, editorial policy, compliance requirements) - {your_task}: what to write (a post, a reply to a customer, an article, an email) - {list_of_violated_principles}: principles that scored 1-2 in the previous step

🚀 Quick start: paste this into the chat:

Here is a Reflect template for principle compliance. Adapt it to my task: [describe the task and principles]. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask which principles matter and what text you need, so it can plug them into the scoring structure. It will take the "generate → self-evaluate → critique" pattern and adapt it to your task.

⚠️ Takes several requests: A full Reflect cycle is 2-3 calls to the model (generation, scoring, revision). For simple tasks it's overkill: if the text is already good, self-evaluation will show 4-5 points and no revision is needed. For mass content generation the overhead can be a dealbreaker.

⚠️ Depends on how the principles are worded: Vague principles ("be creative", "make it interesting") work worse than concrete ones ("use metaphors from nature", "start every paragraph with a question"). The model may interpret abstract rules in different ways.

⚠️ Doesn't replace fine-tuning for permanent requirements: If you generate content every day with the same principles, fine-tuning the model will be more efficient in tokens and time. Reflect is for flexibility and varied contexts.

⚠️ Works worse with contradictory principles: If the principles conflict ("be as brief as possible" + "explain every point in depth"), the model may lower the score on both without finding a balance. You need to either prioritize or split them into two cycles.

The researchers took three models (GPT-4-Mini, Claude-Haiku-3.5, Mistral-7B) and tested how Reflect helps them follow complex and unfamiliar principles, ones the models had NOT been fine-tuned on. For example: "avoid anthropocentrism" or "use historical parallels from non-Western cultures". The idea was simple: if a model learned to be "helpful and harmless" during fine-tuning, can it adapt to entirely different values at runtime?

They tested on 500 prompts from two red-teaming datasets (PKU-Safe-RLHF and Anthropic HH-RLHF), the kind of prompts typically used to probe model safety and surface violations. For each prompt they ran Reflect and scored compliance with each principle on a 1-5 scale using GPT-4 as a judge. They compared three setups: (1) a prompt with no principles (baseline), (2) a prompt with the principles up front (constitution-conditioned), (3) full Reflect (pre-conditioning + post-generation critique).

The surprise: Post-generation reasoning delivered a jump even for advanced models like GPT-4 and Claude. The effect was especially strong on rare but serious violations (tail cases). On average a model might follow a principle at 3.8/5, but 5-10% of answers violated it badly (1-2 points). Reflect cut the share of those violations by 2-3x, which is critical for safety and compliance.

Key insight: Models already know how to identify principle violations; they just don't use that ability when generating from scratch. Splitting "generate → check" unlocked what the model knew but wasn't applying. It didn't add new knowledge; it changed the process, giving the model a chance to "look at itself from the outside".

They also checked whether Reflect can generate data for fine-tuning. They took 2000 prompts, ran Reflect, saved the (base answer, improved answer) pairs, and fine-tuned Mistral-7B with DPO. The result: the fine-tuned model got better at following principles even WITHOUT Reflect at inference. In other words, an inference-time technique produced its own training data, no human labeling required.

Context: The researchers describe the prompt for the "Critique and Revision" step. Below is the structure from the paper (a simplified version; the full one with XML tags is in Appendix G.3).

You are tasked with improving a response to the user's query by ensuring it 
conforms to the following principles:

{Constitution}

Here is the user's query:
{User Query}

Here is the initial response:
{Base Response}

Here is the self-evaluation of the response:
{Self-Evaluation Scores}

The following principles have been flagged as potentially violated:
{Flagged Principles}

Please do the following:
1. CRITIQUE: Explain specifically how the response violates each flagged principle
2. REVISION: Rewrite the response to better conform to all principles

Format your response as:
CRITIQUE: [your critique]
REVISION: [your revised response]

💡 Adaptation for tone-of-voice control:

Instead of ethical principles, use Reflect to control your brand's style and voice.

Tone-of-voice principles for {brand_name}:
1. Casual and direct, but not overly familiar — like a friend, not some random guy at the bar
2. Short sentences (under 15 words) — people read on their phones
3. Avoid corporate jargon ("leverage", "in order to") and clichés ("we're pleased to announce")
4. One emoji per paragraph — for warmth, but don't overdo it

Write {content_type}: {task}

---

(step 2)
Score the text on each principle from 1 to 5. Point to specific phrases for low scores.

---

(step 3, if any are < 3)
CRITIQUE: [what was violated]
REVISION: [corrected text]

🔧 Technique: two rounds of checks for complex tasks

If you have more than 10 principles or they conflict, split them into two cycles:

Cycle 1: Check group A principles (for example, tone of voice and style) Cycle 2: Check group B principles (for example, factual accuracy and compliance)

The model handles 5-6 principles at a time better than 12 at once. After the first revision cycle, run the second one on the improved text.

💡 Adaptation for code review:

Use Reflect to check code against code style guidelines.

Code style principles for the project:
1. Variable names are nouns, function names are verbs
2. At most 3 levels of if/for nesting — anything deeper goes into a function
3. Each function has one responsibility (Single Responsibility Principle)
4. Comments explain why, not what
5. Avoid magic numbers — use named constants

Write a function for {task}

---

Score the code on each principle. Point to the lines of code that violate.

---

CRITIQUE: [what's wrong, line numbers]
REVISION: [corrected code]

🔧 Technique: add "show examples of violations" → sharper critique

In the self-evaluation step, demand specifics:

For each principle scored below 4:
- Score: [points]/5
- What was violated: [specific phrases or fragments]
- Why it's a problem: [how it hurts the goal of the text]

This gives the model hooks for the revision instead of abstract criticism like "the style isn't lively enough". The model sees "phrase X is formal" → replaces exactly that phrase without rewriting the whole text.

Reflect: Transparent Principle-Guided Reasoning for Constitutional Alignment at Scale
Henry Bell, Caroline Zhang, Mohammed Mobasserul Haque, Dhaval Potdar, Samia Zaman, Brandon Fain
Duke University, 2026

Related concepts: - Constitutional AI (Anthropic, 2022): the original critique-revision approach for reward modeling - Chain-of-Thought prompting: structured reasoning improves quality - Self-Refine (Madaan et al., 2023): iterative self-checking to improve output