Apply to my task
arXiv:2507.15015 ★ 79 Jul 20, 2025

EDU-Prompting: four roles in one prompt instead of a shallow answer

Key result

An LLM follows the path of least resistance. It grabs the first solution that comes along and confidently runs with it to the end. That's where cookie-cutter, one-sided answers to hard questions come from. EDU-Prompting lets you walk the model through a full critical analysis in a single prompt, no fine-tuning needed. The trick: split the task into four sequential roles: Brainstormer, Skeptic, Critic, Synthesizer. Each role takes the previous one's output as its starting point. Instead of one confident-but-shallow answer, four steps deeper into the problem.

EduThink4AI: Translating Educational Critical Thinking into Multi-AgentLLMSystems

Key Aspects of the Study:

The study shows that modern LLMs struggle with tasks that require critical thinking and are vulnerable to biased or false information. To fix this, the authors propose EDU-Prompting, a method that simulates the critical thinking process with four virtual "agents" (roles) that work in sequence: brainstorming, checking whether the question itself is valid, critique, and final synthesis.

Key Result: This structured approach, grounded in educational theory, significantly improves the truthfulness, logical coherence and objectivity of LLM answers.


The Core Idea of the Method:

The point of EDU-Prompting is not to just ask an LLM a hard question and wait for an answer, but to walk the model through the same structured critical analysis process that people use. Instead of building this in a technically complex way, an everyday user can simulate the process in a single prompt by making the LLM put on four different "hats" or "roles" one after another.

Here's what the method looks like for the user:

  1. Step 1: The "Brainstorming" role (Brainstorming Agent). You ask the LLM to throw out as many ideas, facts and viewpoints on your question as possible, without worrying about quality or structure. The goal is to collect "raw material."
  2. Step 2: The "Skeptic" role (Validity Agent). You ask the LLM to look at the original question and the generated ideas and ask: "Is the question itself valid? Is there a single correct answer? What hidden assumptions are there?" The goal is to check the foundation of the task.
  3. Step 3: The "Critic" role (Critique Agent). You instruct the LLM to analyze the ideas from step 1 in light of the comments from step 2. The LLM should find strengths and weaknesses, spot contradictions and formulate counterarguments. The goal is to weed out the junk and find the substance.
  4. Step 4: The "Synthesizer" role (Aggregation Agent). Finally, you ask the LLM to act as editor-in-chief and pull everything together: keep the best ideas and the critic's conclusions, resolve conflicts, and present a final, balanced, structured and objective answer.

This approach stops the LLM from taking the first path it stumbles on and makes it explore the problem from different angles, which radically improves the quality and reliability of the final result.


Practical Applicability Analysis:

  • Direct applicability: The user can apply this method directly by building a structure of four sequential role-steps into a single prompt. This turns a complex one-question request into a managed mini-project in which the LLM acts as a team of analysts under the user's direction. It's especially useful for writing essays, developing strategies and analyzing controversial topics.

  • Conceptual value: The study gives the user a key idea: the quality of an answer depends not only on the question but also on the process used to get it. Instead of being a passive recipient of information, the user becomes the "conductor" of the LLM's thinking process. It helps you see that an LLM doesn't have critical thinking by nature, but it can be steered effectively to imitate it.

  • Adaptation potential: The method is very flexible. For simpler tasks you can use a stripped-down version, for example: "Step 1: Sketch out options. Step 2: Pick the best one and justify it." For very important tasks you can add more roles, such as a "Risk Expert" or a "Devil's Advocate." The adaptation mechanism is simple: the user decides which "thinking roles" their specific task needs and describes them in the prompt.


Practical Example:

You are a team of four expert analysts working on my task in sequence. Your goal is to create a detailed, objective and actionable marketing plan for a new business.

**Context:**
I'm planning to open a small coffee shop in a residential neighborhood of a big city. Target audience: local residents, families with kids and freelancers. Key features: high-quality specialty coffee, a cozy interior with power outlets, a small kids' corner. The marketing budget is limited.

**Task:**
Develop a marketing strategy for the first 3 months after opening.

**Complete the task strictly step by step:**

### Step 1: "Brainstorming" Agent
Your job is to generate as many marketing ideas as possible without any criticism or evaluation. Just throw out everything that comes to mind: online and offline, standard and creative approaches.

### Step 2: "Skeptic" Agent
Your job is to evaluate the original request and the ideas from Step 1. Ask critical questions. For example: "Are freelancers and families with kids really compatible audiences?", "How realistic are these ideas given the limited budget?", "What hidden risks do the proposed ideas carry?".

### Step 3: "Critic" Agent
Your job is to take the ideas from Step 1 and the comments from Step 2. Analyze each idea for cost, reach, difficulty of execution and fit with the target audience. Highlight strengths and weaknesses.

### Step 4: "Synthesizer / Strategist" Agent
Your job is to use all the previous analysis (ideas, skepticism, critique) to put together a final, structured and realistic marketing plan. The plan must include:
1.  Key focus areas (the 3-4 most effective).
2.  Specific actions for each area, broken down by month.
3.  A rough budget estimate for the most important activities.
4.  Ways to measure effectiveness.

Present only the final result from Agent 4.

Why It Works:

This prompt works thanks to several practical mechanics described in the study:

  1. Decomposition: The complex task "create a marketing plan" is broken into four clear, sequential stages. This keeps the LLM from "cutting corners" and producing a shallow, cookie-cutter answer.
  2. Guided idea generation (Brainstorming): The first step gathers a wide pool of options without hemming the LLM in with premature criticism.
  3. Built-in sanity check (Skeptic): The second step makes the model find weak spots in the task setup itself and in the initial ideas, which prevents it from heading in the wrong direction.
  4. Structured self-critique (Critic): Instead of just saying "fix the mistakes," we give clear criteria for the analysis (cost, reach, difficulty). This is exactly the "non-raw" critique the study proves effective.
  5. Integration and synthesis (Synthesizer): The last step makes the LLM not just list good ideas but tie them into a single, logical, actionable strategy and throw out everything extra.

Another Practical Example

You are a team of travel-planning experts working in sequence. Your goal is to help me plan the perfect family vacation.

**Context:**
A family of 4: two adults, kids aged 7 and 14. Budget: mid-range, about $5,000 total. Timing: two weeks in August. Interests: the adults love history, nature and relaxing; the younger kid loves the beach and staying active; the older one loves modern entertainment and the internet.

**Task:**
Suggest 3 different vacation options that cover every family member's interests as fully as possible and stay within budget.

**Complete the task strictly step by step:**

### Step 1: "Idea Generator" Agent
Throw out as many destinations and vacation formats as possible that could work in theory (for example, Florida, Mexico, a road trip, a combo trip, etc.). Don't evaluate them, just create a list of possibilities.

### Step 2: "Risk and Constraints Analyst" Agent (Skeptic)
Evaluate the input data and the ideas from Step 1. What potential conflicts of interest are there (beach vs. museums)? How realistic is the budget for different destinations in August (peak season)? What logistical difficulties could come up?

### Step 3: "Critic-Optimizer" Agent
Take the ideas from Step 1 and the risks from Step 2. For each idea, weigh the pros and cons for every family member. Suggest how ideas could be combined or adapted to smooth out conflicts of interest and stay within budget.

### Step 4: "Final Planner" Agent (Synthesizer)
Based on all the analysis, present the 3 best, fully worked-out vacation options. Each option must include:
1.  Destination and type of vacation.
2.  A short day-by-day plan that accounts for everyone's interests.
3.  A rough budget breakdown (flights/travel, lodging, activities).
4.  An explanation of why this option is a good compromise for the family.

Present only the final result from Agent 4.

Why This Example Works.

This prompt works the same way as the previous one and rests on the same principles from the EDU-Prompting study:

  1. Conflict resolution: Planning a family vacation is a classic compromise problem. A simple "plan a vacation" request will push the LLM either to average out the interests (and please nobody) or to focus on just one person.
  2. Uncovering hidden problems (Risk Analyst): The "Skeptic" role here helps flag the main problem early: the conflict of interests and budget constraints in peak season. This steers the rest of the analysis in a practical direction.
  3. Targeted critique (Critic-Optimizer): Instead of a general evaluation, the critique is done from the point of view of specific people (family members). This pushes the LLM to look for solutions that "fit," not abstractly "good" ones.
  4. Building a complete product (Final Planner): The output isn't just a list of ideas but three "products" ready to consider: three worked-out options. That's a much more valuable and useful result for the user than scattered tips. The method makes the LLM travel from a chaos of ideas to a structured, balanced proposal.

Usefulness Score: 92

02

Main Evaluation Criteria

  • A. Relevance to prompting techniques: Very high. The study proposes EDU-Prompting, a complete methodology based on the interaction of several virtual agents (roles), which is an advanced prompting technique.
  • B. Better conversational answers: Very high. The main goal of the study is to increase the truthfulness, logical consistency and resistance to bias of LLM answers, which directly improves answer quality in chat.
  • C. Direct practical applicability: High. Although the study describes a "multi-agent system," an everyday user can easily simulate the process in a single chat by giving the LLM different roles in sequence (brainstormer, critic, synthesizer). No code or special tools required.
  • D. Conceptual value: Very high. The paper gives the user a powerful "mental model" for solving hard problems: don't just ask a question, organize a critical thinking process that includes generating ideas, checking them, critiquing them and synthesizing. It reveals the non-obvious fact that mindless critique ("raw critique") can make results worse.
  • E. New useful practice (clustering): The paper falls into several key clusters at once:
    • #1 (Phrasing techniques): Offers an advanced form of role-play and task decomposition.
    • #3 (Structure optimization): The entire methodology is about structuring a complex request into a sequence of steps.
    • #7 (Reliability and stability): Directly aimed at reducing hallucinations and increasing trustworthiness through built-in verification and critique.
  • Practicality checklist (+15 points): Yes, the paper provides ready-made constructions (agent roles), shows how to structure complex requests, reveals non-obvious LLM behavior (the harm of "raw" critique) and offers a way to improve answer accuracy.
03

Numeric Usefulness Score

Arguments for a score of 92: The study offers not just a "trick" but a full, universal methodology for solving complex and ambiguous problems. It's based on the familiar logic of real-world critical thinking (educational theory), which makes it intuitively clear. A user who masters this approach will get much higher-quality, deeper and more reliable answers to requests that call for analysis, comparison or strategy work. The conceptual value is huge because it teaches the user to "think together with the LLM" rather than just receive answers.

Counterarguments (why the score could be lower): * Hard for beginners: The method takes more effort than a simple one-line prompt. You have to write a multi-step instruction and keep an eye on how it's executed. That may scare off casual users. * Overkill for simple tasks: For simple requests like "capital of Australia" or "pancake recipe," this method isn't needed and would be overkill. Its strength only shows on complex, analytical tasks. * Terminology: The term "multi-agent system" can be misleading and give the impression that this is only for developers, when in fact the method is easy to simulate in an ordinary chat.


Research digest

Key result

An LLM follows the path of least resistance. It grabs the first solution that comes along and confidently runs with it to the end. That's where cookie-cutter, one-sided answers to hard questions come from. EDU-Prompting lets you walk the model through a full critical analysis in a single prompt, no fine-tuning needed. The trick: split the task into four sequential roles: Brainstormer, Skeptic, Critic, Synthesizer. Each role takes the previous one's output as its starting point. Instead of one confident-but-shallow answer, four steps deeper into the problem.

Principle

The Brainstormer collects raw ideas with no limits: the more, the better. The Skeptic takes those ideas and checks the question itself: is the task framed correctly, are there hidden assumptions, is there even a single right answer? Here's the twist: the Skeptic doesn't rate the options, it challenges the task itself. That breaks the "accept the question as given and answer it" pattern. The Critic picks apart the ideas from step 1 using the comments from step 2: realism, cost, risks, contradictions. The Synthesizer assembles the result, using only what made it through the three previous filters.

Why it works

An LLM without structure is like a student in a tough exam. Asked a question, it writes the first thing that comes to mind just to avoid silence. Hence the confident but empty answers. Four roles create internal tension: the Brainstormer generates chaos, the Skeptic casts doubt on it, the Critic takes it apart piece by piece. Each agent disagrees with the one before it, and that's what produces real critical analysis instead of the first idea restated in different words. The model can't cut corners: the structure won't let it.

When to apply

Strategic analysis → when you need a balanced conclusion on a contested topic, especially if the question itself may be badly framed. Complex decisions → a business plan, a marketing strategy, a case breakdown with competing interests. Writing well-argued texts → essays, analytical reviews where objectivity matters. NOT a fit for simple factual questions, where four roles add noise and no value.

Mini recipe

1. Give context: describe the task, audience and constraints. The more specific, the better the roles will work. 2. Role 1, Brainstormer: ask it to throw out all ideas without judging them. The goal is breadth, not depth. 3. Role 2, Skeptic: ask it to check the question itself: is it valid, what's missing, what hidden assumptions are there. Not the ideas, the framing of the task. 4. Role 3, Critic: ask it to analyze the ideas against the criteria that matter to you: cost, realism, risks, audience fit. 5. Role 4, Synthesizer: ask it to assemble the final answer based on everything before. Have it drop the weak parts and tie the strong ones into a single structure. 6. At the end, add: `Show only the last agent's result.` Otherwise you'll get a wall of text from all four steps.

Examples

[BAD]: `Write a marketing strategy for a new coffee shop` [GOOD]: `You are a team of 4 experts working in sequence. Task: a 3-month marketing strategy for a coffee shop in a residential neighborhood, limited budget, audience: freelancers and families with kids. Step 1 (Brainstormer): throw out every possible marketing idea without evaluating it. Step 2 (Skeptic): check the task framing: are freelancers and families compatible as an audience? Is the budget realistic? What's missing? Step 3 (Critic): analyze the ideas from step 1 using the comments from step 2, by these criteria: cost, reach, difficulty of launch. Step 4 (Synthesizer): put together a final plan with specific actions by month and a rough budget estimate. Show only the result of step 4.`

In plain words

Here's the deal: modern models are well-read know-it-alls who actually can't think. They churn out answers on autopilot, just predicting the next word, and often trip over their own logic. The EduThink4AI study suggests we stop waiting for one model to work miracles and turn the process into teamwork. Instead of asking "solve the problem," we build a system of several agents, each responsible for its own stage of critical thinking: one digs up facts, another builds arguments, a third pokes holes in them.

It's like writing your thesis not alone but with a nitpicky professor, a meticulous editor and a skeptical opponent. One throws out ideas, the second stress-tests them, and the third makes you redo everything that sounds like a hallucination. On its own, a model often talks nonsense with a straight face, but with a "supervisor" standing over it, the nonsense level drops sharply. It's the jump from a plain chat to a structured assembly line of thought, where the result isn't random text but a well-checked solution.

Under the hood are specific methods: task decomposition, argumentation and self-reflection. First the system breaks a hard question into small pieces, then gathers evidence for each one, and finally runs everything through the filter of "did I just write garbage?". The researchers built in multi-agent systems (MAS), where every bot has its own role and clear evaluation criteria. That lifts answer quality to a level even the most powerful models can't reach with a plain head-on request.

It was tested on educational tasks, but the principle is universal. This setup is a perfect fit for business analytics, programming or legal review: anywhere mistakes are costly and logic matters more than creativity. You can build an assembly line like this for market analysis or writing code, and it will run far more reliably than a plain "smart chatbot." Critical thinking is now an algorithm you can scale to any field where you need results, not just chatter.

Bottom line: the era of solo prompts is ending, and the time of engineering thinking is here. If you want the AI to stop fumbling, make it follow the rules of classical logic through a system of agents. EduThink4AI proved that structure beats scale: a well-configured group of mid-range models delivers better results than a single top model left unsupervised. Either you build a system of checks, or you keep getting nicely packaged misinformation.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on EDU-Prompting: four roles in one prompt instead of a shallow answer arXiv:2507.15015

Sign in to apply this study New accounts get 50 free credits.

Get the week’s best papers by email

One email on Sundays with the highest-rated new techniques. Unsubscribe with one click.