DiMo: Multi-Agent Debate Through Switching Thinking Modes

Divergent Thinking Mode (for knowledge and common-sense tasks)

STEP 1: Generator produces a first answer → answer text
STEP 2: Evaluator finds logical holes and knowledge gaps → list of problems
STEP 3: Knowledge Supporter offers relevant knowledge → facts and context
STEP 4: Reasoning Path Provider builds a reasoning path → structured logic
STEP 5: Generator produces a refined answer based on the knowledge and the path → new answer
STEP 6: Discussion: all three agents (Evaluator, Knowledge Supporter, Path Provider)
        debate: is the answer correct? → consensus (yes/no)
        If no → back to step 3 (next debate round)

Every step requires a separate request. Each role is its own prompt.

Logical Thinking Mode (for math and logic problems)

STEP 1: Generator produces an answer with a step-by-step solution → solution with steps
STEP 2: Evaluator checks every step → finds a specific error (step #N) or OK
        ├─ If an error is found → STEP 3 (Refiner)
        └─ If there are no errors → STEP 4 (Judger)
STEP 3: Refiner fixes only the problem step → updated solution
        → back to step 2 (re-check)
STEP 4: Judger makes the final call → answer accepted/rejected

The Evaluator → Refiner loop is capped at a set number of iterations (usually 3-4 rounds).

⚠️ Method limitations:

• Heavy token usage: 13,700-19,300 tokens per task vs 100-500 for standard methods
• Requires sequential requests: 5-8 separate prompts instead of one
• Divergent mode does worse on math: 79.9% accuracy vs 90.7% for Logical mode (GSM8K)
• Not for simple questions: overkill for "capital of France"

The method shines on complex tasks where a single model misses errors in its own reasoning.

Divergent Mode: evaluating a business idea

The task: Evaluate an idea: opening a coffee shop + coworking space in a residential Chicago neighborhood. There are two competitors nearby (a Starbucks and a local coffee shop), and the target audience is remote workers and freelancers. Starting budget: $150K.

Prompt 1 (Generator):

Analyze this business idea: a coffee shop + coworking space in a residential Chicago neighborhood,
a Starbucks and a local coffee shop nearby, target audience: remote workers, budget $150K.

Is it worth opening? Give an initial assessment.

Prompt 2 (Evaluator):

Here is an initial assessment of a business idea: [paste the Generator's answer]

Find in this assessment:
1. Logical holes
2. Overlooked risks
3. Missing data needed for the conclusions
4. Weak spots in the argument

Make a list of problems.

Prompt 3 (Knowledge Supporter):

Here is the assessment of the idea and the list of problems:
[paste the Generator's answer and the Evaluator's list]

Provide relevant knowledge:
- Coffee shop market statistics for Chicago
- Rent and operating cost data for residential neighborhoods
- Remote worker behavior (where they work, how much they spend)
- Profile of successful coworking spaces

Facts and figures, no conclusions.

Prompt 4 (Reasoning Path Provider):

Here is the initial assessment, the problems and the knowledge:
[paste everything above]

Build a structured reasoning path:
1. Which questions to ask, and in what order?
2. Which data to verify?
3. Which scenarios to consider?
4. What conclusions should this lead to?

Give a logical framework for the analysis, not the final answer.

Prompt 5 (Generator, refined answer):

Now re-evaluate the idea based on:
- Knowledge: [paste from the Knowledge Supporter]
- Logic: [paste from the Reasoning Path Provider]

Give a final assessment: open it or not? Justify your answer.

Prompt 6 (Discussion, consensus):

Here is the refined assessment: [paste the Generator's answer]

Three roles must vote:

EVALUATOR: Are there logical errors in this assessment? (yes/no + why)
KNOWLEDGE SUPPORTER: Do the conclusions match known facts about the market? (yes/no + why)
REASONING PATH PROVIDER: Is the reasoning logic sound? (yes/no + why)

If all say "yes" → CONSENSUS, final answer.
If even one says "no" → state what to fix and start a new round.

Result: You'll see 3-4 rounds of debate, each bringing new knowledge, corrected logic and refined conclusions. At the end you get a well-grounded decision with an explicit reasoning path: which factors were considered, which risks were found, which scenarios were run. Not just "worth it / not worth it", but an audit of the entire decision logic.

Logical Mode: checking a unit economics calculation

The task: Check the customer LTV (lifetime value) calculation for a subscription service. The model calculated LTV = $450, but there may be an error in the math.

Prompt 1 (Generator):

Calculate the LTV for a subscription:
- Average revenue: $15/month
- Average customer lifetime: 30 months
- Churn rate: 5% per month

Show the calculation step by step.

Prompt 2 (Evaluator):

Here is the LTV calculation: [paste the Generator's answer]

Check every step:
1. Is the formula used correctly?
2. Are the computations correct?
3. Is the data consistent (5% churn rate and a 30-month lifetime)?
4. Are all factors accounted for?

Name the specific step with the error (if any) or say "OK".

Prompt 3 (Refiner, if the Evaluator found an error):

The Evaluator found an error: [state which one]

Here is the original calculation: [paste from the Generator]

Fix ONLY the problem step and keep everything else. Show the corrected calculation.

Prompt 4 (Evaluator, re-check):

Here is the corrected calculation: [paste from the Refiner]

Check again: are there any errors? (yes/no + the specific step)

Prompt 5 (Judger, final decision):

Here is the final calculation after all the edits: [paste the latest version]

Assess:
1. Are all the steps logically correct?
2. Are the computations accurate?
3. Is the formula applied correctly?
4. Does the answer make sense?

Decision: ACCEPT the calculation / REJECT it (and why).

Result: You'll see step-by-step verification that explicitly points to where the error was (for example: "Step 2: at a 5%/month churn rate, the average customer lifetime = 1/0.05 = 20 months, not 30"). The model will fix that step locally, recompute the dependent values and show the correct LTV. The end result is an auditable chain of checks, not just a number.

The LLM's weakness: The model is bad at finding errors in its own output. Once it has generated a solution, it tends to defend it: that's anchoring. It's especially critical in math: an error in step 2 carries through the whole calculation, but the model doesn't notice because it's busy building the next steps.

The LLM's strength: The model is great at critiquing someone else's text and surgically fixing specific fragments when it sees them in isolation. It's also good at following clear role instructions: if the prompt says "you only check, you don't generate", the model focuses on checking.

How the method uses this: DiMo separates the roles physically, through separate prompts. The Evaluator only looks for errors (it can't generate), the Refiner only fixes the flagged step (it can't rewrite everything). This removes anchoring: each role starts from a clean slate, with no previous conclusions to defend. In math, the Evaluator sees the solution as a list of claims to verify, not as its own output. The Refiner gets "error in line 5" and recalculates surgically, leaving the rest untouched.

Prompt control knobs:

1. Number of debate rounds (for Divergent mode):
   • Default: 3-4 rounds
   • ↓ Cut to 1-2 for simple tasks → save tokens
   • ↑ Raise to 5-6 for mission-critical decisions → dig deeper
2. Consensus criterion (in Discussion):
   • Default: "all three agents agree"
   • Switch to "two out of three agree" → faster, less strict
   • Or "the Evaluator + any one other" → logic gets priority
3. Generation temperature:
   • Divergent mode uses a high temperature (0.7-0.9) → diverse hypotheses
   • Logical mode uses a low one (0.1-0.3) → rigorous calculations
   • You can flip them for experiments
4. Depth of checking (Evaluator):
   • Default: "check every step"
   • Relax to "check only the critical calculations" → faster
   • Tighten to "check + propose an alternative formula" → paranoid mode
5. Agent roles (for Divergent mode):
   • Instead of a faceless "Knowledge Supporter", give it a concrete role: "You are a financial analyst at Chase with 10 years of experience" → sharper critique
   • For business tasks: "Optimist" vs "Pessimist" instead of abstract roles

Divergent Mode (knowledge and common sense)

Step 1: Generator:

Task: {describe the task}

Give an initial answer.

Step 2: Evaluator:

Here is an initial answer to the task: {task}

Answer: {paste the Generator's answer}

Find:
1. Logical holes
2. Missing facts
3. Weak spots in the argument

List of problems.

Step 3: Knowledge Supporter:

Task: {task}
Initial answer: {Generator's answer}
Problems: {list from the Evaluator}

Provide relevant knowledge and facts for this task. No conclusions, just data.

Step 4: Reasoning Path Provider:

Task: {task}
Initial answer: {Generator's answer}
Problems: {list from the Evaluator}
Knowledge: {facts from the Knowledge Supporter}

Build a structured reasoning path:
- In what order should things be analyzed?
- Which data should be verified?
- Which scenarios should be considered?

Give a framework for the analysis, not the final answer.

Step 5: Generator (refinement):

Task: {task}

Based on:
- Knowledge: {paste from the Knowledge Supporter}
- Logic: {paste from the Reasoning Path Provider}

Give the final answer.

Step 6: Discussion (consensus):

Task: {task}
Refined answer: {paste from the Generator}

Three roles vote:

EVALUATOR: Are there logical errors? (yes/no + why)
KNOWLEDGE SUPPORTER: Does it match the facts? (yes/no + why)
REASONING PATH PROVIDER: Is the logic sound? (yes/no + why)

If all say "yes" → CONSENSUS.
If even one says "no" → what needs fixing? New round.

Repeat steps 3-6 until consensus or {max_rounds} rounds.

Logical Mode (math and logic)

Step 1: Generator:

Task: {describe the task}

Solve step by step:
1. Break the task into stages
2. Carry out each step with the computations
3. Show the intermediate results
4. Give the final answer

Number every step.

Step 2: Evaluator:

Here is the solution: {paste from the Generator}

Check every step:
1. Is the formula applied correctly?
2. Are the computations correct?
3. Does each step follow logically to the next?

State:
- The number of the step with the error (if any)
- What the error is
- Or "OK" if everything is correct

Step 3: Refiner (if the Evaluator found an error):

The Evaluator found an error: {state which one}

Original solution: {paste from the Generator}

Fix ONLY the problem step {step_number} and keep everything else.
Show the corrected solution.

Repeat steps 2-3 up to {max_iterations} times, until the Evaluator says "OK".

Step 4: Judger (final check):

Final solution after the edits: {paste the latest version}

Assess:
1. Are all the steps logically correct?
2. Are the computations accurate?
3. Does the answer make sense in the context of the task?

Decision: ACCEPT / REJECT (justify it).

Placeholders:

• {task}: the wording of your task
• {max_rounds}: usually 3-4 for Divergent mode
• {max_iterations}: usually 3-4 for Logical mode
• {step_number}: the specific step the Evaluator flagged as the problem

🚀 Quick start: paste into the chat:

Here's the DiMo template for my task. Adapt it and ask me clarifying questions.

My task: [describe the task]

Use this mode:
- Divergent Mode if the task is about analyzing knowledge/ideas/decisions
- Logical Mode if the task is about calculations/verification/math

[paste the matching template above]

The LLM will ask about the task type (knowledge or logic), how many rounds, how strict the checks should be, and which roles to use in the debate. It'll take the structure from the template and fill it in for your task. You'll get ready-made prompts for each role: copy them and run them in sequence.

Context: The researchers tested both modes on benchmarks. Here are examples of their agent prompts.

Generator (math):

You are a mathematical problem solver focused on understanding and solving problems
systematically. Follow these steps:

1. Problem Understanding:
   a. Extract the core question (what exactly needs to be calculated)
   b. Identify all explicit and implicit conditions
   c. List all given values with their units
   d. Describe key relationships between values

2. Solution Planning:
   a. Break down the problem into logical steps
   b. Identify the sequence of calculations needed
   c. Note any special conditions or edge cases
   d. Plan how to validate intermediate results

3. Detailed Solution:
   a. Write clear, step-by-step calculations
   b. Include units in every calculation
   c. Show all work explicitly
   d. Validate each intermediate result

4. Answer Validation:
   a. Verify the final calculation
   b. Check units in the final answer
   c. Confirm the answer makes logical sense
   d. Cross-check against the original question

Evaluator (commonsense):

You are an answer evaluator for multiple-choice commonsense reasoning questions.
Analyze the given answer for:

1. Logical reasoning
2. Consideration of all options
3. Understanding of common sense principles
4. Clear justification for the chosen answer

Please point out any errors, omissions, or areas that need improvement.
Focus on whether the reasoning is sound and leads to a logical conclusion.

Knowledge Supporter (commonsense):

You are a knowledge supporter for multiple-choice commonsense reasoning questions.
Your role is to:

1. Identify relevant commonsense knowledge needed for the question
2. Provide real-world examples and context
3. Highlight any missing or incorrect information
4. Support or challenge the reasoning with factual information

Focus on practical, everyday knowledge that relates to the question and choices.

Reasoning Path Provider (commonsense):

You are a reasoning path provider for multiple-choice commonsense reasoning questions.
Create clear logical paths that:

1. Break down the question into key components
2. Analyze each option systematically
3. Show clear steps leading to the answer
4. Explain why incorrect options are eliminated
5. Connect evidence to conclusions

Ensure each step follows logically from the previous one and leads to a clear choice.

⚠️ Heavy token usage: 13,700-19,300 tokens per task vs 100-500 for baseline methods. On paid APIs (GPT-4, Claude), that can get expensive. With open models (LLaMA, Qwen) via API or locally, it's acceptable for important tasks.

⚠️ Requires sequential requests: 5-8 separate prompts instead of one. This is a workflow, not "one prompt → one answer". To automate it, you need a script or manual copy-pasting between chats.

⚠️ Not for simple questions: Overkill for "capital of France" or "2+2". Use it for tasks where a single model misses errors in its own reasoning.

⚠️ The modes aren't interchangeable: Divergent mode on math gets 79.9% vs 90.7% for Logical mode (GSM8K). Choosing the mode is critical: knowledge/ideas → Divergent, calculations/logic → Logical.

⚠️ Latency: Several sequential requests = wait time multiplies. Not suitable for real-time uses (a chatbot). For asynchronous tasks (analysis, verification), it's fine.

The team took two backbones, LLaMA-3-8B (8 billion parameters) and Qwen-2.5-32B (32 billion), and ran them through six benchmarks: four on commonsense (CSQA, ARC-Challenge, OpenBookQA, StrategyQA) and two on math (GSM8K, GSM-hard). They compared against baselines: a plain LLM, Chain-of-Thought, QwQ-32B (an o1-like model), and two recent multi-agent methods (LLM MAD, MAD).

What they measured: Exact Match, a strict answer match (for multiple choice they normalized the letters; for numbers they stripped spaces/units). The key design choice: they fixed the token budget, so all baselines and DiMo got the same number of tokens per task to keep the comparison fair. DiMo simply spreads those tokens across agents, while the baselines spend them on one long output.

The results were surprising on two fronts. First, DiMo beat even QwQ-32B (a model trained with reinforcement learning + long CoT) on math: 98.4% vs 75.4% on GSM8K with Qwen-2.5-32B. That's odd, because QwQ was specifically trained on long reasoning via RL. The hypothesis: an explicit split of roles (evaluate → refine) works better than an implicit "think longer". Second, on GSM-hard (a harder version of GSM8K with huge numbers) the gap is even bigger: 84.1% vs 51.4%. That suggests DiMo doesn't break down as difficulty scales up.

A curious detail: on commonsense tasks the MAD baseline (multi-agent debate from Liang et al.) flopped: 46% on CSQA vs 80% for DiMo. The breakdown showed that MAD used identical roles for all agents, while DiMo uses specialized ones (Knowledge Supporter ≠ Reasoning Path Provider). Specialization is critical: when every agent does the same thing, the debate degenerates into an echo chamber.

Practical insight: The study reveals protocol–task affinity: not all debate methods are equally useful. Commonsense tasks respond better to the Divergent protocol (parallel hypotheses + knowledge), math tasks to the Logical protocol (step-by-step verification). This isn't an intrinsic property of the model but a prompting pattern: different prompt structures yield different results on different tasks. Takeaway: in production you need a router that automatically detects the task type and picks the mode. Or let the user choose.

Another finding: the number of debate rounds saturates at 3-4 rounds. After that, accuracy barely improves while tokens get burned linearly. The chart shows that on CSQA, the gain from 1 to 2 rounds is large, from 2 to 3 moderate, and from 3 to 4 almost zero. Practical takeaway: 3 rounds is the sweet spot for most tasks.

1. A simplified version for a single chat

💡 Technique: all roles in one prompt → saves time

Instead of 5-8 separate requests, ask the model to simulate the debate within a single output:

Solve the task through an internal debate between four roles:

GENERATOR: produce a first answer
EVALUATOR: find the errors in it
REFINER: fix the errors
JUDGER: make the final call

Show the whole dialogue, then the final answer.

Task: {your task}

Effect: Faster (one request), but less rigorous: the model may phone in the critique, because it generates and checks within one context (the anchoring remains).

2. A hybrid mode for complex business cases

💡 Adaptation: Divergent + Logical in sequence → for multi-layered tasks

If a task requires both knowledge and calculations (e.g., "should we launch a new product?"):

Step 1: Divergent mode → gather knowledge about the market, competitors, audience Step 2: Logical mode → calculate the unit economics, payback and risks based on the gathered knowledge

Task: Evaluate launching a B2B SaaS subscription for HR departments in the US.

PHASE 1 (Divergent):
1. Generator: initial assessment of the idea
2. Evaluator: what knowledge is needed?
3. Knowledge Supporter: data on the HR SaaS market, competitors, pricing
4. Reasoning Path Provider: an analysis framework

PHASE 2 (Logical):
1. Generator: unit economics calculation based on the knowledge from Phase 1
2. Evaluator: check every step of the calculation
3. Refiner: fix the errors
4. Judger: final decision (launch / don't launch)

Effect: The deepest possible analysis, but lots of tokens (20,000+). For critical decisions with an output budget to match.

3. A "red team" to stress-test an argument

💡 Adaptation for texts/pitches: roles → Optimist vs Pessimist

Instead of abstract roles, give it characters with opposing views:

Stress-test this investor pitch through a debate:

OPTIMIST (Steve Jobs style):
- Focus on the potential, the technology, the global market
- Looks for "why this will take off"

PESSIMIST (a seasoned VC with 20 failures behind them):
- Focus on the risks, the competition, how real the numbers are
- Looks for "why this will fail"

Let them debate for 2 rounds, then give a consensus assessment.

Pitch: [paste the pitch text]

Effect: Sharper critique, because the roles are emotionally charged. The model gets into a "Steve Jobs" character more easily than into an abstract "Evaluator".

4. Automating with a script

🔧 Technique: a Python script to run the prompts → no more copy-pasting

If you use an API (OpenAI, Anthropic, local models), you can write a simple script:

def dimo_logical(task, max_iterations=3):
    # Step 1: Generator
    answer = llm.generate(f"Task: {task}\n\nSolve it step by step.")

    for i in range(max_iterations):
        # Step 2: Evaluator
        eval_result = llm.generate(f"Check the solution: {answer}\n\nList the errors or say 'OK'.")

        if "OK" in eval_result:
            break  # No errors, exit

        # Step 3: Refiner
        answer = llm.generate(f"Fix the error: {eval_result}\n\nSolution: {answer}")

    # Step 4: Judger
    final = llm.generate(f"Final check: {answer}\n\nACCEPT/REJECT?")
    return final

# Usage:
result = dimo_logical("Calculate the NPV of a project with a $5M investment, $2M/year cash flow, a 10% discount rate and a 5-year term")
print(result)

Effect: Full automation → run the script, get the result. But you need Python + an API.

"Unleashing Diverse Thinking Modes in LLMs through Multi-Agent Collaboration" arXiv:2501.XXXXX (approximate link; see the original for the exact one)

Authors: Zhixuan He, Yue Feng (University of Birmingham, United Kingdom)

References from the study:

• Chain-of-Thought Prompting (Wei et al., 2022): the foundational step-by-step reasoning technique
• Self-Refine (Madaan et al., 2024): iterative improvement via self-feedback
• Reflexion (Shinn et al., 2024): verbal reinforcement via episodic memory
• MAD: Multi-Agent Debate (Liang et al., 2023): debate with divergent thinking
• LLM MAD (Du et al., 2023): improving factuality through debate