ProSEA: Iterative Problem Solving Through Rich Feedback and Adaptive Re-planning

PHASE 1: ANALYSIS AND PLANNING
└─ Manager → Problem Analyzer: analyze constraints, requirements, assumptions
└─ Problem Analyzer → Planner: structured representation of the problem
└─ Planner → Manager: a plan of N steps (task + goal + success criterion)

PHASE 2: ITERATIVE EXECUTION (loop for each step)
└─ Manager → Expert Agent: step i (task, goal, success criteria)
└─ Expert Agent: goal-directed exploration
   ├─ Reasoning + tool usage + (optional) human collaboration
   ├─ Check whether the goal is reached
   └─ Rich feedback → Manager:
      • Result (success/failure)
      • Why (constraints, learnings, causes of failure)
      • What was discovered (new insights, patterns)
      • Which alternatives were tried

PHASE 3: ADAPTATION (after each step)
└─ Manager analyzes the feedback:
   • Goal reached → next step
   • Failure + useful feedback → Planner: re-plan the remaining steps
   • Problem unsolvable → stop with an explanation

PHASE 4: SYNTHESIS
└─ After all steps: Manager synthesizes the final answer

Important: This isn't a one-shot prompt — it's a multi-turn workflow with a feedback/re-planning loop.

Task: You're preparing an investment memo on Uber for a private investor. You need to analyze the 2023 annual report, find the key risks and compare profitability with Lyft. The report runs 200 pages, the data is scattered, and accuracy matters.

Prompt (Step 1 - Planning):

Task: Analyze Uber's 2023 annual report for an investment memo.

Goal: Find the key risks + compare profitability with Lyft.

Create a plan of 4-6 steps. For each step specify:
- Task: what we're doing
- Goal: the concrete result
- Success criteria: how we'll know we got there

Prompt (Step 2 - Executing the first step):

[Plan from the previous step]

Execute Step 1: [task from the plan]

When done, report back:
1. Result: what you got
2. If it didn't work:
   - Where did you look?
   - What did you find that was close?
   - Which alternative terms did you check?
   - Which constraints did you discover?
3. What did you learn about the document's structure?
4. Do the next steps of the plan need adjusting? Why?

Prompt (Step 3 - Adapting the plan):

[Feedback from Step 1]

Analyze the feedback. Does the plan need to change?

If yes:
- Which steps are we adjusting?
- Why? (based on what in the feedback)
- The new version of the plan

If no:
- Why is the current plan still valid?
- Move on to Step 2

Result:

The model goes through an iterative loop: executes a step → gives detailed feedback on what it found, where it looked, which obstacles it hit → assesses whether the plan needs to change → either continues per plan or adapts the remaining steps. At the end it synthesizes all findings into a coherent memo. Along the way the model won't get stuck on "data not found" — it will explain where it looked and propose an alternative route (for example: "revenue isn't in the Income Statement, but it's in Note 15 on segments").

LLMs handle uncertainty poorly. When they fail, they either hallucinate (make up data) or get stuck repeating the same approach. Binary feedback ("didn't work") gives no information about HOW to adapt — the model doesn't know what exactly is wrong or what to try next.

LLMs are great at explicit reasoning. When you tell them "explain why it didn't work, what you tried, what you learned", they structure their thinking, document the exploration, and spot patterns. This switches on metacognition — the model starts reflecting on its own problem-solving process.

ProSEA turns failures into navigation. Rich feedback builds a map of the explored space: what does NOT work and why. The Manager uses that map for informed decisions: it doesn't force a dead-end path but looks for a way around. Goal-directed exploration gives a clear criterion for when to stop and try something else — not random wandering but a purposeful search.

Control levers:

• Feedback detail → raise the bar on "why it didn't work" for deeper analysis of dead ends
• Re-planning frequency → after every step vs only on failure — a balance between flexibility and stability
• Success criteria → strict vs lenient — affects when a step counts as done
• Breadth vs depth → the Manager can generate alternative plans in parallel (breadth), or the Expert can reason more deeply within a step (depth)

Multi-turn workflow for complex tasks:

=== STEP 1: ANALYSIS AND PLANNING ===

Task: {task_description}

Analyze it and create a plan:
1. What are the constraints and requirements?
2. What are the implicit assumptions?
3. A plan of {N} steps. Each step:
   - Task: what we're doing
   - Goal: the concrete result
   - Success criteria: how we'll know we got there

=== STEP 2: EXECUTION (repeat for each step) ===

[Plan from Step 1]

Execute Step {i}: {task}

Goal: {goal}
Success criteria: {criteria}

When done, report back:

RESULT:
□ Goal reached: [description of the result]
□ Goal not reached: [why]

FEEDBACK:
1. What you tried:
   - Where you looked / which approaches
   - What you found that was close (if you didn't find the exact thing)
   - Which alternatives you checked

2. What you learned:
   - Which constraints you discovered
   - Which patterns you noticed
   - What became clear about the structure of the problem

3. Insights for the plan:
   - Do the next steps need adjusting?
   - What changed in your understanding of the task?

=== STEP 3: ADAPTATION ===

[Feedback from Step 2]

Analyze the feedback:
- If the goal is reached → move on to the next step
- If it failed → is a re-plan needed?
  • Which steps are we adjusting based on the feedback?
  • The new version of the plan
- If the problem is unsolvable → explain why based on the accumulated feedback

=== STEP 4: SYNTHESIS (after all steps) ===

[Results of all steps]

Synthesize the final answer based on:
- The successful steps
- Learnings from the failures
- The constraints you discovered

What to fill in:

• {task_description} — a full description of what needs to be done
• {N} — the number of steps (usually 3-6 for complex tasks)
• {i} — the current step number
• {task}, {goal}, {criteria} — from the Step 1 plan

Key detail: This is a workflow of several requests to the LLM, not a one-shot prompt. A human (or a system) plays the Manager role; the LLM plays the Expert that reports feedback.

🚀 Quick start — paste into the chat:

Here's a workflow for solving complex tasks through iterative exploration with rich feedback.
Adapt it to my task: [your task]

First ask clarifying questions to understand:
- Constraints and requirements
- Available resources/data
- Success criteria

Then create a plan and run the loop: execution → rich feedback → adaptation.

[paste the template above]

The LLM will ask about the task, constraints and resources — because it needs that structure to build a plan. It will take the workflow pattern (analysis → planning → execution with feedback → re-planning) and adapt it to your specifics.

⚠️ Overhead on simple tasks: If a task can be solved in one step without exploration, rich feedback and re-planning are overkill. They add tokens and latency for no benefit. ProSEA is for complex multi-step tasks where dead ends are likely.

⚠️ Requires multi-turn interaction: You can't do it one-shot. You have to run a dialogue: get the plan → execute a step → analyze the feedback → re-plan (if needed) → next step. Automating it requires an orchestration layer.

⚠️ Feedback quality depends on prompting: LLMs don't always volunteer rich feedback. You have to ask explicitly: "explain why it didn't work, what you tried, what you learned". Without a structured request, the feedback will be shallow.

⚠️ Tokens: Detailed feedback + exploration history pile up tokens. For very long tasks you may need to summarize intermediate results or keep context selectively.

The researchers tested ProSEA on FinanceBench — a dataset of 150 questions that require analyzing real financial documents (companies' 10-K, 10-Q, 8-K filings). Tasks are split by difficulty: from simple information retrieval (0-RETRIEVE) to complex multi-step calculations with judgment (4-CALC-AND-JUDGE) and explaining factors (5-EXPLAIN-FACTORS). This isn't a synthetic benchmark — these are real documents hundreds of pages long, where data is scattered, terminology varies, and calculations have to be precise.

They compared against state-of-the-art baselines: LlamaIndex RAG agents, LangChain ReAct agents, OpenAI Assistants, and DANA (domain-aware neuro-symbolic agents). Critically: ProSEA was tested in fully autonomous mode (no human feedback) to make the comparison with automated systems fair.

The size of the gap was striking. ProSEA hit 93.2% accuracy, beating LlamaIndex (56.7%), LangChain ReAct (81.6%), and OpenAI Assistants (42.7%). ProSEA was especially strong in categories that require iterative refinement: 0-RETRIEVE (98% vs 95% for DANA), 1-COMPARE (100% vs 90%), 2-CALC-CHANGE (100% vs 93%). These are exactly the tasks where the initial approach often fails (data in an unexpected place, terms differ) and adaptive re-planning is needed.

The comparison with DANA revealed a trade-off: DANA scored 95.3% (slightly higher), BUT it required domain knowledge engineering and human guidance to build its plans. ProSEA got almost the same result fully autonomously — showing that systematic exploration with rich feedback can replace manual planning. In 4-CALC-AND-JUDGE (70% vs 94%) and 6-OTHER-ADVANCED (43% vs 89%) DANA won, because these categories require domain expertise the LLM doesn't have — but that's a knowledge problem, not a problem with the ProSEA architecture.

Key takeaway: The rich feedback mechanism pays off precisely when a task requires exploration — a static plan doesn't work and you need to adapt on the fly. The more uncertainty and dead ends, the bigger ProSEA's advantage.

ProSEA: Problem Solving via Exploration Agents (Nguyen et al., 2024) Aitomatic, Inc.

Related work cited in the paper:

• MetaGPT (Hong et al., 2023) — the manager-agent paradigm for multi-agent coordination
• Reflexion (Shinn et al., 2023) — memory-driven self-correction through reflection on failures
• Chain-of-Agents (Zhang et al., 2024) — collaborative LLM agents for long-context tasks
• DANA (Luong et al., 2024) — domain-aware neuro-symbolic agents with human guidance
• Voyager (Wang et al., 2023) — skill libraries + episodic memory for long-term exploration in Minecraft