BATTERY: All questions in one prompt → the model answers everything at once

SEQUENTIAL: Question → answer → next question in the same chat → answer...

SINGLE-ITEM: Each question in a new chat, with no context from the previous ones

⚠️ First, the limitations: - Not for creative tasks without criteria - Not for simple one-off questions - The effect shows up with multiple questions that have closed-ended answer options

Where it shines: evaluating something against several criteria, when you need structured answers and want to save tokens.

Task: You're a VC looking at a US edtech startup. You need to score it on 5 criteria: team, product, market, unit economics, risks. Instead of 5 separate requests — one prompt using battery presentation.

Prompt:

You are a venture investor evaluating the edtech startup "CodeSprout" — a platform that teaches kids to code.

Quick facts:
- Team: 2 founders (ex-Google, ex-Meta), 3 years of edtech experience
- Product: online Python courses for kids ages 8-14, gamified
- Market: US, kids' after-school enrichment market at $9B/year
- Economics: average ticket $299/course, CAC $60, retention 40%
- Competitors: Code Ninjas, Tynker, CodeCombat

Score the startup on each criterion. Answer format — JSON:

{
  "team": {
    "rating": "High/Medium/Low",
    "probability_of_success": "60%",
    "key_factor": "brief explanation"
  },
  "product": {
    "rating": "High/Medium/Low",
    "probability_of_success": "60%",
    "key_factor": "brief explanation"
  },
  "market": {...},
  "unit_economics": {...},
  "risks": {...}
}

⚠️ Important: don't give every criterion the same rating. Score each one independently.

Result:

The model will return structured JSON with ratings on all five criteria in a single response. The format lets you see the full picture at once and compare criteria. Savings: instead of 5 requests at ~500 tokens each (2,500 tokens) — one request at ~600-800 tokens.

⚠️ But keep recency bias in mind: If you ask it to pick one option from a list A/B/C, the model will pick C 20 times more often. That's why the example above uses Verbalized Distribution (probability as a percentage) + JSON for structure.

The LLM's weakness: Models generate text left to right, so the last tokens are "fresher" in the attention context. When choosing between A/B/C, the model sees C last and gives it more weight. That's called recency bias — not a bug, but an architectural trait of transformers. On odd-numbered scales (1-5), models treat the middle option as the "safe neutral" one and pick it more often.

The LLM's strength: Models hold context well and can handle several related tasks in one prompt (multitask in a battery). That saves tokens on repeated instructions and lets the model see the full picture instead of answering questions in isolation.

How the method uses that strength: Battery presentation loads all the questions at once — the model sees the full structure of the task and can account for connections between questions. Structured outputs (JSON) force the model to follow the format rather than generate free text. Verbalized Distribution ("60% confident") sidesteps recency bias — instead of picking one option, the model spreads probabilities in words, which reduces the influence of each option's position.

Control knobs:

• Question format — battery to save tokens, single-item if you need the questions isolated
• Option order — to test robustness, ask the question twice with the order reversed (A-B-C → C-B-A); if the answers differ, there's recency bias
• Structured output — JSON for structure, but you can skip it if you want to see the reasoning
• Odd vs even scale — avoid 1-5 scales if you don't want a pull toward the middle; use 1-4 or 1-6

{role_and_context}

{description_of_object_or_situation}

Score it on the following criteria:

1. {criterion_1}: {description}
2. {criterion_2}: {description}
3. {criterion_3}: {description}

Answer format — JSON:

{
  "{criterion_1}": {
    "rating": "{options separated by /}",
    "confidence": "{percentage}",
    "rationale": "{brief explanation}"
  },
  "{criterion_2}": {...},
  "{criterion_3}": {...}
}

⚠️ Important: score each criterion independently; don't give them all the same rating.

Placeholders: - {role_and_context} — who you are and why you're evaluating (investor/recruiter/expert) - {description_of_object_or_situation} — what's being evaluated (startup/candidate/product) - {criterion_N} — which parameters to score on (team/experience/quality) - {options separated by /} — the answer options (High/Medium/Low or 1-4) - {percentage} — degree of confidence (Verbalized Distribution)

Why "score independently"? Models are prone to consistency bias — giving every criterion a similar score. An explicit instruction dampens that effect.

🚀 Quick start — paste into a chat:

Here's a battery-presentation template with bias protection.
Adapt it to my task: [your task — what to evaluate and on which criteria].

Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what you're evaluating, which criteria matter, and what answer options you need — because the JSON structure and Verbalized Distribution require a clear format. It'll take the pattern from the template and fit it to your task.

🔧 Technique: Test for recency bias → see how robust it is

If you suspect the model is choosing by the option's position rather than its meaning:

[ask the same question twice with the options in reverse order]

Question 1: How important is this factor?
A) Very important
B) Moderately important
C) Not important

Question 2: How important is this factor?
A) Not important
B) Moderately important
C) Very important

If the answers differ (C the first time, C again the second time), that's recency bias.

🔧 Technique: Drop the JSON → see the reasoning

If you want to understand WHY the model scored it that way, drop the JSON and ask for free text with a probability distribution:

Score it on the criteria, and for each one state:
- How confident you are in each rating, as a percentage
- Which key factor drove the decision
- What data would be needed for a more accurate assessment

Write in free text, but organize it by criterion.

🔧 Technique: Even scale instead of odd → removes middle category bias

If the model picks the "middle" option too often:

❌ 1-5 scale (has a middle option, 3)
✅ 1-4 or 1-6 scale (no clear middle)

The model is forced to choose — either positive (3-4) or negative (1-2).

⚠️ Not for creative tasks: Battery presentation reduces answer diversity on creative tasks. If you need 5 different content ideas, you're better off with 5 separate requests (single-item); otherwise the ideas will all look alike.

⚠️ Recency bias is stronger on small models: On Llama-3.1-8B the effect is 20x; on Llama-3.3-70B and Gemini-1.5-Pro it's weaker, but still there. The smaller the model, the stronger the bias.

⚠️ Not for complex multi-step reasoning: If each question depends on the answer to the previous one, use sequential, not battery. Battery is better when the questions are independent or loosely related.

⚠️ Structured outputs can break: If a model wasn't trained on structured outputs (JSON), it may return invalid JSON. Verified to work on: GPT-4, Claude, Gemini, Qwen, Llama 3+. On older/smaller models it may take several attempts.

A team from the University of Mannheim and GESIS took four real sociological surveys — the American National Election Study (ANES), the German Longitudinal Election Study (GLES), the American Trends Panel (ATP) — 16 questions in total about political views and social attitudes. They created 7,530 synthetic personas with demographic characteristics (gender, race, age, region) drawn from real respondents. For each persona they ran 10 models (Llama, Qwen, Gemma, Phi, Gemini) with 3 different seeds and 4 prompt variations (question order, answer option order). Total: 40+ million answers.

They compared these with real human answers on two metrics: Mean Absolute Error (how much each individual answer differs) and Wasserstein distance (how much the answer distribution of a synthetic subgroup differs from a real subgroup of people with the same characteristics).

The surprise: Battery presentation saves 2-6x the tokens (sequential: 8,216 input tokens, battery: 723 tokens for one persona) while also giving 8% better alignment with human answers for large models. For small models (<8B parameters), sequential or single-item works better — apparently they struggle to hold many questions in context at once.

The 20x recency bias in Llama-3.1-8B was the most striking result: the "Not important" option was chosen 5% of the time when it was first (A), and >100% of the time when it was last (E) — in other words, practically always. For larger models the effect is weaker (2-3x), but still significant.

Practical insight: If you're doing data annotation or surveys with LLMs, use battery to save money, but always test for recency bias by reversing the option order. And it's better to use Verbalized Distribution or even-numbered scales to reduce the impact of these biases.

QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models — Maximilian Kreutner, Jens Rupprecht, Georg Ahnert, Ahmed Salem, Markus Strohmaier (University of Mannheim, GESIS - Leibniz Institute for the Social Sciences, CSH Vienna)

Live GUI — a no-code interface (for researchers)

Video presentation