Control version (for comparison):

Task → Prediction: Yes/No → Result

Incentive version (with the betting mechanism):

START: Bankroll = 1,000,000 LLMCoin

FOR EACH TASK:
  1. Prediction: Correct/Incorrect
  2. Stake: from 1 to 100,000 coins
  3. Brief rationale (optional)

PAYOUT RULES:
  - Prediction right → +stake to bankroll
  - Prediction wrong → -stake from bankroll

CONSTRAINTS:
  - Total stakes in a round ≤ current bankroll
  - Bankroll < 1 coin = bankruptcy (game over)

AFTER EACH ROUND:
  - Feedback: which predictions were right
  - Statement: wins, losses, new balance

Everything happens in a single conversation with the model. Each task = one output with a prediction + a stake.

⚠️ The method works best when you need to rank confidence across many assessments/forecasts, not get one correct assessment. It doesn't fit subjective creative tasks with no verifiable outcome.

Task: You have 20 edtech startup ideas. You want to know which directions to dig into — you need prioritization by confidence, not just a pros-and-cons list.

Prompt:

You are a venture analyst with a bankroll of 1,000,000 VCCoin.

Your task: for each idea on the list, predict whether it can raise a seed round of $2M+ from US VC funds in 2025.

Rules:
- For each idea, give a prediction: YES (will raise) or NO (won't raise)
- Attach a STAKE of 1 to 100,000 VCCoin to each prediction
- If the prediction is right, you win the stake
- If it's wrong, you lose the stake
- The sum of all stakes can't exceed your current bankroll
- Goal: maximize your final bankroll

Output format for each idea:
{
  "idea": "[name]",
  "prediction": "YES/NO",
  "stake": [number from 1 to 100000],
  "rationale": "[1 sentence]"
}

Here are the 20 ideas:
1. A corporate training platform with an AI assistant...
2. A mobile app for SAT prep...
[... the rest of the ideas]

After the predictions, show:
- The top 5 ideas with the biggest stakes
- The top 5 ideas with the smallest stakes
- The total amount staked

Result:

The model will return a JSON list with a prediction and a stake for each idea. The stake size shows the model's confidence: whale bets of 50k-100k coins go to ideas with obvious product-market fit and a clear business model (e.g., "corporate training with measurable ROI for HR"). Small bets of 100-5,000 coins go to highly uncertain ideas (e.g., "a gamified social network for high schoolers"). At the end you'll see a list ranked by confidence and can focus on the top, where the model "went all in."

You get not just an assessment of each idea but a numeric priority scale, grounded in the model's calibration. It works better than "rate on a 1-10 scale" because the stake is tied to the bankroll and risk — the model can't bet 100k on every idea, it has to choose.

The weakness: LLMs are bad at calibrating confidence in binary or verbal answers. "Most likely" and "probably" are fuzzy words. Even if you ask "rate your confidence 1-10," the model doesn't grasp the relative weight of scores within one request: everything may get 7-8.

The strength: LLMs are great at ranking and can work with numeric constraints when the constraints are explicit and tied to context. When the model has a finite resource (a bankroll) and explicit rules (win/lose the bet), it starts differentiating tasks by confidence.

How the method exploits this: The "maximize your bankroll" framing turns the task into a constrained optimization problem. The model can't put 100k coins on every prediction — it has to choose where confidence is high. The stake becomes a proxy for internal probability: if the model sees clear signs of correctness (e.g., an easy problem + a strong model), it bets big. If there's uncertainty, it bets pennies.

Bonus effect: The feedback mechanism through a numeric balance speeds up learning. After each round the model sees "+45,000" or "−12,000" — a brighter signal for the attention mechanism than a plain "wrong." In the study, models in Incentive mode improved accuracy by 12 percentage points between round one and round four, while the control group improved by only 2.9. A numeric loss/gain is louder to the model than verbal feedback.

Prompt control knobs:

1. Bankroll size (1M → 100k or 10M) — changes the granularity of bets. A bigger bankroll = more room to differentiate.

2. Stake range (1-100k → 1-1k or 1-1M) — affects how aggressively the model can express confidence. A narrow range = smaller gaps between "sure" and "not sure."

3. Payout rules (even odds → different multipliers for different tasks) — you can make it trickier: "a bet on a hard task pays ×2." That forces the model to recompute the risk/reward ratio.

4. Bankruptcy condition (remove it or replace with a "soft penalty") — drop the hard limit and the model gets bolder. Add "max stake = 10% of balance" and it gets more conservative.

5. Feedback between rounds (balance only → balance + error breakdown) — a detailed "where and why you lost" review speeds up learning even more.

These knobs let you tune the sensitivity of the method to your task: want aggressive differentiation — widen the stake range; want safe ranking — add a per-bet cap.

You are a {role, e.g.: expert/analyst/evaluator} with a bankroll of {number} {currency name, e.g.: AnalystCoin}.

Your task: for each {item being assessed: idea/forecast/option/solution} on the list, give a prediction {criterion: "will work/won't work", "correct/incorrect", "profitable/unprofitable"}.

Rules:
- For each item, give a prediction: {option A} or {option B}
- Attach a STAKE of {minimum, e.g.: 1} to {maximum, e.g.: 100000} {currency name} to each prediction
- If the prediction is right, you win the stake (added to your balance)
- If it's wrong, you lose the stake (subtracted from your balance)
- The sum of all stakes can't exceed your current bankroll
- Goal: maximize your final bankroll

Output format for each item:
{
  "name": "[item name]",
  "prediction": "{option A/option B}",
  "stake": [number],
  "rationale": "[1-2 sentences]"
}

Items to assess:
{paste your list of tasks/options/ideas}

After all predictions, show:
- The top 5 items with the biggest stakes (highest confidence)
- The top 5 items with the smallest stakes (low confidence)
- The total of all stakes
- The bankroll remaining after the bets

What to plug in: - {role} — the expert context (venture analyst, technical reviewer, editor) - {number} — starting bankroll, usually 1,000,000 or 100,000 - {currency name} — make up a name (VCCoin, EditorCoin, TechToken) — it creates game framing - {item being assessed} — what you're evaluating (startup ideas, code snippets, copy variants) - {criterion} — a binary criterion with a verifiable outcome (will work/won't work, has a bug/no bug) - {option A/B} — the two possible prediction outcomes - {minimum/maximum} — the stake range, usually 1–100,000 - {list} — the specific items to assess

This isn't programming, it's framing. The LLM doesn't "execute code" — it follows game rules described in words. The betting mechanism is a structured instruction that creates:

1. A numeric constraint (the bankroll) — the model is forced to rank tasks and can't hand everyone the max.

2. An explicit notion of risk (win/lose the bet) — the model understands that a big bet = a big cost of being wrong.

3. A feedback loop (balance updates) — a numeric change in the balance is louder to the model's attention than a verbal "wrong."

This structure turns the model's internal uncertainty into an explicit output — the stake size. Without bets the model generates "yes" or "no" with no gradations. With them it's "yes, and I'm betting 80k" vs "yes, but I'm only betting 500."

⚠️ The arithmetic is unreliable: In three of nine experimental runs the models slightly exceeded the per-round stake limit (~0.2% overspend). Internal "bookkeeping" of wins/losses was correct, but planning the total stake glitched. If precise balance control matters, check the math outside the model (e.g., sum the stakes yourself or with a script).

⚠️ You need a verifiable criterion: The method works when there's an objective truth (a correct answer, a real-world outcome). For subjective assessments ("how creative is this text"), the model can't get "bet won/lost" feedback — calibration breaks down.

⚠️ Best for multiple assessments: If the task is a single assessment or forecast, the stake adds a confidence signal, but ranking won't work (nothing to compare). The method pays off most when you need to prioritize 10-100+ items.

⚠️ Doesn't boost accuracy much: In the study accuracy rose by +2.4 percentage points (79.1% → 81.5%, p = .089, not statistically significant). The main value isn't accuracy but calibration — you see where the model is confident (whale bets: 99% accuracy) and where it's in doubt (small bets: 74% accuracy).

The researcher generated 100 math problems with algorithmically verifiable answers (divisibility, primes, modular arithmetic, linear equations) using GPT-5 Extended Thinking + verification code. That ruled out "guessing" from memorized training data.

Six Baseline models (three current-gen: Gemini 2.5 Pro, Claude Sonnet 4.5, DeepSeek R2; three prior-gen: Gemma 2 2B, Claude 3 Haiku, DeepSeek R1) solved all 100 problems. Current-gen models answered 93.3% correctly, prior-gen 52.3%.

Three Predictor models (Grok 4 Expert, Mistral Large 2 Thinking, Qwen 3 Max Thinking) predicted, for each (problem, Baseline) pair, whether the Baseline would answer correctly. That's 600 predictions per run (100 problems × 6 Baseline models).

Each Predictor model did three runs in the control group (plain "yes/no" predictions) and three in the Incentive group (prediction + a stake of 1 to 100k coins from a 1M bankroll). In total, 9 runs per condition = 5,400 predictions per group.

The problems were split into four rounds of 25 questions. After each round the Predictor got feedback: which Baseline models answered correctly. The Incentive group also saw an updated coin balance (how much it won/lost).

The results showed: - Accuracy in Incentive was slightly higher (81.5% vs 79.1%), but not statistically significant (p = .089). - Learning was 4× faster: Incentive models improved accuracy by 12 percentage points from round 1 to round 4, control models by only 2.9 (p = .011, a significant effect). - Whale bets were nearly flawless: stakes of ≥40k coins were right 98.8% of the time (168 of 170), small stakes <1k only 73.9%.

This showed that stake size is a clean calibration signal. The model "sorts" its predictions by confidence on its own via the financial mechanism.

Why did it come out this way? Numeric feedback ("+45,000" vs "−12,000") is visually higher-contrast for an LLM's attention mechanism than a verbal "correct/incorrect." After a few losses on weak Baseline models, the Predictor quickly recalibrates: it starts betting pennies on Gemma 2B and Haiku (which often get things wrong) and whale bets on Gemini/Sonnet (which are almost always right). In the control group this pattern shows up too, but more slowly — there's no vivid numeric anchor for "how bad was that."

A fun detail: in one run Qwen bet the minimum of 1 coin on all 25 first-round problems, then opened up sharply in round 2 — it looks like a "scout the risks first, then go aggressive" strategy. Grok and Mistral behaved more evenly. This shows that different models interpret the game framing in their own way.

Context: The researcher used the following prompt for the Incentive group (with the betting mechanism). Below is a verbatim excerpt of the instruction in English, as it was given to the Predictor models:

You are an expert forecaster with a bankroll of 1,000,000 LLMCoin.

Your task: For each (question, baseline model) pair, predict whether the baseline 
model will answer the question correctly.

Rules:
- For each pair, output:
  • "predicted_correct": "Yes" or "No"
  • "stake": an integer between 1 and 100,000 LLMCoin
  • "rationale": brief explanation (~20 words max)

- If your prediction is correct, you WIN the stake (added to bankroll).
- If your prediction is incorrect, you LOSE the stake (subtracted from bankroll).

Constraints:
- The sum of stakes in a round must not exceed your current bankroll.
- If your bankroll drops below 1 coin, you go bankrupt and the game ends.

Goal: Maximize your final LLMCoin balance.

After each round, you will receive:
1. Correctness summary (which baselines answered which questions correctly)
2. Updated bankroll statement (wins, losses, new balance)

Output format (JSON-like):
{
  "round": [round number],
  "question_id": [id],
  "base_model_id": [id],
  "predicted_correct": "Yes" / "No",
  "stake": [integer 1-100000],
  "rationale": "[brief reasoning]"
}

For the control group the instruction was identical, but without the "stake" field and without any mention of a bankroll/bets. The goal was phrased as "Focus on being accurate" rather than "Maximize your bankroll."

💡 Adaptation for work scenarios: rating your own ideas

The classic example from the study is predicting whether other models' answers are correct. But you can apply the same mechanics to rate your own options in any multiple-choice task.

Scenario: You have 15 headline variants for a landing page. You need to prioritize which ones to A/B test first.

Prompt:

You are a marketing analyst with a bankroll of 500,000 MarketCoin.

Task: for each headline, predict whether it will get a CTR above 8% 
on Google Ads traffic (B2B SaaS, audience: heads of sales).

Rules:
- For each headline, give a prediction: YES (>8% CTR) or NO (≤8%)
- Attach a stake of 1 to 50,000 MarketCoin to the prediction
- If the prediction is right → you win the stake
- If it's wrong → you lose the stake
- Total stakes across all headlines ≤ 500,000
- Goal: maximize your final balance

Format:
{
  "headline": "[text]",
  "prediction": "YES/NO",
  "stake": [number],
  "rationale": "[why]"
}

Headlines:
1. "Grow sales 40% in one quarter"
2. "A CRM your team doesn't need training to use"
3. "How we helped 50+ companies automate their sales team"
[... the other 12 variants]

After the predictions, show the top 5 by stake size.

Result: The model will place whale bets (30k-50k) on headlines with clear triggers for decision-makers (concrete numbers, pain points, fast results). Small bets (1k-5k) go to abstract or creative phrasings. You get a ranked shortlist for testing, based on the model's confidence in each variant.

🔧 Technique: multi-round feedback to train the model on the fly

In the original study, after each round the model received feedback (correct answers + an updated balance), and that sped up learning fourfold. You can simulate this in your own tasks, even if you don't have an objective truth.

Prompt modification:

[Base prompt with the betting mechanism]

After you give your predictions, I'll give you PARTIAL feedback 
(e.g., results for 5 items out of 20).

Then you'll revisit your predictions for the remaining 15 items and may 
change your stakes based on the new information.

Goal: maximize your final balance after all rounds.

Effect: The model adapts after the first wave of feedback. For example, if you were rating 20 business ideas and gave the model feedback on 5 (which ones actually raised money), it will recalibrate its stakes on the remaining 15 — lowering bets on similar ideas that flopped and raising them on those that took off.

This turns the method into iterative calibration, where the model learns from your data within a single conversation.

💡 Extrapolation: confidence scoring without a verifiable truth

The original method needs feedback (right/wrong) to update the balance. But you can use just the betting mechanics to get a confidence score, even when there's no objective truth.

Scenario: You're writing a content plan for the month. 30 post ideas. You need to see where the model is confident and where it's just winging it.

Prompt:

You are a content strategist with a bankroll of 1,000,000 ContentCoin.

Task: for each post idea, predict whether the post will get >500 likes 
in a marketing newsletter on Substack (30k subscribers).

Rules:
- For each idea: a YES/NO prediction + a stake of 1 to 100,000 coins
- You will NOT get feedback (this is a hypothetical bet)
- But imagine you're maximizing your balance — bet more where you're confident

Format: [standard JSON]

30 post ideas:
[list]

Show the top 10 by stake size (highest confidence).

Result: Even without real feedback the model will distribute stakes according to its internal confidence. You'll get a ranked list of ideas — the top 10 with whale bets = "the model thinks these will definitely land," the bottom with penny bets = "uncertainty here." It's better than a plain "rate 1-10" because the limited bankroll forces the model to choose.

Going All-In on LLM Accuracy: Fake Prediction Markets, Real Confidence Signals
Michael Todasco, James Silberrad Center for Artificial Intelligence, San Diego State University
Preregistered at OSF: https://osf.io/dc24t/