TL;DR
The study found: when you ask an LLM to generate creative ideas, which prompt you use explains 36% of the variance in quality, about as much as the choice of model itself (41%). But the prompt barely controls the number of ideas (4%): that's baked into the model (51%) and into the randomness of each generation (34%).
The headline finding: 10-34% of the result is a lottery inside a single model. Run the same prompt on the same model 100 times and you get ideas ranging from mediocre to brilliant. One generation might score a 3, another a 5. The problem: most LLM evaluations run a single query and call it "the model's ability". That's like judging a marksman by one shot: maybe their hand shook, maybe the wind helped.
The fix: for creative tasks, generate 3-5 versions and pick the best. Prompts like "You are the most creative person" or "Think across different domains: art, survival, business" shift the quality distribution upward, but they don't remove the randomness. A query isn't deterministic: it's a sample from a probability distribution, and the spread can be huge.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
What was studied
The researchers used a creative task called the Alternate Uses Task: "Come up with unusual uses for a plastic bottle." It's a standard test of divergent thinking.
They tested 12 models × 10 prompts × 100 generations = 12,000 queries. Each prompt-model pair was run 100 times with identical settings. That's the crucial part: studies usually do 1 generation and say "here's the model's result." This one showed that a single result is noise.
They scored two things: - Originality (1-5): how unusual and creative an idea is - Fluency: how many ideas the model produced
They used an automated creativity scoring system (there's no way to hand-grade 12,000 responses).
Key findings
1. Prompts drive quality, not quantity
For idea quality (originality): the prompt explains 36.43% of the variance, the choice of model 40.94%, internal randomness 10.56%.
For idea quantity (fluency): the prompt explains a measly 4.22%, the choice of model 51.25%, and internal randomness a whopping 33.70%.
Takeaway: With a prompt you can strongly influence WHAT ideas the model produces (better/worse). But you have almost no influence over HOW MANY ideas it produces. That's baked into the model and the randomness of generation.
2. One query proves nothing
10-34% of the result is within-model variability. The same prompt on the same model can produce: - Generation #1: average ideas (score 3.2) - Generation #47: brilliant ideas (score 4.6) - Generation #89: average again (score 3.4)
The problem with most studies and benchmarks: they run one query and say "GPT-5 is more creative than Claude." What they actually compared is one random sample from GPT's distribution against one random sample from Claude's distribution. That's like comparing the average height of men and women by measuring one person of each.
3. Which prompts improved quality
The top 3 strategies (significantly raised originality):
- Persona (P6): "You are the most creative person in the world..."
- Heuristic/Domain (P3): "Think across different fields: art, survival, science, everyday life..."
- Anticipatory (P4): "Avoid cliché ideas like 'vase' or 'cup'..."
What did NOT work:
- One-shot example (P2): giving one example → didn't improve creativity
- Chain-of-Thought (P5): "Think step by step" → didn't help creativity (though it helps with logic)
- Small variations (paraphrasing, typos, word order) → no effect
Why Persona works: It isn't magic; it's a discriminative stress test. A prompt like "You're super creative" separates the models that can actually play the role (and produce genuinely creative output) from the ones that just repeat templates. Grok 4.1 delivered the best results on the Persona prompt (1st place) but dropped to 11th on other prompts. Gemini 3 Pro stayed in the top 2 on every prompt.
4. Models fall into types
Reasoning models (quality > quantity): - Gemini 3 Pro: average originality 4.06 (the best), but few ideas (13.7 on average) - Qwen 3 235B Thinking vs regular Qwen: the thinking version gets +3.4% quality but −38% quantity
High-fluency models (quantity > quality): - GPT-5.1 and GPT-5.2: >150 ideas on average, but lower average originality because of dilution
Stable vs volatile models: - Gemini 3 Pro: rank stability = 0.18 (always top 1 or top 2) - Grok 4.1: stability = 8.32 (1st place one time, 11th the next, depending on the prompt)
5. Longer ideas ≠ better ones, but there's a link
They found a correlation of 0.1-0.4 between idea length and originality. Gemini 3 Pro produced the longest ideas (125 characters on average).
They checked: could the model just be padding and getting credit for it as creativity? After statistically removing the length effect, the model rankings barely changed. Gemini stayed #1. Qwen 3 235B Thinking climbed from #6 to #3: it turned out to produce the most original ideas with the fewest words.
Takeaway: the top models are creative in substance, not by rambling.
What to apply at work
Principle 1: Generate several times for creative tasks
What to do: For brainstorming, idea generation and creative naming, run 3-5 queries instead of one. Pick the best.
Why it works: 10-34% of the result is sampling randomness. One query can land in the tail of the distribution (a weak result), another at the peak (a strong one). Multiple attempts raise your odds of catching a good sample.
Example:
Bad:
Prompt: "Come up with 10 names for a podcast about AI"
→ One query → took whatever came out
Good:
Prompt: "Come up with 5 creative names for a podcast about AI and its impact on business"
→ Three queries → compared them → picked the best out of 15 options
Principle 2: Persona prompts for creativity
What to do: Give the prompt a creative-expert role, or tell it to think across different domains.
Why it works: It shifts the quality distribution upward. The model generates ideas from less common regions of the probability space.
Examples:
You are a seasoned creative director with 15 years of experience. Suggest
unusual formats for launching an online design course that will
surprise the audience.
Think across different fields: marketing, psychology, gamification,
storytelling. How could a standard career coaching session be packaged
so that people actually want to buy it?
Principle 3: Say what you DON'T want
What to do: Add "Avoid..." for the clichéd options.
Why it works: Models tend to generate high-probability (= common = clichéd) options. An explicit ban pushes them toward rarer (= more original) ideas.
Example:
Come up with 7 ways to use an empty glass jar.
Avoid the obvious: vase, cup, storage for odds and ends.
Look for uses in the home, in creative work and in science.
Principle 4: Chain-of-Thought is NOT for creativity
What NOT to do: Don't use "Think step by step" for creative tasks.
Why: CoT helps with logic and reasoning (math, analysis), but for creativity it showed no effect in the study. Creativity is divergent thinking (widening the options), while CoT is convergent (narrowing down to the right answer).
Principle 5: The prompt doesn't control the number of ideas
What to know: If the model gave you 5 ideas and you wanted 20, rephrasing the prompt probably won't help. It's baked into the model (alignment, temperature, training style).
What to do: - Either switch models (GPT produces more ideas than Gemini) - Or run several queries and pool the results
Why it works
The problem: An LLM isn't an "input → output" function; it's a probability distribution. For a given prompt the model doesn't produce "the answer"; it _samples from a distribution of answers_. Every time, a new sample. Even at temperature=0 there's nondeterminism from GPUs, parallelism and rounding.
The consequence: A single query shows a random sample, not the "true ability." It's like rating a marksman on one shot: maybe they missed, maybe they hit the bullseye. You need a series.
Why prompts work for quality: A prompt shifts the center of the distribution. A basic prompt → the distribution centers at 3.5 out of 5. A Persona prompt → it centers at 4.0 out of 5. But the spread remains: individual samples can be 2.8 or 4.7.
Why prompts DON'T work for quantity: How many ideas to generate is baked deep into the model: alignment (training for "conciseness" vs "detail"), temperature, architecture. A prompt doesn't rewrite those parameters.
Why Persona works: The model was trained on texts where "creative people" write more originally. By assigning the role, you activate that pattern. But it works only on models with the capabilities: Grok 4.1 opened up, Gemma 3 27B didn't (it had nothing to open up).
Why CoT doesn't work for creativity: CoT = structured, step-by-step reasoning. It narrows the search space toward "the right answer." Creativity needs the opposite: widening the space and exploring rare regions. CoT pushes toward likely paths; creativity pushes toward unlikely ones.
How to apply it: templates
Template 1: Multi-pass creative brainstorm
You are {expert_role}. Suggest {number} ideas for {task}.
Think across different fields: {domain_1}, {domain_2}, {domain_3}.
Avoid clichéd options like {cliche_examples}.
Look for unexpected connections and unconventional uses.
How to use it: Run it 3-5 times. Collect all the ideas. Pick the best or mix elements together.
Example fill-in:
- {expert_role} = "a seasoned product marketer"
- {number} = "5"
- {task} = "getting B2B clients to sign up for an analytics webinar"
- {domain_1, 2, 3} = "decision psychology, storytelling, gamification"
- {cliche_examples} = "'free registration', '20% off', 'a gift for attending'"
Template 2: Anti-cliché
{Task}.
Requirements:
- Avoid obvious solutions: {cliche_list}
- Look for uses in unexpected contexts
- Combine ideas from different fields
Give me {number} of the most original options you can.
How to use it: Do 2-3 passes and compare originality.
Template 3: Role + stress test
You are {specific_role_with_expertise}.
Task: {task_description}
Condition: every idea must be so unusual that
90% of people would never think of it.
{Number} ideas, no explanations, just the core.
How to use it: Run it on different models or several times on one. Persona + a stress test ("90% would never think of it") amplifies the effect.
Limitations
⚠️ Open-ended tasks only: The "generate several times" principle works for creative tasks with a fuzzy definition of correct (ideas, names, concepts). For factual questions ("the capital of France") or tasks with clear criteria (code that has to compile), variability is lower and repeats make less sense.
⚠️ Tokens × count: Generating 5 versions instead of 1 = 5 times the tokens. For long texts (articles, long reads) that gets expensive. The principle pays off for short generations: ideas, headlines, slogans, names.
⚠️ Not every model responds to Persona: The study found that Grok 4.1 took off on the Persona prompt while other models didn't. The model needs the capabilities (built-in abilities) for the prompt to unlock. If the model is weak, Persona won't help.
⚠️ Automated creativity scoring isn't perfect: The study used an automated scoring system (there's no other way to check 12,000 responses). It may have biases. Be careful extending the conclusions to other kinds of creativity (music, visuals, poetry).
⚠️ Variability depends on temperature: The study used each model's default settings (different temperatures). At temperature=0 variability is lower but not zero (GPU effects). At high temperature it's higher. The "10-34% within-model variability" finding holds for standard settings, not extreme ones.
Resources
Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks
Jennifer Haase, Jana Gonnermann-Müller, Paul H. P. Hanel, Nicolas Leins, Thomas Kosch, Jan Mendling, Sebastian Pokutta
HU Berlin, Weizenbaum Institute, Zuse Institute Berlin, University of Essex, TU Berlin
The study used the Alternate Uses Task (Organisciak et al., 2023, 2025), a validated divergent-thinking test with automated scoring.