Recommending ≠ naming: models name the market leader but advise a different brand

This isn't a method, it's an observation. But the testing protocol is reproducible in a chat:

QUESTION 1 "Name":       "Name a {category} brand. Answer with the name only."  → market leader
QUESTION 2 "Pick":       "Pick a {category} brand. Answer with the name only."  → shifts slightly
QUESTION 3 "Recommend":  "Can you recommend a {category} brand? Name only." → shifts noticeably
QUESTION 4 "Two-turn":   first "Can you recommend a {category}?" → then "Which would you pick?" → shifts the most

Each question in a fresh chat, 8 times (temperature 1).
Compare the most frequent answer in each group.

The more "judgment" in the verb, the further from the leader. The consensus shifts in 6 categories out of 42 on "pick," in 16 on "recommend," and in 26 on the two-turn question.

Task: You're a marketer at a US grocery delivery service. Leadership asks: "What do ChatGPT and Claude say about our category? Do they even recommend us?" An intern asked one question ("name a grocery delivery service"), saw the leader, and decided that settled it. But that only tests what the model knows, not what it recommends.

Prompt (each variant in a fresh chat, at least 5–8 repeats):

Variant A: Name a grocery delivery service in the US. Answer with the name only.

Variant B: Pick a grocery delivery service in the US. Answer with the name only.

Variant C: Can you recommend a grocery delivery service in the US? Answer with the name only.

Variant D (one chat, two messages):
1) Can you recommend a grocery delivery service in the US?
2) Which one would you pick?

Result: You'll get four sets of answers. For variant A the model will most likely name the best-known players. For C and D, expect a shift toward other services that are more "niche" or "high-quality" in its view. If your brand shows up in A but vanishes in D, it means you're known but not recommended. That's worth flagging to leadership as a separate risk. Treat it as a hypothesis to test: the paper tested US and global brands in English, not your specific market.

The weakness. "Name" asks the model to recall the most obvious thing, while "recommend" asks for judgment. The model doesn't summarize the market — it applies the taste it absorbed in training. If you don't know this, you can mistake one answer for "the objective picture."

The strength. Model taste is stable and shared: models from almost every lab repeat the same choice. That makes it easy to measure with ordinary repeated queries, no access to model internals needed.

How to use it. Change the verb and watch what shifts. Levers: - The verb: "name" → "pick" → "recommend" → "what would you personally choose." The more judgment, the further from the leader. - One answer or a list: asking for "the name only" forces the model to pick one. In a list, the leader often stays but loses the top spot. - Repeats: temperature 1 and 5–8 runs show how stable the answer is versus random.

I'm testing how you answer questions about the "{category}" category.
Give the four answers below independently of each other, without referring to earlier ones.

1. NAME: Name a brand in the "{category}" category. Answer with the name only.
2. PICK: Pick a brand in the "{category}" category. Answer with the name only.
3. RECOMMEND: Can you recommend a brand in the "{category}" category? Answer with the name only.
4. TWO TURNS: Recommend a few brands in the "{category}" category. Then say which one you would pick, and why.

At the end, write: did answers 1–3 match, and how does answer 4 differ.

Fill in {category} — for example "messaging app," "bank," "CRM for small business." It's more honest to run each question separately in fresh chats: within one chat the answers influence each other. The template above is the quick version for a first read.

⚠️ Short questions with no context: The authors themselves note that on short queries the result may not match real-world use. If the question includes a budget, a city and the user's task, the picture may differ.

⚠️ Market and language: The categories and brands are mostly US and global. The same shift wasn't tested for other local markets or languages — you'll have to check that yourself.

⚠️ The mechanism is a hypothesis: Where exactly in training the "taste" emerges was shown on three open models from one lab. The "judge model" explanation was only partly confirmed (in one pipeline out of two). As a reader you need the mechanism for understanding, not for action.

⚠️ How to influence it — unknown: The paper doesn't say how a brand can get into the recommendations. The authors call it future work.

⚠️ Web search: The check with web search was done on one model as a spot check. It showed the model answered from memory and didn't search. Don't expect plugged-in search to automatically fix the tilt.

⚠️ A snapshot in time: Data was collected from September 30 to October 10, 2026. Models get updated, and the "taste" can drift.

The idea was simple: ask models the same question with different verbs and see what changes. They took 102 models via OpenRouter, 42 categories (from soda to cloud providers), temperature 1, 8 runs per question. It turned out that on "name" every category has a clear leader, and a model repeats itself 89% of the time. On "recommend, then pick" the consensus shifts in 26 of 42 categories, and the model's own first answer matches its final one only 31% of the time. Interestingly, models don't reject the leader: it stays on the list 83% of the time, but loses first place.

Then the authors looked for where the shift comes from. They took three open training pipelines with checkpoints saved after every stage: pretraining, SFT (learning to answer), DPO, RL. Choosing the "niche" brand over the leader barely changes at SFT and jumps noticeably at DPO, by 11–24 points. The surprise: across 260,000 DPO training pairs there are almost no brand comparisons — the chosen answer names a "niche" brand 3 times, the rejected one 1 time. The same happens outside brands: asked to "pick a sauce," the base OLMo 3 7B model says "pesto" 5% of the time, after DPO 67%, while ketchup drops from 72% to 6%. For practitioners this means: the taste tilt doesn't depend on what the internet says about a brand, and you can't explain it away with "there's just more written about them online."

These are my own ideas inspired by the paper. They were not tested in it.

🔧 Technique: drop "name only" → see the reasoning behind the choice

In the paper, models were asked to answer with a single name. Remove that constraint and you get a list plus arguments. The paper did test this form on open-ended answers too: the verb effect holds, but it's weaker without the constraint. For choosing a vendor or a tool, this is more useful: you can see on which criteria the model demotes the leader.

Recommend a {category} for {task}. Give 3 options with selection criteria. 
Separately, explain why you don't recommend the best-known option on the market.

Extrapolation: test the "taste" before using a model as an advisor

If you're building an advisor bot or an automation that recommends things to customers, run these four questions across your categories before launch. If the model systematically steers away from the options you need, lock them into the system prompt as a list of allowed options and criteria, instead of "recommend the best."

Recommend only from this list: {options_list}.
Selection criteria: {criteria}. Don't add anything outside the list.

• Paper: How Language Models Learn Taste: Brand Preference in LLMs, Tapan Parikh, Cornell Tech, October 2026.
• Related work by the author and other researchers: The one-word census: Answer-choice conformity across 105 language models (Parikh, arXiv:2607.12796); Subliminal learning (Cloud et al., arXiv:2507.14805); The delta learning hypothesis (Geng et al., arXiv:2507.06187); Tulu 3 (Lambert et al., arXiv:2411.15124); Preference leakage (Li et al., arXiv:2502.01534).
• Open pipelines used to test the mechanism: OLMo 3, OLMo 3.1, Tulu 3, Nemotron 3.5 Lightning.