Apply to my task
arXiv:2507.04751 ★ 72 Jul 7, 2025

M-OS: specs + reviews in one prompt give you an expert-level review

Key result

Without technical data, an LLM retells reviews and fills in the blanks with guesses. Ask for a laptop review and you get “users say it's powerful.” The M-OS method (multi-source opinion summarization) gets you a product review that actually helps you decide whether to buy: not review fluff, but a coherent expert analysis. The trick: give the model not only reviews but also the manufacturer's technical specs. The model stops guessing and starts matching opinions against facts. Instead of “it's loud,” it writes “the grinding noise is explained by the 1450 W grinder,” and the final review reads like it came from a consultant, not an anonymous stranger online.

LLMsas Architects and Critics for Multi-Source Opinion Summarization

Key Aspects of the Research:

The paper shows that to get a high-quality, genuinely useful summary of a product, you need to give the LLM not just subjective opinions (user reviews) but also objective data (technical specs, the manufacturer's description). This approach, called Multi-Source Opinion Summarization (M-OS), produces far more complete, trustworthy and balanced answers.

Key Result: Combining factual product data and user reviews in a single prompt dramatically improves the quality of the final summary and makes it vastly more useful for decision-making.


The Core of the Method, Explained:

The method changes how you approach opinion summarization. Instead of asking the LLM to simply “summarize the reviews,” you step into the role of an “information architect” and give the model full, multi-angle context.

The problem with the traditional approach: When the LLM sees only reviews, it can draw overly general conclusions, miss important technical details or even hallucinate while trying to explain why users like or dislike something. The result is subjective and incomplete.

The fix (the M-OS method): In a single request, you give the LLM several sources of information at once: 1. Objective data: Technical specs, the official description, a list of key features. 2. Subjective data: A set of real user reviews (ideally both positive and negative). 3. Overall score: The product's average rating.

With all these “building materials” in hand, the LLM can't just retell opinions; it has to synthesize them into a single, balanced report. It gets “grounded” in the facts from the specs, which makes its conclusions more accurate and reliable. For example, instead of “users say it's powerful,” the model can write: “Model X's Y-GHz processor delivers strong performance, which is backed up by user reviews noting smooth gameplay.”

For the user, this means spending an extra 1-2 minutes gathering information, but the result is an order of magnitude better.


Practical Applicability Analysis:

  • Direct applicability: Maximum. Anyone can open ChatGPT, copy a product's specs, description and a few reviews from a website, then paste it all into one prompt with the task “write a comprehensive review.” The method requires no special knowledge or tools.

  • Conceptual value: Huge. The paper teaches a core principle of prompt engineering: “Context is king.” It shows clearly that output quality depends directly on the quality and completeness of the input. Users come to see that an LLM isn't an oracle but a powerful information synthesizer, and that it needs quality “ingredients” to work well. This builds the right “mental model” for working with LLMs.

  • Adaptation potential: Very high. The mechanism transfers easily to any task where you need a balanced opinion:

    • Choosing a hotel: Mix the official description and amenities list with traveler reviews.
    • Analyzing a movie: Mix the synopsis and cast information with reviews from critics and audiences.
    • Deciding on a job: Mix the official job posting with employee reviews of the company. The adaptation is simple: decide which data sources are “objective” (facts, numbers, descriptions) and which are “subjective” (opinions, experiences, ratings), then combine them in one prompt.

A Practical Example:

You are an experienced home appliance consultant. Your task is to help me choose an espresso machine.

Analyze all the information provided below and write a balanced, structured review (about 200 words). The review must integrate the technical data with the opinions of real users. At the end, give a short conclusion: who this machine is the best fit for.

---
### INFORMATION TO ANALYZE

#### 1. Official description and specs
*   **Name:** De'Longhi Magnifica S
*   **Type:** Fully automatic espresso machine
*   **Pressure:** 15 bar
*   **Power:** 1450 W
*   **Bean hopper:** 250 g, built-in grinder with 13 grind settings
*   **Milk frother:** Manual (panarello)
*   **Water tank:** 1.8 L
*   **Features:** "Compact & Fresh" system, can use pre-ground coffee, self-cleaning program.

#### 2. Key user reviews
*   **Review 1 (Emma, 5 stars):** "Great machine! The coffee is as aromatic as at a coffee shop. I love that you can adjust the strength and grind to your taste. Easy to clean. The only thing is the frother takes some practice, but I got the hang of it fast."
*   **Review 2 (Mike, 4 stars):** "Been using it for six months, happy overall. Reliable, easy to operate. But for latte lovers the manual frother is a downside. Frothing milk by hand every time gets a bit tiring. Kind of noisy while grinding."
*   **Review 3 (Greg, 5 stars):** "Best money I've spent. Saves me time in the mornings. Push a button, get an espresso. For the price, the features are outstanding. Compact, fit perfectly in my tiny kitchen."
*   **Review 4 (Sarah, 3 stars):** "Good coffee, but the plastic housing feels a bit flimsy. And yes, it's very loud. If someone's asleep, I don't dare turn it on."

---
Write a review based on this data.

Why It Works:

This prompt is effective thanks to several mechanics that follow from the paper's findings:

  1. Multi-source context (M-OS): The prompt combines objective facts (specs) with subjective experience (reviews). This forces the LLM not just to retell opinions but to check them against reality. For example, the model will connect the “manual frother” in the specs with Mike's review calling it “a downside for latte lovers.”
  2. Grounding: Having exact data (15 bar pressure, 13 grind settings) keeps the model from inventing or exaggerating. It has to work with facts, which raises “faithfulness” (fidelity to the sources).
  3. A clear role and task: The instruction “You are an experienced consultant” and the task “write a balanced, structured review” set the tone and format of the answer, steering the model toward useful rather than merely descriptive text.
  4. Structuring the prompt: Using headings (#### 1. Official description..., #### 2. Key user reviews...) helps the model understand the structure of the input and separate facts from opinions, which improves how well it synthesizes them.

Another Practical Example

You are an experienced travel agent who specializes in family vacations. My clients, a family with two kids (ages 5 and 10), are considering the "Sunny Breeze" hotel for their summer vacation.

Analyze all the information below and prepare a short, honest hotel brief for this family. List the strengths and potential drawbacks, based on both the official information and real guest experiences.

---
### INFORMATION ABOUT THE "SUNNY BREEZE" HOTEL

#### 1. Official description from the website
*   **Location:** Beachfront, private sandy beach.
*   **Grounds:** Large and green, with a pine grove.
*   **For kids:** "Pirate Island" kids' club (ages 4-12), playground, shallow pool with a water slide.
*   **Dining:** All-inclusive, 3 restaurants, 5 bars.
*   **Rooms:** Standard rooms of 25 sq m; two-room family suites of 40 sq m. Every room has air conditioning and a minibar.

#### 2. Excerpts from guest reviews
*   **Review 1 (Laura, family with a 6-year-old):** "The grounds are gorgeous, plenty of room to roam! The kids' club staff are fantastic; we could barely drag our kid away. But the food in the main restaurant is really repetitive; we were sick of it after a week."
*   **Review 2 (Dan, couple):** "Great beach, clean water. The rooms are looking 'tired', the furniture is dated, the AC was noisy. Fine for a couple, but with kids I'd look for something newer."
*   **Review 3 (Jessica, family with kids 5 and 9):** "The kids loved the pool with the slide and the evening mini-disco. A lifesaver for parents! But be ready for lines at the restaurant at peak times. And yes, Wi-Fi barely worked in the room, only in the lobby."

---
Prepare the brief for the family based on this data.

Why This Example Works.

This prompt runs on the same principles as the previous one, adapted to a different field:

  1. Synthesizing promises and reality: The prompt makes the LLM weigh the hotel's marketing promises (“kids' club,” “all-inclusive”) against real guest experiences (“the staff are fantastic,” “the food is repetitive”). That creates a balanced picture.
  2. Target audience: Naming a specific audience (“a family with kids ages 5 and 10”) lets the LLM prioritize information. It will understand that “tired rooms” (Dan's review) and “lines at the restaurant” (Jessica's review) are important downsides specifically for a family, while for a couple without kids they might matter less.
  3. Higher relevance: Instead of a generic summary, the model will produce targeted recommendations. It will highlight the pros (kids' activities, the pool) and cons (room condition, lines, Wi-Fi) that matter most for the family's comfort, raising the “relevance” of the answer.
  4. Preventing one-sidedness: Without reviews, the LLM would produce a glowing retelling of the website copy. Without the description, it might form an overly negative impression from a few isolated flaws. Combining sources forces it to find the happy medium, which is exactly the goal of the M-OS method.

Usefulness Score: 94

02

Main Evaluation Criteria

  • A. Relevance to prompting techniques: Yes, the paper offers a concrete and powerful method for structuring a prompt (M-OS) by combining several data sources to get a high-quality result.
  • B. Better conversational answers: Yes, the main result of the paper is more complete, accurate and useful summaries, confirmed by a user study (87% preferred the new method).
  • C. Direct practical applicability: Yes, the method can be used immediately in any chatbot, with no code and no special tools. The user just needs to gather a few text sources and paste them into the prompt.
  • D. Conceptual value: Very high. The paper clearly demonstrates the principle of “grounding”: how supplying factual, objective data (product specs) alongside subjective data (reviews) dramatically raises the reliability and usefulness of the LLM's answer while reducing hallucinations.
  • E. New useful practice: The work falls into several key clusters at once:
    • Cluster 1 (Prompt-writing techniques): Offers Multi-Source Summarization as an advanced technique.
    • Cluster 5 (Extraction and structuring): A direct example of advanced extraction and synthesis of information from different sources into one structured answer.
    • Cluster 6 (Context and memory): Teaches how to feed heterogeneous information into the context effectively to solve a single task.
    • Cluster 7 (Reliability and stability): The method directly targets higher “faithfulness” (fidelity to facts) and a lower risk of fabrication by relying on objective data.
  • Practicality checklist (+15 points): Yes: the paper gives a ready approach for structuring complex summarization requests, shows how to raise answer accuracy and completeness, and offers an effective summarization method.
03

Numerical Usefulness Score

Arguments for a high score (94/100): The paper offers not just a theoretical model but an extremely practical and universal prompt-engineering pattern: “to get quality output, mix objective and subjective data.” You can carry this principle straight from the domain described (product reviews) into any other: analyzing reports, planning trips, choosing online courses and so on. The example in Table 1 vividly shows the huge difference in quality, which motivates users to apply the approach right away. The prompt design principles in Appendix B are a ready-made instruction manual for the user.

Counterarguments (why not 100/100): * Academic focus on evaluation: A large part of the paper is devoted to building the M-OS-EVAL dataset and developing prompts for evaluating summaries (SPECTRA-PROMPTS, OMNI-PROMPT). This part has little direct value for an ordinary user who wants to generate a good answer rather than grade it with scientific precision. * Narrow domain: Although the method is universal, all the examples and data focus on e-commerce. Users have to draw the analogy and adapt the approach to their own tasks themselves, which takes a little effort.


Research digest

Key result

Without technical data, an LLM retells reviews and fills in the blanks with guesses. Ask for a laptop review and you get “users say it's powerful.” The M-OS method (multi-source opinion summarization) gets you a product review that actually helps you decide whether to buy: not review fluff, but a coherent expert analysis. The trick: give the model not only reviews but also the manufacturer's technical specs. The model stops guessing and starts matching opinions against facts. Instead of “it's loud,” it writes “the grinding noise is explained by the 1450 W grinder,” and the final review reads like it came from a consultant, not an anonymous stranger online.

Principle

Reviews are subjective. Specs are objective. On their own, both sources work poorly: the official description is a glowing retelling of the marketing, raw reviews are a jumble of impressions. Put all three layers into one prompt: objective data (specs, the manufacturer's description), subjective experience (real user reviews) and the overall rating. The model can't pick which matters more; it has to synthesize, linking the “manual frother” in the specs to the review saying “frothing milk every time gets tiring.” Separate the sections with headings; that helps the model see where the facts are and where the opinions are.

Why it works

When the model sees only reviews, it has literally nothing to stand on. No data about 15 bar of pressure? It will write “makes good coffee.” Technical specs act as an anchor: they keep the model from drifting into generalities. This is called “grounding”: the model ties subjective impressions to concrete parameters and stops inventing things that aren't in the sources. Structuring the prompt with headings strengthens the effect: the model clearly sees what's fact and what's opinion, and synthesizes them more precisely.

When to apply

Choosing expensive or complex products → especially when you need to understand what actually sits behind a review that says “great!” and whether it's worth paying extra. Works well for: home appliances, smartphones, laptops, hotels, online courses, jobs (the official posting + employee reviews). It transfers to any task with a source of facts and a source of opinions: mix both. NOT a fit for: products without technical data (a novel, a movie). There simply are no specs, and the method loses its point.

Mini recipe

1. Gather the facts: copy the technical specs and official description from the manufacturer's site, literally ctrl+c from the product page. 2. Gather the reviews: take 3–5 real reviews with different ratings; make sure to include both positive and critical ones. 3. Give it a role: `You are an experienced [topic] consultant. Your task is to write an honest, balanced review.` 4. Structure the data in the prompt: paste the specs first under a `### Specs` heading, then the reviews under `### Customer reviews`; this helps the model separate facts from opinions. 5. Set a synthesis task: ask it not just to retell, but to explicitly link customer opinions to specific product parameters. At the end: who it's for and who it's not for.

Examples

[BAD]: `Here are some reviews of the De'Longhi Magnifica S espresso machine. Write a review.` [GOOD]: `You are an experienced home appliance consultant. Analyze the information below and write an honest 150–200 word review. Be sure to connect the technical specs with customer opinions. At the end: who this machine is right for, and who should keep looking. ### Specs - Pressure: 15 bar - Power: 1450 W - Grinder: built-in, 13 grind settings - Milk frother: manual (panarello) - Water tank: 1.8 L ### Customer reviews - Emma, 5 stars: “Coffee like at a café, the frother takes some practice” - Mike, 4 stars: “Reliable, but the manual frother gets tiring, noisy when grinding” - Sarah, 3 stars: “Good coffee, flimsy housing, very loud”`

In plain words

Here's the deal: today's AI models still fumble when they have to boil hundreds of contradictory reviews down into one picture. The usual “just summarize it” approach yields either watery mush or loses key details. The researchers propose an LLM-as-Architects setup in which the model first works as a designer: it doesn't write the text right away but builds a structural plan based on clusters of opinions. That forces the AI to sort out the hierarchy of ideas first and only then pick the words, which drastically lowers the risk that an important complaint about a “broken button” drowns in raves about the “gorgeous color.”

It's like trying to build a giant shopping mall without blueprints, just hiring a crowd of builders and telling them, “Eh, make something nice.” You end up with a crooked shed with the restroom in the middle of the food court. The Architect method means you first draw a detailed plan of every floor, then make a critic model check that nobody forgot the elevators. Technically it's the same job, but the results differ like a luxury development versus a DIY shack.

Two key processes run inside the system: structure generation and iterative critique. First the model picks out the key aspects (say, price, quality, service) and builds a tree of arguments. Then the Critic steps in, compares the draft with the sources and slaps the author's wrist for hallucinations or omissions. In numbers this gives a serious boost: methods like CoT (Chain-of-Thought) are cranked up to the max here, forcing the model to justify every sentence with a reference to a real review.

It was tested on hotel and product reviews, but the principle is universal. You can and should plug the same logic into any complex analysis, from processing legal documents to sorting through software feedback. Wherever you have a chorus of conflicting data, a plain summarizer chokes, while the architectural approach pulls out the essence. GEO and the search of the future will work exactly like this: synthesizing thousands of opinions into one sane answer you can trust.

Bottom line: stop asking the model to “keep it short”; make it design the structure first. Without a clear plan, any LLM turns into a chatty neighbor who kind of heard something but got it all mixed up. The “Architect + Critic” combo is the only way to get a sensible digest out of chaotic data without losing critical facts. Anyone who skips this two-stage check will keep feeding users hallucinations and useless fluff.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on M-OS: specs + reviews in one prompt give you an expert-level review arXiv:2507.04751

Sign in to apply this study New accounts get 50 free credits.

Get the week’s best papers by email

One email on Sundays with the highest-rated new techniques. Unsubscribe with one click.