Key Aspects of the Study:
The researchers compared different prompting methods on the task of deciding whether a sentence in a text is an objective fact or a subjective opinion. They found that the most effective approach combines a detailed prompt that clearly defines the criteria for objectivity and subjectivity with the "few-shot" technique, where a few examples of correct classification are added to the prompt. This approach turned out to be more reliable and simpler than fancier methods such as "debates" between several LLMs.
Key Result: A carefully crafted prompt with clear instructions and a few examples often beats complex and exotic prompting techniques.
The Core of the Method, Explained:
The essence of the best-performing method is not to just give the LLM a command, but to act as a mentor who explains the task in detail. Instead of a short request like "Determine whether the text is subjective", you use a multi-part prompt that breaks down into three pieces:
Role assignment (Role-playing): The prompt opens by giving the LLM an expert role: "You are a linguistics expert specializing in...". This puts the model in a more serious, academic frame of mind.
Detailed instructions and definitions (Extended Prompt): This is the heart of the method. You don't just ask it to classify the text; you give clear, thorough definitions for each category. For example, what exactly counts as "objective" (facts, even disputed ones; quotes) and what counts as "subjective" (value judgments, assumptions, intensifiers like "very" or "incredibly"). It's also important to describe the edge cases (for example, how to classify direct speech).
Learning from examples (Few-Shot Learning): After the instructions, you give the model 2-6 examples in the format "Text -> Correct category". The study showed that even randomly chosen examples significantly improve performance. The model "sees" the pattern and better understands how to apply the rules it was given in practice.
The main takeaway for the user: don't be lazy about writing detailed prompts. Spend the time to spell out clear rules and definitions, and you'll get a far more accurate and predictable result from the LLM than by cycling through dozens of short but vague requests.
Practical Applicability Analysis:
Direct applicability: Users can immediately apply this approach to any task that involves classifying or analyzing text. Just copy the "Extended Prompt" structure from the study and swap the "subjectivity/objectivity" definitions for your own. For example, to sort emails into "Urgent/Not urgent" or to analyze reviews as "Positive/Negative/Neutral".
Conceptual value: The study clearly demonstrates that an LLM is not a "black box" but a tool that follows the logic of its instructions. It gives the user the right "mental model": to get a quality result, you need to provide quality, unambiguous instructions. The most valuable takeaway: clear and complete instructions matter more than "magic words" or overly complex constructions.
Adaptation potential: The method adapts easily. For instance, the "debate" technique, even though it didn't come out on top in the study, can be simplified and used within a single prompt. You can ask the LLM: "First give arguments for why this review is positive. Then give arguments for why it's negative. Based on these arguments, reach a final conclusion and assign a category". This forces the model to do a deeper analysis before it delivers a verdict.
Practical Example:
# Role
You are an experienced customer success manager who needs to sort incoming requests quickly and accurately.
# Task
Your task is to analyze the text of a customer request and classify it into one of three categories: "Technical Issue", "Billing Question", "General Feedback". Respond in JSON format only.
# Classification Criteria
* **Technical Issue:** A request falls into this category if the customer reports that something isn't working, is broken, throws an error, or they can't use a product feature.
* *Sample keywords:* "can't log in", "404 error", "button doesn't work", "freezes", "won't load".
* **Billing Question:** A request falls into this category if it concerns money, invoices, plans, subscriptions, refunds or payment methods.
* *Sample keywords:* "I was charged", "payment didn't go through", "get a refund", "plan price", "renew my subscription".
* **General Feedback:** A request falls into this category if it's a general opinion, a suggestion for improvement or a thank-you that isn't tied to a specific problem or billing question.
* *Sample keywords:* "thank you", "it would be great if...", "I love", "awkward to use".
# Response Format
Provide your answer as a JSON object with the following keys:
- "category": one of the three categories ("Technical Issue", "Billing Question", "General Feedback").
- "summary": a one-sentence summary of what the request is about.
---
# Request text to analyze:
"Hi! I'm trying to download last month's report, but the page just hangs with a spinning wheel and nothing happens. I've tried different browsers. What should I do?"Why It Works:
This prompt is effective because it follows the principles from the study to the letter:
- Role assignment: "You are an experienced customer success manager" sets the context and the expected level of expertise.
- Clear instructions and definitions: Instead of simply asking "Which category does this text belong to?", the prompt provides exhaustive definitions for each category ("Technical Issue", "Billing Question", "General Feedback"). This removes ambiguity.
- Sample keywords: Adding sample keywords works like a mini version of few-shot learning, helping the model match the request text to the right category faster.
- Strict output format: Requiring the answer in JSON makes the result predictable and ready for further automated processing, which is a common need for users.
Another Practical Example
# Role
You are a professional HR specialist reviewing resumes from candidates for a "Sales Manager" position.
# Task
Your task is to evaluate a resume excerpt and determine whether the candidate's experience meets the key job requirement: "3+ years of B2B sales experience". Respond in JSON format only.
# Evaluation Criteria
* **Relevant:** The candidate's experience is considered relevant if the text directly or indirectly indicates work with corporate clients (businesses, companies, organizations) and the total length of that work is 3 years or more.
* *Signals:* Mentions of "B2B", "corporate clients", "business accounts", "managing large contracts". Tenure at companies that clearly operate in the B2B sector.
* **Partially Relevant:** The candidate's experience is considered partially relevant if B2B sales are mentioned but for less than 3 years, or if the experience is mixed (B2B and B2C) and it's hard to isolate pure B2B tenure.
* **Not Relevant:** The candidate's experience is considered not relevant if the text describes only retail sales (B2C), work with individual consumers, or if the field has nothing to do with sales at all.
# Response Format
Provide your answer as a JSON object with the following keys:
- "assessment": one of three ratings ("Relevant", "Partially Relevant", "Not Relevant").
- "justification": a brief rationale for your decision based on the text.
---
# Resume excerpt to analyze:
"2020 - present. Industrial Supply Partners LLC. Key Account Manager. Negotiated with directors of manufacturing plants and construction companies, closed long-term equipment supply contracts. Grew the corporate client base by 40%."Why This Example Works: The Mechanism.
This example works for the same fundamental reasons as the previous one, showing how universal the method from the study is:
- A narrowed-down task: Instead of the abstract request "evaluate this resume", the task is narrowed to checking one specific but important requirement ("3+ years of B2B sales experience").
- Detailed rules: The prompt doesn't just list the categories; it describes in detail what goes into each one. The "Partially Relevant" category is especially important: it handles the edge cases and makes the classification more precise.
- Contextual hints: Pointing out the "Signals" (keywords and company types) helps the model focus on the relevant information in the resume while ignoring the "noise".
- Required justification: The "justification" field in the JSON response makes the model not just output a label but spell out the chain of logic that led to its conclusion. This boosts reliability and lets the user quickly check the LLM's reasoning.
Usefulness Score: 95
Main Evaluation Criteria
- A. Relevance to prompting techniques: Very high. The study directly compares the effectiveness of different prompting techniques: a simple prompt, an extended prompt with detailed instructions, few-shot with different example-selection strategies, and multi-agent (debates). The appendix contains the exact prompt wording.
- B. Better conversational answers: High. Although the task is classification rather than dialogue, the findings directly affect the accuracy and reliability of LLM answers in text analysis and classification, which is a common chatbot use case.
- C. Direct practical applicability: Very high. A user can immediately take the "Extended Prompt" structure from the paper's appendix and adapt it to their own classification tasks without any code or special tools.
- D. Conceptual value: Very high. The study delivers key insights:
- Detailed instructions and clear definitions work significantly better than generic commands.
- More complex strategies (debates, picking "dissimilar" examples) don't always beat a well-written basic prompt with a few random examples (few-shot).
- LLMs with few-shot prompts can be more robust to "noisy" or poorly labeled data than fine-tuned models, which matters for real-world tasks.
- E. New useful practice (clustering): The paper falls into several key clusters:
- Cluster 1 (Formulation techniques): Yes (compares zero-shot, few-shot, role-play, detailed instructions).
- Cluster 3 (Structure optimization): Yes (shows the benefit of structuring a prompt with clear definitions and rules).
- Cluster 7 (Reliability and stability): Yes (it studies methods for improving classification accuracy, and the "debate" method is a form of self-checking).
- Practicality checklist: Provides ready-made constructions, shows how to structure requests, reveals non-obvious aspects of LLM behavior and offers ways to improve accuracy. (+15 points to the base score).
Numerical Usefulness Score
Arguments for a high score (95/100): This study is the quintessence of practical prompt engineering. It doesn't just propose a method; it compares several approaches and reaches a very practical conclusion: fundamental principles (clarity, detail, examples) matter more than exotic techniques. For an everyday user, that's the golden rule. The full prompt texts in the appendix make the study literally an "instruction manual".
Counterarguments (why not 100): * A narrow task: The focus on binary "subjective/objective" classification is academic. The user has to mentally transfer these principles to their own tasks (e.g., "spam/not spam", "urgent/not urgent"). * A "negative" result: One of the key findings is that advanced techniques (debates, choosing examples by dissimilarity) gave no significant advantage. While that's very useful, time-saving knowledge, it isn't the discovery of a new, revolutionary technique.
The final score of 95 reflects the huge practical value and conceptual clarity for any user who wants more accurate and reliable LLM answers in text classification and analysis tasks.