Key Aspects of the Research:
The research looks at how to use GPT-4o to generate high-quality follow-up questions during interviews. The authors found that if you simply ask the model to generate a question, the result is no better than a human's. But if you give the model a clear list of "typical interviewer mistakes" and ask it to generate a question that avoids those mistakes, the quality (clarity, relevance, informativeness) becomes significantly higher than a human's.
Key Result: Telling an LLM which mistakes to avoid is a more effective strategy for getting a quality result than simply asking it to "do a good job."
The Core Idea of the Method, Explained:
The essence of the method, which you could call "steering through constraints" or "mistake-guided generation," is a shift in focus when writing the prompt. Instead of trying to describe the ideal result, the user first defines what a bad, failed result looks like, and then turns those "failures" into clear rules and constraints for the LLM.
This approach works because it's much easier for an LLM to follow specific, measurable rules (for example, "the question must not contain technical jargon," "the answer must be shorter than 50 words") than to interpret abstract wishes ("make the answer good and clear").
The practical method for the user looks like this: 1. Define the task: What do you want from the LLM? (For example, "a reply to a negative customer review"). 2. Analyze the "failures": Think about what typical mistakes people (or LLMs) make on this task. (For example: "the reply sounds robotic," "blames the customer," "offers no solution," "too long"). 3. Turn the "failures" into positive rules: Rephrase each mistake as a clear instruction or criterion. * "Sounds robotic" → "Use an empathetic, human tone." * "Blames the customer" → "Show understanding and never shift the blame." * "Offers no solution" → "Always propose a concrete next step to solve the problem." 4. Structure the prompt: * Role (Persona): Start by assigning a role ("You are an experienced customer success manager"). * Context: Clearly provide all the necessary information (the text of the review). * Task: State the main goal. * Rules/Criteria: Add a separate block with your positive rules, which will steer the generation.
This method turns the user from a mere requester into the "architect" of the answer, building "guardrails" that keep the model from going off track.
Practical Applicability Analysis:
Direct applicability: A user can start using this method immediately. Just add a block titled
QUALITY CRITERIA:orRULES TO FOLLOW:to your prompts and list constraints there based on the "typical mistakes" for your task. It works in any chat with no technical skills at all.Conceptual value: The research gives the user a fundamental understanding: an LLM isn't a "thinker" but an "executor." It works better when given clear boundaries and constraints rather than abstract goals. This helps you get past the frustration of "why doesn't it understand me?" and move to a more engineering-minded approach to writing prompts. The key idea: control the process, don't just demand the result.
Adaptation potential: The method is universal. The user just needs to replace "interviewer mistakes" with mistakes relevant to their own task.
- For writing an email: Mistakes: "informal tone," "no clear call to action," "grammar errors." Rules → "Keep a professional style," "End the email with one clear question or call to action," "Check the grammar carefully."
- For building a content plan: Mistakes: "repetitive topics," "no tie-in to current events," "too complex for the audience." Rules → "Suggest 3 different content formats (article, video, post)," "Include 1-2 topics tied to current news," "All topics must be understandable to a beginner in the field."
A Practical Example:
You are an experienced customer success manager at the online electronics store "TechWorld." Your task is to respond to negative reviews while keeping the customer's loyalty and protecting the store's reputation.
**CONTEXT:**
Here is a review a customer left on our website:
**CUSTOMER REVIEW:**
"I ordered a new smartphone from you. The courier was three hours late and didn't even apologize! The phone itself seems to work, but the box was crushed like someone had been playing soccer with it. For that kind of money, the service is just awful! I'm never buying anything from you again."
**TASK:**
Write a polite, empathetic and professional reply to this review.
**REPLY QUALITY CRITERIA (follow them without exception):**
1. **Show empathy:** Start with a sincere apology for the bad experience. Acknowledge our mistakes.
2. **Don't be a robot:** Avoid boilerplate phrases like "Your opinion is very important to us." Write like a real, caring person.
3. **Don't make excuses:** Don't explain why the courier might have been late. Just acknowledge that this kind of service is unacceptable.
4. **Offer a concrete solution:** Don't stop at apologies. Offer the customer a small bonus (for example, a promo code for their next purchase) as compensation.
5. **State the next step:** Clearly explain how the customer can get this bonus (for example, "we've already sent a promo code to your email...").
Why This Works:
This prompt is effective because it directly applies the research's "steering through constraints" method:
* Role and Context set the right frame for the model.
* The REPLY QUALITY CRITERIA block is the heart of the prompt. Instead of just saying "write a good reply," we listed the specific "failures" to avoid (being a robot, making excuses, offering no solution) and turned them into positive, actionable rules.
* Rule #2 ("Don't be a robot") prevents soulless, canned brush-offs.
* Rule #3 ("Don't make excuses") blocks the model's defensive reflex and makes it focus on the customer.
* Rules #4 and #5 ("Offer a solution" and "State the step") ensure the reply is not only empathetic but constructive, which is the ultimate goal of good customer service.
Another Practical Example
You are a creative copywriter who comes up with ideas for short ad videos (up to 30 seconds) for social media.
**CONTEXT:**
Our product is a "Smart Notebook" that syncs handwritten notes to cloud storage through a smartphone app. The target audience is students and young professionals.
**TASK:**
Come up with 3 different concepts for a short ad video. Describe each concept in 2-3 sentences (scene, action, key message).
**RULES FOR THE CONCEPTS (follow them without exception):**
1. **Focus on the problem:** Each video must open with a relatable problem (for example, you lost an important note, spilled coffee on your notes, can't find the lecture you need).
2. **No jargon:** Don't use the words "sync," "cloud storage," "integration." Speak the user's language: "all your notes are on your phone," "save it forever."
3. **Show, don't tell:** Describe the visuals instead of just listing features. Instead of "the notebook has a search function," describe a scene where the hero instantly finds a handwritten formula on their phone.
4. **A clear call to action:** Each concept must end with a clear call (for example, "Learn more at the link in the description!", "Write smarter with our notebook!").
5. **Variety:** All three concepts must differ in mood (for example, one funny, one about productivity, one about calm and order).
Explaining the Mechanism: Why This Example Works.
This prompt works on the same principle as the previous one, adapted for a creative task. It prevents the typical mistakes in creating ad concepts: * Rule #1 ("Focus on the problem") makes the model create relevant content that "hooks" the viewer from the first seconds, instead of a boring product description. * Rule #2 ("No jargon") keeps it understandable for a broad audience and prevents a "techie" video that would scare off non-technical viewers. * Rule #3 ("Show, don't tell") is a classic principle of good storytelling. It steers the LLM toward visually rich ideas that suit the video format, rather than dry text. * Rules #4 and #5 ensure practical value and variety, making the model's output immediately ready for discussion with the team.
So the prompt doesn't just ask "come up with an ad"; it creates a "creative corridor" within which the model can generate high-quality, relevant ideas while avoiding the most common failures.
Usefulness Score: 96
Main Evaluation Criteria
- A. Relevance to prompting techniques: Very high. The research directly provides prompt templates (Fig. 1), describes specific techniques (persona, one-shot examples, positive phrasing) and explains why they work (section V.A).
- B. Improving the quality of conversational answers: Very high. The main finding of the research: guided question generation (with a list of "mistakes to avoid") beats human-written questions in quality (clarity, relevance, informativeness).
- C. Direct practical applicability: Maximum. All the proposed methods (role assignment, structuring, listing criteria) can be used by any user in any chat interface (ChatGPT, Claude, etc.) without a single line of code.
- D. Conceptual value: High. Reveals key features of LLM behavior: sensitivity to formatting (capitalized roles), trouble with negations, improvement from step-by-step instructions and examples. Gives the user a powerful "mental model": instead of asking it to "do well," clearly define "what bad looks like" and forbid it.
- E. New useful practice (clustering): The work falls into several key clusters at once:
- #1 (Phrasing techniques): Explicitly demonstrates persona-based prompting and one-shot learning.
- #2 (Behavioral patterns): Reveals sensitivity to formatting (capitalization) and problems with handling negations.
- #3 (Structure optimization): Shows the effectiveness of clear role separators (INTERVIEWER/INTERVIEWEE) and lists of criteria.
- #7 (Reliability and stability): The whole mistake-guided generation method is an extremely powerful technique for raising the reliability and quality of answers.
- Practicality checklist (+15 points): Yes, the work provides ready-made constructions, shows how to structure requests, reveals non-obvious LLM behavior and offers ways to improve the accuracy and quality of answers.
Numerical Usefulness Score
Arguments for a score of 96: This work is a real gem for prompt-engineering practitioners. It doesn't just offer an abstract idea; it proves its effectiveness in a controlled experiment and, most valuable of all, reveals concrete mechanics for implementing it in a prompt. The key takeaway, "steering through constraints" (or "mistake-guided generation"), is a universal and extremely powerful principle. The user gets not just a tip but a whole methodology: define the failure criteria for your task, phrase them as positive rules and put them in the prompt. This immediately raises the quality and consistency of results.
Counterarguments (why the score could be lower/higher): * Why not 100? The main domain of the research (requirements gathering in IT) is fairly specialized. An everyday user who wants to write a social media post or plan a trip will have to put in a bit of mental effort to adapt "interviewer mistakes" to their task. The methodology is universal, but the examples aren't. * Why not below 85? Despite the narrow domain, the findings and techniques described in section V.A ("Prompts Design") are completely universal. Sensitivity to capitalization, positive phrasing of instructions, using one-shot examples: this is fundamental knowledge that will improve any complex prompt, regardless of the task. The practical value of these insights outweighs the specificity of the main experiment.