TL;DR
HACHIMI-II is a way to get a model to convincingly portray one specific student. The model doesn't play a role; instead it predicts, in the third person, what this person will do next. It gets there in four steps: it builds a profile of the student from their history, assesses their state on the current question (what they know, where they'll slip, how confident they are), turns that state into hard constraints, and only then writes the answer.
If you just say "answer like a weak student," you get competence leakage. The model knows the right solution, so it either solves the problem correctly despite the role or drops an error in some random spot. The result: every "student" looks the same. The weakest third of real students solve 22% of the tasks, while the role-playing model portrays them solving 78–83%. The root cause is that "pretend you don't know" asks the model to suppress what it knows. It can't do that, and the role never says where exactly this person is going to trip.
The method flips the task: "predict where exactly this student will go wrong" requires knowing the correct solution, which the model has. Four steps: profile → state → constraints on the answer → generation with a check. The piece that matters most is the explicit state-assessment step: without it, the effect nearly vanishes.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1: Student profile (3rd person) → from answer history: level, habits, confidence, style
STEP 2: State on THIS question → 4 fields: reachable knowledge, active error, confidence, move (attempts / asks for help)
STEP 3: Behavior frame (harness) → hard rules for the answer: depth, exactly where the error goes, length, degree of doubt
STEP 4: Generate the answer → strictly inside the frame; if it's violated, regenerate
In the original, steps 1, 2 and 4 are separate model calls and step 3 is a fixed rule with no model involved. In a chat you can run them back to back in one conversation or as a single structured prompt (see the template).
Example
Task: An online learning platform like Khan Academy or IXL is testing an AI algebra tutor. They need to see how it handles one specific 7th grader, Leo. He's generally sharp, but when expanding brackets he keeps forgetting to multiply the second term. A "weak student" role produces either perfect answers or mistakes in random places, so the tutor ends up training on an unrealistic "student."
Prompt:
You are not playing a student. You are predicting the behavior of a specific person in the third person. Work strictly step by step and output the result of each step.
<StudentModel>
Leo, 7th grade. History: 68% correct over the last 25 tasks. Linear equations: 85%. Expanding brackets with a sign or a factor: 30%; most often he multiplies only the first term. Writes briefly, all lowercase, no punctuation. Unsure of himself; on hard items he answers cautiously ("maybe", "right?"). Rarely asks for help, usually just tries quietly.
</StudentModel>
<Context>
Tutor's question: "What do you get if you expand the brackets in 3(n − 4)?"
Previous turns: Leo earlier wrote: "ok so we gotta do something with the 3".
</Context>
STEP 2. Assess Leo's state on THIS question (fields):
- reachable_knowledge: which prerequisites he has mastered, how far he can get
- active_error_pattern: will he make a mistake; if so, at which step and by what mechanism
- confidence_state: how firmly he will give his answer
- local_move: will he try to solve it or ask for help
STEP 3. Turn the state into a behavior frame (hard rules for the answer):
- which knowledge he is allowed to use
- reasoning depth (how many steps, is there a full derivation)
- if an error is predicted: the answer must contain exactly one specific error at the predicted step
- degree of uncertainty
- format and length (number of sentences)
STEP 4. Write Leo's reply strictly inside the frame. Then check: is any item of the frame violated? If so, rewrite.
At the end, output only Leo's reply.
Result: The model will show four blocks: the filled-in state (with the predicted error "multiplied only 3·n"), the list of frame rules, and a draft reply. Then it runs a self-check and returns a short reply in Leo's style with one specific error at the right step. The answer comes out as one or two lines, hesitant, with no perfect solution.
Why It Works
The LLM weakness. The model is trained to be correct and helpful. When asked to "play someone weak," it has to know the solution and not use it at the same time. That doesn't work: the model's competence leaks through. A role names a level but doesn't say what exactly this person knows and where they'll break. So the model fills the gap with its own knowledge and a generic picture of "a student."
The strength. Models are good at prediction and analysis. To say where a person will drift from the correct solution, you need to know that solution. So the model's knowledge turns from an obstacle into a tool. It can reason about errors in the third person ("he'll multiply only the first term") and follow strict constraints on the answer format.
How the method connects the two. It splits the task into "decide where the error is" (analysis) and "write the answer with that error" (execution). The explicit state locks in the decision before generation, and the frame stops the model from fixing the error "out of habit." Ablations showed that the state is the main source of the gain, the frame keeps the answer consistent with the state, and the persona sharpens the differences between students.
Control knobs: - Student history → the more concrete data (percent correct by topic, typical mistakes), the sharper the profile. Without history the method is barely different from a role. - State fields → add your own (motivation, fatigue, fear of mistakes) to fit your task. - Frame rules → "exactly one error," "no more than three sentences" set the level of realism. Loosen them for livelier, less deterministic replies. - Check step → keep regeneration if the error must land exactly in the right spot.
Prompt Template
You are not playing a role. You are predicting the next action of a specific person, describing them in the third person. Do the steps in order and show the result of each.
<StudentModel>
Who they are: {profile}
What their history shows: {answer_history_and_typical_errors}
Speech and behavior style: {style}
Confidence and habit of asking for help: {confidence_and_help}
</StudentModel>
<Context>
Current task or the other person's message: {current_question}
Visible conversation history: {dialogue_history}
</Context>
<LearnerState>
STEP 2. Determine the person's state at THIS step:
- reachable_knowledge: which prerequisites are mastered, how far they can get in their reasoning
- active_error_pattern: will they make a mistake; if so, at which step and by what mechanism
- confidence_state: how firmly they will give the answer
- local_move: will they try to solve it themselves or ask for help
</LearnerState>
<BehaviorHarness>
STEP 3. Turn the state into rules for the answer:
- knowledge: what the person can use and what they can't
- depth: how many reasoning steps; is a full derivation allowed or not
- error: if an error is predicted, exactly one specific error at the predicted step; otherwise, no errors
- uncertainty: degree of doubt in the wording
- format: number of sentences, style
</BehaviorHarness>
<Generation>
STEP 4. Write the answer strictly within the rules.
Check: is any item of the BehaviorHarness violated? If so, rewrite.
At the end output: "Answer:" and only the reply.
</Generation>
Fill in:
- {profile}: who this is (age, level, role);
- {answer_history_and_typical_errors}: real data; the more specific, the better;
- {style}: how the person writes;
- {current_question}: the task they're getting right now.
🚀 Quick start: paste into the chat:
Here's the HACHIMI-II template for simulating a specific person by predicting their actions. Adapt it to my task: [your task, who you're simulating and why].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask about the person's history and typical mistakes, their style and confidence, and what task they're getting right now. That data feeds the "state" step: without specifics the model slides back to a generic picture. It will take the structure from the template and assemble a ready-to-use prompt for your case.
Limitations
⚠️ You need the history of a specific person: without it the method is barely different from a plain role. With no answer history, the model can't tell students apart.
⚠️ Errors are reproduced only partially: even the best variant doesn't always hit the real error with the same knowledge type, location and mechanism. It's far better than roles, but not an exact copy of the person.
⚠️ Tested mostly on math and one model: the data is school problems and a science exam, and the answers were generated by one large model. Transfer to other subjects and models is plausible but unproven.
⚠️ Pricier than a plain role: several sequential calls plus regeneration when the frame is violated. For a one-off task it's overkill.
⚠️ A high "average match" is misleading: the variant without the state step is good at guessing common correct answers. Yet it barely reproduces real errors. Evaluate on the wrong answers specifically.
How They Tested It
The researchers used five real-world datasets. Three are task solutions by actual students (Chinese 3rd-grade problems, Eedi diagnostic questions, Brazil's ENEM exam). Two are real student–tutor dialogues (Eedi QATD2k and Khan/CoMTA). Constraints and state were built by a single large model, Qwen3.5-397B. The method was compared with three or four variants on the same model: a plain role, a role with constraints in the prompt, a role with a student profile, and, for dialogues, also an agent with memory.
The evaluation is set up cleverly. For tasks, they checked whether the accuracy distribution of simulated students matches the real one (Wasserstein distance). For dialogues, they checked whether the simulator reproduces the error itself: the same knowledge type, location and mechanism. The judge was GPT-5.4, whose agreement with three human annotators was verified in advance.
Result: the ability distribution on tasks is 2.4–8.4× closer to reality. The weakest third of ENEM students solve 22% in reality, 78–83% under the role-play baselines, and 15% under HACHIMI-II. In dialogues, real errors are reproduced 1.8–5.1× more often. The baselines answered correctly in 82–93% of the turns that should have been wrong.
The key finding came from ablations (switching off one component at a time). Without the state step, everything snaps back to the model's own level: on ENEM average accuracy jumps from 0.38 to 0.80. It's not that the method gets more data about the student: the model still received the profile and persona. What matters is the explicit prediction of the state before answering. One more surprise: putting constraints directly in the prompt (Prompt-C) did almost nothing unless they follow from the predicted state.
Adaptations and Extrapolations
🔧 Technique: drop the check requirement → faster, but rougher. If you need a quick draft, keep steps 1–3 and let the model write the answer right away without self-checking. The frame still holds back the "leak," but it lands the error in the right spot less often.
Extrapolation: simulating a customer to test a support bot. The "predict, don't play" principle carries over to shoppers. The paper's method wasn't tested on this; it's an idea built on its logic.
You are not playing the customer. You are predicting the next message of a specific online-store customer, describing her in the third person.
<CustomerModel>
Customer: Megan, 38, has ordered on Amazon 14 times, 2 returns. Writes briefly, gets irritated by canned replies, quickly escalates to a manager if she doesn't get specifics.
</CustomerModel>
<Context>Bot's reply: "Your order has been handed over to the carrier. Please wait."</Context>
STEP 2. State: what she'll understand, which objection is active, how persistent she is, will she keep chatting or demand a human.
STEP 3. Frame: length, tone, exactly one objection, level of irritation.
STEP 4. Write Megan's message strictly inside the frame; check for violations.
Resources
- HACHIMI-II: From Role-Play to Learner Behavior Modeling for Faithful Student Simulation: Yilin Jiang, Fei Tan, Xuanyu Yin, Jing Leng, Aimin Zhou. East China Normal University; Hong Kong University of Science and Technology (Guangzhou); Shanghai Innovation Institute.
- Related work cited in the paper: HACHIMI (Jiang et al., 2026), a corpus of a million student personas; Yuan et al., 2026 (the "competence paradox"); Song et al., 2026 ("expert knowledge leakage"); Scarlatos et al., 2026 (limited fidelity of role-play simulators).
- Datasets: XES3G5M, Eedi, ENEM 2024, Eedi QATD2k, Khan/CoMTA.