Structured Evaluation Framework: Personalized Grading of Work with ChatGPT

Single-step prompt:

INPUT:
1. Assignment (assessment brief)
2. Grading criteria (rubrics) — a checklist of requirements
3. Student's/employee's work (submission)

OUTPUT (structured):
- Strengths: what was done well
- Shortcomings: mistakes + how to fix them
- Score: X out of Y points
- Justification: why this score

All in a single ChatGPT request.

Task: A corporate blog editor is grading a copywriter's article about the company's new product.

Prompt:

Assignment: Write a corporate blog article about a new product launch. Requirements: grab attention, explain the value for the customer, include a call to action.

Grading criteria:
- Catchy headline (10 points)
- Clear explanation of the product's benefits (30 points)  
- Use cases or customer stories (20 points)
- Clear call to action (20 points)
- Style and grammar (20 points)
Maximum: 100 points

Article text:
[paste the copywriter's text]

Response format:
- Strengths: what the article does well
- Shortcomings: specific weaknesses with an explanation of how to improve
- Score: X out of 100 points
- Justification: why exactly this score

Result: ChatGPT returns a structured assessment in blocks. Under Strengths it lists what works (e.g., “strong headline, benefits well described”). Under Shortcomings it flags specific problems — “no use cases, the call to action is vague (write a concrete ‘Sign up for a free trial’ instead of ‘learn more’)”. Score is a number broken down by criteria. Justification explains the logic: why points were deducted and what outweighs what (strengths vs. flaws).

The LLM's weakness without structure: Ask ChatGPT to “grade this” and it drifts into generic phrases and inconsistency. Without explicit criteria, the model invents its own, and they change from request to request. Without a required format, it produces a stream of consciousness where strengths and weaknesses are mixed together and the score's rationale is fuzzy.

The LLM's strength: The model follows structured instructions extremely well when they're spelled out. Rubrics work as a checklist — the model goes through each item and assesses it. A fixed output format (Strengths / Shortcomings / Score / Justification) removes ambiguity — ChatGPT knows where each part of the assessment goes.

The mechanics: You give the model three anchors — what the work should contain (the assignment), what to grade on (rubrics), and what to grade (the submission). The output format separates praise from criticism and facts from opinions. That forces the model to balance — it can't write only about flaws (there's a Strengths block) or only praise (there's a Shortcomings block). Justification raises quality — the model explains its conclusion, which reduces randomness in the score.

Control knobs:

• Rubric detail — the more specific the criteria (with points for each), the more objective the score. You can add weights: “main criterion X — 50 points, the rest 10 each.”
• Shortcomings format — you can demand not just “what's wrong” but “what's wrong + concrete steps to fix it” or “prioritize issues from critical to minor.”
• Score granularity — you can ask for an overall score or a breakdown by each criterion (so you see exactly where it falls short).
• Tone in Strengths — you can add “no generic phrases, only concrete examples from the work” to avoid boilerplate compliments.

Assignment: {task_description_or_requirements}

Grading criteria:
- {criterion_1} ({points_or_weight})
- {criterion_2} ({points_or_weight})
- {criterion_3} ({points_or_weight})
Maximum: {total_score}

Work to grade:
{work_text_or_document_link}

Response format:
- Strengths: what was done well (with concrete examples from the work)
- Shortcomings: weaknesses and how to fix them (concrete steps)
- Score: X out of {total_score}
- Justification: why this score, breakdown by criteria

How to fill it in:

• {task_description} — what the author was supposed to do (e.g., “write a quarterly sales report,” “create a use case diagram for an ordering system”)
• {criterion_N} — specific requirements for the work. The more detailed, the better. Add points or weights if you need a numeric score.
• {work_text} — copy the text or paste a link to the document (if ChatGPT can read it)
• {total_score} — the maximum on your scale (100, 10, any number)

You can drop the Score block if you don't need a grade — keep just Strengths + Shortcomings for pure feedback without numbers.

⚠️ Hallucination risk: ChatGPT can invent elements that aren't in the work if the criteria are too abstract. The fix: make rubrics as specific as possible (“includes three examples” instead of “enough examples”).

⚠️ Needs validation: ChatGPT doesn't replace a human expert, especially in highly specialized domains (medicine, law, advanced engineering). Use it as a first pass; the final call belongs to the expert.

⚠️ Rubric quality = grading quality: If the criteria are vague or incomplete, ChatGPT fills the gaps with its own assumptions. Garbage in, garbage out. Invest time in clear criteria — they're the foundation of the whole method.

⚠️ Subjective criteria: For criteria like “creativity” or “emotional impact,” ChatGPT's grading is less reliable than for objective ones (presence of elements, format compliance, grammar).

The researchers took 23 students from a Software Architecture course (who gave consent, out of 290) and graded their work two ways: manually (4 instructors) and with ChatGPT-4o using a zero-shot prompt. The assignment included text + diagrams (use case, class, 3-tier architecture) — an interesting case for testing multimodality.

Experiment design: They gave ChatGPT the student's assignment, the rubrics, and the work itself, and asked for structured feedback (strengths / shortcomings / score / justification). Then they showed the results to the instructors and asked them to rate their confidence in ChatGPT's assessment on a 1-5 scale.

The surprise: Instructors gave ChatGPT an average confidence of 4 out of 5 (ratings: 4, 5, 4, 3). That's a high level of trust. On top of that, ChatGPT's feedback turned out more detailed than the humans' — instructors wrote briefly (“Good attempt, but incomplete coverage”), while ChatGPT gave concrete examples and paths to improvement.

Key insight: ChatGPT didn't just save time — it raised the quality of feedback through enforced structure. Instructors often skip spelling out details (especially with large classes); ChatGPT doesn't. This shows that the right prompt structure matters more than how “smart” the model is.

What it means in practice: The approach works without fine-tuning, right in the web interface. So it applies to any domain that has grading criteria. The study's limitation is a small sample and a single domain (Software Architecture), but the principle is universal: give criteria + demand structure = get quality feedback.

Personalized and Constructive Feedback for Computer Science Students Using the Large Language Model (LLM)
Javed Ali Khan, Muhammad Yaqoob, Mamoona Tasadduq, Hafsa Shareef Dar, Aitezaz Ahsan
University of Hertfordshire, UK; Information Technology University, Lahore, Pakistan; University of Gujrat, Pakistan