Multiscript Numeracy: How LLMs Trip Over Number Formats and What to Do About It

THE PROBLEM: - Standard format (922,436.38) → high accuracy - Different format/script → accuracy drops by 66-87%

FIXES (in order of effectiveness):

Technique 1: Mapping Add a correspondence table to the prompt: {Persian digit} = {Arabic digit}

Technique 2: Few-shot Show 2 example problems with numbers in the target format, along with their solutions

Technique 3: Full translation Translate the entire prompt (instructions included) into the language of the corresponding writing system

Combo: Mapping + prompt translation Gives the maximum effect, recovering up to 50-70% of the lost accuracy

Task: You're working with documents from your India office, where numbers are written as ₹9,22,436.38 (a comma after every two digits past the thousands, the standard for Indian rupees). You need to total three payments.

Prompt without techniques (low accuracy):

Solve:
9,22,436.38 + 4,359 + 12,847.50

Prompt with few-shot (high accuracy):

Numbers are in Indian format: a comma separates every 2 digits past the thousands, and a period is the decimal separator.

Examples:
Problem: 5,43,210.50 + 2,100
Answer: 5,45,310.50

Problem: 7,89,000 - 45,321.75
Answer: 7,43,678.25

Now solve:
9,22,436.38 + 4,359 + 12,847.50

Result: The model picks up the pattern from the examples: where the commas go, what the period means, what format to expect in the answer. Accuracy rises from ~30-40% to 70-90%. The model returns the number in the same format: 9,39,642.88.

The LLM's weakness: Models were trained on text where 72% of all numbers are in standard Hindu-Arabic format (0-9), 12% are Persian digits, 9% are Devanagari, and every other script is a fraction of a percent. Formats with a period as the decimal separator are the overwhelming majority. When the model sees ۱۲۳ or 9,22,436.38, it doesn't recognize it as a "number to compute with"; it simply has too few such examples in memory.

The LLM's strength: Models are excellent with patterns if you show them explicitly. Few-shot creates a local pattern in context: "here's what the numbers in the task look like → here's how to solve them". Mapping removes ambiguity: the model sees "۱=1", so ۱+۲=۳ turns into the familiar operation 1+2=3. Translating the full prompt activates the language context: the model switches into working with that language, where the non-standard digits become the standard ones.

How few-shot works here: Two examples are enough for the model to "see" the structure: which symbols are separators, which are digits, how to write out the answer. It works because models are trained to find patterns in context; few-shot hands them that pattern explicitly instead of hoping the model recalls a rare format from its training data.

Control levers: - Number of few-shot examples: 2 are enough for most tasks, 3-4 for complex formats (say, mixed systems). More isn't needed; redundancy adds nothing. - Full vs partial mapping: You can map only the digits that appear in the task (۱=1, ۲=2, ۳=3 for the problem "۱۲+۳") or all of them (۰-۹). Full is more reliable but longer. - Prompt language: If you translate the whole prompt, use simple instructions ("solve", "calculate"), not complex constructions. The model handles direct commands better.

Basic (mapping):

The digits are written in {system_name} (example: {digit_examples}).
Mapping: {digit_1}={arabic}, {digit_2}={arabic}, ... {digit_n}={arabic}

Solve:
{arithmetic_expression}

What to fill in: - {system_name}: Persian digits / Devanagari / Thai digits / Indian format - {digit_examples}: ۱۲۳ / १२३ / ๑๒๓ / 9,22,436.38 - {mapping}: a correspondence table, only for the digits in your task - {arithmetic_expression}: your problem with numbers in the non-standard format

Advanced (few-shot):

Numbers are in {format_description}.

Examples:
Problem: {example_1}
Answer: {solution_1}

Problem: {example_2}
Answer: {solution_2}

Now solve:
{your_problem}

What to fill in: - {format_description}: briefly describe what the symbols mean (comma = thousands separator, period = decimal separator / Persian digits where ۱=1, ۲=2) - {example_1, example_2}: two simple problems (addition/subtraction) with numbers in the target format - {solution_1, solution_2}: the correct answers in the same format - {your_problem}: the expression you need computed

🚀 Quick start: paste this into the chat:

Here is a template for working with numbers in a non-standard format. Adapt it to my task: [describe the task and the number format]. Create 2 few-shot examples for my format.

[paste the template above]

The LLM will ask: which format exactly (separators, grouping), which operations the task involves, what answer format you expect. It will create few-shot examples that match your pattern, because few-shot acts as a reference for the model, showing "here's how to handle this specific format".

⚠️ Doesn't work for low-resource scripts without mapping: For very rare writing systems (Adlam, Osmanya, Balinese, Javanese), models simply don't recognize the digits, even with few-shot. You need an explicit mapping for every digit. Without one, accuracy is close to zero.

⚠️ Multiplication and division suffer more: Models "pull off" addition/subtraction with non-standard formats better. Multiplication shows significantly more errors (a coefficient of -1.136 in the regression): long calculations with many intermediate steps break more often.

⚠️ More digits, worse results: Each additional digit in the operands lowers accuracy (a coefficient of -0.027 for multiscript, -0.154 for formats). For numbers longer than 6-7 digits in a non-standard format, always use few-shot or mapping.

⚠️ Gemini 2.5 Pro is especially weak: This model scores ~20-30 percentage points lower than the others on all non-standard formats. The reason: its tokenizer splits every digit separately, even in the standard format. For working with numbers, Claude or GPT-4o is a better bet.

⚠️ Context matters: Models mostly saw the format "922.436,38" (period for thousands, comma for decimals) in IP addresses, not in arithmetic. That's why accuracy is especially low for this format (a -2.259 coefficient): the model doesn't connect the pattern with computation. Few-shot is critical here.

Researchers from Kensho Technologies and MIT built a dataset of 336 basic arithmetic expressions (addition, subtraction, multiplication, division) with 4-8 digit numbers. Each expression was rendered in 21 writing systems (from widespread Persian and Devanagari to the very rare Osmanya and Adlam), for 7056 variants in total. Separately they created 6 format variations of the same problems: changing the decimal separator (period/comma/space), the group separator (comma/period/apostrophe), and the grouping pattern (3-3-3 vs the Indian 3-2-2).

They tested 9 models: four large ones (GPT-4o, Claude Sonnet 4.5, Gemini 2.5 Pro, Llama 3.3 70B) and five small ones (Qwen, Phi, Ministral, Gemma). The small models failed almost completely (accuracy near zero for most scripts), so the analysis focused on the large ones.

Why the results look like this: They analyzed the training corpora: 72% of all numbers are in standard Hindu-Arabic format, Persian 12%, Devanagari 9%, the rest fractions of a percent. Gemini failed harder than the others, and a tokenization analysis showed why: its tokenizer splits every digit separately (922,436.38 → 18 tokens), while GPT-4o groups them (922,436.38 → 9 tokens). The more tokens per number, the harder it is for the model to "keep the whole number in its head", hence the errors.

The surprise: Few-shot examples work disproportionately well: just two examples raise accuracy by 20-60%. That runs against the intuition that "the model should know numbers from training". It turns out the knowledge is there but gets activated by context: few-shot sets up the pattern "in this conversation we're working with this format", and the model switches over. The second surprise: translating the full prompt into the script's language (the task and instructions in Hindi for Devanagari) works better than mapping. That shows models learned not just "digits" but "digits in the context of a language": when the language and script line up, the right "processing mode" kicks in.

Practical insight: The problem isn't that models can't compute in other formats; they don't recognize a non-standard pattern as a number. The fix is to give a reference in context (few-shot) or remove ambiguity (mapping). This works for any underrepresented pattern, not just numbers.

1,729 vs. १७२९: The Effect of Scripts and Formats on LLM Numeracy Varshini Reddy, Craig W. Schmidt, Seth Ebner, Adam Wiemerslage (Kensho Technologies, Cambridge, MA), Yuval Pinter (Ben-Gurion University, Beer Sheva, Israel), Chris Tanner (Kensho Technologies & MIT, Cambridge, MA)