Logical Obfuscation: Testing Real Understanding with Logically Equivalent Rewordings

STEP 1: Ask the question in its standard form → get answer A

STEP 2: Reword it in a logically equivalent way 
        (via negation / indirect relations / symbol swaps)
        → get answer B

STEP 3: Compare answers A and B
        ┣━ They match → the model understands the logic
        ┗━ They differ → the model is pattern-matching, NOT reasoning

Run all the steps in one chat session, or in two separate ones for a cleaner experiment.

Task: You're building a financial model for an investor pitch. You want to check whether ChatGPT actually understands your business logic or is just generating answers off a "sounds convincing" template.

Prompt 1 (direct form):

My service brings in $120K a month in revenue, 
operating expenses are $85K, growth is 50% per quarter. 
Is the company profitable?

Prompt 2 (obfuscated form):

It is false that my service is unprofitable 
if operating expenses of $85K a month 
do not exceed revenue of $120K with 50% quarterly growth. 
Is the company profitable?

Result:

The model should give an identical answer ("yes, it's profitable") to both questions, because they're logically equivalent. If it said "profitable" the first time but got confused or contradicted itself the second time, it doesn't understand the financial logic; it's reacting to the keywords "unprofitable"/"profitable" and the words around them. A mismatch = a red flag: the model is pattern-matching, not reasoning. For a high-stakes financial model, that's your cue to double-check the numbers by hand or ask for a detailed step-by-step explanation.

An LLM trains on billions of text examples and memorizes frequency patterns: which words go together, which phrasings are typical of answers. When a question is phrased the standard way ("A is B's father, how are they related?"), the model recognizes a familiar structure from its training data and gives the right answer. That doesn't mean it built an internal model of family relationships; it has just seen similar examples thousands of times.

A logically equivalent rewording changes the surface, not the substance. "A is B's father" = "A is the only son of B's grandfather": the same thing! But the second version rarely appears in training data. The model has never seen that pattern, can't slot in a familiar template, and starts making mistakes. The study showed that at every layer inside the model, a probability for the next token is computed. On a normal question that probability climbs toward the final layers (that's where the "reasoning" happens) and the model heads confidently to the answer. On an obfuscated question the probability drops by 50-80% in the later layers. The model literally gets lost inside and can't find a familiar path.

How to use it: if answer reliability matters to you (finance, legal wording, high-stakes decisions), ask the question twice in different forms. Same answer = the model understands the logic. Different answers = the model is guessing from keywords. You can also explicitly ask the model to explain its reasoning for both versions; if the logic falls apart on the second one, don't trust it. This is especially relevant for contracts (a single clause can be phrased with a double negative), technical specs and ROI calculations.

Control knobs: - Obfuscation type: negations (De Morgan's law), indirect chains ("mother" → "father's wife"), symbol swaps (numbers → planets → hashes), extra steps that cancel out - Number of rewordings: one for a quick check, 3-5 for high-stakes tasks (a million-dollar contract) - Requiring an explanation: "show your reasoning step by step" on both versions, and track where the logic breaks

Check your logic: answer the same question in two forms.

FORM 1 (direct):
{your_question}

FORM 2 (reworded, but logically equivalent):
{reworded_question}

Answer each question SEPARATELY. Show your reasoning for each.
Then tell me: do the conclusions match? If not, explain why.

Fill-ins: - {your_question}: the original wording of the problem - {reworded_question}: a logically equivalent version using: - Double negation: "it is false that it is NOT..." instead of a plain statement - Indirect relations: "father's wife" instead of "mother," "grandfather's only son" instead of "father" - Symbol swaps: "Planet Mars" instead of "4," "Venus" instead of "2" - Extra canceling moves: "go 5 km north, 3 km south, 2 km north" (net 4 km north) instead of "go 4 km north"

🚀 Quick start: paste this into the chat:

Here is my task: [describe the task]. 

Phrase it in TWO logically equivalent versions:
1. A direct, clear form
2. An obfuscated form (using negation, indirect relations or substitutions)

Then solve BOTH versions with step-by-step reasoning and compare the answers.
If the conclusions differ, explain where and why.

The LLM will write the rewording itself, solve both versions and check for consistency; if the answers differ, it will notice and point out the contradiction. You get a reliability signal: match = you can trust it, mismatch = double-check by hand or simplify the task.

⚠️ Some tasks are hard to reword: For abstract or creative questions ("how do I improve my homepage design?") it's not obvious how to build a logically equivalent version without changing the meaning. The method works for tasks with unambiguous logic: math, formal relationships, facts, contract terms.

⚠️ Subjective tasks produce legitimate differences: On creative and evaluative questions ("is this a good business idea?") different phrasings can legitimately shift the emphasis. "Is the idea profitable?" ≠ "Is the idea NOT unprofitable?": the second sounds more skeptical and can shift the tone of the answer. That's not a model bug, it's the nature of subjective tasks.

⚠️ Manual work or an extra call: If you don't ask the model to write the rewording (via the Quick start), you'll have to do it yourself. For complex multi-step logic that takes effort and a grasp of formal logic (De Morgan, contraposition).

⚠️ Not a guarantee of correctness: Matching answers on both versions do not guarantee the answer is right: the model can be wrong consistently on both (for example, not knowing the specifics of state tax rules). The method exposes unreliability (pattern matching), not incorrectness. High-stakes tasks still need an expert review.

Researchers from Indian universities (Manipal, IIT Patna, IISER Kolkata) assembled 1,108 questions split into 4 task types: formal logic (272 questions), blood relations (300), number sequences (284) and navigation (252). For each base problem they created a logically equivalent obfuscated version. In formal logic they applied De Morgan's law ("not (A and B)" → "not A or not B") and double negation ("A" → "not not A"). In blood relations they replaced direct relationships with indirect chains: "father" → "grandfather's only son" (two hops instead of one, same result). In number sequences they swapped numbers for planet names (2→Venus, 4→Mars), ASCII sums of letters or MD5 hashes. In navigation they added extra steps that cancel each other out (5 km north + 5 km south = 0).

They tested 9 models: GPT-4o, GPT-5, o4-mini (reasoning), Claude 3.7, Gemini 2.5 Pro, Qwen QwQ-32B (reasoning), GPT-4o-mini, Gemma-3-27B, LLaMA-4-Maverick-17B. Every task ran in three modes: zero-shot (no examples), few-shot (with worked examples) and chain-of-thought (with step-by-step reasoning required). They measured accuracy: the share of exact matches with the correct answer (normalizing for case and punctuation).

Why did the results come out this way? Obfuscation pulls the model out of its comfort zone of familiar patterns. The team looked inside the LLaMA model: they plotted the next-token probability at every layer (a model is a stack of ~30-40 layers, each one refining the prediction). On a normal question the probability rises layer by layer: the model heads confidently to the answer, and the final layers do the "reasoning" and pick the right token. On an obfuscated question the probability drops 50-80% in the later layers. The model literally "gets lost" inside, can't find a familiar pattern, and the final layers dampen the signal instead of amplifying it.

The surprise: even reasoning models (GPT-5, o4-mini, Qwen QwQ-32B), which supposedly think deeply and spend more compute deliberating, drop by 20-27%. Regular models (GPT-4o, Claude, Gemini) lose 40-50% accuracy. Meanwhile undergrad students solved the same obfuscated problems with 99% accuracy; to a human, logical equivalence is obvious. The takeaway: LLMs don't build a deep logical model, they lean on language statistics.

Practical insight: Chain-of-thought (CoT) helps somewhat: for GPT-5 the drop shrinks from 27% (zero-shot) to 18.7% (CoT), but it doesn't fully fix the problem. If the model fundamentally doesn't get the logic, "reason step by step" only masks the problem with nice-sounding words rather than fixing it. Testing via rewording is the only way to catch a model pattern-matching before it hands you a confident but wrong answer on a high-stakes task (a contract, a financial model, a technical spec).

🔧 Technique: hand the obfuscation off to the LLM → save time

Instead of rewording by hand, ask the model to create logically equivalent versions of the question:

Here is my task: {your task}.

Create 3 logically equivalent rewordings:
1. Using double negation (instead of "A is true" → "it is false that A is false")
2. Using indirect relations or swapped terms
3. Reordering the conditions (if possible)

Then solve all 4 versions (original + 3 rewordings) 
with step-by-step reasoning and compare the answers.
If the conclusions differ, point out where and why.

The model tests itself. If the answers diverge, it will notice and can flag the contradiction (or get even more confused, which is also a sign of unreliability).

🔧 Technique: combine with Chain-of-Thought → transparent reasoning

Answer the question in two forms:

FORM 1 (direct):
{direct form}

FORM 2 (obfuscated):
{obfuscated form}

For each, show your reasoning STEP BY STEP: 
which facts you used, which conclusions you drew.

Then compare the LOGIC of the two lines of reasoning: 
do the steps match? If not, where did they diverge and why?

This exposes divergence not just in the answers but in the reasoning process too. A model can stumble onto the same answer by two different routes (one correct, one flawed), and CoT will show you that.

🔧 Technique: multi-round checks for high-stakes decisions

For million-dollar contracts, technical specs with serious risk and legal wording:

Check this clause in 5 logically equivalent versions:
1. A direct statement
2. Via negation
3. Via contraposition (if A→B, then not-B→not-A)
4. Using synonyms for the key terms
5. Split into two conditions

For each version, write:
- What it means in plain words
- What obligations it creates
- What risks you see

Then compare all 5 interpretations: are there contradictions?

If even one interpretation diverges, there's a high risk of ambiguity, and you need an expert legal review or a rewrite of the clause.

Don't Judge a Book by its Cover: Testing LLMs' Robustness Under Logical Obfuscation
Logifus (obfuscation framework): https://github.com/abhilekhborah/logical-obfuscation
LogiQAte (dataset of 1,108 questions): https://huggingface.co/datasets/abhilekhborah/LogiQAte

Abhilekh Borah, Shubhra Ghosh, Kedar Joshi, Aditya Kumar Guru, Kripabandhu Ghosh
_Manipal University Jaipur, Indian Institute of Technology Patna, Indian Institute of Science Education and Research Kolkata_