A fact-checking comparison is useful only when the test can separate a polished answer from a well-supported one. For this Siftano hands-on evaluation, we gave ChatGPT and Claude the same synthetic source packet, the same fixed prompt, and the same 12 questions. We preserved the first response only and scored every case against an answer key written before the product runs.
The result was deliberately narrow:
- ChatGPT: 24/24
- Claude: 23/24
- both products reached the correct final answer on all 12 cases
- neither used a superseded value as the final answer
- neither invented an unsupported retention period or mobile-app result
- neither made a calculation error
- the only scored difference was source-attribution completeness in F1-07
This does not establish that ChatGPT is generally more accurate than Claude. It is evidence about one controlled source-grounded verification fixture, run once per product on 2026-08-30.
What we tested
F1 uses a fictional Northstar Workspace Pilot source packet created only for evaluation. It contains a charter, budget memo, early evaluation summary, correction notice, final addendum, and operations note. No real customer or company data is used.
The 12 cases target practical verification failures:
- keeping an earlier number after a later correction
- calculating a percentage from the wrong denominator
- confusing a budget ceiling with actual spend
- dropping one condition from a multi-condition rule
- inventing a policy that the source never states
- inferring that a native mobile app was tested when only desktop web is documented
- giving an incomplete structured source list even when the conclusion is correct
Because the fixture is synthetic and fixed, the result does not depend on changing web information.
Test conditions
Both runs used the same Windows machine and Codex in-app browser environment.
| Condition | ChatGPT | Claude |
|---|---|---|
| Plan | Pro | Pro |
| Model/version visible in UI | Not shown | Sonnet 5 · 중간 |
| Session | Temporary Chat | Incognito |
| Web search/research tools | Not used | Off |
| Attempts | 1 | 1 |
| Follow-up messages | 0 | 0 |
| Retries | 0 | 0 |
The source file SHA-256 was identical in both runs:
A5E6C0E592D00CAE345858C9A1591205AF1F346D516BDB591673CA49EB66784B
The fixed prompt SHA-256 was also identical:
C826AC81128CD0A4759CD109A60D0250B9A07AF1BF00C787E6F50FBD1B4C61BF
We did not infer the ChatGPT model from the account type, response style, or interface. The exact model was not shown in the tested UI, so it remains explicitly unknown.
Scoring rubric
Each case was scored independently.
| Score | Meaning |
|---|---|
| 2 | Final answer, source selection, and required calculation or reasoning are correct |
| 1 | Final answer is correct, but source attribution, calculation detail, or qualification is incomplete |
| 0 | Incorrect answer, stale value treated as final, unsupported claim invented, or required unsupported state not recognized |
Maximum score: 24.
We also checked six failure flags: STALE_VALUE_USED, UNSUPPORTED_CLAIM_INVENTED, SOURCE_ID_INVENTED, CALCULATION_ERROR, CONDITION_DROPPED, and FORMAT_CONTRACT_BROKEN. Neither product triggered any of them in this run.
Results by case
| Case | What the case checks | Expected result | ChatGPT | Claude |
|---|---|---|---|---|
| F1-01 | corrected eligible enrollment | 84 | 2 | 2 |
| F1-02 | unchanged completion count | 71 | 2 | 2 |
| F1-03 | corrected completion rate | 84.52% = 71/84 | 2 | 2 |
| F1-04 | corrected escalation count | 96 | 2 | 2 |
| F1-05 | corrected escalation rate | 3.98% = 96/2,410 | 2 | 2 |
| F1-06 | actual spend vs budget ceiling | USD 15,480 | 2 | 2 |
| F1-07 | all-condition success rule | No; severity-1 incident occurred | 2 | 1 |
| F1-08 | severity-1 incident count | 1 | 2 | 2 |
| F1-09 | human approval before external release | Yes | 2 | 2 |
| F1-10 | unsupported retention period | NOT SUPPORTED | 2 | 2 |
| F1-11 | unsupported native-mobile test claim | NOT SUPPORTED | 2 | 2 |
| F1-12 | enrollment cap vs actual enrollment | 90 vs 84; not the same | 2 | 2 |
| Total | 24/24 | 23/24 |
Claude's F1-07 conclusion was factually correct. The one-point difference is about the completeness of the structured source field.
The only scored difference: F1-07
F1-07 asked whether the pilot satisfied the charter's complete success definition and required the decisive reason. The charter required all three conditions: at least 80% completion, factual-error escalation below 4.0%, and zero severity-1 security incidents.
The corrected completion rate was 84.52% and the corrected escalation rate was 3.98%, so the first two conditions passed. One severity-1 incident occurred, so the pilot did not meet the complete success definition. Both products reached that conclusion.
ChatGPT
ChatGPT's structured source_id field listed A1,E1. A1 defines the three-condition rule; E1 records the corrected final result and the incident. That source selection received full credit.
Claude
Claude's structured source_id field listed only E1. Its reasoning, however, explicitly said that all three conditions were required and referenced A1. The reasoning therefore used the charter source while the structured citation field omitted it.
Under the prewritten rubric, the final conclusion was correct but the source attribution was incomplete, so F1-07 received 1/2.
This distinction matters when a person or downstream system consumes a structured source field without rereading the entire explanation.
What both products handled correctly
They rejected superseded values
The early draft used 86 enrolled participants, 82.6% completion, 102 escalations, and a 4.23% escalation rate. Later sources corrected enrollment to 84 and removed six duplicate escalation tickets. Both products used the corrected final values.
They recalculated from corrected inputs
Both products reached 71 / 84 × 100 = 84.52% and 96 / 2,410 × 100 = 3.98%. No calculation-error flag was triggered.
They separated a budget ceiling from actual spend
The procurement memo authorized up to USD 18,000, while actual invoiced software spend was USD 15,480. Both products returned the actual spend.
They preserved unsupported states
The packet did not define a vendor data-retention period and did not state that a native mobile application was tested. Both products returned NOT SUPPORTED rather than filling the gaps with a plausible guess.
For verification work, knowing when not to answer is part of accuracy.
What this test does not prove
F1 is one synthetic fixture. It does not measure coding, open-web research, creative writing, agentic tool use, or general reasoning quality.
Each product received one first attempt. We intentionally did not test whether follow-up prompting could repair an error.
Product behavior also changes. This result is tied to the recorded conditions on 2026-08-30. Claude displayed Sonnet 5 · 중간; ChatGPT did not display an exact model name in the tested interface.
Finally, the one-point difference is a citation-structure issue, not a wrong final answer. Turning 24/24 versus 23/24 into a broad claim that one product is categorically more factual would exceed the evidence.
Practical takeaway
In this fixture, both products produced strong first-attempt source-grounded answers. The meaningful difference appeared in whether every source actually used in the reasoning was also represented in the structured citation field.
For a verification workflow, check four things separately:
- Is the final statement correct?
- Did the answer use the newest applicable source?
- Were calculations made from the corrected inputs?
- Are all sources relied on in the reasoning also present in the structured citation output?
A natural-language explanation can satisfy the first three while still missing the fourth.
For a broader verification process, see How to Verify AI Answers, Sources, and Factual Claims.
Next tests
F1 alone is not enough for a general ChatGPT-vs-Claude recommendation. We will run independent fixtures that target different failure modes:
- F2 — long-document summary fidelity: corrections, omissions, exceptions, metric scope, and unsupported ROI claims
- F3 — structured CSV analysis: weighted aggregation, missing values, cost ranking, filtering, and arithmetic
A broader product judgment should wait until the same pattern is observed across multiple independent fixtures.