A CSV analysis comparison is easy to make look decisive by asking only arithmetic questions with complete data. F3 deliberately included one missing cost value and required weighted rates, team filtering, ranking, and reproducible calculations.
On the preserved outputs, the numerical result was a tie:
- ChatGPT recovery run: 24/24
- Claude preserved first run: 24/24
- every one of the 12 expected answers matched the fixed answer key
- neither product treated the blank cost as zero
- neither called USD 1,343 a complete dataset cost
- neither included the incomplete Epsilon team in the full-period cost-per-success ranking
- both used weighted rather than unweighted success rates
The important qualification is the run history: this is not a clean first-attempt-versus-first-attempt comparison.
What we tested
F3 uses a synthetic CSV with 18 rows covering six teams across three months. Every request, success, review-minute, and incident field is populated, but one cost_usd cell is intentionally blank: Epsilon in February 2026.
The fixed prompt asks 12 questions covering:
- total requests and successes
- overall weighted success rate
- highest team weighted success rate
- highest incident count
- cost per successful request for teams with complete cost data
- the sum of known costs without pretending it is complete
- identification of every missing cost row
- March-only weighted performance
- review minutes per successful request
- incident rate per 100 requests
It also requires a short Data quality section that explicitly mentions missing values affecting cost analysis.
Test conditions
| Condition | ChatGPT | Claude |
|---|---|---|
| Preserved run date | 2026-08-30 | 2026-08-31 |
| Plan shown in UI | Pro | Not shown |
| Model/version shown in UI | Not shown | Sonnet 5 · 중간 |
| Web search or research | Off | Off |
| Built-in data analysis observed | Yes | Yes |
| Follow-up messages | 0 | 0 |
| Retry count for scored preserved output | 1 authorized recovery | 0 |
The CSV SHA-256 was identical:
3DD5D56F100DA9B98A02FD363701EE30CBA0EBEBE87A3C4C3F531C5605C0B5B8
The fixed prompt SHA-256 was identical:
5986629BCAA35F39E7546C94E1D0BBE039572D8BAC73BEA9954365C8DAAF9245
Why the ChatGPT result is labeled a recovery run
The original ChatGPT response completed, but the Temporary Chat tab was lost before the raw response and screenshots could be preserved. We did not score an invented reconstruction of that output.
A recovery rerun was explicitly authorized. It used the same CSV and exact fixed prompt, and the recovered raw response and screenshots were preserved. The 24/24 ChatGPT score refers to that recovery output.
Claude's scored output was preserved from its first completed run after a previous product usage-limit blocker had reset.
Because the preservation histories differ, F3 should not be described as a first-attempt race even though both scored outputs are valid hands-on evidence.
The 24-point rubric
Each case is worth two points:
| Score | Meaning |
|---|---|
| 2 | answer, arithmetic, and required data-quality qualification are correct |
| 1 | final result is correct but a calculation or material qualification is incomplete |
| 0 | wrong result, wrong filter, unweighted rate, missing cost treated as zero, or unsupported complete-total claim |
Results by case
| Case | Expected result | ChatGPT recovery | Claude |
|---|---|---|---|
| F3-01 | 1,860 requests | 2 | 2 |
| F3-02 | 1,602 successful requests | 2 | 2 |
| F3-03 | 86.13% weighted success rate | 2 | 2 |
| F3-04 | Gamma, 93.21% | 2 | 2 |
| F3-05 | Delta, 6 incidents | 2 | 2 |
| F3-06 | 13 incidents | 2 | 2 |
| F3-07 | Alpha, USD 0.78 per success among complete-cost teams | 2 | 2 |
| F3-08 | USD 1,343 known cost; not complete | 2 | 2 |
| F3-09 | record 14, Epsilon, 2026-02 | 2 | 2 |
| F3-10 | 645 requests; 561 successes; 86.98% | 2 | 2 |
| F3-11 | Zeta, 4.25 review minutes per success | 2 | 2 |
| F3-12 | Delta, 1.33 incidents per 100 requests | 2 | 2 |
| Total | 24/24 | 24/24 |
The most important row was the blank one
A blank cost can produce several plausible but wrong analyses:
- silently treat the missing value as zero
- include Epsilon in a complete-period cost ranking anyway
- sum the known costs and label the result a complete total
- drop the incomplete row without disclosing it
Both products avoided all four errors.
They identified Epsilon 2026-02 as the only missing cost, excluded Epsilon from the complete-cost ranking, and described USD 1,343 as the sum of known costs rather than the full dataset cost.
Weighted rates were also correct
The overall success rate is not the average of six team percentages. The fixed rule required sum(successful) / sum(requests).
Both products returned:
- 1,602 successful requests out of 1,860
- 86.13% overall weighted success rate
- Gamma as the highest team at 93.21%
- March at 561 / 645 = 86.98%
They also reproduced the Zeta review-minutes calculation and Delta incident-rate calculation rather than returning only unsupported final numbers.
No official failure flag triggered
For the scored outputs, neither product triggered the predefined flags for:
- missing cost treated as zero
- incomplete cost called complete
- incomplete team included in cost ranking
- unweighted rate used
- filter error
- arithmetic error
- wrong extreme team
- data-quality note dropped
- format contract broken
What this result does not prove
F3 is a small synthetic table. It does not test messy spreadsheets with merged cells, formulas, multiple sheets, ambiguous date formats, very large datasets, external joins, or business rules that require domain expertise.
It also cannot establish that the products are equally reliable in general. Both got this fixture right, but the ChatGPT evidence comes from an authorized recovery run while Claude's score comes from its preserved first completed run.
The test is therefore useful as a bounded observation about weighted calculations and missing-value discipline, not as a universal data-analysis ranking.
Practical takeaway
Before accepting an AI-generated table analysis, check at least four things:
- Is an overall rate weighted by the underlying counts?
- Are blank values distinguished from zero?
- Are incomplete records excluded only where the metric requires completeness?
- Is a subtotal of known values clearly labeled as incomplete?
F3 shows that both tested products can satisfy those rules on this controlled dataset. A separate workflow review is needed to learn what happens when a user asks for a decision-ready deliverable, corrects it, and tries to export the result.
For the preceding long-document test, see ChatGPT vs Claude for Long-Document Summaries.