GPT-4 and Enterprise SQL: What the Benchmark Numbers Mean

There is no single "GPT-4 score". The live leaderboard, read on 30 September 2026, has no row for plain GPT-4, and OpenAI lists gpt-4-0613 for shutdown on October 23, 2026. A score belongs to a model snapshot, a wrapper, a benchmark and a date.

Which model does a benchmark number belong to?

The Spider 2.0 paper (Lei et al., ICLR 2025 Oral, peer-reviewed) gives its reason for a new benchmark: the classic ones often use non-industrial databases with few tables and columns and simplistic SQL. The leaderboard then names each entry as a wrapper plus a model snapshot. On the Snow setting, read on 30 September 2026, Spider-Agent with GPT-4o-2024-11-20 scores 12.98 and with GPT-4o-2024-08-06 scores 10.79, two snapshots of one model family. Spider-Agent with o1-preview scores 23.58, and the top entry, a named system from the organisation listed beside it, scores 96.70.

The page says scores "may change slightly over time" while the evaluation metrics are checked.

What has changed for marketing analytics?

The benchmark you pick can flip the conclusion. AD-Bench (Hu et al., arXiv preprint, revised 22 June 2026) is built from real marketing-analysis requests on a production advertising platform, and its best model, Claude-Opus-4.7, attains Pass@1 = 76.9% overall and Pass@1 = 61.4% on its hardest level, L3. Those are descriptive analytics requests with tool use, graded against ground truth regenerated from expert trajectories, so the two benchmarks are not comparable either.

What else moves under a model's name?

1. The model's lifetime. OpenAI says it is deprecating a set of older models, and lists gpt-4-0613, gpt-4-turbo, gpt-4o-2024-05-13 and o1 for shutdown on October 23, 2026 (official documentation). Anthropic says it regularly retires older models, and its page for a model released on June 30, 2026 already marks it legacy.
2. The service behind a name.

How do you audit a benchmark claim?

Run these steps on any "X% of SQL tasks" headline before it moves a decision:

1. Copy the exact model string. Take it from the tool's settings, the model field in an API response, or the vendor. "GPT-4" or "AI" is not a model string; "GPT-4o-2024-11-20" is.
2. Find that string on the benchmark's own page. No row means no published score for that model.
3. Check the benchmark, setting, wrapper and date. Which tasks (Snow, Lite or DBT), which system (Spider-Agent or a vendor's own), read when.
4. Check the provider's deprecation page. A shutdown date inside your planning period means the model behind your tool will change without any change on your side.
5. If there is no row, test it yourself. Use the known-answer set in How to Test an LLM on Your Own Attribution Data.

Pass: a row for your model, in a setting that looks like your questions, read this month. Fail: no row, another setting or an old date. The headline is then about something else.

An LLM still helps with drafting and explaining SQL, and the leaderboard lists systems far above the plain GPT-4o baseline. "Check the query" stays: run it, and reconcile its totals to Shopify and GA4 (the benchmark post has the check).

Sources, 30 September 2026: Spider 2.0 full text (Lei et al., arXiv, 2025); Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows (Lei et al., arXiv, 2025); Spider 2.0 leaderboard (XLANG Lab, read 30 September 2026); AD-Bench (Hu et al., arXiv, 2026); OpenAI API deprecations (OpenAI, 2026); Claude Sonnet 5 model page (Anthropic, 2026); Anthropic model deprecations (Anthropic, 2026); How is ChatGPT's behavior changing over time? (Chen, Zaharia and Zou, arXiv, 2023).

Related answers

• What the Spider2 SQL benchmark says about marketing data
• How to Test an LLM on Your Own Attribution Data
• Why an LLM Gives Different Answers to the Same Data Question
• Causal attribution, explained