Does Your LLM Know Your Schema? A GA4 and BigQuery Check

A model can only be right about the columns it knows, and a GA4 export keeps part of its schema in repeated records that differ from one implementation to the next. Google's schema page says the set of event parameters "is unique to each implementation", so your export's parameter list isn't something a model has seen unless you show it. Spider 2.0's databases average between 743.5 and 812.1 columns each, depending on the setting, against 54.2 in BIRD, and its authors attribute 27.6% of the errors in a 300-example review to wrong schema linking. Give the model your real columns, dry-run what it writes, then reconcile the totals.

What do the studies and documentation show?

Two studies measure schema difficulty for models, and Google's documentation says what a GA4 export contains.

• Spider 2.0 (peer-reviewed, ICLR 2025 Oral) draws its databases from applications such as Google Analytics and Salesforce. Its Table 2 gives 812.1 columns per database in the Snowflake setting against 54.2 for BIRD. In its review of 300 randomly sampled examples, "Wrong schema linking (27.6%)" covered wrong tables and columns: column linking was 16.6% of the errors and table linking 10.1%.
• BEAVER (preprint, revised May 2026) is built from private data warehouses: 9128 question and SQL pairs over 812 tables in 19 domains. Agentic systems built on GPT-5.2 "achieve only 10.8% accuracy", and supplying the answers to five subtasks, among them multi-table retrieval, join key detection and column mapping, raises it to 30.1%.
• Lost in the Middle (peer-reviewed, TACL 2023) tested language models on multi-document question answering and key-value retrieval, not on SQL. It found that "performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts".
• Google Analytics Help (official documentation) says the event_params record "is repeated for each key that is associated with an event", and that the set of parameters "is unique to each implementation". The user_properties and items records are repeated too.

The schemas and questions in these studies are not yours: BEAVER's three warehouses come from an academic institution, a research laboratory and a residential facility, and Lost in the Middle is about long text, not schemas. No study cited here measures how often an LLM-written query fails against a GA4 export with its own parameters.

What does a GA4 export look like to a model?

Events sit in one table per day, events_YYYYMMDD, plus a streaming table that Google says not to use for a finished day: "You should query events_YYYYMMDD rather than events_intraday_YYYYMMDD so you're querying a stable dataset for the day." The width sits in nested records. To read a parameter such as a session ID, a query has to UNNEST event_params, and which keys exist depends on your implementation. A model that hasn't seen your keys will guess them. A guessed name is an error a dry run can catch. A real name used at the wrong grain is an error only a reconciliation catches.

How do I show a model my schema and check its query?

List the real column paths and the parameter keys your purchases carry, and paste both into the prompt.

SELECT field_path, data_type
FROM `your_project`.analytics_123456789.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS
WHERE table_name = 'events_20260927'
ORDER BY field_path;

SELECT param.key AS parameter, COUNT(*) AS times_seen
FROM `your_project.analytics_123456789.events_20260927`, UNNEST(event_params) AS param
WHERE event_name = 'purchase'
GROUP BY parameter
ORDER BY times_seen DESC;

Add one line of grain: one row per event, with event_params, user_properties and items as repeated records. Then dry-run whatever the model writes, in the BigQuery console or with bq query --dry_run. BigQuery says a dry run provides a "validation of your query" and that "Dry runs don't use query slots, and you are not charged for performing a dry run."

Read the outcome like this:

1. Pass. The dry run validates, every column path in the query appears in your first list, and the totals reconcile with Shopify and GA4 for the same dates.
2. The dry run fails. The message names a column or a type the model guessed. Paste the lists again and ask for a corrected query.
3. The dry run passes but the totals do not reconcile. The names are real and the meaning is wrong. A dry run checks structure only, so look at joins, grain, NULLs and the time zone, each covered in the related answers.

An LLM can write the UNNEST pattern once it has your real paths, and explain what a nested field is. The real paths are what it lacks.

Sources, 30 September 2026: Spider 2.0 paper (Lei et al., ICLR 2025); BEAVER and its full text (arXiv preprint, 2026); Lost in the Middle (Liu et al., TACL, 2023); BigQuery Export schema and BigQuery Export (Google Analytics Help, 2026); Run a query, dry run and COLUMN_FIELD_PATHS (Google Cloud, 2026).

Related answers

• How a join drops or repeats orders
• What the same SQL means in each warehouse
• Natural language to SQL for marketing
• GA4 BigQuery export: which field to use