Which AI model for which job in Google Ads? We tested 8 models on a real account

We also checked the judges themselves. Two of the three model judges inflated the scores of models from their own company. In a blind review by a practitioner, Claude Opus 5.5 won 8 of 10 pairs.

We ran the test in early October 2026 because, working with AI on Google Ads accounts, we kept running into truncated results and hallucinations: answers that sound confident but have no support in the data.

We explain where they come from in the post on wrong Google Ads numbers. Instead of asking which model is best, we asked: best at what?

Results: Claude, GPT or Gemini, which AI model won which Google Ads task

In our test of 8 AI models on a Google Ads account, Claude Opus 5.5 led in diagnosis, optimisation and campaign design, GPT-6.1 Sol led in strategy, and GPT-6.1 Sol and Gemini 3.8 Flash were best at judging search terms to exclude.

Scores 1-5 are averages from judges made by companies other than the one behind the model being scored. Six models took part in the search terms task, without Claude Opus 5.5 and Gemini 3.1 Pro.

The search terms task used 200 search terms from this account's report. Each had a label: 83 do not match the offer, 21 match, and 96 are ambiguous, meaning generic, like "toys for a 3 year old". "Bad terms caught" is the share of the 83 non-matching terms that the model said to exclude.

A wrong exclusion means excluding a matching term. Excluding an ambiguous term is not an error, but it risks losing traffic.

Full results of 8 AI models in Google Ads: scores and response time

In the full results of 8 AI models in Google Ads, only Claude Opus 5.5 and GPT-6.1 Sol score at least 4.2 out of 5 at all five task levels. The fastest to answer were Gemini 3.5 Flash-Lite (7 s) and Claude Haiku 4.5 (10 s), which took the last two places in the total score.

Order by total score from cross-scoring. Time is the median response time in written-answer tasks.

The main practical takeaway: if you use one model for everything, in a regular chat or in one script, some of your tasks are done by a model that scored lower than others on exactly those tasks.

How we tested AI models on a Google Ads account

Eight AI models from Anthropic, OpenAI and Google got the same data pack from a real Google Ads account and 17 questions at five difficulty levels. Three model judges from different companies scored the answers.

What data we tested on

We tested on a real Google Ads account of a small online store, with one main product campaign. We do not share its name or sales figures.

Each model got the same pack: a description of the client and their goals, and the results of 25 Google Ads API queries, meaning reports pulled from the account. Plus the same prompt, the same reasoning level and the same answer length limit.

Which AI models we tested

Anthropic: Claude Opus 5.5, Claude Sonnet 5.5, Claude Haiku 4.5. OpenAI: GPT-6.1 Sol, GPT-6 Luna (the same models run in ChatGPT, but in the chat with different settings than in our test). Google: Gemini 3.1 Pro (preview version), Gemini 3.8 Flash, Gemini 3.5 Flash-Lite.

What tasks the models got

17 questions at five difficulty levels. We dropped two extra general knowledge questions, because with the instruction "use only the data" all models refused to answer.

A separate part checked yes/no decisions: judging 200 search terms from the account (exclude or keep), recognising the question type and catching prompt injection attempts. The six models in this part recognised the question type correctly in 86-93% of cases, and all of them caught the prompt injection attempts.

How we scored the answers

Three model judges scored the answers, each from a different company: Claude Opus 5.5, GPT-6.1 Sol and Gemini 3.1 Pro. The scale was 1-5 for five criteria: correctness (matches the data, nothing made up), completeness, relevance of recommendations for this account, usefulness (can it be applied right away) and language.

Code, not a judge, checked some things: whether the requested numbers were given, whether ad headlines fit the character limit, and whether the model excluded the store's brand.

Who prepared the test

The prompt, tasks and criteria were prepared by Claude, one of the tested providers. So we removed each judge's scores for models from its own company from the ranking, checked how answer length affects the score and added a blind human review.

The Opus and Sol judges slightly penalised long answers, Gemini slightly rewarded them. After correcting for length, the order of the models did not change.

In an audit we found and fixed two errors in the code checks. The fact check lowered the scores of two OpenAI models. The brand detection in exclusions at first also wrongly flagged Claude Opus 5.5 and GPT-6.1 Sol.

Two of the three model judges inflated scores for their own company's models

In our test of AI models on a Google Ads account, two of the three model judges, Claude Opus 5.5 and GPT-6.1 Sol, raised the scores of their own company's models by about 0.5-0.8 points on a 1-5 scale. We checked this before we calculated the ranking.

The third, Gemini 3.1 Pro, did not favour its own, but it was lenient: Claude Opus 5.5 got 5 out of 5 from it in every task.

Averages of all scores on a 1-5 scale, including scores for models from the judge's own company, which we do not count in the ranking.

That is why only cross-scores count in the ranking: a model is scored only by judges from other companies. Each judge also had a different scale, so we normalised the results separately for each of them.

The lesson for anyone reading AI tool rankings: check who did the scoring. If a model scores itself or models from the same company, the result may be inflated. In our test, two of the three judges did this.

A simple question will not show the difference between AI models

When reading numbers from a Google Ads account, the gap between the best and the weakest AI model is one point on a 1-5 scale. In optimisation, strategy and campaign design it is more than two.

Average cross-scores (a model is scored only by judges from other companies), scale 1-5.

Claude Haiku 4.5 gave all the numbers we asked for in the reading tasks correctly. Yet in the Search campaign design it suggested excluding the store's brand.

So the question "show me yesterday's costs" only checks whether the tool has access to the data. Only a hard question about an account you know well tells you about quality.

The practitioner saw differences where model judges saw a tie

In a blind review of 16 answer pairs from the Google Ads test, Robert Szymański, founder of Roition and former performance specialist at an agency, picked Claude Opus 5.5 in 8 of the 10 pairs where this model appeared. He did not know which model wrote which answer.

Claude Opus 5.5 appeared in 10 pairs: it won 8, one was a tie, and in one the reviewer found both answers unsatisfactory. In five of those wins, the model judges saw a tie.

What the practitioner wrote in his notes:

• the best answer was the one that recalculated the data itself and noticed a duplicated conversion (same day, same value),
• the user should see campaign and product names, not their IDs,
• for exclusions he expected lists grouped by topic with the cost of each term. In two of the four exclusion pairs he found both answers unsatisfactory, because terms were missing.

The GPT-6.1 Sol judge, on the other hand, lowered scores for answers that questioned whether conversion tracking was correct. The practitioner valued these answers most.

Caveat: one person and 16 pairs. Part of the review is style, which the reviewer knows from daily work with Claude models, and Opus's answers were the longest. This is a hint, not proof.

Every AI model made mistakes, just in different places

No model in the test of 8 AI models on a Google Ads account was free of errors. The weakest, Claude Haiku 4.5, added free delivery to an ad, though it was neither in the data nor in the client description. In the Search campaign design it also suggested excluding the store's brand.

The best models made subtler mistakes. They excluded broad terms that also block converting search terms, or proposed large bid changes based on a few clicks (for example, a 70-100% device bid adjustment based on 5-7 clicks).

Claude Opus 5.5, the leader in diagnosis and campaign design, also has flaws. It wrote the longest and slowest answers: a median of 64 seconds, compared with 18 seconds for GPT-6.1 Sol. It also proposed exclusions that block relevant traffic and settings that no longer exist in a given campaign type.

AI mistakes in Google Ads that repeated across many models

Each of the eight AI models made at least one of six recurring mistakes in Google Ads, including exclusions that block relevant traffic and conclusions stronger than the data.

So choosing a model alone will not guarantee quality. The system around it has to guard quality: code that does the calculations and checks that test proposals before a person sees them.

What the AI model test changes in Marketing Workspace

After the AI model test, Marketing Workspace will move the judgement of whether a search term matches the offer from Claude Haiku 4.5 to Gemini 3.8 Flash in the next release. The test also confirmed two earlier changes: a separate model for each job and calculating exclusions with code.

Marketing Workspace, our AI assistant for Google Ads, Meta Ads and GA4 accounts, assigns a separate AI model to each job, calculates exclusions with code and checks whether the numbers in an answer have a source in the account data.

Today it uses Claude models (Anthropic) and, since October 2026, also Google models (Gemini), with OpenAI as a fallback, always through paid APIs, with no training on client data. We give any new provider access to client data only when its paid API does not train on that data and runs under a data processing agreement. What to check before you paste client data into a chat yourself, we cover in the post on client data in AI chats.

We introduced the split of work between models and the reliability check on 29 September, before the test. Calculating exclusions with code went live on 7 October, the day we closed the test report. The test confirmed the direction of both changes.

What already works:

• Each job has its own model. Claude Sonnet 5.5 collects data from the account, and Claude Opus 5.5, which won the practitioner's blind review, writes the diagnosis and proposals.
• The system calculates exclusions, not the model. When looking for negative keywords, code pulls and calculates the account data, and the model only judges whether a term matches the offer. Search terms with conversions and brand terms saved in the client card never go on the list.
• Answer reliability check. The system checks whether the assistant pulled data from the account and whether the numbers in the answer have a source in it. If not, you see a warning.

What we are changing after the test: today Claude Haiku 4.5 judges in MW whether a search term matches the offer. On its own, the model caught 72% of bad terms in the test. Gemini 3.8 Flash caught 96%, with no wrong exclusions.

The paid Gemini API runs under Google's data processing agreement and, according to Google's terms, is not used to train models.

What we are working on: aggregating products by ID in code, exclusion lists grouped by topic with the cost of each term, and answers without campaign and product IDs, only with names.

We describe every release in the public Marketing Workspace changelog, and how the check works in the description of answer reliability.

What this AI model test does not tell you

The result of our test of AI models in Google Ads has five limitations.

• One account. A small store, mainly a product campaign. Large accounts, lead generation and Search campaigns may give a different result.
• Each model answered once. We did not measure how much the answers vary.
• The same data for all. The test does not measure how a model collects data from the account on its own. That is a separate skill. What goes wrong when AI gets incomplete data or does not pull it at all, we explain in the post on wrong Google Ads numbers.
• The search term labels were set by Claude. GPT-6.1 Sol agreed with them most, and Claude models less, so the search terms task shows no advantage for Claude. Disputed terms need a human review anyway.
• Versions from October 2026. Models change every few months; the result applies to the versions we tested.

See how Marketing Workspace works

About the author

Robert Szymański

Founder of Roition, independent Google Ads API developer

Founder of Roition, with almost 3 years of Google Ads experience. Designs and builds process automation for Polish companies: system integrations, handling enquiries and quotes, AI assistants working on company data.

All articles by the authorLinkedIn