I tell all my clients, “We can try anything you want, but we’re going to measure it!”
This philosophy has led to some surprising victories and some surprising defeats. It also tells us where to spend money so we can be most effective with our marketing dollars.
Now that we’re entering the world of LLMs, this is still important- but more difficult.
How can we measure the impact of LLMs on our marketing efforts? I learned some things at the 2025 Google Search Central Live (SCL) event in NYC that can help.
I know what you mean when you say “AI” here, but bear with me as I ensure we’re using the term to mean the same thing.
John Mueller made a helpful distinction at SCL: AI involves two major types: machine learning (ML) and learned language models (LLMs).
Google has long used ML in its algorithms. ML helps Google better “understand” what you’re asking for when you give it a query.
What’s new in the AI world (over the last couple of years) is LLMs like ChatGPT. LLMs give you a long answer that resembles a human conversation. Rather than sending you to a page that might answer the question, the LLM answers it for you. When most people say, “I want to measure AI,” they mean an LLM. How can I tell if ChatGPT, Google’s AI Overviews, Caude, or whatever is recommending me?
As we discuss this, I want to make these differences clear. It’s easy to get confused. Even Mueller used the wrong terms throughout his presentation.
Before we talk about the “how,” let’s pause and remind ourselves why this is important. There is a massive amount of hype around AI/LLMs these days.
In my marketing experience, this is nothing new. Whenever something new comes along, everyone gets excited about it. Sometimes, companies make knee-jerk reactions to the excitement, taking dramatic action on their overall marketing strategy. “Let’s move all our marketing dollars into TikTok!” they say because it’s the cool, new thing.
Some companies want to do the same regarding LLMs. You can try anything you want, but we’re going to measure it. Do LLMs deserve this attention? Is it worth the focus? We won’t know unless we measure our results.
Before we get into the how, we must also understand what we’re measuring.
Too many SEOs are stuck in measuring “rank.” If this is you (or your SEO campaign), LLMs are terrifying! You can’t measure “rank” when it comes to an LLM.
But, if you’re honest, you’ve been unable to measure “rank” consistently for years. Besides personalized results, Google is constantly experimenting with new positioning, making it impossible to measure rank accurately. Besides, machine learning in the algorithm means you rarely get the same result for any query. Even Google Search Console gives you an “average rank” because you never consistently rank in any one position.
Unfortunately, many SEO customers are stuck in the rankings, too. They insist they’re paying for the “first position.” They tell their SEO team or providers, “That’s your job.” The problem with this thinking is that you can rank for anything, but if nobody is searching for it, it will never bring you new business, and that’s the point of marketing.
When I attended Google Search Central Live in NYC (2025), Mike King, an SEO expert I respect, expressed his frustration with Google’s inability to show “rank” in AI overviews. His problem: his clients insisted his job was rank, and their job was to take that traffic and create the conversion.
That is shortsighted for his clients—no wonder they (and he) are frustrated. However, if they stop siloing the SEO team to only focus on rank and take credit for conversions, it will help everyone.
It’s not Mike’s fault. Recently, I had a client receive a lead from ChatGPT. This exciting lead led to a frustrating afternoon asking ChatGPT different questions and trying to replicate the “search result.” That’s not how this works! There is no “rank” in LLMs.
While rank has been difficult to objectively and accurately measure in search for years, it’s still part of the process. You have to rank if you want traffic. Traffic alone is not the goal, however. You can get traffic for many things, and never get a customer. So, you want to measure the rank that leads to traffic that leads to customers.
Google told us how to measure “rank” for LLMs- at least, for AI Overviews in a Google result. Google Search Console’s “average rank” includes the entity’s position for any query on a search result page. So, if you appear in an AI Overview, your position for that query will be #1. In other words, if you’re #1 in the organic search results (blue links), you’re now #2 in this report, since AI Overviews (at least, at this point) appear at the top of each search result.
Even if we can’t measure rank for most LLMs, we want to follow any traffic we receive from an LLM. That tells us important information: that an LLM is recommending us in the first place. If we trace traffic to a page, we can get an idea of how it’s recommending us (for which kinds of queries).
But we need more information than just traffic. Just like a search, we can get irrelevant traffic from an LLM. What good is a ton of traffic from an LLM if it’s not bringing customers? That’s why lead or sales attribution is key here. It tells us that an LLM brings a valuable visitor because they become customers.
I’ve set up Looker Studio dashboards for all my clients, showing them the traffic (and leads/sales) they get from LLMs. Let me know if you need help setting this up for your site.
Conversations at SCL between sessions pointed out that, although small, LLM traffic has a much higher conversion rate than conventional search traffic. This phenomenon makes sense: while we may lose long-tail traffic to the LLMs if someone ultimately needs a product or service we’re offering, an LLM cannot provide it. They need us! So, when an LLM recommends us for the solution to their users’ queries, we have a good chance of getting that customer.
Back in the day (I feel old just saying that) when we got a visitor, we could get the conversion. These days, however, customers are more savvy. They often make multiple connections with a company before becoming a customer. This limitation is even more true for the LLM world.
The other day, a client received a call about their services. Despite the call, the receptionist told the caller we couldn’t help them. “But,” the caller retorted, “ChatGPS [sic] told me to call you!” This rejection immediately caused the receptionist to reconsider blowing the caller off. Instead, she passed her contact information on to the sales team.
The most interesting part of this interaction was that I could listen to the call because it came through a Google Local Service Ad, not from our conventional phone call tracking system. In other words, here’s what happened: the customer asked ChatGPT for help; ChatGPT answered their question; the customer asked whom they should call to get that help; ChatGPT mentioned our company; the customer Googled our company and was served a Local Service Ad- so they called.
Which marketing channel should take the credit for this lead? ChatGPT? Google? Local Service Ads?
That’s the complexity of modern lead attribution: people connect with us in many different ways before they reach us. Had we not been able to listen to the call, we would never have known this came from ChatGPT—and we would have assumed the Local Service Ad was the winner.
Google Analytics is ahead of the game here. Despite all the hate with GA4, it uses data-driven lead attribution as the default model. That means it gives partial credit for a lead to other channels during the buying cycle. In this circumstance, GA would not be able to directly attribute this lead to ChatGPT—it can still catch other touches along the way.
In short, you can (and should) monitor your company’s effectiveness in reaching people through LLMs by tracking traffic and new customers from these LLMs.
If you’re not seeing any traffic from LLMs, you must take action to get that traffic. From my data, LLMs really like blogs. Blogs should provide answers to questions people want to know about your products and services. When the LLMs see you’re providing valuable and unique information (not generated by LLMs- since an LLM doesn’t want to use LLM-generated content to feed its LLM), it will use your information to provide answers.
Of course, that means your website and blog should be readable by LLMs. How can you ensure that? Google Search Console can tell you if Google’s AI tools can read your site. Bing Webmaster Tools can tell you if Copilot (and ChatGPT, which uses Bing’s index, among others) can read your site. If not, fix the errors these tools suggest.
Of course, you can have a readable site with good content, but how will anyone (even an LLM) know you’re an expert? If anyone else says, you’re an expert, of course. Make sure other sites speak favorably about you (and your site).
In other words, to reach LLMs, do good SEO!
And measure it like you would good SEO, not with rank, but with traffic and customers.
With that information, you will know if your LLM (or AIO) efforts are paying off.
Don’t be afraid of marketing over LLMs- but don’t believe the hype either. Let’s give it a try. I’ll try anything, but we’re going to measure it!
If you need help setting up a report to measure the impact of LLMs on your website, let me know.
Reliable Acorn will help you create a custom digital marketing strategy that does just that.
Ready to Talk?