How do you find answers on the internet?

If your business relies on search engine optimization to acquire new customers, you might be sweating when you hear about the shift toward AI. Will AI hurt your ability to receive new customers from search engines?

Large Language Models (LLMs) and generative AI have grown faster than expected! With quick changes, people sometimes panic. I’m not just talking about the fear of Skynet killing all humanity. I’m talking about conjecture and speculation that accompanies change.

Rather than rely on conjecture and anecdotal stories, we should look for data to inform our decisions. However, AI has been changing so quickly that I have not found any data about LLM’s adoption over other ways to find information.

So I decided to collect my own.

To understand the current state of search, I created a survey that asked: “If you have a question, where on the internet do you turn to find an answer to that question?”

If you have a business, you’re offering an answer to your customer’s question. If someone finds your business has the solution they’re looking for, you have acquired a customer. For example:

• Is there a good plumber who can make emergency calls in the evening?
• Is there a pizza place near me?
• Who manufactures blue widgets?
• Does anyone provide maintenance on a Mark 3 model 7 sonic screwdriver?
• Is there a reliable SEO consultant near Charlotte, NC?

A potential customer may search for the answer to their problems in several ways- on the internet. Some might ask a search engine. Others might look toward their social network. Recently, people have a third option: turning to their favorite generative AI or LLM model for answers.

Which of these options do people use to find answers today (as of July 2023)? If their first option fails, what is their second option to find an answer? Has that trend changed in the last six months?

Here’s what the survey tells us:

1. Some people are using LLM models to find answers to their questions, and that number has increased in the last six months.

2. Today people still prefer search engines to answer their questions.
   
3. If their first search fails, people resort to social media if they need help finding a solution.

4. AI is replacing social media as a way to find answers to questions- but only if their first attempt fails to answer their question.

TL;DR: As people try to find answers to their questions, LLMs are not replacing search engines but social media- but only if they can’t find the response on a search engine first.

What does this tell us about our marketing strategy moving forward?

Keep your SEO campaign- people still prefer search engines to find answers to their questions. If your company has a solution- that’s where your customers are first looking.

You should shift your marketing budget from social media campaigns to search. The data of this survey suggests that people are losing confidence in their social networks as a way to find reliable answers to their questions. They’ve lost faith in Facebook, and LLM models are starting to replace it as a way to find information.

Now is the time to double down on your SEO campaign by taking the time and effort to produce new and unique content for your website. Although Google has said that AI-generated content is acceptable to them, when they move their search engine into an LLM (Bard), they won’t want AI-generated content fed into their AI-generated answers. That’s just an echo chamber they wish to avoid. Instead, write unique content on your website today. If you don’t, you might need to rewrite it all again once the LLM model search engines have rolled out completely. A good SEO strategy is not about cutting corners but about doing good marketing.

Now, if you want to get further into the data, I’ll present the details below:

By the numbers

The survey received 6,434 responses. After cleaning the data (by removing people who submitted an answer using the same email more than once), there were 5,723 responses. Then I limited respondents to those living in the United States- leaving 4,358 responses.

I asked the following questions to these participants:

1. If you have a question TODAY, where on the internet do you FIRST turn to find an answer to that question?

Most people (52.4%) use search engines as their first way to find an answer to their questions. Social media is the second most popular choice (32.9%).

This data doesn’t surprise me; I’m an SEO professional. SEO is a great marketing channel to connect with customers looking for an answer to their questions. “Who’s the best plumber near me?” SEOs work to get their clients to be the answer to that question.

But social media is a search engine too. I’m not talking about the SEO tactic where you stack the search results with your social media profiles- using them to be another way someone can find your business. I’m talking about the phenomenon that many people “search” their social network for a recommendation for the question: “Who is the best plumber near me?” This question is still a search for an answer- although outside the scope of SEO. Many people turn to their social network (using social media sites) to find the answer to this question- rather than start from Google (or even Bing).

What surprised me is the number of people who claim that AI is the first place they turn to to answer a question: 14.8% of respondents. That seems high until you realize how quickly ChatGPT grew in its user base.

2. If you have a question TODAY, where on the internet do you turn to find an answer to that question if the first option doesn’t help?

Only some people find an answer to their question the first time they ask. That’s why I wanted to know where their second choice to find a solution. The second choice for most people was social networks 38.5%), followed by search engines (33.9%).

One interesting piece of data came from the fact that, for many, their first and second place to look was the same (28.6%). In other words, if they didn’t find an answer in a search engine, the second place they looked was a search engine.

At first, I thought this was an error in the data (people not paying attention) because the survey did not prevent people from selecting the same answer for their first and second choice. As I looked at the data (and spoke with some random respondents), I realized they turned to another search engine if the first didn’t help. Some also turned to another social network if their first attempt didn’t help.

From this, I learned that although LLM AI solutions are the talk of the town, they’ve not replaced search engines or social networks (yet). If you combine the first and second choice of people looking for an answer, most people (60.5%) did not turn to an AI solution for help to answer their questions. In other words, most people still turn to a search engine or social network.

3. If you had a question 6 MONTHS AGO, where on the internet would you FIRST turn to find an answer to that question?

LLM models’ popularity has grown extremely fast. Could I baseline this data against trends six months ago?

I learned that people didn’t have a statistically significant difference six months ago vs. today.

4. If you had a question 6 MONTHS AGO, where on the internet would you have turned to find an answer to that question if the first option didn’t help?

While people’s first choice way to find an answer stayed the same, in the last six months, their second choice did not. More people today (than six months ago) use an AI-LLM model for their second choice to find an answer to their question (from 15.8% to 27.6% of respondents, respectively).

Where did these people come from? Did they stop using Google? No- the percentage of people using search as their second source of information remained statistically similar. Instead, they stopped going to social media for answers: the percentage of people asking social media sources for their second source of answers declined from 52.5% to 38.5%.

This data is confirmed by looking only at people who included an LLM model in their choices to find answers. Six months ago, 72.4% of respondents did not use any form of AI to find answers to their questions. If we compare this to the previous data, the number of people who used AI as any part (first or second choice) to find answers increased in the last six months from 27.6% to 39.5% of respondents. More people are using LLM to find the answers they need- but only if their first source fails to answer their question.

Anecdotally I can see this in my own life. For one, social network algorithms have all but muted my friends and people who I trust to give me good answers to my questions.

In addition, with the political polarization we’re experiencing (at least in the United States), people are getting increasingly frustrated with social networks. Some are abandoning them altogether. It might be better to use ChatGPT to find the answer to “a good plumber near me” than to ask your Facebook feed, weeding through the latest, crazy political conspiracy theory before you can get there.

If this data is accurate, AI might not be the death of SEO- but the death of social network’s ability to be the answer to questions people want to know. People might still turn to social networks for entertainment. People might still see ads on social networks- pointing them to a solution to their problem. However, it might not be the lead generator it once was.

5. Which web browser do you typically use?

Real statisticians (not me) will include related questions in their surveys to help test the data. I had a couple of questions like this to help verify the quality of my data, so I asked about their web browser usage.

As you can see from this chart, the web browser usage was random or evenly distributed amongst the main choices. Unfortunately, that’s not a good thing. Other sources suggest that Chrome is the most popular browser today. This graph tells us something about this data’s quality- or lack of quality. While we understand that some people have no idea how to answer this question, it should more closely resemble third-party data if we want to be confident in the data of this study.

6. In which country do you live?

I wanted to add this data for the corroboration of the other data points and allow parsing of AI usage by country.

Unfortunately, this also shows that the overall data is suspicious. Although, unsurprisingly, most respondents are from the United States (in that I live in the US and share this survey with a US-based audience), there are untrue data points here. For example, the first choice of country was “Afghanistan.” You could have left it so if you were lazy in filling out this survey. Many did because there was nearly the same number of respondents from Afghanistan as France- despite millions more internet users in France than in Afghanistan.

Other suspicions came when people claimed to be in a country that banned ChatGPT but still used it. While there are ways around these restrictions, that data is suspect. However, this was a relatively small number of respondents.

The good news is that to choose the most common option, people had to scroll to the end of the list to select “United States” as their country of residence.

It’s for these reasons I limited the data in this final report to only people who identified themselves as living within the U.S.

7. Are you currently working in an internet-based career?

As someone who works on the internet all day, I understand that I have biases. I’ve already mentioned one: I’ve become an early tester of LLMs to see how this might affect my career and clients. Acknowledging this bias, I wanted to collect data for only people who do not work on the internet all day.

Unfortunately, 83.4% of people who responded to this survey self-identified as working on the internet. We can parse the data for only non-professionals, but if we do, we are working with a small sample size, making it hard to have confidence in the (already suspicious) data.

If the most significant proportion of surveyed people were internet professionals, you’d expect them to identify their browsers correctly. Internet professionals use browsers differently than an average user, thus showing an atypical result. However, people misidentify their browser in this survey: many claim that IE is their browser, but that has long since been discontinued.

Room for Improvement

The Scientific Method encourages us to repeat our experiment to see if we get the same results. Please repeat this survey to determine if your conclusions are similar.

If you attempt to replicate this survey, there are some things I’d recommend you do differently. I would have liked to remove additional data from this response because some of the data makes me suspicious that there were automated responses (with the hopes of earning the $100 gift card). I could not remove the following:

• Questionable geographical data. There were allegedly nearly as many submissions from Afghanistan as in France. Since “Afghanistan” is the first choice in the drop-down, this might be people attempting to claim the prize (although since Amazon doesn’t deliver to Afghanistan, the award is no good there)
• Some people claim to use AI in countries in which ChatGPT is illegal. They might be able to get around it- but they were a minimal number.
• Some of the submitted emails were suspiciously similar. This coincidence makes me think that people submitted answers multiple times to increase their chances of winning the drawing. Unfortunately, there were so many submissions that they diluted their chances of winning.

If you would like the raw data (without email addresses), I’d be happy to share that data with you so you can do your analysis.