Question the Data: Should we publish AI articles on our website?

I’ve been advising people not to publish AI-generated content on their websites for a while now. Then, Ahrefs published a study suggesting that AI content is okay. Is it? I’m still not wholly convinced. Here’s why I’m resistant.

Here’s the study to which I am referring: https://ahrefs.com/blog/ai-generated-content-does-not-hurt-your-google-rankings/

Problem #1: Rank as Success

The first problem in this study is that it judges success based on rank: “Most of the top-ranking pages are AI-assisted,” the study states.

Why is this a fundamental problem? Because rank is baloney. No tool, even Ahrefs, really knows where you rank for anything. This limitation is because the rank fluctuates. No wonder both Google and Bing webmaster tools show an average rank. This study, however, did not use this data, but their ranking tool. Note: I’m an Ahrefs customer and think they have the best SEO tool available, even if I don’t use their rank data.

Second, why are we concerned about rank when LLMs dominate the future? Even if you can objectively measure rank today, who cares when the ten blue links go away?  

In an LLM world, we want to know if we receive visitors from the LLMs. Since an LLM doesn’t want to add LLM-generated articles to its index to generate LLM answers, I’d suggest that AI-generated pages will not help in the future because they will be ignored and not bring you traffic.

Even if rank can be measured objectively, the result of this study is temporary at best.

Problem #2: Accurately Detecting AI Content

The second problem in Ahrefs’ study is in detecting AI content. Anyone who has worked with AI content has experienced the difficulty of AI tools in determining whether an article was written by AI or not. I’m not the only person who has tried these tools on my articles to get false positives.

In light of this, how can Ahrefs be sure of its conclusion on AI content? Don’t believe me? Look at Ahrefs’ data about AI detecting tools. The best tools got a “C.”

Some might read this and say, “Great! If nobody can detect AI content, let’s publish all the AI content!”

To that, I’d suggest a couple of problems.

For one, just because our tools can’t detect it doesn’t mean Google (or other LLMs) can’t detect it consistently. These tools possess massive computing power, significantly more than the AI detection tools. Since LLMs have an inherent interest in not adding LLM content to their index, they can (and do) apply these powers to double-check for accuracy.

Second, remember that AI is not really “intelligent.” It can’t generate new ideas, but only reiterate existing ideas. So, when it comes to writing articles, it’s just fancy article spinning (if you’ve been doing SEO as long as I’ve, you know what that’s like). I find it no coincidence that Google rolled out the “Helpful Content Update” around the same time as ChatGPT was gaining acceptance. ChatGPT is unable to say anything new; it merely rearranges existing information about a topic. It looks new to us (and might be able to trick an AI-content test), but it’s not really “helpful content” per Google’s criteria. In light of this, I think it’s better to take time to write something new and helpful, rather than publish more of the duplicate content, assisted by AI.

Unfortunately, article-writing has been a staple of SEO strategies for years. Some clients even expect X articles a month. This rate of production, combined with inflation, has affected us over the last few years. I understand the pressure to find cheaper ways to maintain the same pace of article production. However, if we’ve learned anything about search recently, it’s that quality trumps quantity. I’d rather reset my clients’ expectations for the number of articles so we can take more time to write something new, unique, and helpful, rather than keeping up the pace with AI-generated content.

You might dodge the AI detectors, but AI-written articles don’t say anything new or helpful. Better to produce fewer quality articles than more AI junk.

Problem #3: The Echo Chamber

The third problem with the Ahrefs study is not inherent in the data itself, but rather in how people utilize it: as a justification for publishing AI content on their website.

This reason stems from the root of many SEO strategies: the notion that SEO is about tricking Google or cutting corners. Some will read this study and interpret it to suit their own needs, concluding that it’s acceptable to continue publishing AI-generated content on their site.

However, please read the rest of this study. The conclusion isn’t to encourage the publication of AI articles. Only 4.6% of the articles they studied were written entirely by AI- and those articles rarely reached the #1 position. The vast majority (81.9%) were written mainly by humans, with AI as an assistant. The conclusion is that it’s okay to use AI to help you write articles, but not to publish AI content.

This conclusion reminds me of John Mueller’s plea at Search Central Live in NYC this February. He clarified Google’s stance on AI, reminding us that we had taken the recommendation too far. We thought Google advised against publishing AI content, but it never really stated that. It was suggesting that it’s okay to use AI to help write a page. This is what Google has been saying all along. This conclusion aligns with Ahrefs’ study.

Unfortunately, too many people are in the “trick Google” model of SEO, so they read this to justify their continued cutting of corners. What I’m advocating for is SEO-as-marketing, where we use the principles of SEO to help us reach the customers looking for what we have to offer.

But even those who are not trying to trick Google are falling for this fallacy. Let’s face it, we are all under pressure to do more with less. Whether you’re an independent web developer who needs to populate a website with words or the largest, Fortune 500 company, we all have limited resources. LLMs promise a fantastic solution: let’s write more for less time and money. It sounds good, and we want it to be true, so we find the article that tells us what we want to hear.

There are many ways AI can productively help you write articles, but that doesn’t mean AI should only write your articles.

For example, I used Grammarly to help write this article by checking my grammar. Guess what? That’s a form of AI (even if I didn’t use their AI suggestions, the grammar check is still an AI-powered tool). What a hypocrite! No. These are my words, but AI helped me make sure those words are clear. That’s an acceptable use of AI in content- that both this study and Google agree is legitimate, acceptable, and beneficial.

Problem #4: The Scientific Method

Along with the idea above, rather than just looking for a study to tell us what we want to hear, we should be pushing for more research. Can anyone else corroborate this study? Can anyone else replicate the results?

I’d like to see someone improve on this study with the following data:

• Does ranking (according to objective data like Google Search Console or Bing Webmaster Tools) correlate with AI-written articles? To measure this, someone would need access to thousands of GSC and WMT accounts- but I’m sure someone does have access (I’m looking back at you, Ahrefs).
• Do AI-written articles still bring traffic from LLMs? To measure this, I wouldn’t want crappy, estimated traffic (I’m glaring at you, SEMrush) but real data from a tool like Google Analytics.

If someone can produce results using this data, I’d be more willing to accept the conclusions, not only because the study would be more comprehensive but also because it’d be an entirely new study.

Of course, each study will have its limitations. That’s why we discuss them and iterate again, producing new research. This is the Scientific Method. This is how we should draw conclusions, rather than just waiting for another article that confirms what we already want to hear.

So, please don’t take any one article- whether from Ahrefs, Google, or even me- and run with it as the gospel truth. Be critical. Ask questions. Do your own research. Follow the scientific method if you want to know if you should be publishing AI content.