How to Build Internal Links (with Screaming Frog)

One of the most powerful, but underrated, tactics in SEO are internal links. Build some internal links to an important landing page on your website to improve it’s ability to be found on Google. Updated October 2020.

Do You Need Internal Links?

Have you ever done a competitive SERP analysis? That’s just a fancy way of asking if you’ve looked at search engine results to see why Google ranks websites. We know there are lots of (600+) ranking factors. Despite this, SEO mostly comes down to two important factors: content and links. I’m oversimplifying here, but don’t get distracted by the details.

It’s easy to look at content. Is the keyword in the title, h-tags, content, etc. Heck, check those ALT tags, too, while you’re at it.

Links can be harder to check without a third-party tool (I like Majestic, but AHREFS or Moz will do in a pinch). These can show you how many links a page has, a domain has, and the quality of those links. Links are still a major ranking factor.

After this analysis, have you ever felt stumped? Does it seem that a page is ranking high, despite these factors? More often, than not, when I see this, it’s due to internal links from that website.

If your website seems to have all the right on page and off site factors but doesn’t rank well, I have two pieces of advice:

1. Get over yourself. Rank doesn’t really exist. Google’s results are personalized and always in flux. Rank is a terrible way to measure the success of an SEO campaign. Don’t be like Donald Trump and to compensate for your tiny hands in the SERPS. Sorry- had to get that off my chest.
2. You need to build some internal links to your page.

Internal Linking Guidelines

Some of you’re thinking, “If I need more internal links, I’ll just add every page of my site to the menu or on the footer of my site.” This is why SEOs can’t have nice things. Google specifically warns us to keep links from pages to a “reasonable number.” Also, as I read Google’s guidelines, Google downplays the value of navigational links, anyway. That means you’ll need to build links from within your content.

Also, be sure you only link once to a landing page, from any given page. More links from one page won’t help you. Not only will it make your page look spammy, it dilutes the value of the link. Remember, keep those links to a “reasonable number.” How many is “reasonable”? If you have to ask, you’ve used too many.

How I Used to Build Internal Links

If I decided a page needed more, I’d turn to a Google Search:

site:mywebsite.com “keyword phrase”

This would tell me which pages, on my site, mention this specific phrase. I’d then have to go through each page in the SERPs and see if the page already linked to my destination page, or not.

What’s nice about this method is that almost anyone can do it. It’s also good because you are building links from pages that Google has indexed. Sure, I could use a CMS search to do this but that risks building links from pages that aren’t indexed.

At the same time, it has some limitations:

• This is a pain. It’s slow. Many times it leads to false positives because it doesn’t tell me if I’m already linking to my page.
• It also breaks an important, personal rule: don’t click on my own site in the SERPs. That inflates my organic search traffic.
• It’s hard to determine if internal links are what I need. I’d have to rely on my SEO-senses. I’d rather use data.

This is why I turned to one of my favorite SEO tools, Screaming Frog, for help.

Use Screaming Frog to Build Internal Links

Thanks to Screaming Frog you can find pages (even specific paragraphs) from which you an build internal links to your landing page. Their Extraction feature makes this possible.

Step 1: Identify the page to which you want to build a link.

Start with a simple crawl of your site. From the main dashboard, you can scroll to the right and see how many pages link to each page. If Screaming Frog can’t find a page, it has no internal links and Google might not be able to find it either (note: this is where XML sitemaps come in handy but don’t rely on that alone).

This will also help you approximate how many links you need. The most important page on any site is the homepage. This probably also has the most internal links. The least important pages probably have the least number of links. Where do your landing pages fit in this scale? If they are among the least linked-to pages on your site, you need to build some more.

Now that you have data telling you which pages need internal links, you can…

Step 2: Identify a couple phrases from which you’d like to get a link.

Ever since Google started cracking-down on links, people are afraid of anchor text links. They should be- except for internal links. The good news: we can get away with many more keyword-rich anchor text from internal links. Don’t go overboard here- still use some variation. My point: you can do a lot more with internal links than you would ever do with external links.

Once you know the anchor text phrase from which I would like to build a link…

Step 3: Create Your XPATH phrase to find your new anchor text.

XPATH is a way of reading XML text. Since HTML is a form of XML, you can use XPATH to read your web pages by specific elements. Sure, you could simply search your site for a particular phrase. With that, you risk wasting time with false positives. In this case, for instance, you want to know what’s in your content- not your menu. XPATH will allow you to specify a search within certain HTML elements.

For example, if you want to find a particular keyword that occurs within a paragraph tag, I could say:

//p[text()[contains(.,’particular keyword’)]]

That’s cool, huh? We can make this even better with a case insensitive search:

//p[text()[contains(translate(.,’PARTICULAR KEYWORD’,’particular keyword’), ‘particular keyword’)]]

This assumes your content is contains paragraphs. Some CMSes still use BR tags (ugh! but not an SEO problem). Check your HTML to see what HTML container holds your content. Change the XPATH to fit your site. Remember: we’re looking for places within your content (not just our menu) that mention a particular phrase.

Step 4: Create your XPATH phrase to the page to which you’d like to build a link.

Now you need to create a similar XPATH query to look for pages linking to your desired page. For example:

//p/a[@href[contains(., ‘/parent/page/’)]]

This asks for pages with links to a particular page contained within a paragraph. You might need to do some testing, to make this fit your site’s URL structure. For instance, if you have daughter pages (like /parent/page/daughter/) this XPATH query will show you links to that page, too.

Step 5: Enter your XPATH Extraction searches into Screaming Frog.

1. Open up your copy of Screaming Frog. Go to Configuration > Custom > Extraction.
2. Click “+ Add” at the bottom of the window that pops up.
3. A line will be added within the window. by default it will show XPath as the search option. This is where you’ll enter both of your XPATH queries.
   1. One of your XPath queries will search for your keyword on pages. Unlike the image (below) select “Function Value” and NOT “Extract Inner HTML”
   2. One of your XPath queries will search for pages linking to your desired destination page. Unlike the image (below) select “Function Value” and NOT “Extract Inner HTML”
4. Click “Okay” and then crawl your site as normal.

Step 6: Crawl your site.

Since you’re only interested in HTML, change the Spider Configuration to not crawl images, javascript, CSS, external links, etc. You can see the spider in action by going to the bottom of the right column under Custom > Extraction.

Once the crawl is complete I start the real work and…

Step 7: Build the internal links.

Looking at the Extraction results, I now know what to do:

• If the second column (searching for your phrase within the content) has words but the third column (searching for a link to your page) is empty, you can build a link from there.
• If the third column has some text, there’s already a link to your page. If you’ve setup the XPATH right, this is not just a navigation link.
• If both extraction columns are blank it contains neither your phrase or your link. It could mean your XPATH has a typo and is not working right. I’ll admit, I had to test things a few times before I got it right.

If you can’t find enough linking opportunities, what should you do?

• Go to pages relevant to your topic and mention something related to your landing page’s topic. This will freshen-up your old page and give you a linking opportunity.
• Write some blog posts that relate to the topic of your landing page. This will bring long tailed traffic and give you an internal linking opportunity.

How do you decide whether you need more internal links, to a landing page? Have a better XPATH query example? Can you think of other uses for the Extraction feature? Leave your thoughts in the comments, below.