Internal Site Structure, and Perceived Page Importance


An often overlooked factor for your own SEO strategy is how your sites architecture and navigation can affect the way the link juice flows through your site.

We know that the 'authority' is diluted as we move away from the home page (or indeed any page that has the most external links built to it), but our visualization and understanding of page depth is often muddied due to fancy 'popular-posts' widgets, automatically generated menus, in-content links as well as site wide footers and side bars.

Screaming Frog is an awesome tool that should be in your tool box already, and if you can't afford the small yearly cost, then you can accomplish similar things using Xenu, although personally I don't use Xenu.

More...

Screaming Frog is an incredible crawling tool that you can use to crawl your own (or indeed someone else's) site, to generate a mind blowing array of reports and information.

Screaming Frog and Xenu will both, for a standard site crawl provide you with 'level' and 'links in' for every page on your site.

Level is the depth of the page from the start page (number of ‘clicks’ away from the start page).

In Links are the number of internal inlinks to the URI. ‘Interal inlinks’ are links pointing to a given URI from the same subdomain that is being crawled.

As a side note, Screaming Frog is a great tool for discovering broken links (both internal and external) and it's well worth running a quick site scan every couple of weeks or so just to ensure that your site is 404 free! You can also use it to find broken links on sites which may be useful for some more advanced (yet white hat) link building tactics.

After running a crawl on my site, I export the results and paste them into Excel or Google Spreadsheets to quickly analyze how my site authority is being distributed across my site.

Using Excel or Google Spreadsheets means I can just paste in the data and hit refresh, no matter what site I'm looking at. It's an ideal way to get a good handle on what pages or posts are the most linked to (i.e. the pages the webmaster thinks are the most important).

​I'll be running through a quick site audit in this blog post, detailing the steps that I take, so you can follow along with your own site. Please bare in mind that there is no 'right' or 'wrong' answer, but this is more a sanity check to ensure that your pillar articles or money articles are receiving a generous proportion of your links.

Crawl Your Site.

You only need to crawl internal html pages (so you can untick, images, javascript etc in the settings). This will make the crawl a lot faster. Depending on how big your site it, it will take anywhere from a few seconds to hours (if you're the Huffington Post).

Export to Excel

Once the crawl has completed, just select the 'Internal' tab, choose 'HTML' from the filter and click on the 'Export' button.

​Open up your file in Excel and you can see there is a whole swathe of data. For this exercise you can delete all the columns except for the following ones:

  1. Address
  2. Level
  3. Inlinks

In addition to these columns, create three new columns called 'Level Score', 'Link Score' and 'Total Score'.

In the new 'Level Score' column, we'll be giving each result a value based on how far away from the home page it is. Enter in the following formula and copy it down so each row now has a level score. The home page (with level of 0) should have a value of 1. Everything else should have a number less than 1.

​=1-($B2)/(MAX($B:$B)+AVERAGE($B$2:$B$46))

Next in the Link Score column, we'll calculate a value derived from the number of links the page has pointing to it. The value is a proportion, so if your home page had 100 links pointing to it, and your /blog/ page had 75 links pointing to it, the value would be 0.75.

=$C2/MAX($C:$C)

Now for the third new column, 'Total Score' we're simply going to add up the level score and the link score values.

=SUM(D2:E2)​

Now, select ALL your data and click Data and then Filter in the Excel menu. Then sort the column 'Total Score' from Largest to Smallest.

​Your table once completed should look very similar to this one:

The higher the value in the Total Score column, the more important that page is.

Assumptions and Caveats

Now as I'm sure you are already seeing there are some clear issues and shortcomings with this over simplified method, the biggest one being the lack of external link weight in the formula - but it's the point of this exercise to simply consider the 'internal' weight of importance, as dictated by architecture, widgets and menus that are used on our sites.

Questions?

Hopefully you'll have found this super quick tutorial really useful - if you find a page that has way more links that you think it deserves, you can click on the URI in Screaming Frog and dig into it some more to find out what other pages are linking to it.

Any questions or comments, feel free to add them below. I read and respond to every single comment.


4 Comments

  1. Brad December 7, 2015
    • Steve Rendell January 13, 2016
  2. Gokhan Tunc May 12, 2016
    • Steve Rendell May 17, 2016

Leave a Reply

Your email address will not be published. Required fields are marked *