The Problem with Related Post Plugins

Showing related content to your users is important. I don’t think there’s anyone disputing that (at least not that I’m listening to). The real question is “how?” How can you show your user good related content without adding a ton of extra work for yourself? This is where related posts plugins come into play.

There are a lot of options out there. So many that it’s quite time consuming to try them all until you find one that suits you. Two of my favorites are WordPress Related Posts and Yet Another Related Posts Plugin (YARPP). Yarpp gives you more control over how matches are made, but for that very reason it’s also less efficient. Joost de Valk referred to it as a “heavy plugin” in his article on Optimizing WordPress database performance, and it definitely is. WordPress Related Posts is far more efficient, but offers you a little less control over how matches are made. Unfortunately they share the same problem.

So what is this problem? They all find matches to a post in the front end rather than the back end. They do it when a user views a specific post, rather than when a post is created or modified. On a brand new site I launched, which has only 7 posts, we’ve received roughly 2000 pageviews. That’s pretty low, but lets take a look at it. About 700 of those visits were to the home page and about 1300 were to single post pages. If you only show related posts on single post pages (which is how we currently do it) then the related posts plugin has been run over 1300 times for only 7 posts, which is roughly 185 times per post! If I were to show related posts for each post on the front page then it would have run another 4000 times (which is a conservative estimate), bringing it to 757 times per post. If you think this seems excessive, lets take a look at the stats for Web Developer News. It has had over 13,500 page views in the last 30 days. About 750 were to the home page, about 140 were to other static pages, roughly 550 were to tag pages, and another 250 were to miscellaneous pages such as search pages. That leaves 11,810 visits to single post pages and 21 posts during that same time. That’s about 562 times per post! If I added related posts to the home page, tag pages, and search pages it would need to be run roughly another 15,000 or 1,276 times per post.

In addition to the simple waste of memory and cpu cycles, putting the calculation of related posts on the front end means that when they start to slow down your users are left with a bad experience and slow loading pages. And they will slow down eventually, if your site gets big enough. I just recently worked on a site with more than 6,000 posts and over 1,800 tags (they offer investing advice, and have a lot of tags that correspond to ticker symbols). At that point, some of the related posts queries were taking 10 seconds to run. That’s really show. We can do better than that right? Well it’s not easy but I think we can.

I would always prefer that I (or my content creators) have to wait, rather than make my users wait. So, the logical solution would be to match related posts when a post is saved, and store them in the post’s meta data. Simple and solved right? Wrong. If that’s all you did, then posts would only ever be related to posts older than themselves. This would mean that part two of a series would likely relate to part one, but part one would never relate to part two. We solved the speed issue, but we took a huge hit in functionality. However, a plugin that I’ve developed and am testing takes it farther than that. When a post is saved, it’s related posts are saved, but all possible related posts are spidered and checked to see if they need to have new matches calculated for them.

For extremely large sites, the plugin still causes delays when you save a post. However, better you than your user. If you’re interested in using this plugin, I’ll be releasing it soon and I’ll add a link to it here when I do ((Efficient Related Posts)). However, for now I need a name for it! I originally called it Related Posts – High Volume because it was meant to fix issues on high-volume sites. However, since it can apply to almost any site, I’m looking for an inspired name that really lets people know what it can do. You can leave any ideas you have in the comment here or send them via twitter to @aaroncampbell or @wpinformer.

UPDATE:
The WordPress plugin has been released. You can download Efficient Related Posts from wordpress.org and you can read more about it at Efficient Related Posts at Xavisys.com

About Aaron D. Campbell

Owner and lead developer at BlueDog, Aaron has 10+ years of web development experience, it a regular core contributor to the WordPress project, and has released many WordPress plugins.
This entry was posted in Wordpress and tagged , , , . Bookmark the permalink.

20 thoughts on “The Problem with Related Post Plugins

  1. Dorothy says:

    Your plugin sounds good. How about: ‘Intelligent related posts’?

  2. Dorothy: I like “Intelligent Related Posts”

    Joost: I already sent it to you (04/02/2009 from aaron@xavisys.com). There’s been a lot of testing on sites since then, but not many changes at all.

  3. Sounds like something I need to try :)

  4. Jason says:

    Spidered Posts
    Future-Proof Related Posts
    Granular Related Posts

  5. Jihad Haidar says:

    The idea and reasoning behind this plugin is very good.
    Suggested names:
    “Who goes there” (or what)
    “Link Mill”
    “Linking Path” (rhymes with Linkin park lol)

  6. Peter says:

    Super Related Posts

  7. Yes it would, but even then someone is going to be the poor user that generates the cache the first time. That user’s experience will be extremely poor.

    Also, the more posts/pages and categories/tags you have, the more unique URLs you have. The more unique URLs you have, the more cache needs to be generated, which just means more users will have a bad experience.

  8. efikim says:

    If you used one of the cache plugins wouldn’t this substantially reduce the impact? (That’s not to say that fixing the problem where it starts isn’t a better idea, but there are probably far more wordpress users able to install a plugin than able to rewrite related post plugins)

  9. Pingback: Efficient Related Posts WordPress Plugin

  10. BGR says:

    fastter loading pages is what I want. so I am game and will give it a try (currently using contextual related posts—works ok actually)

    One of the problems I have had with such things is where it shows up and I don’t see anything to config.

    also not sure difference between auto insert into posts and auto insert into single posts only

  11. BGR says:

    I tried it and got a long table of numbers up to like 9248 in .02 seconds and processing stopped and all the checks had gone back to default like nothing happened.. WT?

  12. cja: If you’re talking about the actual plugin that I created as a solution to this problem, there’s a brand new support forum (there’s one for each of my WordPress Plugins).

  13. Gravity says:

    I just disabled Yet Another Related Posts Plugin because my site was suspended for high CPU usage. Its queries were insanely large and slow, and done on every page view.

    Your new plugin looks like the solution I need.

  14. Chet Payne says:

    On my new site I got rid of yarpp and installed yours. Trying to get anything perfect I can see is a real task.

  15. hello to you, thanks for this plugin, I setted options but I don’t have any related articles, I have many similar tags, so what is it wrong?

    Another thing I hope solve first one, but can you create a plugin that search the related article not finding in my blog but directly on google search?
    Thanks in advance
    :-)

  16. please how can I hide my email and show only name to avoid spam?
    thanks :-(

  17. Lachlan says:

    Running your excellent plugin on my iPad site. Very nice work. Thx.

  18. Ben says:

    Hello Aaron,
    thank you very much for this great article. I only downloaded WordPress last week and the tremendous amount of plugins can be a bit overwhelming.
    So an article like yours, detailing the important distinctions to consider when choosing a plugin, really brings enlightenment.
    The plugin works great, many thanks again!

  19. Alexey says:

    The greenest related posts plugin, i guess.

Leave a Reply

Your email address will not be published. Required fields are marked *

Note: If you are replying to another commenter, click the "Reply to {NAME} ↵" button under their comment!