Day: April 17, 2024

There’s no real original news article or Wikipedia article to link to here. This is the original story, and I figure there’s no better place to put it than here.

As many of you may know, I blog about my arcade and pinball adventures (mostly pinball these days) over on SKQ Record Quest. Over the past few weeks, it’s been painful to do much with the site as it had become increasingly unreliable. First it was an occasional “500 Internal Server Error” and then they came increasingly frequent to the point where, at times, the site was basically unusable.

To make matters worse, the host I am on bills by resources used. Taking a look back, the resources used started skyrocketing at the start of the year. I did a few things that appeared to fix the problem at first, but it would eventually come back with a vengeance.

That is, until I finally took a look at what was going on and noticed the CPU for SKQ Record Quest was much higher than the other WordPress sites I have running. I finally began going through a guide that recommended a plugin called WP-Optimize. So I started going down through the list of options. At some point it told me about some cron tasks which had not been completed. Something like over 100,000 (I didn’t notice the exact number, and honestly I thought this was a glitch). So I followed the instructions and did a wp cron event run --all (after some blundering around to find this exact command). I was then greeted with:

Executed the cron event 'wsrs_update_blacklist_twicedaily' in 6.925s.

and many more lines like it, with different times on the end. Turns out that this was left over from a plugin I used to run called Stop Referrer Spam. Not only did this plugin do little, if anything, to get rid of the referrer spam problem I was having, apparently it left behind thousands of cron task turds behind.

I’m not waiting for nor paying for all of that junk to run, so I had to play Go Fish in the database server. Turns out deleting the “cron” key under wp_options (well, what my wp_options table is renamed to) is enough to fix this. Or so I thought, as after the first time it came right back. A second nuke of this key appears to have permanently fixed the issue.

I hate referrer spam, but right now I’m afraid to try any other plugins to fix the issue given what has just happened, and what it took to find and fix the issue. The truly horrifying thing is that I was only a couple more weeks away from taking an extended break from blogging to finally abandon WordPress and migrate almost a decade’s worth of content to a static site. Now, this blog and the other WordPress sites I currently have would be much easier. SKQ Record Quest, however, makes extensive use of Jetpack’s image galleries feature, meaning a large number of posts and images would need to be migrated to whatever is used by the static site generator I am migrating to. On top of this, I would have needed to find an alternative solution for analytics, and it would become much more difficult to post from my mobile phone should the need ever arise again. Also, just for good measure, I would likely have to edit many other links to WordPress-specific things like wp-content/uploads/2024/04/{image filename}. We’re talking easily a month or two of spare time involving a considerable amount of caffeinated beverage consumption and a nontrivial uptick in stress and uttered profanities. Oh, and this is spare time that would come out of bar/arcade visits (i.e. less time playing pinball) and possibly other social activities.

Now there would be advantages to such a move, the least of which would be that backup copies of static sites can be mirrored much more easily and on censorship-resistant platforms. Hopefully the potential censorship won’t ever be an issue; it’s a nontrivial amount of work to go from WordPress to a static site and there’s no guarantee I would be able to easily reverse the process should the original reason no longer make sense from whatever standpoint, be it technical, social, etc.

I’d like to try to make this a learning experience for as many people as possible. So here’s what I learned:

1. Be mindful of what plugins and themes you install. Most of the time, especially if you stick to plugins available from WordPress’s own directories, you shouldn’t have issues.
2. Take it seriously when weird things start happening on your website. You should never get a “500 Internal Server Error” and even the occasional one means something is going really wrong. I assumed a later plugin update was going to fix the issue eventually. It did not; manual intervention was needed to get things back to a sane state.
3. When troubleshooting, always thoroughly investigate anything and everything that is out of the ordinary. As I remember it, there were errors in a previous round of troubleshooting that should have pointed me to the problem had I investigated them more thoroughly. What I actually did was slap a quick bandage fix on it, which of course didn’t solve the real problem.
4. If you still have the Stop Referrer Spam plugin installed, get rid of it! Like this, if you have WP-CLI installed:
   wp plugin delete stop-referrer-spam
   and then, just to be sure, delete the entire cron row from your wp_optionstable. (Generally, open phpMyAdmin, go to your wp_options table, sort on option_name and look for cron, then delete that row. I’m not sure how to do this by typing in SQL commands if that’s your only option, but that’s the basic idea.)

Hopefully you’ll never need this advice, but it’s there if you do.