The tale of the WordPress XML-RPC attacks

I can’t believe I’m writing this post. But, I am.

This concerns something in WordPress that a lot of people don’t even know is there called XML-RPC. The XML-RPC feature has been turned on by default since version 3.5 (released 2012 December 11) and which I think has been in WordPress itself since the beginning even if not enabled by default.

Without getting too technical, XML-RPC is responsible for two main features of note: pingbacks and external blog post editor support. Unfortunately, it has recently become the vector of choice for what I can only assume is some kind of worm/malware which sends thousands of requests in an attempt to either break into a site or just plain cause havoc and increased resource usage. That’s where the problem begins.

This wouldn’t be that vexing of a problem except for one thing: I actually use XML-RPC legitimately. This post, for example, was made via the XML-RPC function, not WordPress’s internal editor (though I have made posts via the internal editor before). Most of the posts you’ve seen me make in the last three years have had the XML-RPC interface come into play during editing at some point. So, just turning this thing off isn’t an option.

I tried a couple of plugins which promised to “secure” XML-RPC and they did so by completely breaking it. (I can do a better job of this with “rm xmlrpc.php” or “chmod go-rx xmlrpc.php” myself, I don’t need a plugin to effectively do something like this for me.) It is difficult to know for sure, but given the relatively low traffic I have, it does not make sense that my resource usage is what it is. (Resource usage which I pay for, I might add.)

So, what the h-e-double-hockey-sticks am I, or other WordPress site admins, supposed to do?

Well, right now (actually, by the time I post this, as of a couple of weeks ago), my solution is akin to a duct-tape-quality fix: copy the existing xmlrpc.php to something else, and use that for my editor’s XML-RPC endpoint. It’s the only thing I could think of to do, and thankfully almost everything still appears to work as designed.

I say “almost” because this doesn’t yet solve the problem of legitimate pingbacks potentially getting dropped. My host has XML-RPC protection, but unfortunately that protection is to change permissions on xmlrpc.php at the first sight of an attack (breaking the functionality until I manually restore it). Unfortunately I don’t have a way to tell sites sending me pingbacks to send them somewhere else automagically just yet. It also means I have to repeat this step on each upgrade (well, at least if xmlrpc.php changes). Like most actual duct tape repairs, it’s probably not the best solution. But, for now, it works, and to me that’s what matters.

Libreboot and the Leah Rowe follies

Hopefully, this story has developed enough that it’s safe to write about now.

Not too long ago I was on IRC (Freenode #fsf) and someone pasted this URL linking to the Libreboot mailing list (all typos are as in the original, very slight snippage made for brevity):

Hi,

The Free Software Foundation recently fired a transgendered employee of the FSF, just for being trans, because some transphobic cissexist people wrote negativly about her. The FSF fired her because they thougdt she, rather than the assholes bullying her, was causing the FSF potential damage. As a result, she was fired from the FSF.

As a trans person myself, I find this disgusting.

I’m declaring here and now to the whole world that Libreboot is no longer part of the GNU project. I do not believe that the FSF or the GNU project deserve to exist.

[…]

Long live the LGBT community, and long live the free software movement. Meanwhile, FSF and GNU can both go fuck themselves.

-- 
Leah Rowe

Libreboot developer

Later that day, John Sullivan of the Free Software Foundation responded with this statement (quoted below in part):

This morning, an open email circulated in which the author said that the Free Software Foundation ended a relationship with one of our employees for discriminatory reasons.

Although it is our usual policy not to comment publicly on internal personnel matters for privacy reasons, we felt it necessary to state unequivocally that the allegations made in that email are untrue.

It is part of our job to celebrate and improve the diversity of the free software world. We have strong anti-discrimination and anti-harassment policies to help provide a safe and supportive working environment. […]

(Not quoted or linked above was a reply from Dr. Richard M. Stallman himself also denying Ms. Rowe’s accusations. You know stuff just got real when RMS has to take time out of whatever he usually does to respond.)

In the days since, no further details have come forth. Of the two, I am personally much more inclined to believe the FSF. Especially in light of the fact that Ms. Rowe claims to speak for the “libreboot community”, when in fact it is more likely she’s really only speaking for herself, per this post by Darnien Zammit on zammit.org:

Given the recent kerfuffle, and in spite of my vested interest in wanting to continue being paid to continue this important work [on the Libreboot project], I find it necessary to spell out a couple of facts I find important about the libreboot project and the libreboot community:

1) I have recently noticed that Leah Rowe is the only person who has git commit access to the website, libreboot.org, and also the only person who has git commit access to the codebase, which has only become a problem recently.

2) The codebase is a deblobbed coreboot repository, with patches from libreboot contributors (but committed by Leah), and a bunch of install scripts for ease of use.

3) We (the contributors) are not consulted about any of the views expressed on the libreboot.org website when they are hastily published by Leah.

So, whenever you read “We believe….” or “We say that…” on the lists and websites, Leah has ultimate control of the libreboot project currently. It is clear that this person has been misusing control of the project to spew out irrelevant personal opinions on behalf of the “libreboot community”, a singleton group of people consisting of … yes you guessed it, Leah Rowe.

I find it quite distasteful that someone like Ms. Rowe would take it upon herself to speak on behalf of the community which uses Libreboot. I doubt that the vast majority of the users approve of her profanity-filled exit from the GNU project. Honestly, not only does this look bad on Libreboot’s users, incidents like this add up to look bad on free software users in general.

It’s not really the use of profanity I mind as much as deciding to dissociate with a larger umbrella project like GNU over something like this. Especially given that it’s impossible to verify due to the fact names weren’t named, and the overwhelming majority of free software users are going to give the FSF the benefit of the doubt. Even if there is a rogue manager at the FSF, I’d rather, we as a community, give the FSF a chance to rectify the situation than have it splattered all over the net the way Ms. Rowe did.

I hope that someone forks Libreboot so the users who don’t want to be associated with Ms. Rowe in any form may choose otherwise. This is absurd and nobody who uses a free software project like Libreboot should have to put up with it.

The chilling effect of a DoS attack: why the KrebsOnSecurity incident should alarm all of us

Ars Technica recently reported on a troubling situation involving well-known security blogger Brian Krebs and his blog KrebsOnSecurity. Brian’s blog is now back online, thankfully, and he wasted no time firing off this post entitled The Democratization of Censorship which I will quote in part:

More than 20 years after [John] Gilmore first coined [the] turn of phrase [“The Net interprets censorship as damage and routes around it”], his most notable quotable has effectively been inverted — “Censorship can in fact route around the Internet.” The Internet can’t route around censorship when the censorship is all-pervasive and armed with, for all practical purposes, near-infinite reach and capacity. I call this rather unwelcome and hostile development the “The Democratization of Censorship.”

Allow me to explain how I arrived at this unsettling conclusion. As many of you know, my site was taken offline for the better part of this week. The outage came in the wake of a historically large distributed denial-of-service (DDoS) attack which hurled so much junk traffic at Krebsonsecurity.com that my DDoS protection provider Akamai chose to unmoor my site from its protective harbor.

The blog post goes on to describe the details, namely that a denial of service (DoS) attack was aimed at Brian’s blog that was so massive that Akamai could no longer afford to protect him from it. Rather than allow his own hosting provider to get swamped with the traffic, Brian instead chose to take his blog offline by pointing it at the loopback address; given that the vast majority of users are not running a web server on their own PCs, and certainly not on their phones or tables if they are using such a device, this had the effect of taking the blog down. Brian finally put it back up under the protection of Google’s Project Shield.

As to what figurative fireant mound Brian managed to step in, I’m not even sure it really matters that much. The problem is that this happened. If it happens to someone like Brian Krebs, it can just as easily happen to any of us. The tragedy here is not that Akamai and Google are willing to protect sites like Brian’s from crippling denial of service attacks. No, the tragedy is that it’s even necessary that such protection even has a need to exist.

In this case, Google happens to be the good guys. However, it is worth mentioning I have been highly critical of Google in the past; always for a good reason of course, but it’s inevitable that a corporation like Google (or Akamai, for that matter) will get one wrong once in a while.

That’s not to say that Akamai are the bad guys here; they protected Brian’s site for as long as they could, until it became completely unfeasible to do so. To give you an idea of the scale of the chessboard Brian is playing on, here is another chilling quote from his blog post:

In an interview with The Boston Globe, Akamai executives said the attack — if sustained — likely would have cost the company millions of dollars. In the hours and days following my site going offline, I spoke with multiple DDoS mitigation firms. One offered to host KrebsOnSecurity for two weeks at no charge, but after that they said the same kind of protection I had under Akamai would cost between $150,000 and $200,000 per year.

[…]

That annual figure breaks down to $400 or so per day. For most of us that’s a crippling sum; I would think even Bill Gates, Warren Buffett, Jeff Bezos, or most other multi-billionares would probably throw in the towel if keeping a blog or website up cost that much (and was operated purely for free speech and not profit purposes; obviously that’s a drop in the bucket for Microsoft, Amazon, or most of the companies owned by Berkshire Hathaway, for their public-facing and profit-generating websites).

It almost goes without saying, but if it cost that $400 per day to keep this site online, it would be gone. I’m not even sure I could take the two free weeks in good conscience, because I’d be quitting immediately thereafter. I would likely even be afraid to try to distribute the archives via BitTorrent for fear anywhere I tried to seed it from would be hit with the same type of attack. I certainly couldn’t use BitTorrent for posting new updates; maybe I could use something like Freenet or ZeroNet. That would likely mean giving up WordPress, which I really do not want to do.

There has to be something that can be done about this cybervandalism at the Internet service provider (ISP) level. I don’t get why the heck any decent ISP still allows the kind of garbage in a DoS attack outside of its own network. It’s one thing for someone to take control of a botnet on, say, Comcast’s or AT&T’s network, and take someone offline that’s on that network. It’s another entirely to take control of botnets across the planet and blast someone right off of Akamai’s network. I don’t even know if cybervandalism is still the right word at that scale; it’s more like cyberwarfare at that point.

Why is “rolling coal” still tolerated at all?

A recent New York Times story covered the practice of “rolling coal” which is perhaps the most flagrant disregard for the environment this country has seen in the past couple of decades. For the uninitiated, I will cite Wikipedia’s explanation:

Rolling coal is the practice of modifying a diesel engine to increase the amount of fuel entering the engine in order to emit an under-aspirated sooty exhaust that visibly pollutes the air. It also may include the intentional removal of the particulate filter. Practitioners often additionally modify their vehicles by installing smoke switches and smoke stacks. Modifications to a vehicle to enable rolling coal may cost from [US]$200 to [US]$5,000.

Rolling coal is a form of conspicuous air pollution. Some drivers intentionally trigger coal rolling to taunt environmentalists, such as those who drive hybrid vehicles (when it is nicknamed “Prius repellent”), as well as bicyclists and pedestrians.

The article goes on, of course, but that’s enough to get the gist of what is going on here. I will summarize by saying the environmental concerns are only part of the case against rolling coal, and there is also a road safety impact as the black smoke obscures visibility as well.

The Clean Air Act prohibits the modifications necessary for rolling coal, though rarely is that act enforced by itself. Only New Jersey has actually passed a law which outlaws rolling coal. I’m honestly surprised why the states which require vehicle inspections haven’t cracked down on this. (Incidentally, not all states require inspections, most notably Florida and Minnesota, per this Wikipedia article.)

We’ve already had this discussion as a society when it came to smoking. Those who wanted to breathe clean air have pretty much already won. The pharmacy chain CVS does not sell tobacco products any longer (I’m surprised how long it’s taking Walgreens to follow suit). Where I live (Houston, Texas, US), restaurants, bars, and pubs cannot allow indoor smoking, and only a few designated “cigar bars” are permitted to do so (I don’t think any new cigar bars are allowed inside city limits, but I could be wrong).

And so it is with rolling coal. The difference is, cigarette smoke has never been opaque enough to cause a traffic or safety hazard just from its opacity. Soot from diesel exhaust does. From the New York Times article:

The owner [of a Ram 3500 fitted with two steel smokestacks], Pryce Hoey, insisted his truck was emissions compliant, but nevertheless agreed to demonstrate its smoke-generating prowess.

“I just wanted something different,” Mr. Hoey said, revving the engine and releasing two black pillars of smoke into the evening air before Sgt. Worthington shut him down. “People who see it giggle. They think it’s funny.”

I wonder how many people giggle when they find out someone they know has lung cancer or another health issue related to inhaling this garbage. Or when they hear about a traffic accident. This is not funny, people. This is dangerous. I can only hope that Mr. Hoey got a huge fine from showing off his “emissions compliant” truck. It should not take a genius to figure out that anything that belches forth black smoke is at the very least unethical, and almost certainly is not “emissions compliant.”

It’s time to put a stop to reckless and senseless air pollution once and for all. I encourage everyone reading this to document rolling coal incidents and write their state (and federal) legislators, linking to photos and video so they can see for themselves.

Weighing in on the Brock Turner controversy

While a bit old, the basic issue behind the controversy surrounding this case will remain timely for quite some time, and is similar to the central issues in prior posts to this blog.

For those unfamiliar with the Brock Turner case, this Wikipedia article offers a summary, as well as the news articles it links to therein. In the following paragraphs, I am assuming at least a basic layman’s familiarity with the case so I’m just going to get right into my thoughts.

My first thought centers around the sentence as approved by Judge Aaron Persky. I am only vaguely familiar with his history, mostly what is mentioned in the Wikipedia article. The only other case which was noteworthy to make the Wikipedia article about him was regarding a civil suit about a gang rape of a female victim by male athletes from De Anza College. That case was found in favor of the defendants, notably after Judge Persky allowed seven photos of the plaintiff taken at the party to be viewed by the jury.

With that in mind, the facts are that the probation department recommended a relatively lenient sentence:

Santa Clara County probation officials, including his probation officer Monica Lassettre, recommended that Turner is given a “moderate” county jail sentence with formal probation:

During the presentence interview, the defendant expressed sincere remorse and empathy for the victim. In determining an appropriate recommendation, this officer considered myriad factors, including the impact of the crime on the victim and the safety of the community. Other factors included the defendant’s lack of a criminal history, his youthful age, and his expressed remorse and empathy toward the victim. Based on the aforementioned information, a moderate county jail sentence, formal probation, and sexual offender treatment is respectfully recommended.

With that in mind, Mr. Turner serves his jail time (and gets credit for good behavior). Then he comes back to people protesting out in front of his house and threatening him. I get the impression these people are actually angry at the lenient sentence approved by Judge Persky. They need to take it up with Judge Persky or add their voices to the people seeking to recall him from the bench. Mr. Turner’s role in this is to complete his sentence and abide by his conditions of probation for the next three years.

I don’t condone what Mr. Turner did. I find rape and sexual assault rather revolting, but as I have said before this country is a country ruled by law, not by vigilante justice. I am disgusted that people have taken it upon themselves to protest in front of his house. All that does is make the protesters look like a bunch of animals. This isn’t the jungle, and the human race as a whole is more intelligent than that.

As a very brief aside here, there are two things about this case that are unusual compared to the law and the way things are done here in Texas. First, it is unusual to me here in Texas to read about a judge presiding over both criminal and civil cases in the same court. Here in Texas there are civil courts and criminal courts and different judges preside over each. Though it is possible for a criminal court judge to later be elected to preside over a civil court and vice versa, it would be for different terms and judges are only up for re-election once every six years.

The second, more on topic, is that county jail time given as a condition of probation is served day-for-day here in Texas; probationers here do not get the good behavior credit that Mr. Turner got.