Sky's Blog

This morning at 12:30am I shut down my last Apple X-Serve  — intentionally.

In the early days of the public Internet, I exclusively used cloud servers. Cloud servers in the sense that they were virtual private servers whose location was unimportant. This was the most economical way to set up “your own server” and operate it without having to purchase hardware and colocate it somewhere in Santa Clara or San Jose. Or even L.A.

Then, in the early 2000s I bought Apple X-Serves and colocated them at a little Mac-only ISP called Maccius. I liked their premise and I liked the fact that they were hosting part of Apple’s developer network for Apple. My $3000 X-Serves were good investments, running 24/7 from 2003 until 2011 with only a reboot here and there (maybe once a year). I upgraded storage to RAID, and I boosted their RAM memories, but other than that these machines were real troopers with plenty of compute power. I saved a lot of money by owning my own boxes and just buying electricity and bandwidth from Maccius.

But by 2008 it had become cheaper (and more secure) to purchase computing as a commodity — cloud computing was coming of age. So I started launching virtual private servers again, at Slicehost and Rackspace (and eventually some at Amazon AWS). I could pay $12 a month for a tiny virtual server at Rackspace and not have to worry about some disk drive failing on me. And I could size ’em up or down as necessary.

By 2010 I was paying more for the virtual servers than for the real hardware boxes, but I was hosting far more sites in a more flexible manner. And Apple decided to exit the dedicated server business. So no hope of replacing hardware, and no more server software upgrades.

So in 2011 I shut down my X-Serves, one at a time, until I pulled the last plug (figuratively) out of the wall this morning.

And I’m entirely back in the cloud again.

I wrote about this – a few months ago in “Are hungry searchbots eating your site alive?” – but the saga continues! I need a rescue mission, so please will someone send in the SWAT team?

[Geek warning—this post is really for geeks only]

Here’s the short version:

• If you tweet your blog posts, there are hundreds of bots reading the twitter feed and waiting for your post;
• These bots immediately descend on your web server (following a tweet) and spider all over the place;
• If your blog is WordPress-powered or requires significant CPU or database resources to generate a page, this can slow your server at exactly the time when you most need the capacity for human visitors;
• The majority of these swarming bots do not properly identify themselves to your server; and
• The majority of them are coming from AWS now.
• It’s time to firewall unidentified bots hosted at AWS out of our blogs!

(continue reading…)

The San Francisco Oakland Bay Bridge will contain a unique new suspension span when it opens in 2013. Its tower began to rise above the horizon of Yerba Buena Island a few months ago. In this photo you can see three of the existing suspension span towers on the right, and behind the San Francisco skyline toward the left you see a construction crane and scaffolding holding the new tower while it is being pieced together and rising skyward.

A few weeks ago, the catwalks were constructed that will hold the workers while the suspension cables are woven.

Much as my best days are those on which I “learn a lot,” I find that some of my best days are also “totally scattered and almost devoid of billable hours.” In the last three days I’ve probably addressed ten problems for ten different people, and although I’ve billed out a good number of hours to a couple of clients, the majority of the others aren’t getting a bill at all. How do you feel when you’re in this kind of situation—is this extreme attention deficit disorder, or is there something useful to be learned from this kind of behavior? (continue reading…)