I wrote about this – a few months ago in “Are hungry searchbots eating your site alive?” – but the saga continues! I need a rescue mission, so please will someone send in the SWAT team?
[Geek warning—this post is really for geeks only]
Here’s the short version:
- If you tweet your blog posts, there are hundreds of bots reading the twitter feed and waiting for your post;
- These bots immediately descend on your web server (following a tweet) and spider all over the place;
- If your blog is WordPress-powered or requires significant CPU or database resources to generate a page, this can slow your server at exactly the time when you most need the capacity for human visitors;
- The majority of these swarming bots do not properly identify themselves to your server; and
- The majority of them are coming from AWS now.
- It’s time to firewall unidentified bots hosted at AWS out of our blogs!
Digital nomads, you can finally and really be the system administrator for your cloud (and other) servers from your iPad. Since December, each time I’ve left town, I have intentionally left my MacBook Pro at home in favor of my iPad. I found that just having a few specific apps allowed me to fully administer my cloud servers from the pad. Please note that a bluetooth (or other) keyboard is required for some of these apps to function fully. But generally I can do everything I need to when I’m on the road.
When I got my iPad and started carting it around everywhere with me, it first went into the big backpack along with my MacBook Pro (15”), and since I’m used to carrying 20+ pounds in the pack, adding the iPad didn’t bother me at all. It’s a good workout. And when I’m flying internationally, I take one wheeled bag and the backpack, so it’s standard-issue for me.
Far from solving all your problems,