Here’s my history of hosting on the Internet. Before 1994 I built software for everything from supercomputers to home computers (I’ll tell you more about DesignWare some day).

• 1994- Virtual hosts on Best.com; 1999- Virtual hosts on Verio (bought Best.com)
• 2003-2005: New X-Serves on Maccius (Mac-only host in San Jose)
• 2009: Slicehost, and then Rackspace cheap virtual hosts- “slices” in the cloud
• 2012: WPEngine — specializing in WordPress, with caveats

Why this particular migration and what would I recommend that you do today?1994-2003: Virtual hosts were the way to go when the Web was first ramping up. They’re easy to start up (click a few buttons) and you can use FTP to upload, and most of the provide root access so you can load any software you want to. But they were relatively expensive, given that they are on shared computers and you’re at the mercy of the other users who’re on that same computer – if someone runs away with the CPU you end up with a very slow site. (And I’ve been guilty of messing up the other users too…)^[1]

2003-2008: Later on, I tired of paying $95 a month for a slice of a larger computer and bought my own Apple servers. These machines really rock. Lots of power, and dedicated to me alone. No need to worry about other users getting in my way. For $125 a month I get a space in a rack and I can run my server at 100% CPU if I want to. Cranks out sites really quickly.

2009: Virtual servers have come down in price. I looked around early in the year and chose Slicehost, which has a simple $25/month server with 256MB of “RAM” and 10GB of RAID storage, and they give me 100GB of data transfer per slice per month. I wouldn’t have gone back to virtual hosting except for the low price. Slicehost was acquired by Rackspace, and made a part of their cloud late this year, and I also had a few virtual servers on Rackspace. So they’re all consolidated under Rackspace now.

2012: WordPress specialists, such as WPEngine, have come along, providing service for those who use WP but don’t care to do the security and system upgrading.

Only for the brave: The pros and cons of slicing up a big server: Well, 256MB of RAM is pretty small, and that’s what Slicehost allowed you to start with. Rackspace starts at 512MB now. You gotta install a lean operating system (Ubuntu 8.04 was my choice — but now Ubuntu 12.04 LTS), and you gotta really tune it up so it functions well in tight places. The nicest thing about the old Rackspace slices is the 4 powerful CPUs, which give you lots of headroom if you have to crank out a page. If you buy a new Rackspace VPS now (December, 2012) you’ll get a single CPU allocated, with 512MB of RAM as the minimum.

Yup, if you like getting your hands really dirty, then a slice might be for you. Ubuntu has gotten pretty easy to install – and on Rackspace you just click a button and a few minutes later they send you a hostname and IP address and it’s running. And adding new features is about as simple as running “apt-get install foo” and then you do a few configuration changes. Well, almost that easy. Every once in a while I run into something that I think is going to take an hour and it takes two days. But I can configure a new host from scratch in about 25 minutes (Apache, PHP, MySQL) and bring up new web sites in about 20 minutes each.

WordPress specialists: I’ve got to say something about WPEngine here because it’s a love/hate relationship. First, WPE provides a great service because they give you a functioning WP engine in one click, for as little as USD $25 a month. They provide a one-click staging version of your site where you can test new plugins and do your own PHP coding without upsetting your live site. They host under your own domain name or theirs — you pick. They “curate” common plugins, taking care of any problems and updating them so you’re always up to date. The one caveat is that you can’t use certain banned plugins (mostly those that are database intensive), and they sometimes push upgrades without warning you, which might break your non-curated plugins! I recommend them if you have plain vanilla WordPress needs, but my jury is still out in terms of whether more sophisticated sites should use them or not.[2]

[1] A Virtual Server or Virtual Host or Virtual Private Server [VPS] is a piece of software that subdivides a big server into smaller “servers” that from a user’s viewpoint look like independent machines, but in real life are just logical (thus “virtual”) subdivisions of the larger computer.

[2] I’ve brought up several sites that needed large bursts of speed and high reliability. WPE is great for the bursts, which would have bogged down a VPS. However, we’ve been surprised several times when our sites just stopped working due to unexpected WPE upgrades. They do not tell you in advance when they’re going to change some plugin out in the middle of the night, so you have to be on call 24/7 watching for such things. You just don’t have the same degree of control here in terms of staging your upgrades and improvements and then rolling them on particular days because they might do one suddenly that you didn’t expect.

The post Hosting- Rackspace on steroids and on to WPEngine (for some) appeared first on Sky's Blog.

If your web site requires a lot of CPU time to generate pages, then nginx may not hold any advantage for you. [2. I define “A lot” as more than a second.]. You can test to see what your page-generation time is using webpagetest.org — and what you want to look at is the bar that shows how much time elapsed between the browser’s request and the delivery of the first byte of the page. The time it takes to serve that first HTML file is pretty much composed of CPU and MySQL (database) time. If the time between the HTTP request for the page and the first byte served is long (a couple of seconds) then your site is probably too CPU-intensive and nginx may not help you out very much.

That said, more and more people are moving toward specialized WP-hosting, where they don’t have to worry about what web server is used at all. And within a few years this may be a moot point, as nobody may be self-hosting their own WP any more!

Footnotes:

The post Nginx may not improve your performance compared to Apache appeared first on Sky's Blog.

In the early days of the public Internet, I exclusively used cloud servers. Cloud servers in the sense that they were virtual private servers whose location was unimportant. This was the most economical way to set up “your own server” and operate it without having to purchase hardware and colocate it somewhere in Santa Clara or San Jose. Or even L.A.

Then, in the early 2000s I bought Apple X-Serves and colocated them at a little Mac-only ISP called Maccius. I liked their premise and I liked the fact that they were hosting part of Apple’s developer network for Apple. My $3000 X-Serves were good investments, running 24/7 from 2003 until 2011 with only a reboot here and there (maybe once a year). I upgraded storage to RAID, and I boosted their RAM memories, but other than that these machines were real troopers with plenty of compute power. I saved a lot of money by owning my own boxes and just buying electricity and bandwidth from Maccius.

But by 2008 it had become cheaper (and more secure) to purchase computing as a commodity — cloud computing was coming of age. So I started launching virtual private servers again, at Slicehost and Rackspace (and eventually some at Amazon AWS). I could pay $12 a month for a tiny virtual server at Rackspace and not have to worry about some disk drive failing on me. And I could size ’em up or down as necessary.

By 2010 I was paying more for the virtual servers than for the real hardware boxes, but I was hosting far more sites in a more flexible manner. And Apple decided to exit the dedicated server business. So no hope of replacing hardware, and no more server software upgrades.

So in 2011 I shut down my X-Serves, one at a time, until I pulled the last plug (figuratively) out of the wall this morning.

And I’m entirely back in the cloud again.

The post End of an Era – back in the clouds again appeared first on Sky's Blog.

MY TOP APP PICKS FOR SYSTEM ADMINISTRATION ON iPAD

• iSSH— gives you secure shell (SSH) access to your servers using name+password or digital certs. If you use a command-line editor on your server (I use vi), be aware that up-down-right-left arrows won’t really function if you use the onscreen keyboard, but from a bluetooth keyboard they do work! Recently I’ve also had trouble with ESC, and I’ve had to tap its onscreen “button” instead on the physical key. You can also configure iSSH to emit true function keys (which are needed for some configuration work—in htop, for instance).
• 1Password— what a great way to keep all those passwords in one place! And encrypted too. 1Password for iPad syncs with 1Password on my Mac through Dropbox. When I make a new password, or change one, it is always available on the iPad as soon as I need it. This way I can use those 20-character random passwords that I’d never remember if I had to commit them to memory.
• Dropbox— Well of course you already know I use Dropbox for sync’ing 1Password across devices. And you can do without it if you sync the two devices “locally” on wi-fi, but I would never remember to do it—Dropbox lets it happen more in real-time and effortlessly.
• DropDAV— (Not an iPad app, but essential nevertheless) I need DropDAV because I have a buddy who watches my back and serves as sysadmin when I’m on those long air flights or otherwise indisposed, and he and I need to share documents, which we do through DropBox. DropDAV isn’t an app, it’s a service. Sign up and it makes your DropBox documents available to Pages and Keynote through WebDAV services on DropDAV.
• WordPress app— HTML textboxes don’t scroll properly on Safari on the iPad. This is a really big problem if you’re trying to admin a WordPress blog in Safari. So the WordPress iPad app is a necessity, though you don’t really have access to all of the WP admin features (it’s designed for bloggers, not admins), which means I’m constantly back and forth between this app and Safari even when I’m working on a single blog. This needs improvement, but I can make it work well enough for now.

PROBLEMS WITH THE iPAD

• No Flash. This means I can’t fully utilize a lot of tools, like Cloudkick, when on the road because they use Flash extensively. (However, I can log in at CloudKick even with my Yubikey one-time-password USB device, as long as I have the iPad USB camera adapter with me. That’s a trick to be explained elsewhere.)
• There’s no PGP mail encryption/decryption for the iPad mail app. Although I have other ways of dealing with encrypted mail when I’m on the road, this is still a big problem. If you rely on encrypted mail, be sure you have an alternative available when you’re traveling with your pad.

The post Top sysadmin tools for iPad appeared first on Sky's Blog.

Cloudkick runs a slick service that lets you manage your Slicehost and Rackspace/Mosso cloud servers (both brands are owned by and housed in Rackspace facilities), providing some data visualization in addition. Of course you could manage your servers using the control panels provided by Slicehost and Rackspace, but Cloudkick’s one-layer-removed management adds some unique extras. And they can “automatically” migrate your Amazon EC2 servers over to Rackspace if you’re interested in that. I have nine virtual servers on S&R (as I’ll call the two services) and signed up for the free Cloudkick services almost three months ago. They were voted Best in Show at Under the Radar (see Cloudkick writeup and video). The company is a little under a year old. It was kick-started by a seed investment by Y-Combinator, and recently obtained an angel round of investment.

They start by giving you a nice little control panel that shows the status of all of your servers (you have to “register” each service with Cloudkick – giving them your API information from Rackspace or Slicehost). They determine the status by pinging a handful of services on each virtual host.

It seems to me that ultimately they’d like to take over the complete management interface so you don’t ever have to go down to Slicehost or Rackspace interfaces. Currently you can fire up a Rackspace virtual server directly from Cloudkick, without ever touching the Rackspace Cloud control panel.

I separately monitor my own servers to be sure they’ve not been hacked, but the Cloudkick service adds a dimension I like, by integrating the control together in one place. And today they described some future services that I know I’ll be able to utilize. I particularly like the idea of additional data visualization, and that’s on its way. Here’s the kind of graphing they provide right now… this shows web page load times for two Slicehosted sites:

And here’s the UStream video of Alex describing CloudKick.

[swfobj width=”480″ height=”386″ flashvars=”autoplay=false” allowfullscreen=”true” allowscriptaccess=”always” src=”http://www.ustream.tv/flash/video/1426508″]

The post How will I manage? (the cloud) appeared first on Sky's Blog.

A virtual host looks like an actual server. You (or your programmer/sysadmin) can use it as if it were your own dedicated server. In fact, however, it is only a portion of a much larger server. Rackspace/Mosso, and Slicehost are two I’ve discussed and actually use. The focus is on the virtual server.

An automatically-scalable hosting solution is a service or set of services which are hosted on one or more computers, and you can’t actually tell how big the server is or for that matter, whether it’s a whole array of servers. The focus is on the virtual service (not the server itself).

What’s good: Virtual servers are a more secure environment than shared servers because you are only dependent on your own security efforts. (On a shared server, if another user picks a poor password, or doesn’t upgrade their PHP software when security upgrades are released, you can be hacked if their account is compromised.) Automatically-scalable hosts may also be secure in this same way if accounts are adequately protected from each other.

What’s bad: A root compromise of a virtual server may be possible. In fact, it’s probably inevitable that such things will happen. And if you don’t update your underlying software (like WordPress, for instance), they you’re likely to be in trouble anyway. So ultimately any server can be compromised.

I’ve written about “economic denial-of-sustainability” attacks, in which an attacker causes a cloud user to so scale up their server usage that it becomes economically impossible for the defender to survive. These wouldn’t be possible if there were no cloud computing.

The post Security in the Cloud – Matey, there be challenges ahead appeared first on Sky's Blog.

In the sense that data was being stored somewhere and the customer didn’t know where it was, then yes it was a cloud failure. But I contend that it was also a failure caused by the existence of a single-point-of-failure. (As system developers, this is our constant nightmare.) The product/service was set up to use a single data service with (apparently) inadequate backup. That created the possibility of this single-point-of-failure. The real failure was that T-Mobile didn’t provide the option to back up your data in a location under your control, so that you could later on restore it if the central service went down.

When I was working at Leapfrog to construct systems that would support “online” toys that would get and store data in the cloud (yes, in 2000 we were doing that too), we considered these same possibilities. We had not only a distributed Oracle database running on more than one computer, but we were streaming our data out to a backup in “real time” so we could reconstruct the data if something nasty happened. And once before we launched we had to take advantage of this when a database administrator‘s fingers slipped and he destroyed the data in a critical part of the database. These were multi-gigabyte databases that would have taken a long time to restore. We really didn’t want to lose them!

That’s why I still purchase CDs before uploading them to my iPod. At least mostly. The few albums I purchase and download, I always write off to CDs for safekeeping. When considering purchasing any system – computer or smartphone – consider how you’re going to back up your data, and maybe in more places than just one.

The post Is the Sidekick failure a cloud failure? appeared first on Sky's Blog.

Geek alert! This article is for Slicehost geeks only.

Talk about playing on the edge, I almost fell off this time. I have eight slices (virtual servers, that is) on Slicehost (which is now owned by Rackspace) and believe me they are tiny!^[1] But they can serve low-volume to medium-volume web sites pretty well. For instance, Shaping Youth (several thousand visitors a day) and GirlsHorseClub are on these little servers and they serve up pretty rapidly.^[2] Nice thing about Slicehost is that any host can be scaled up from the tiny size to 16x that size, and because it’s cloud computing, you can also bring up multiple instances of any server and have them share load. This allows me to implement the traditional three-tiered architecture (web/application/database) quite well without having to worry about whether my servers will be able to handle the load because I can scale ’em up whenever I need to.

But, the catch is that when you start with the tiniest slice (256MB RAM) you really have to fine-tune your operating system and applications in order to get good performance out of them.

This has led me to some fun discoveries about 1) how to make Apache work in a really tight space; 2) how to add additional web serving using NGINX (engine-X); 3) how to run MySQL in less space; and finally 4) how to use Java and Tomcat wedged into a non-existent memory slice.

Today’s lesson for me was installing Tomcat 5.5 on Slicehost. I started by reading How to Install Tomcat on Ubuntu in the mkyong blog. Not bad at all. The essence is to first find out what version of Tomcat is available on your slice:

sudo apt–cache search tomcat

Then install that version and the admin app for it:

apt–get ––fix–missing install tomcat5.5
apt–get ––fix–missing install tomcat5.5-webapps
apt–get ––fix–missing install tomcat5.5-admin

Then I like to have /usr/local/tomcat defined (like on all my other servers):

ln –s /usr/share/tomcat5.5/server/ /usr/local/tomcat

That was about all it took. It serves by default on port 8180 – and the admin app was there and ready to go. My next step was to port a Java app that I have been running on bigger servers over to this little server. That actually worked quite well, requiring about 6 hours to port, recompile (to eliminate a few warnings) and test.

[2] These two slices actually had to be boosted to 512MB recently in order to handle anticipated high traffic. But I can reduce their size after the stress-out period passes.

The post Slicehost- Tomcat installation on a tiny virtual server appeared first on Sky's Blog.

First, here’s my own take on cloud computing in the future. I can see that within a very few years (maybe now for some of us), many of us will not know (or care to know) where our data reside. Instead, we’ll be using our home computers as netbooks, connecting to our databases, friend-networks, profiles and documents, and we really won’t care where the data live. Today we see this happening with the rise of Google Docs (documents), gmail (Yahoo and Hotmail too, of course for years), and I see more and more people content to “just Google” to find answers to questions and no longer needing to have all of the books sitting on bookshelves at home.

For years I have been replacing my worn-out old computers with new and more powerful computers, making a 2x to 5x improvement in speed each time^[1]. However, I’m increasingly putting my documents online (Google docs), and my email is being retained on servers for online use. My most powerful computer is the iMac in my music/video studio, which is not my primary workstation (I can keep it cranking for hours converting video from one format to another). My primary is a laptop system. And at times I’m going out to meetings with just my iPhone and no computer at all – because the iPhone has enough connectivity and power to do what I need. I know I’m not unique, and in fact as a power user I may be carrying far more compute power with me than 90% of the populace, who have moved into the cloud even faster than I have.^[2]

Why does JD bring identity into the equation? Well, if my data is no longer physically on my computer, then what’s left to me is identity. And in a sense identity has also become disconnected from my physical computer(s) and consequently harder for me to locate. And identity is rapidly becoming portable, though many companies would prefer that it not be.

Some ideas from JD’s report:

“The Internet and a host of new communications technologies have transformed the concept of identity and redefined our relationships…” “Growing numbers of digital natives now define themselves by their Web presence as well as their real-world presence.” “…mass markets give way to millions of niche markets and … culture changes from one in which companies push products to passive consumers into … empowered users pull down products and services that meet their needs…” “It appears that the full cloud, in its startling richness, remains at least a decade away,” “The public may not be familiar with the term, but many are already doing cloud computing. We have been using Web applications for years without any concern about where the applications actually run.” “The cloud has become our entertainment network: we are spending hundreds of millions of hours on sites like YouTube, Hulu and Flickr.” “The term cloud computing, which came into wide use in tech circles only in early 2007, does have a specific, technical meaning. It refers to a collection of resources—applications, platforms, raw computing power and storage, and managed services (like antivirus detection)—delivered over the Internet.” “While Amazon’s utility computing solution chiefly targets small businesses and consumers, companies such as Salesforce.com and Netsuite are targeting businesses of all sizes with both virtual hosting and software suites to run their data.” “More than a decade ago Oracle CEO Larry Ellison declared that the network would become the computer, …”

Some final thoughts from me:

I am concerned about the fragility of the network. When our data live in the cloud and networks fail, we lose access to our data. This happens to me more often than I’d like – probably a couple of times a month in my home office! What will happen in an age of netbook^[2] computers? I think that ultimately the cloud is going to be partially collocated in your home or office, as well as off in some big server farm in Oregon (or pick any other state, you’re probably right). I think this will be solved by a combination of local storage (Network Attached Storage [NAS] perhaps, which is what I use right now), local processing power (I have a big computer in the studio and my laptop computer can be used to control it), and remote processing and delivery of certain services that really belong out in the cloud closer to the customer.

The full cloud … a decade away? I doubt it. We’re all using aspects of the cloud every day. It will progress rapidly. However, I do think that full acceptance of cloud computing is going to require a generational shift — in other words, today’s digerati over the age of say 40 may not be able to shift to cloud computing at all. Those in their 50s and 60s are still sending documents around via email rather than collaborating online.

Cheap computing and cheap communication? Well, I don’t find communication cheap right now. My phone bill used to be $25 a month. Now it’s $100 for a landline and close to that for a mobile plan with adequate data capabilities. I buy cloud computing and they nickel and dime me to death…the starting price has gotten better (around $25), but as I use more bandwidth and CPU power, it becomes more expensive than owning my own servers. Someone has to make a profit somewhere, and the guys operating the cloud are first in line, which is OK by me, but I don’t want anyone to think I’m promoting services that are by their nature the cheapest solution to all problems.

Abstracted and scalable are the two adjectives I like to use the most when comparing cloud computing to other computing. And those are the characteristics I most like about cloud computing.

Yeah, it is where we’re going. Without question. Most certainly. And we’re going to get there soon.

[2] One of the odd forces pushing people toward cloud email, in my opinion, is the unreliability of their home computers. Get a virus and your email might be destroyed. Hard drive breaks down and your email is gone forever. Upgrade to a new computer and start all over again? Nah. It’s far easier if your email is at gmail because when you get a new computer your mail is still in exactly the state it was before you switched. And when you get off the plane in New Delhi, your mail is still available to you at the hotel (or Internet café) even if your laptop computer is back in San Francisco.

[3] Bonus: A recent Infoworld article on netbooks.

The post Identity in the Age of Cloud Computing appeared first on Sky's Blog.

What will happen in an age of netbook computers? When the computing power and the data live off in the cloud somewhere, and my little computer is used primarily to connect, and not to process?

Well, I think that ultimately the cloud is going to be partially collocated in your home or office, as well as off in some big server farm in Oregon (or pick any other state – you’re probably right). I think this will be solved by a combination of local storage (Network Attached Storage [NAS] perhaps, which is what I use right now), local processing power (I have a big computer in the studio and my portable computer can be used to control it), and remote processing and delivery of certain services that really belong out in the cloud closer to the customer.

Oh, and maybe I’d better look at the fine print in my Comcast contract – I am purchasing a “home” Internet service, and of course I am doing business-related tasks here every day. They probably don’t deliver the same kind of reliability to homes as they deliver to businesses. But I’m not the only one doing this, and if you combine the current rage for netbook computers and the current state of affairs in connectivity, it leaves us pretty vulnerable to outages.

Some netbook references:

• Netbook on Wikipedia (this is not the best or most complete article it could be, but it’s a start)
• Larry Ellison’s ideas about the network computer
• The ASUS eee PC (I saw three teens opening the box and oogling a new one outside Best Buy yesterday on a warm afternoon – they couldn’t wait to get it going) – look at their web site – this machine is clearly for (young) teens – these kids in the ads must have an average age of 14
• The OLPC laptop, which in many ways, but not all, is a netbook
• Is the iPod Touch a baby netbook waiting to grow up?
• Dick Tracy wrist-radio (is it a cellphone? is it a netbook? aw shucks, is it anything at all?)

This whole article is a precursor to a longer article about cloud computing’s role in our future, and inspired by a series of reports by JD Lasica.

The post Netbooks and the Fragility of the Internet appeared first on Sky's Blog.