User-agent: *
Disallow: /

Instead, I discovered that the googlebot may still spot the site and then put up a message saying that the site exists but is not indexed. i.e. the Googlebot still publicizes the existence of the site. It makes Google look like the good guys and us look like the bad guys for putting up a robots.txt. Yay for Google liberating all online information! Boo for us trying to keep our site un-indexed until we’re ready to make it public.I suppose if the site is public, they reason it’s OK to mention its existence. However, most of us did not intend for any results whatsoever to show up in Google, so having it say “the site exists but I can’t index it” is a big of a revelation! Beware of this if you are creating a pre-production test site — your site may still show up in Google searches. Instead, turn on some other protection — like the “Maintenance mode” plug-in for WordPress, so that not only sites but humans can’t use the site. Here’s kind what the Google result looks like:

Mork-A-Bork » Uncategorized
mork-a-bork.info/

A description for this result is not available because of this site’s robots.txt — learn more

The post Even Robots.txt won’t keep the googlebot away appeared first on Sky's Blog.

Just for example, if you look at traffic to YouTube from Iran, you’ll see this chart… showing high traffic until June, 2009, when YouTube was blocked in Iran. The scale runs from 0 to 100 and is “normalized” rather than showing absolute bandwidth that’s being used in each country. So it quite nicely illustrates  various cases of heavy-handed content blocking.

To see how censorship effectively blocks YouTube in other countries, try looking at Bangladesh, China, Libya, and maybe some others you can discover in the data.

The post Google and YouTube censorship appeared first on Sky's Blog.

They’re experimenting with putting up a page that shows the number of requests they’ve received, and (partially) the action they took, for the most recent six months.

You can view their map and click the pushpins to see country-specific data. For China, it says

Chinese officials consider censorship demands to be state secrets, so we cannot disclose that information at this time.

Isn’t it interesting that rather than saying “removal requests” Google used the word “censorship” in this case?

To read more about legitimate (legal) requests and requests that do not have the force of law behind them and may simply be trying to intimidate a web site owner, visit the Chilling Effects Clearinghouse.

The Open Net Initiative seeks to identify and document Internet filtering and surveillance.

The post Google and government requests for data or removal appeared first on Sky's Blog.

This June 16, 2009 c|net article —  Google’s censorship struggles continue in China describes many of the ups and downs. I’ve written a few articles about this relationship, including Censorship as Punishment and Search engines’ blocking in China “improves.” [See also the Related articles section below]

Google is reported to have around 15% of the online search market in China, compared to 75% for Baidu.com. Both companies comply with Chinese government site blocking (censorship) regulations. [Network World article cites the China Internet Network Information Center (CNNIC) as the data source – they publish periodic reports on Internet use in China]

CNNIC’s reports are quite interesting, and for a statistician they’re a lot of fun to puzzle through. For instance, of 338 million Internet users, the “accounts or passwords of 110 million were stolen.” And among all users, 320 million (94%) are on “broadband” – but note that 46% of users access the Internet from “mobile” connections…which seems to me like they would in no way be considered broadband. Also, 28% of mobile Internet users said they would access the Internet by 3G mobile phone – what are the rest using? Maybe wi-fi is considered mobile Internet? Anyone have the answer?

Previous articles I’ve written on Google and censorship in China:

Censorship as Punishment

The post Google and censorship in China – the tale continues appeared first on Sky's Blog.

Recently I became my own private OpenID provider. I did this using phpMyID (see below). I created a private page that sits on my personal web site, and when I’m at a site that uses OpenID I supply the URL of this private page, type my name and password, and bingo I’m logged in at the third-party site. I used phpmyID, which is listed in the OpenID directory.

On the utilization side, when DiSo Project created an OpenID plugin for WordPress, I immediately installed it on my blog. And it works. Lets you log in at my blog using your Yahoo or gmail (or other OpenID provider)^{[3] [4]} information without creating a new account.

OpenID eliminates the need for multiple usernames across different websites, simplifying your online experience

You get to choose the OpenID Provider that best meets your needs and most importantly that you trust.  At the same time, your OpenID can stay with you, no matter which Provider you move to.  And best of all, the OpenID technology is not proprietary and is completely free.

[2] AOL, Google, MySpace, Yahoo!, Blogger, Flickr, LiveDoor, LiveJournal, Orange, SmugMug, Technorati, Vox, WordPress.com … and probably some others. See the OpenID site for a current list.

[3] Do you know your gmail Google OpenID? You can find it with just a few clicks.

[4] If you have a WordPress.com blog, (like let’s say you have a blog called myblogname.wordpress.com) you use the URL http://myblogname.wordpress.com/ as your OpenID. You will have to log in at WordPress.com in order to validate your password, but once you’ve done that, you can log in using this URL.

The post Use OpenID on your WordPress blog! appeared first on Sky's Blog.

Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. — Doug Bowman, Goodbye Google

I love this farewell post by Google’s visual design lead because it confirms a theory I’ve long held: that Google is a company where engineers are the ruling class, while at Apple it’s designers. – Buzz Andersen, Sci-fi Hi-fi

This just blew me away because of my experiences with people from both Apple and Google.

After I recovered from rolling on the floor laughing about Buzz Andersen’s comparison, I thought just how true it is. My experience with Apple goes back to trying to convince Steve Jobs that the Apple II could be used for distance education (in 1978) – interesting that I drove Steve to a meeting where we looked at Plato-IV and shortly thereafter he visited Xerox PARC and well, that became history when the Lisa and the Macintosh were invented. But, I digress, my experience with Steve is that he understands technology really, really deeply, but he is at heart an artist and his reputation as being extremely demanding is well deserved, and stems from his artistic temperament. Then there are the Googlers, especially those I’ve met recently, who do indeed want to subject every web site to a Google Analytics interrogation to determine by a percentage point which size and color of button works best – but even before they develop any traffic to the site! (Good grief, let’s get some traffic first and then let’s see what is working with visitors.) But this business about testing a half dozen different buttons is beyond me. And I’m an engineer!

Deep breath.

Why did I say different approaches to life? (In the first paragraph.) Well, it’s because I run across both of these types in life. There are people who do a great design right up front and move straight to product (even though it may take a long time). And there are people who design something and then twiddle the bits to death before finally starting to build product. In my case, I design quickly and then jump into implementation as soon as possible. I always change the design after I see the first prototype of a product.  “Release early and release often” is certainly a motto I understand. I have very little tolerance for situations where everything has to be meticulously planned up front.

The post Google ruled by engineers, Apple ruled by designers appeared first on Sky's Blog.