[Updated 2013] No Longer Recommending Fastmail.fm

[This post seems to come up in search engines, so I've updated in 2013. It was written in 2006. I left Fastmail.fm for three years and hosted my own email. During that time, my wife and father both remained on Fastmail, and it was acquired by Opera Software, which makes browsers. In 2009, I decided to stop the hassle of running my own email again, returned to Fastmail, and have been generally delighted for four years! So I can again re-recommend them, and even provide a referral link. I leave the post below for historical reasons. I don't like to change history.]

I can no longer recommend Fastmail.fm, a service provider specializing in email that I have been using for years, and that I have found generally reliable until recent months. Probably through growth and some wonky hardware, they have had intermittent downtime that sometimes stretches into hours over the last few months. Today, I am finally fed up. Why? Because a new server, designed to help them avoid downtime and replicate their email, failed, and they revealed that all the new server equipment they purchased doesn't meet their spec, so months after the previous outage, they have only future plans in progress for providing robust replication.

In addition, they have no "software" backup plan. After being told for hours this morning that it would take "two hours" to rebuild a filesystem and start mail back up, they now believe it will be a full day. Great. I understand how that can happen, of course. (You can read their statement about this on their forums along with my reply.)

But they have nothing in place to help their users. After previous outages, you'd think they might have developed a replicated configuration system, so they could replicate just our accounts (not our email) to receive email in the meantime. A Webmail interface that would allow us to look at new queued email. A tool to redirect email temporarily to an off-Fastmail.fm account. A new account set up on a non-affected server to redirect email to.

They apparently have nothing.

I'm afraid this is what comes of trusting a niche player that I thought could handle this kind of performance and load. I can understand when a firm is plagued by hardware troubles, and they have had more than their share. However, having put apparently no effort over the last several months into any kind of contingency plan for their users is unacceptable.

I'm working to migrate my incoming email elsewhere so I can actually get work done today. Fat chance.

Another update: I finally had my account restored on Sunday night. This was 3 1/2 days of downtime, with no good communication except for brief, typically inaccurate estimates of the time completion. Their solution at the end was to use a procedure that copied account settings and stored email over to a new server. They recovered all the data, but despite their consistent statement that incoming email to our accounts was queuing, all email sent after the crash and before I migrated to my own servers was bounced.

I liked Fastmail.fm very much. But this was too much. After significant outages and failures, they had not established a total failure position. Fortunately, this was a single partition on a single server (apparently one of four servers). I thought these folks had the resources and expertise to cope with problem and even respond.

I will give them credit for ultimately being able to recover the data. That's great. But their ability to communicate a position, apologize, and explain were very poor. They weren't listening to the discussions at the forums to which they point for talking about their service. They didn't try to do anything interim that would have allowed short-term access to new email.

I'll be helping to move all other folks I know that I've recommended to Fastmail.fm off to other services.

Get the Name of the Dog

I'm a working reporter, but I don't pretend to have all the answers about my profession or my specialty. This is why I sometimes interview dozens of people. Not to get more detail, but to get more understanding. Sometimes I need 10 hours of interviews to walk around a subject and make sure, to use an example favored by an old boss and friend, that I'm not saying that a rhinoceros has a skin comprised of bony plates.

Recently, I learned a great tip from a veteran journalist and teacher from his blog over at the Poynter Foundation. It's a long-standing mantra of Roy Peter Clark to "get the name of the dog." This means that you need what he calls "the telling detail" in a story because it takes you down from the ivory tower into the specific. It brings the story home.

I've often asked for the name of the dog, but wasn't consistent. Since reading this column a few weeks ago, I've learned a lot of dogs' names. A few weeks ago, I interviewed a fellow for a story that's coming out in a few weeks about him watching a World Cup game via streaming media. Before I was done, I said, what match was it? Who won? Turns out it was Argentina versus the Netherlands in an up-to-the-end tense bout that ended naught-naught in a tie. That's an interesting dog, right there, and explains why he was rapt through the game.

I noticed in BoingBoing today that they call out an error in a San Francisco Chronicle article in which the gas emissions from Burning Man are listed and discussed, and cupcake scooters are mentioned in a list of gas-burning items at the festival. Not so, the cupcake creators write in letters to the Chronicle.

What happened here? I know, because I've been there. The reporter either made an assumption, was told incorrectly, or read an incorrect account. For daily reporters, this is incredibly tough. You cannot run down every fact without going insane. (It's hard even for those of us who are working on a feature over six weeks that involves, say, calling three or four continents, five time zones, and 20 interviews, as one recent article that will be in print in a few weeks entailed.)

If the reporter had had the time, he or she would have called or contacted the cupcake makers and gotten the answer. This is more like, "Make sure it's a dog, not a cat," but it's the principal of fine detail that's the same.

I made an error in a piece I wrote in the New York Times a few months ago that came out in editing. It was clearly my fault for not carefully noting the change during editing that caused the error. (I had a fever, was on antibiotics, and was generally in pretty poor shape while reading the edit. Nevertheless!) Instead of a network being described as having many nodes, it was described as having a single node. I had the name of the dog right, just the size of the litter wrong. The subject involved was horribly offended, and the Times did run a correction. (In context, it was really a single word--"the"--that made the difference in interpretation.)

So we have to not just get the name, breed, and quantity of dog right, but we have to make sure that our 9-year-old twin Russian wolfhounds don't become Russian wolves before they see the light of print.

Aug. 29: Another great example from an article on a report about the alleged misuse of office by Kenneth Tomlinson, a Republican appointee who handles US broadcasting abroad, such as the Voice of America: "The State Department report noted his use of his office to oversee a stable of thoroughbreds but did not mention one specific way in which his professional responsibilities and personal interests appear to have intersected. The horses, according to track records, include Karzai, as in Hamid Karzai, and Massoud, from the late Ahmed Shah Massoud) references to Afghan leaders who have fought against the Taliban and the Russians, as well as Panjshair, the valley that was the base used by forces to overthrow the Taliban."

On Getting PHP, MySQL, and Apache To Play Nice

Technical post, so beware. This is another of my technical notes to my future self, when I've forgotten what I learned today.

I intended, this morning, to very briefly update PHP from 5.1.0 to 5.1.6, Apache from 2.0.55 to 2.2.3, and OpenSSL from 0.9.7something to 0.9.8b. That turned into three hours, with Web sites dead during most of that time, or at least misfiring.

Two critical problems occurred in the upgrade, that, once solved, seem to solve a number of other odd behaviors I've seen over the last couple of years.

First, I've been using the glibc 2.2 static version of MySQL. I should have been using the plain 686 version, which is built against glibc 2.3. That allowed PHP to compile and work at long last against the client libraries.

Second, I should have had this directive in the httpd.conf files:

AddHandler php5-script .php

Third, I needed to have zlib installed to (it seems) appropriately compile PHP with MySQL's client libraries.

Now, everything not only works, but things that were erratic--I had to keep PHP 4 and 5 running for different Web servers--seems to have totally disappeared.

What I Learn When I Interview

Creativetoy 1912 24384850I was talking to a fellow in Australia for an article I'm writing for The Economist, and he started talking about a Heath Robertson machine. I had to stop and think--who is Heath Robertson? Do I know a Mr. Robertson? Did he invent some part I should know about? The fellow stopped and said, "Oh, do you call it that in America? We call something that has many parts--like the game Mousetrap." I said, "We call that a Rube Goldberg machine in America."

Frank and Ernest Creator Thaves Dies

Bob Thaves, the fellow behind the extremely benign, punny, and unusually funny comic strip Frank and Ernest died last week. I interviewed Thaves in August 1998 for a feature in the New York Times about the effect of the Internet on cartoonists. The piece led to four other articles on cartooning, three of which were about The Cartoon Bank or New Yorker cartoons.

I spoke to Thaves and his daughter Sara back in 1998 about the strip. Here are some rough verbatim notes from the conversation.

While Scott Adams is widely credited with being the first cartoonist to put an email address into a daily strip, he was syndicated in relatively few papers at the time--150, according to the article I wrote, based on information he and his syndicate gave me. Thaves, in contrast, was in well over 1,000 newspapers when he added an address. The mailbox exploded.

The cartoonist received thousands of email messages. Thaves said, "We had not anticipated that there would be that big a response. … it was the first strip in over a thousand papers… one of the first chances for people to communicate with a strip’s creator."

At the time, Thaves didn't want to broadcast that his kids, Sara and his son Tom (who is now taking over the strip entirely), were answering email on his behalf, as he didn't want to disrupt the relationship he had over decades with his readers.

Sara Thaves said a few prescient things back in 1998: "The newspaper will continue to be the medium for the foreseeable future. The Web will not provide comparable revenue [yet]. The newspaper market in general…has a real challenge in front of them." Truer words never spoken. I still don't see a revenue stream good enough to support more than a handful of online-only cartoonists, although it's the combination of print and online that has made more daily strip artists more successful.

She also noted, "[the] key opportunity is to capitalize on the fact that the Web is distinct from the newspaper. The same people will use both, the same people will see both. The challenge for comic properties to be successful on the Internet, is to find a way to bridge the newspaper and the Web."

Bob was fascinated by the potential for three-dimensional comics and immersive environments. He said, "I have read the comics all my life., and when I was a kid, and when reading the Sunday comics, somebody had said to me…now you can turn to the computer…and move right into that Sunday comic page, and suddenly you are surrounded by it,…and now you can walk around in it. See from different perspective, and see different elements of it environmentally."

Here's to you, Bob, frankly and earnestly.