Dotcom Web Services Commitment

If you want to see a funny juxtaposition of me and an earlier post's subject, read Leander Kahney's Web Services story at Wired News. Web services was a huge topic at Emerging Technology: it's essentially a backchannel method of querying a Web site's data store. So I can retrieve pricing and other information from through a lightweight XML transaction instead of a heavy, ugly Web page. Not only do I get more information, I get more explicit permission to use it.But my fear has been that Web services is a one-way street: you sign a license agreement to use the site's data a certain way or not use it in others, but you don't get a commitment (in these dotcom cases) in which the site promises availability, persistent, or other service-based agreements. It's lopsided because the sites, like Google, have the information, and we're supplicants. The amusing part is that I had the conversation with Leander standing next to Jeff Bezos, who might have been pointedly ignoring me or just oblivious of my existence (I worked at several years ago and was directly hired by Jeff). Odd to be juxtaposed in the story with Jeff while not having any human contact with him.

Emerging Tech Takeaways

The big messages at EtCon in order of importance (most to least).1. Swarms/social modeling. Groups follow rules. Failure to accept the fact that groups follow rules guarantees failure because regardless of your personal view of reality, groups follow rules. Anticipating group behavior allows you to create rules that can mitigate, not eliminate emergent patterns. 2. ____ ______ (his absence, his politics, his force-of-nature personality, his influence). I suggested putting an empty chair next to Chris DeBona at the Journalism BOF on Wednesday night because his presence was so palpable. 3. Technorati rocks, but bloggers rule. 4. Those who forget the past are condemned to rewrite it. 5. Google! Google! And a triple! 6. Web services has reached overhype, but its influence and deployment grows, just on the downside of the hype curve as its utility increases. My best personal soundbite about my success in achieving Net.fame from "I'm following the pheromone of my own ego."

The Whuffie Game

I'm not a particularly interesting person in my own right. I'm very good at collecting stories and facts and turning them into compelling non-fiction stories. This leads people to believe that I might, in fact, be interesting.At the Emerging Technology conference, I've been watching my own interactions and those of people I meet. I'm relatively well known in this crowd and I have a speaker badge. At the risk of being one of those people who says, "There are two kinds of people: people who categorize people, and those who don't," I'm seeing a lot of emergent Whuffie-based behavior. (What's Whuffie? It's a catchall term for your relative reputation based on many factors that Cory Doctorow used in his recent novel, and which derives from an even older source he was involved with.) The more well-known the person you're near, the more likely you are to be deemed interesting. I now know a number of relatively to extremely well-known people. I'm not very opportunistic. I like people who engage on ideas, and many of the folks here who are well known are the kind of folks who talk and are interactive. Well-known people tend to avoid interacting beyond a superficial level. I've met many people I've wanted to meet for a while, and had great conversations with some of them. Others, even though we've corresponded or I'm on their radar, they had a sort of learned avoidance behavior in crowds that I was either sucked into, or I gave out signals that triggered that behavior. I may be too effusive, complementary, or cognizant of someone's written or spoken works. The last conversation that ends is the one conducted by someone who has a constitutional inability to stop talking. They always win unless you're rude or Tim O'Reilly. This might offend some people I've talked to, but I've had a number of excellent, long conversations, some of them over an hour, and I'll leave my colleagues guessing which ones of them I'm referring to. Let me explain my Tim O'Reilly crack: it's not a crack, it's rather a compliment. Tim has a natural, quiet ability to extricate himself from a conversation when that conversation is done. He's not offensive about it; it's very subtle. But it allows him to maximize conversation efficiency. He's well known enough that the degree of deference paid to him combined with this natural ability allows him to leverage this conversational/social advantage. Man, am I tired.

Laugh Distortion Field

A couple of hours ago, I spotted Jeff Bezos here at Emerging Technology talking to J.C. Herz (currently sitting in the row in front of me at a session) and Kevin Kelly (one of Wired's early gurus among many other projects). I approached to say hello, as I hadn't seen him in a couple of years, and then the laughter began. The three of them, triggered by Jeff's remarkable, contagious bray, went into a kind of laugh escalation.I was standing nearby, looking for a quick chance to say hi, and started talking to Leander Kahney. He's working on a short piece about Web services, which are a big topic of conversation here at Emerging Technology, and whether applications will wind up increasingy relying on Web services. My take is that Web services fed by companies that don't offer two-way contracts -- contracts that promise continiuity, uptime, consistency, and persistence -- will make application developers nervous about relying on them. My site relies on lots of Web services, but I can't rely that any of them will exist in the future, that I won't be cut off from them, etc. The irony, of course, is that I'm standing next to's founder and CEO talking to Leander about the fact that isn't offering any commitment to Web services, it's a large experiment, but that that's within their rights. Meanwhile, the laugh jag continued. I never did get a chance to say hello, either.

Google's Director of Technology Talks

Craig Silverstein from Google, employee number one and director of technology, is talking here at Emerging Technology about how Google accomplishes innovates, rather than how they technically carry out their tasks.Craig says that the goal is to do things that matter. He used to believe that "for a company to be successful, it really had to be evil." But Google has proven that you can do things that make a difference to people and succeed. "It's one of the reasons we're still around after the bubble." Google resisted pressure to run popup ads in the early days, as well as banner ads, because they wanted to create lightweight unobtrusive text ads. "It's successfully bringing in money for us now." At Google Labs, the company puts out ideas that they aren't quite nailed down on yet. They don't know the precise feature set or how to accomplish a certain goal. The feedback allows them to "do something that's really useful to people" when and if they turn it into a product. The cost of switching search engines for the user is zero. "We have to have it that users would rather go to Google than other search engines just to survive."' Craig emphasizes hiring: "it's the key to what makes our process successful." His slide reads, "brilliant people have good ideas." Google trusts that its people will do the right process because they've hired the right people. "They realize the value of trying to do the right thing." A creative environment creates creativity. Design is a first and primary component of their development process. "Our site looks simple, and looks artless almost, but it takes a lot of work to get there." He shows the first beta search interface for Google, which he says an article described as "anorexic." Hard to get things wrong with a single form, and fast to download. They counted the bytes on the home page, and they still do. Google keeps "the top 100" list: the top projects that they'd like to do. It's not 100, might be 130. They're not working on all of them, but by keeping this list, and ordering them vaguely by priority, they can keep their mission statement in mind -- they don't wind up working on something unrelated. He shows a page of the list. The first item is, naturally, Build a Search Engine. Number two: Crawl the Web. Number three; Google News. "Just last month we got the idea of building a search engine, so we'll be working on that." (laughter) There's an iterative process of figuring out what works. Shows a chart of wireless (cell phone) traffic. It meandered for a while, and then jumped when they signed a partner last Christmas. It was way down on the top 100 list, and now it's maybe number 12. They meet weekly and review priority of the "living document." "It's very much essential for this to be a success." Their approach is to use small teams: "small teams are fast and agile," the slide says. The whole group works on a project, including design, testing, launch, and continuing engineering. The Web crawler team is Kingson and Jill. They crawl 3 1/2 billion pages per month, and "these two people are the ones responsible for making it happen." Other teams help, but they own it. Hierarchical communication is antithetical to their small group approach, so they have to constantly ensure that communications continues across the entire organization. When a project is ready to start, the entire company can be involved, but a few people are responsible for dealing with comments. "Make sure the groups are sharing experiences, sharing technology, sharing ideas." They have an interteam group, which is really 10 groups of three, not one group of 30 people. One group helps make sure that new people can understand how to get into the code base. These positions are "volunteer": people take time out of other projects. They have a weekly tech talk in which people talk about what's in progress or finished, and then they put the talks up on the Web site as videos. It helps new employees understand what happened with the project. Google creates tools to help them foster communications. They have an internal search engine -- "you can guess which one" -- over their internal Web sites. They have a status page that's created from emails sent to it and that maintains state for a week. When Google acquired Pyra Labs (Blogger's creators), none of the press speculated that they might use blogs internally, but that's the first thing the group suggested when they came on board. In the old days, they put stuff up and waited for comments. But now they do user studies. Their first stab post the initial home page was, "I'm waiting for the rest of it." "Is this some guy's home page?" "How many people work there?" Google Labs allows them to test ideas that aren't ready for prime time. "My particular favor is Google sets." You enter the first part of a set, like the names of some of the Seven Dwarfs, and it helps complete them. "I encourage anyone who is thinking about switching jobs or starting a new one to apply to Google." They hired a compiler expert, which they didn't exactly need, but the first thing he did was massively improve the speed of part of the back-end. They have one hiring committee, just a few people, and this committee hires everyone. "It manages to keep us consistent" and it separates hiring from head count. Someone might say we need 10 engineers to make the ad system up to date, but that person isn't making the hiring decision: the committee doesn't have those pressures. "It's much much more important to use that we hire people who fit this criteria we have than to miss people who are great." "Bad employees are the time sink that keep companies from being as successful as they could be." It allows them to run all the projects they do at the same time without worrying about these hiring issues. Only a few people at Google haven't worked out. A Google News home page can't be lightweight: sort of the opposite of news. They wanted something that was "googly." He walked through an iteration and how they decided on several elements, including top stories. During the bubble, people said, "The Web changes everything." That true? Not exactly. But before the Web how would they have created what they do? Google wouldn't have existed, of course, but before the Web, simple search engines with unsophisticated search engines worked fine. Wouldn't be able to have small groups because you couldn't talk broadly enough efficiently enough. Sharing information on Web pages that archive information "that allows us to have the kind of communication we need to have these small teams." The company runs the way you think you might run a company if you weren't actually running one. Especially unlimited food. (I visited Google yesterday, and their biggest problem in scaling has nothing to do with their search engine, but rather with their dining room. The Grateful Dead's former road chef runs the free restaurant, serving lunch and dinner, and they've got so many employees at Google now, that even with an overflow outdoor tent, they can't quickly handle the full-on lunch load. And, man, is that food good.) Q: With new office in New York, how do you maintain culture? A: Groups there working through same processes. Q: [inscrutable question on groups] and then why not more than 3.5 billion pages. A: I didn't understand your first question, so I'm just going to answer the second. Some finite resources, but will be solved over time. Q: Applied Semantics acquisition? A: Can't comment directly, but name implies what it does. Q: How do you rein people in? A: Happy to have unrelated ideas, although they might not get selected to discuss. Gatekeepers are technologists who stand at different parts of the process to see what's considered. Q: RIAA sued students who built search engine for college intranet. Google is better at searching MP3s...if you type "MP3s and..." Craig: Stop it! Right there! [laughter] A: Why don't we have a music search? Exactly this reason: intellectual property. Not much legal music and video on the Web, and "we don't want to make it easy to find this." Can only imagine illegitimate uses for it, then won't do it. But if it has a good goal, they don't try to secondguess. Only nix pages that the content owner asks for or that are stolen.

Clay Shirky Has a Posse

Andrew Orlowski wrote this hilarious, non-fact-based account of how the Emerging Technology conference's agenda was set. Apparently, Clay Shirky is the god we all worship.I was one of the track chairs for the event, focusing on wireless, and the process of soliciting and encouraging proposals, and reading and commenting on them was entirely consensus driven with about 7 to 10 people involved in conference calls and online collaboration. It was a great way to build a conference. The only problem was that there were too many excellent proposals. And a few proposals in which excellent emerging technology was wrapped in marketing instead of clearer talk. I had two proposals rejected myself, and several folks I'd suggested also had their talks rejected. But a few made it through into a very tight schedule that's bursting with interesting ideas. The danger of soliciting proposals is that when proposals are rejected, people have sour grapes because their particular hobbyhorse doesn't wind up fitting into the overall vision for an event. I'm sure academics deal with this all the time. (Update: I just met Clay Shirky and another track chairperson Geoff Cohen. We discussed the article, which Clay had heard about but not read yet, and we're all quite amazed at the Register publishing this piece. It reads like an account of the Protocols of the Learned Elders of Wi-Fi. We were there. We know what happened.) Even later: Tim O'Reilly wrote why the Register article was a hack job. Meanwhile, Clay approach a group of his friends and colleagues, and they prostrated themselves.