We’ve always had multiple social graphs, even offline

Maybe there was a time in college where our friends, colleagues, and interests were all melded into one group of people (Maybe it’s still like this for people who work at college type companies like Google and Facebook?). But, as your life gets more complicated, that one social graph tends to sort of branch as our interests in life evolve.

When I take my son to a birthday party, I don’t go there expecting to talk about linked open data with all the other parents. And on the other hand, I didn’t go to Chirp last week expecting to talk about how to manage children’s video game time or what’s going on with school redistricting. And the family reunion isn’t a good place to talk about why Activity Streams don’t just use RDFa.

The same dynamics are happening online. My Twitter graph, for example, is all about my professional interests. I follow people who have interesting things to say about software development, the internet, open government, Gov2.0 and so on. On the follower side, excluding the spammers, it’s the same picture. It’s all people who share those particular interests. So I’m not going to share a camera I want to buy or some clothes I’m looking at or some blocks code I’m writing with those people, because all I would be doing is adding a lot of noise.

And so I participate in online forums that are specific to certain interests, I keep my Twitter world more professional, I use a personal email addr for certain friends and family, I use LinkedIn for other things, and I can’t see myself ever wanting to put all of that together into one big hose. Just looking through my browser history, I can’t find a single page that I would want to “like/share” with everyone I know.

It would be a lot of work to tag every thing I read or write about, so I do what a lot of people do which is just maintain different social graphs on different sites, online and offline, to “tag” things for me.

This is why social aggregators haven’t caught on with regular users

If everyone I follow on Twitter shared every online interaction, every checkin, every personal interest, every dish they liked, every political statement, I would stop using Twitter. The noise would just be ridiculous. And I think that’s the real struggle with tools like Buzz and what Facebook is building now. Right now the interface isn’t there yet for me to be able to quickly say “this is something of interest to these 5 camera geek friends”, “this is something interesting for my iPhone friends who code”, “here’s something for iPhone fans who don’t code”, and so on.

This is why Google Buzz wasn’t immediately interesting to me, especially with the initial cut at sharing settings. This is also why I have 4 different email addresses. It’s also why most people use SMS, multiple phone numbers, email, IM, Facebook messages, gists, and dozens of other modes of communications, all in the same day.

Again, I can think back in my own life in school where my friends were my friends and my colleagues and I didn’t have a lot of interests outside of what we did together as a group. But I think the general consumer or worker is not like that.

Now the problem with that is I may have 10 different actual social branches, each of my friends has 10, and they have varying degrees of overlap. So the real interesting work will be in coalescing conversations from these different graphs, not in real time, but at the right time around a particular subject like health care reform or Grand Central Dispatch.

To me that is a much more interesting problem to solve, and more applicable to real lives, than creating the one big graph or the one big firehose.

Ok, that’s the part I don’t quite understand. I understand that OAuth support is a major advantage in protecting users from rogue applications. More importantly, as a practical matter it allows users to more seamlessly change their Gmail passwords, something we should all do more regularly. But this all does come at some expense to app developers; just look at how many Twitter apps still don’t support OAuth.

But Gizmodo and others seems to be implying that OAuth will enable application capabilities that are not possible today. Can anyone think of any examples of this?

Or are people just thinking that better security will lead to more people adopting more third party apps? I don’t know about that, since most users still don’t know or care about OAuth, and the OAuth experience for native mobile apps still leaves much to be desired.

So what am I missing?

I think the general consensus is that there needs to be positive steps taken by the government to promote real and fair competition of ideas in IT programs. Many of us who have worked in this field or observed this field agree that innovative, disruptive (read cheaper for the taxpayers) ideas tend to get smothered more often than not. Some attribute this to fear of change, others to the fact that government contractors provide value to shareholders by selling lots of expensive hours and lots of expensive products from their partners. It is hard for fresh ideas to compete in that dynamic, and we’ve all experienced that in one way or another.

I think everyone agrees that fixing this systemic issue is a necessary step for sustainable innovation in E-Gov, Gov2.0, OpenGov, Gov^2, and so on.

Right now, innovation is not the rational choice

The way contracts are structured right now, innovation is often not the rational choice for a government program. I’m not talking about incremental investments like social media or website redesigns and that sort of thing; I’m talking about major IT programs like financial systems, CRM, document and application management and the sorts of programs where governments often end up with underperforming expensive systems.

In a lot of those cases, the government department buying such an IT solution is both a customer and an investor in that product. There are very few cases where a product can be bought off the shelf and just put to work. In almost all cases there are data and service integrations that need to be done, security constraints that need to be met, accessibility mandates that need to be met, etc. So for all of that, the government program is the investor.

So now you have an investor and customer who is asked to choose one solution, one team, one architecture and one outcome. Any person who is asked to put all her eggs in one basket is going to choose the old, staid, proven basket. That’s human nature, but it’s also, I would argue, completely rational. Innovation involves risk, and our current procurement processes, which are almost always “winner takes all” make that risk virtually impossible to manage.

So there needs to be a way for a government agency to choose more than one solution so that the risk of innovation is actually palatable. The question is how do you structure that in a way that works for conservative buyers and powerful contractors and vendors? Solving that is, I think, key to returning value to taxpayers in this Gov2.0 world.

As always, I look forward to your thoughts…

I think, in reality, the barriers to innovation in large private sector companies are really not all that different from what we see in the government. If you worked for a large hotdog bun maker and your big idea is to give away the bun recipe and rent the hotdogs instead, you are going to face an uphill battle at MegaBuns, Inc. And that battle will be just as hard as you would face at MegaGovContractor, Inc. who has a strong business relationship with MegaLegacySystems, LLC.

The difference, though, is that if you are sufficiently fanatical about this great idea, you can leave MegaBuns and start your own company. That works because your disruptive business model can find investors who are willing to take that bet. But in the government, if you have a brilliant idea to automate the Cans for Beans program, you can’t just leave and start your own Department of Agriculture. Unfortunately, it also turns out, you usually can’t even get a parallel contract for the program either.

One way to address this issue is to evangelize vendor independence, open standards, SaaS, open source and other great ideas from the top down in government. Those efforts are to be lauded, but I wonder if there aren’t also some more structural ideas that can accelerate innovation in general.

The issue is that right now we, as taxpayers, if we need a program to track Cans for Beans, will invest $X million in a single program to build that system. We have decades of great work in the FAR to make sure the program is awarded competively and fairly, but at the end of the day we invest in one team, one idea, one architecture, and one outcome, and then hope that works out well for us. The Department of Defense has a better way of approaching huge weapons programs, where they will at least fund a substantial “bake-off” and pick between two prototypes for the next fighter jet or armored vehicle. We could adapt that idea to government IT, but that could be a big challenge for program directors and acquisitions staff that would then have to manage twice as many contractors. This may or may not scale well to smaller programs.

Innovation Set-Aside

But what if the government took a smaller step first? What if for every new IT program of size greater than $X, (X doesn’t have to be very big), we tell the chosen contractor that they must set aside some amount $Y to fund a prototype “Plan B” architecture for the program using different technologies or models than “Plan A”. So the program and acquisition staff will still choose one contractor and one technical approach like they do today, but then the government invests in a built-in hedge on a consistent basis.

This doesn’t need to be as complicated as putting two hyper-competitive contractor firms in the same pen and expecting them to play nicely together. And it doesn’t have to put the government in the position of having to manage two programs. (Imagine a VC investing in two directly competitive companies, knowing one will kill the other, and then trying to sit on the two boards and help where he can.)

Instead, the winning contractor will invest the $Y internally (this can involve subcontractors, or not) and if the “Plan A” fails or the “Plan B” is just a rocking success as a prototype, then the government can switch to “Plan B” with no harm done to the prime contractor (and therefore no incentive to suppress it). “Plan B” becomes the new “Plan A” and gets built out to scale. No contract cancellations, no scathing GAO report, not even a re-compete required.

We have a lot of experience in using financial set-asides (incentives for women-owned business, minority-owned businesses, firms started by service-disabled veterans, etc) to effect social good and encourage more open competition between companies. Can we apply a more limited version of this to encourage competition between ideas as well?

UPDATE: Bill Brantley also wrote about some other barriers to innovation in government IT

UPDATE: Some thoughts on why innovation isn’t rational for a lot of government programs today.

This blog post which is making rounds on Twitter tonight highlights a problem with FourSquare, which sounds pretty scary. But if that’s scary, you should see many of the other APIs that mobile apps use — including many apps you probably use every day. This problem is a lot more common than people may realize.

In fact, it affects most Twitter apps that we use on our phones. Twitter realizes this, and has tried to phase out the use of Basic Authentication in their API. At one point they even announced an end to Basic Auth for all new apps, but had to reverse that decision because of developer resistance. Their official policy now is they really want this to go away some day:

“OAuth is the Twitter preferred method of authentication moving forward. While we have no plans in the near term to require OAuth, new applications should consider it best practice to develop for OAuth.  We eventually would like to suspend Basic Auth support. However we realize that Basic Auth has been a large part of the API’s success, and that the barrier to entry if OAuth is the only solution is substantially higher.”

OAuth is, of course, the right answer to this issue. But it can be tricky to implement (for developers) in native apps. And, perhaps more significantly, it can be a little confusing (arguably even a little disconcerting) for users as they get kicked out to the Twitter site to “approve” access for this new application they just downloaded. If you understand OAuth, it’s easy to see this is more secure than giving some app your Twitter password, but explaining that to a user and having them go through the process is often difficult.

So, yes, FourSquare and Twitter and everyone else should implement and require OAuth (and deprecate HTTP Basic), but it’s not their fault that this is not really feasible yet. One thing that I think would really help is OS level support, especially in the user experience for OAuth. I would envision something similar  to how Apple implemented seamless login to restricted WiFi hotspots. If OAuth can be seamlessly weaved into the OS like that (with direct API support), then I’m sure most mobile developers and web services would leverage that quickly. Nobody likes Basic Authentication, but right now it’s just the easiest way to do things.

The reality is that if you visit http://www.browserchoice.eu/, it is far from fair if you define “fair” as “every browser has an equal chance of appearing in the first slot, and an equal chance of appearing the second slot, and so on.” In other words, by any reasonable definition of fair, given the context of the case to begin with, what the EU probably wanted is a uniform distribution of the world’s top 5 browsers across the five columns on that page.

What they got was something totally different. And arguably, the IE team may have gotten the worst placement of all.

Now, part of it is that Microsoft’s programmers picked an awful algorithm. First of all, let me say that Microsoft employs some of the most brilliant minds in software that exist. Some (ok, not all) of the research and products that come out of Redmond are brilliant. But, unfortunately, when you’re a really huge company, not all of your programming teams will meet those highest standards. And even the best programmers can be sloppy when on a deadline. (although in this particular case, I don’t think they can claim the “we were in a rush” defense!)

Maybe Microsoft outsourced this particular bit of work. In any case, someone needs to be called into the office on Monday.

But what about the role of the EU’s negotiators and policy-makers in this? Here is the agreement, and the relevant section:

(13) The Choice Screen will prominently display the Final Releases of the five highest ranked web browsers based on usage share in the EEA (i.e. only these browsers will be immediately visible without requiring any user action under typical user settings). These five web browsers will be displayed in random order each time the Choice Screen is presented. The remaining seven browsers will be displayed if the user scrolls sideways and will also be displayed in random order.

There’s that word “random”. What they really meant to say is that the browsers should be uniformly distributed across the five possible positions. Now, again, I think any reasonable person would realize that “random order” in this case implies that the browsers should be uniformly distributed.

But if that is what was meant, then why not just say that? You can’t assume that the programmer sitting down to write this Javascript understands the legal and policy context. Somewhere in there is a good lesson for requirements analysts.

But another interesting aspect is the definition and recognition of “external official presences”. The directive defines these and lays out some specific requirements for safeguards around those official presences. This is where there will be a lot of interesting innovation in the next couple of years:

From the directive:

“External official presences shall:

a. Receive approval from the responsible OSD or DoD Component Head. Approval signifies that the Component Head concurs with the planned use and has assessed risks to be at an acceptable level for using Internet-based capabilities.

b. Be registered on the external official presences list, maintained by the Assistant Secretary of Defense for Public Affairs (ASD(PA)), on www.Defense.gov.

c. Comply DoD 5200.1-R, and with References (a) and (b) as well as DoD Directive (DoDD) 8500.01E, DoDI 8500.2, DoDD 5230.09, DoDD 5015.2, DoD 5240.1-R (References (c) through (j), respectively).

d. Use official DoD and command seals and logos as well as other official command identifying material per ASD(PA) guidance.

e. Clearly indicate the role and scope of the external official presence.

f. Provide links to the organization’s official public website.

g. Be actively monitored and evaluated by DoD Components for compliance with security requirements and for fraudulent or objectionable use (References (d), (g), and (i)).”

The directive goes on to discuss the records management requirement, which I’ve always felt is very important. Government agencies have invested a lot in policy and tools around records management of all sorts of communications, social interactions should be no different.

There are some early tools out there in the commercial world to help implement and even automatically comply with policies like this. But I think there is room for a lot more innovation in both tools and strategy here. I look forward to seeing that unfold over the next year.

But when you look at the Android ads and billboards and particularly the Verizon Droid ads, is it any wonder more young men buy them? Who else would be attracted to a phone that is used by robots?

and one that can apparently punch a hole in your wall if you should need that:

Of course, for all that great engineering, things haven’t quite gone according to plan for our friends at PALM. Lots of people have piled on in the past couple of days, some looking at where Palm goes from here, some looking at how they got to this point (for example, did Palm get here because they don’t have Steve Jobs?).

Well, as much of a fan of Apple (and Jobs) as I am, I’m not sure I subscribe to the idea that a mobile electronics company simply cannot succeed without Steve Jobs at the helm. For me, I keep coming back to two big mistakes that helped to throttle what should have been the best competitor Apple has:

1. The launch date

Palm launched the Pre in Sprint stores on June 6, 2009 (a day that will go down in infamy?). This was on a Saturday, two days before the keynote at WWDC, where Apple introduced the third generation iPhone. Two days. Zero business days. So on Monday the news was all about Apple’s impending announcement, the keynote, and then reaction to the keynote, then reactions to the reactions, and so on. The entire news cycle that you would expect if you’ve ever heard of Apple. Or if you, for example, had worked there.

So, of course, the innovative and very interesting new Palm Pre wasn’t even the first headline in mobile phones, let alone gadgets, for the whole week. And Palm got not nearly the coverage they would have had two weeks later after everyone had come to terms with the fact that the iPhone 3GS didn’t cure cancer. So why release a phone two days before the hottest company in your segment? I presume it was a statement that this phone was so great that it could actually drive the 3GS off the front page. If that’s so, someone should have run that thought by someone outside of Palm. The reality is the public release ended up being entirely forgettable (and forgotten).

(Actually, there is a 1b here, which is Palm’s obsession with Apple. That’s understandable given the history of the people working there. But still probably not smart, given that even as WebOS is arguably better than iPhone OS, it totally blows away other mobile OSes. Perhaps their energy would have been better directed at those other competitors.)

2. The development environment

This is where I think Palm had a great idea that was probably better on paper than in practice. The idea was that if the development environment was Javascript and CSS, then there would be millions of potential WebOS developers, immediately leapfrogging the small number of experienced Objective C developers. In theory, that makes sense. In practice, though, I think what people realized is that most CSS and Javascript work in the world is simple little tricks in webpages. A lot of that work is copy/pasted and evolved from scripts and stylesheets borrowed from around the web. And, so, as a result most web developers have never done anything as complicated as building a full app and don’t even know Javascript to that kind of depth that you need to work with something like Mojo.

So what happened in practice is that Mojo development actually ended up being more daunting for web developers (the target market) than Android development, for example, is to Java developers (their target market). Or that Cocoa development is to someone who knows C. And that’s hard to overcome when you’re competing with the AppStore gold rush. I wonder how this story would have played out if Palm had offered a native kit from the beginning that allowed the easy port of some popular OpenGL games. Or a new framework that looked and felt a lot like Flash, with the target being Flash developers rather than general web developers.

Again, the good news is that the Palm team built a very nice OS, at least from the user’s perspective. And the UI is so much better than the iPhone knockoffs. Licensing WebOS to other handset manufacturers may be a challenge against the essentially cheaper than free Android. But becoming part of Nokia could really save Palm (and probably Nokia as well in the smartphone market).

What do you think? What’s your theory on how such a great smartphone sold so poorly when everyone is talking about and buying smartphones?