What the social web can learn from Lehman Brothers: social systems architecture

We did a car boot sale on the weekend.. and amongst the pile of books that we keep trying to sell but never do was one I bought a couple of years ago.  The topic of that book was how to price various kinds of exotic derivative contracts, as used in a range of complex ways to buy, sell, spread and mitigate risk and make loads on money for investment bankers.

Derivatives start out as nice simple ideas, but before you know it, things get very complex.  And if you trade lots of derivatives with lots of other parties, you end coupled to your trading parties in lots of weird ways.

So, what started out as a possibly sensible risk management exercise becomes so complex and you end up so tied to everybody else’s success that you’ve got a Lehman-sized problem.

The trouble here is:

Complexity:  it gets too hard for humans to understand what is going on. Everybody kind of pretends they do, but nobody really does.

Coupling:  it gets very hard to work out how to untangle the knot of relationships you’ve created.

Trouble. So the investment bank solution was to create more and varied derivatives to cover all of that, making it worse.  Pass the coke and let’s do some business.

What has this got to do with the social web?  Lots.

If we layer API on top of API, and couple all our social web services together without a lot of thought about systems architecture we run the risk of making something complex, over-coupled, and utterly unstable.

And when we start building businesses on top of all of that, things get risky for those businesses indeed. And if we start relying on complex, over coupled web services, what happens when something’s business model fails and it goes into the deadpool.  Could the whole lot fall over and be hard to get working again?

So, I’m thinking we need to start thinking about the discipline of social systems architecture, where we manage and look at the interconnected web as we do large distributed systems, taking note of single points of failure, instability, reliability, complexity and coupling.

Sure, we can integrate lots of social web stuff, but we need to keep a systems engineering mind while doing it.  Otherwise it is a bit like running your billion dollar derivative contracts out of an Excel spreadsheet.

Google Chrome’s chrome is on the inside

There’s been a bunch of speculation as to just why Google has taken on the browser market with Chrome.  Everybody has a good idea as to why.  Now it is my turn.  And while I’m at it, I want to give a quick review of the features of Chrome and why they might be important.  We don’t get a new browser to play with every day of the week, and this one is different.

Okay, so what is Chrome all about?  Why?

My take is this:  The important features are not in the UI.  Chrome usually refers to pretty UI stuff that isn’t important.  Google’s being ironic here.  Chrome looks ok but is nothing startling or beautiful.   It is low key.  Few menus, calm UI features.  As if the served page matters more than the browser.  Yup.

Basically, Chrome is the first browser that is designed from the ground up to serve web applications fast and seamlessly.  Engineering-led Google has built something to serve Gmail, Google reader, Gears, and other web apps without getting in the way.  In fact, accelerating these applications and all other web applications to make them snappy.

Why?

Beacuse Google’s web app developers have been telling the bosses that what they need is speed, proper memory allocation, decent security and proper multitasking to deliver web apps that really work.  They’ve got a long way with the existing toyish javascript implementations but the limits are clear.  All the other browsers developments were off in user chrome land.  Adding more features and menus and gradually getting faster javascript execution but not quickly enough to keep new applications coming forwards.  

And once you’ve got a good, fast, secure engine in the browser, then you can implement all the chrome you like in javascript in the browser, not in the menus and in the browser executable.  Hooray.

Strategy:  Raising the Bar and Elevating Best Practice

My take on Google’s strategy is to elevate best practice in all browsers so web apps can succeed. That’s why all the components are open source.  We all win (and Google wins more) if we have quick, secure browsers.  So changing the game, raising the bar, and setting a new technical standard for browser best practice is the way to do this.  Packaging it all up in a browser shows it off and gets the early-adopters along for the ride, but I’d guess that Google can handle having the V8 javascript engine and other features show up in Mozilla and elsewhere.

Oh, and it has to nicely unnerve Microsoft.  Always a good thing.

Chrome is fast and tidy and usable, and that’s using it on a Mac using Parallels to run it under Windows XP.  Can’t wait to see a native OS X version. It ought to fly.  Best feature for me, apart from the Internals, is the ability to turn any web app into a single-window application.  We’ve been messing with this in OS X a bit and it is a nice idea, but Chrome makes it easy.

Neat little browser.  But don’t forget the real chrome is on the inside.

 

 

 

Twitter stumbles, and there goes the neighbourhood

Witness the emotional committment of Twitter users.  Wow, people love it, really want this thing to work, and really love to moan about it as the fail whale displays more and more often.

Twitter looks back on track after a shaky few days back there, which shows that all is not well in microblogging land, and there’s something wrong with the microblogging model, but that’s a topic to take up later.

Having Twitter get slow, turn off features, or just not respond has started to get really annoying.   We’re inclined to include Twitter as an emerging tool to use to build and attract community. But without stability, it’s not going to work predictably . How can we recommend building twitter into a social media campaign?  Well, we can’t really.  Or we have to accept Twitter as a somewhat flaky, sometimes useful tool.

And worse, with Twitter going up and down, there goes the neighbourhood.  People pick up and leave to one of the fifty other microblogging services that are growing up in the shadow of twitter and waiting for users to fall out of the Twitter tree.

Trouble.  We’re never going to find each other if we’re spread across tens of different services.

But then again, we want Twitter, in its lovely cuteness, to work.  But that makes it a monopoly with a secret or currently secret business model.

Tricky.

So, my big needs in microblogging are:

1. I want something reliable that works
2. I want something that accesses most people (that want to be involved)
3. I want it to be long term sustainable, not a monoculture or monopoly with a secret business model

To meet these three, we’re going to need to do some internet-level architecture work to support microblogging and ambient status.  Basically, we’re going to need to:

1. Develop some standards for microblogging messaging
2. Develop standard ways to connect microblogging services together
3. Allow users to migrate from one service to another easily– and use more than one service at once
4. Ensure some level of reliability in messsaging
5. Make sure the whole thing can scale up to the current level of global SMS usage and beyond

This looks a lot like what we have for the internet email architecture.  It took a long time to get organised, and it has some problems, but it is a mostly universal service with lots of servers, providers and clients.

There are a bunch of people talking about these sorts of standardisation. I’ll review the efforts in a later post and see where we are headed.  My guess this is going to take a while and we are going to have some early-adopter pain in the meantime.

Key point:  At some point Twitter is going to have to open up and interwork with other microblogging services.  And that is the moment, in my opinion, when they will really succeed.

That set of faces, top left: FaceMap

For the new nodestone banner, I made a little application that goes and builds that set of faces that you see on the top-left banner of the nodestone.com. This post is all about the why and how of that.

We were looking for a way to visually represent the human nature of social media. We’ve got the nice, nodey logo, but where are the people? All the people. All the faces, so that leads us to a bunch of avatars or icons with people’s faces.

Now there is a facebook app called FriendGrid which goes some way there:

But, too big and not enough faces for what we are looking for.  We needed something wide to fill that banner space.  Also, those question marks send the wrong message, no?

So, then, how hard is it to get that set of avatars photos, scale em down and place them in a single image?   Not hard at all, it turns out.

I turned to Twitter.  Twitter users are prety good at uploading avatars, and accessing them via the API is pretty straightforward, so I wrote a quick app with Google AppEngine to accept a twitter username and password and then fetch the avatar URLs and display them in a block.  It worked nicely, so I meddled with the display of the images, got them in a suitable sized block in a browser and screen captured them. I used the Gimp to manipulate the image a bit and make the fractured right hand end. Done.

Next features I’ll add:

• Add more source twitter IDs, so the starting set can be our friends, not my friends.
• Follow friends of friends until we have enough unique faces, this avoiding duplicates
• Remove the ‘no avatar’ images
• Do the actual image manipulation to build a single image from all these.
• Auto-update the nodestone banner once a week or something as friends change,

Sometime over summer I’ll tidy up and publish the app over to AppEngine and let you all know.

Slowly and gently we create the new

We’re in the middle of building a new web presence/website/blog combining the best of presencelabs and authentic blogging into one great blog to inform to world.

We start with concept of what we want to be, then comes a name, and then comes new site, logo, tagline, stakeholders, new architecture for the site/blog and then the endless technical configuration of it all from feeds to email to subscription options, taxonomy of categories, tags, redirects, SEO, plugins and widgets

Typically, it is taking longer than we thought. It might voodoo the whole process, but I’ll say we should have something to show this week in its imperfect still-in-beta form. And I’m writing about the process from the technical side. I want to get a sense of what totally is involved in the new site and just what it has taken to get there.

So, more on that soon. And if presencelabs.com turns into a redirect to something else soon, you’ll know why.