Future of Web Apps: Thursday morning summary

I’m at the Future of Web Apps conference today and tomorrow in London.  Here are a few summary notes from this morning.

Note: this gets pretty technical in places.

The mornings’s theme seems to have been about operations and development isuses.

Digg Recommendations

Kevin Rose, from Digg, talked about Digg and the new Digg recommendations.  I’ve done lots of work in this area with our Recommendation Ventures web services, so it was really interesting to hear Digg’s experiences. A few points:

• When Digg added recommendations, they saw a 4 times increase in the number of people ‘friending’ other people, and a 40% increase in diggs (votes on news stories).  This goes along with the conventional wisdom regarding recommendations — they help keep visitors on your site longer and encourage interaction.
• Digg generates recommendations by clustering around keywords in their existing taxonomy. This generates better recommendations, by allowing a person to have differing interests, and generating and blending speciific recommendations for for those topic areas.  I suspect the do the clustering/bucketting to make the calculations less expensive, too.
• Digg have built custome graph stuff in Python to generate recommendations.  Nice to hear the Python namecheck there.

Web Applications

Edwin Aoki from AOL talked fairly generally about the Web Application Ecosystem..  A few points:

• Web apps have probably suffered from the release of a lot of device development kits this year:  iPhone, Android, more Flash development.  So, a step back into putting programs on devices rather than developing apps to run on all devices.
• Basically, end user consumers don’t care about open web standard and that.  They just want ot do stuff in usable apps.
• Web Services are important for building the fundamental services for creating enduring value, rather than another website.  (I think I got the point of that comment…)

Languages and Scaling and Operations

I guess every tech conference has to have a session to poke fun at programming languages.  Jokes cast at Ruby, PHP, Python and Perl by Joe Stump (Digg) and Blaine Cook (ex-Twitter). But some important points as well:

• languages don’t scale.  Scaling is something else, comes from actual systems architecture. (Therefore, who cares what language is used, keep developers happy)
• Web Apps need to be able to scale horizontally onto lots of small cheap boxes. Architect this in from the beginning to avoid pain later, but don’t sweat it too much.
• Capacity management matters.
• Use message queues.  Defer tasks into the background if you can.  This is essential when systems grow, and add lots of flexibility.
• Use caches such as memcache, but do it intelligently: cache invalidation is often a hard problem to solve.  Easy to add to the cache, harder to keep it consistent.
• Look out for herd-effects on cache invalidation: All servers then go and re-fetch data at once. Stagger invalidation times across servers.

Message Queues

Matt Biddulph from Dopplr talked about using message queues. Interesting stuff.  Basically, this is all about moving server processes into queues, so you can have one or more worker save servers to handle less-time-critical parts of the application in the background.

A few notes:

• Queues make life easy because:
  • Easy to add and remove slaves, which means easy scaling
  • Improve application performance by delaying things that don’t matter now to lower priority background processing
  • Easy performance monitoring .. look at the queues
• “Enterprise Integration Patterns by Hohpe, Woolf et al.” is worth reading.

Email Marketing Masterclass

21 August, 2008
6:30 pm	to	8:30 pm

One of our special Nodestone colleagues Mr Jim Callender is running a session soon that can’t be missed. Well, except by us as we will be swanning around in France having a well earned break!

For those of you who will be around, do get along. Email is one of the most tried and true, low cost and high response online marketing / social media tools to use. Do it well, and it might even go viral.

Jim will be co-presenting with the now-famous pure360 - total email marketing solution people. Great resources available on their site.

Thursday 21st August, 18:30 - 20:30

Payment by donation to The Werks.

Full details and register here.