Future of Web Apps: Thursday morning summary

I’m at the Future of Web Apps conference today and tomorrow in London.  Here are a few summary notes from this morning.

Note: this gets pretty technical in places.

The mornings’s theme seems to have been about operations and development isuses.

Digg Recommendations

Kevin Rose, from Digg, talked about Digg and the new Digg recommendations.  I’ve done lots of work in this area with our Recommendation Ventures web services, so it was really interesting to hear Digg’s experiences. A few points:

• When Digg added recommendations, they saw a 4 times increase in the number of people ‘friending’ other people, and a 40% increase in diggs (votes on news stories).  This goes along with the conventional wisdom regarding recommendations — they help keep visitors on your site longer and encourage interaction.
• Digg generates recommendations by clustering around keywords in their existing taxonomy. This generates better recommendations, by allowing a person to have differing interests, and generating and blending speciific recommendations for for those topic areas.  I suspect the do the clustering/bucketting to make the calculations less expensive, too.
• Digg have built custome graph stuff in Python to generate recommendations.  Nice to hear the Python namecheck there.

Web Applications

Edwin Aoki from AOL talked fairly generally about the Web Application Ecosystem..  A few points:

• Web apps have probably suffered from the release of a lot of device development kits this year:  iPhone, Android, more Flash development.  So, a step back into putting programs on devices rather than developing apps to run on all devices.
• Basically, end user consumers don’t care about open web standard and that.  They just want ot do stuff in usable apps.
• Web Services are important for building the fundamental services for creating enduring value, rather than another website.  (I think I got the point of that comment…)

Languages and Scaling and Operations

I guess every tech conference has to have a session to poke fun at programming languages.  Jokes cast at Ruby, PHP, Python and Perl by Joe Stump (Digg) and Blaine Cook (ex-Twitter). But some important points as well:

• languages don’t scale.  Scaling is something else, comes from actual systems architecture. (Therefore, who cares what language is used, keep developers happy)
• Web Apps need to be able to scale horizontally onto lots of small cheap boxes. Architect this in from the beginning to avoid pain later, but don’t sweat it too much.
• Capacity management matters.
• Use message queues.  Defer tasks into the background if you can.  This is essential when systems grow, and add lots of flexibility.
• Use caches such as memcache, but do it intelligently: cache invalidation is often a hard problem to solve.  Easy to add to the cache, harder to keep it consistent.
• Look out for herd-effects on cache invalidation: All servers then go and re-fetch data at once. Stagger invalidation times across servers.

Message Queues

Matt Biddulph from Dopplr talked about using message queues. Interesting stuff.  Basically, this is all about moving server processes into queues, so you can have one or more worker save servers to handle less-time-critical parts of the application in the background.

A few notes:

• Queues make life easy because:
  • Easy to add and remove slaves, which means easy scaling
  • Improve application performance by delaying things that don’t matter now to lower priority background processing
  • Easy performance monitoring .. look at the queues
• “Enterprise Integration Patterns by Hohpe, Woolf et al.” is worth reading.

Shine weekend

We’re just back from the Shine Unconference. Lib was there for the lot, but I was only there for Sunday. I’m a bit tired to say much about it this evening.

But.. it was great and inspiring hanging around and chatting with a lot of people who are really going with their passions and doing great and good things. It was a very warm and friendly space, and I really enjoyed the sessions we did today.

Lib ran a short discussion on Authenticity Online which sparked a robust discussion of a bunch of issues around social media, free content, risk management, online reputation and online tools. There are a lot of questions out there around social media and the third sector. We collected some meaty questions which we’ll post over on the authentic blogging community in the next couple of days along with the beginnings of some answers. Thanks to all who participated in that discussion and hope you join us over on the community to work through the issues.

Busy afternoon — we then whizzed over to the “What’s in a Name” session being run by Neru. Lib and I are working on rebranding, so wanted to get some tips on choosing a new name for us. Interesting ideas from there. Thanks to the Neru team and all who threw in their thoughts. We’ll keep working on that new name and see what happens.

It always seems a long train ride home to Brighton on a Sunday, but when we got home it was still sunny and the air was fresh. It was quiet. You got the sense a lot of people had satisfying weekends and were now relaxing at home. I sure did.

Lib at Lift08

Lib has headed off to Lift08 in Geneva. Here she is giving some reflections so far…

I’ll be at Hack Day

I just got the word that I’m accepted into the BBC/Yahoo Hack Day. Which sounds like a phenomenal event.
So, now to cook up something interesting to do. Hmm. Something with Pipes and BBC content and a recommendation engine or two perhaps… I wonder.
[Update: I've added the Python language category to the unofficial hack-day wiki to see what sort of group forms there.]

Brighton web and tech events

Raj Anand had made a really useful summary of the web and tech events that go on in Brighton.  So, I’ll be getting to a bunch of these in the next few months.

BarCamp London coming in June or July

Something else to do in London on a summer day…

Bar Camp London.

No date set yet, but promised sometime in June/July.  I’m looking forward to this one.

Geek day/night/dinner in London

I’m going to head into London next Saturday June 17th for the @media Social and then stay on the for the Geek Dinner. Om Malik is also talking about doing something in London that day. So, will have to try and fit it all into the one afternoon.

[Update Fri 16/6:  Just planning my day tomorrow; it looks like the Geek Dinner and Om Malik event are indeed separate and both on tomorrow evening.  The Om event, codenamed Londom, is happening at at the Princess Louise (Holborn) from 7pm onwards, while the Geek dinner is scheduled for 6pm onwards at The Livery, 130 Wood Street, in the City of London, EC2V 6DL. Maybe the plan is to @media social first, catch the Geek dinner then head over to the Princess Louise later.  i'm sure the plan will evolve...]

reboot

The reboot8 conference looks really interesting. One of the many reasons to move to the UK is to be close to things like this and actually be able to get there without it taking weeks and costing a fortune.

Think I might try and get up to Copenhagen for the weekend of 1-2 June. We arrive in the UK on May 16th, so we’ll just be getting settled.

Here’s what they say reboot is about:

reboot is a community event for the practical visionaries who are at the intersection of digital technology and change all around us… 2 days. 400 people. A journey into the interconnectedness of creation, participation, values, openness, decentralization, collaboration, complexity, technology, p2p, humanities, connectedness and many more areas.
Applied towards us as individuals, citizens, teachers, culture workers, entrepreneurs, creators and change makers.

Update 23/4:  I’m about to start booking flightsand stuff for this.  It just seems so much a good thing!