Pipes to the rescue

I’ve always though that Yahoo Pipes was a pretty cool thing. I’ve done a lot of work inside Scouta working with incoming and outgoing RSS, and the idea of doing ‘arithemetic’ on feeds is intriguing.

So, a problem just came up: for Widgety Goodness, we wanted to feed in all posts about Widgety Goodness and Widgets and Brighton into the WG07 backnetwork, but backnetwork would only accept tagged posts from registered bloggers that have activated their accounts. That limits the amount of blog articles that can be seen in backnetwork, which is a shame. Here’s how I got around it:

1. I constructed some searches in Google Blog Search for appropriate keywords
2. For each of these blog searches, I got an RSS feed, and then fed them into a Yahoo Pipe Fetch Feed block
3. Made a Union of these feeds
4. Sorted by ascending publish date
5. Removed duplicates
6. For each item in the resulting aggregated feed, I added the tag to the item.desciption with a Loop/String Builder
7. Fed the resulting into backnetwork, via a new user created called WG Feed.

Because backnetwork will aggregate all tagged posts from participant’s feeds, these posts now appear in the backnetwork Posts page.

It works nicely. One little issue is that all these posts show up as authored by the WG Feed user, but clicking through to get the full article goes to the right place.

Look at the pipe on Yahoo Pipes.

Mashup fatigue

This is mostly just a feeling at this point, but I’m starting to think that overenthusiastic mashup-making is a waste of time.

Mashups, I’m now thinking, are a path to, or perhaps even only (as Ivan suggested across the round table today) practice for, a solution that helps to solve some human problem. When we go to develop social and sociable systems, we are trying to solve some problem in the hands of users.Practice. Like a textbook problem. To build skills and test ideas. A demo. To mix shit together and see what happens. To test openness and transparency. A place on the way somewhere.

So, let’s go and mash up all these APIs, but let’s not waste all our lives on it.

In searching for real problems we’ll find elegant things to do that help.

Hack Day

I didn’t get to write a summary of Hack Day.. Fell into a week of busyness and a bunch of real work to get out of the way before Glastonbury. But that’s another story.

Hack Day was good fun:

• Lots of interesting people
• Lots of encouragement
• Great inclement weather and systems problems to make us work together
• Lots of great ideas flowing around. One of the things I love about the hacker crowd is the willingness of pretty much everybody to put forward crazy, mad schemes to test all the boundaries in sight. And how delightful to have somebody to listen to my mad schemes as well.

With a couple of annoyances for me:

• The WiFi never quite got reliable, at least on the Saturday. Very frustrating, and pretty quickly made me attempt to do something that didn’t need a net connection.
• I had an offer of a comfortable bed and good food close by, so that made it easy to sneak off on Sat and get a good night’s sleep. (Not really an annoyance, but the temptation got to me and i didn’t stay up hacking all night :-)

I got a good way through making a python APML library at Hack Day. I had intended to use it to do some attention-based searches on BBC content, but that will have to wait for later. I’ll put the finishing touches on that and post the code in the next couple of weeks after a bit more testing.

Hack Day lightning strike

A lightning strike to Alexandra Palace caused a fire system failure in the building housing Hack Day, opening air vents in the roof and letting rain into the room.

So, we all have had to evacuate to the palm court and hack together power and internet networks to keep Hack Day going again.

We’ve found a corner, have got some power strips, and I’ve set up a WiFi access point via my 3G mobile.  We’re off and running again, but the whole event is pretty disrupted at this point.

I’ll be at Hack Day

I just got the word that I’m accepted into the BBC/Yahoo Hack Day. Which sounds like a phenomenal event.
So, now to cook up something interesting to do. Hmm. Something with Pipes and BBC content and a recommendation engine or two perhaps… I wonder.
[Update: I've added the Python language category to the unofficial hack-day wiki to see what sort of group forms there.]