Getting started with APIs: using the Twitter API

API is one of those abbreviations that’s thrown around a lot but can seem a bit abstract.  Application Programming Interfaces (APIs) basically define a way for you to interact with a particular application (All clear now? No?). The best way to get your head around what this actually means is to use an API for something.

The Nerdary has a clear introductory guide to APIs, using the Twitter API. And the Twitter API really is a great place to start.

While Twitter may have bumped RSS off the homepage, you can still subscribe to Twitter using the API and, using Yahoo Pipes, combine and filter these feeds (and even clean up the data a little). This is a particularly handy way of monitoring feedback and mentions on Twitter and combining these into a super social media feed.

The Twitter API documentation will provide generic URLs as a guide which you can then use as RSS feeds in Yahoo Pipes (for example).

For starters to subscribe to a Twitter user’s lists:

(If you have any problems, there are always cheats available).

You can also subscribe to a Twitter search using the following format

There’s also a Basic Twitter Scraper available on ScraperWiki which you can fork to use as the basis of more advanced Twitter searches – but that might be a story for another time.

Continue reading “Getting started with APIs: using the Twitter API”


Getting started with Yahoo Pipes

Some rights reserved by alto maltés on Flickr

Using Yahoo Pipes has fast become a cornerstone of mashups in the library world. This popularity is largely down to just how accessible it makes the process of mashing data. It provides a nice, visual interface for remixing and reusing information from multiple sources.

At its simplest, Yahoo Pipes lets you combine multiple RSS feeds into a single, comprehensive news or information source. The introductory video seems to have disappeared from the website, but there are some great guides and pretty extensive documentation available in other places. We’ve also put together a walk-through for creating a basic pipe from multiple feeds. This is available as a word document which you’re welcome to modify and/or reuse.

Another way to get to know Yahoo Pipes is by viewing the source and cloning existing pipes to see how they were constructed. And in that spirit, I’ve listed a few that demonstrate some of the main uses of Yahoo Pipes. Click on ‘Edit Source’ to see what’s happening behind the scenes or clone the pipe to use it as a starting point for your own creation.

Starter pipe: A very simple pipe to combine RSS feeds and sort items by publication date.

Library Technology: A Different Version– Combines multiple feeds and returns results for these that match particular keywords.

UK Academic Library Blogs – Combines multiple feeds, sorts into ascending order and limits the feed to 5 items at a time.

This is hopefully enough to get started and, like most things, the best way to learn about something like this is by trying it out yourself.

In a future post, we’ll take a look at some of the more advanced features such as geocoding.

Data, pipes, geeks and cake: Pancakes and Mash at Lincoln

In lieu of someone posting some decent photos from the day

Pancakes and Mash, the eighth (!) Mashed Library event was held yesterday, organised by the gang at University of Lincoln. It was a great day looking at various existing projects that involved mashing data together (in various ways and by various means) as well as giving attendees a chances to get involved in library-related mashups.

The keynote speaker was Gary Green (Surrey County Council), one of the key people behind the ‘Voices for the Library‘ campaign. Gary gave an overview of how this project began and how it facilitates sharing info between the many local library campaigns underway. He also gave an overview of the various tools that have helped make it such a success. Among these, the VftL team credits Twitter with playing a major role in bringing the project together while they also use Facebook, Delicious, email lists, Google Fusion tables, wikis and blogging to help get the word out.

One aspect of this I hadn’t fully explored until now was the map of libraries under threat — which is now generated using Google Fusion tables from data on Public Library news. You can see the map on If you haven’t explored the Voices for the Library website yet, it’s a great resource of campaign news and advice.

The second session was split into a few different workshops. I attended the ‘Metadata Forum: building a community around metadata‘ session led by Stephanie Taylor from UKOLN — while trying my best to also stealthily follow the other sessions on Twitter. The Metadata Forum is a JISC-funded project that aims to build a community around metadata for those who work with it in any capacity. They’re interested in all levels of metadata users, not just the specialists and based on attendance, a *lot* of us are working with metadata. You can read more about the forum on their blog on the UKOLN website.

The other sessions were “We Can Haz Ur Data!?” with Alex Bilbie & Nick Jackson from the University of Lincoln and “Using Web2.0 tools to save libraries” with Gary Green, building on his keynote talk. Both sounded really interesting and hopefully notes and/or slides will be circulated soon.

After lunch, Alison McNab (De Montfort University) gave a talk on ‘mash at lunchtime’ for those looking for shorter, onsite events and Stephanie Taylor (UKOLN) led the discussion on ‘Across the divide: how geeks and non geeks can have meaningful conversations with each other, and how we’re all the same, really‘. There was also the option to get some mashing done in the other spaces open to attendees — which turned into a great walkthrough session on Yahoo Pipes by Paul Stainthorp.

It was a productive and creative day and I think it’s safe to say that everyone went home brimming with ideas and armed with an extensive list of web apps and tools to try out. Thanks to Paul and the rest of the gang at Uni of Lincoln for organising such a great event (and to Elif Varol for the cakes). Only tentative rumours about the next mashed libraries day at the moment but keep an eye on the wiki as I’m sure there be more news soon and there’s plenty to get started with in the meantime.

RSS technologies: around the web

ReadWriteWeb has printed a top ten guide to RSS and Syndication Technologies for the year, including mobile and web-based RSS tools  (the article has been republished on but the original has a better comments section).

A surprise omission is LazyFeed which is a great way to monitor specific topics in real time and now includes more sharing features. They’re also working on more filtering options to be rolled out.

Not sure I quite agree that Yahoo Pipes has been usurped by YSQL (do they mean  YQL?) as I’m still hoping that the two tools can co-exist but there are lots of other great RSS tools to check out.

Middlemash: the third Mashed Library UK event (part two)

QRCode created using SplashURL

The afternoon sessions of Middlemash were a chance to collaborate. It was loosely divided into 3 groups, although participants were free to break off into their own groups to develop and exchange ideas.

The main sessions were

  • Javascript User Interface Componentised Extensions (JUICE)
  • Yahoo Pipes
  • Mapping libraries

I had originally intended to attend the JUICE session (led by JUICE creator, Richard Wallis), as I think a move towards a standardised way of managing and implementing web extensions for library systems is a great idea. JUICE has just moved to a new website and new documentation is being added everyday so if you are interested in finding out more, visit

But in the end I decided to  join the Yahoo Pipes group (while still trying to keep an eye on the JUICE discussions – with only limited success). Yahoo Pipes formed the basis of many of the mashups mentioned during the day and was a much more powerful tool than I’d previously realised. The session, led by Tony Hirst, from the Open University, gave us an overview of the possible uses for Yahoo Pipes. You can view the slides for the session via slideshare. Also, it’s worth checking out some of Tony’s previous posts about pipes, such as the 2D journal pipe . One of the many interesting things about Yahoo Pipes is that, as well as RSS, it allows you to import data from other sources, such as XML and CSV files.

While Tony Hirst’s knowledge of Yahoo Pipes is incredibly advanced (and then some), his presentation is well worth viewing even as an introduction. As with everyone else who attended, I left the session blown away by the potential for data manipulation using pipes. There are also introductory videos available for those just getting started. You can sign up for an account and experiment with some simpler options such as merging and filtering RSS feeds. Another option is to ‘clone’ existing Pipes, which is a great way to find out what’s ‘under the bonnet’ of other people’s creations.

And to finish, a non-pipe but very useful technique for using to create QRCode. These are graphical representations of URIs, to save us all from having to copy down long web addresses in the midst of (un)conference fever.

Unfortunately I didn’t manage to catch any of the ‘mapping the library’ discussion but details about the project can be found at the Mashed Library wiki. There is also a bit about the session on the Panlibus blog.

Thanks again to Damyanti Patel and others at Birmingham City University for organising such an inspiring event. Can’t wait for Mashed Library 2010.