Categories
Research

Why can't AIP/Scitation do RSS feeds right?

I follow a number of scientific journals through their RSS feed of new articles. Google-reader makes it easy to stay on top of the 20 or so most interesting journals that each publish maybe 10 papers per week.

But now AIP and their website provider Scitation are messing it all up. They've decided to expand their archives and are putting scanned pdf articles from around 1979-1981 online. The trouble is these papers are showing up as new articles in the RSS feed! Not only that, but the text in the RSS feed shows the papers as appearing now, in 2008. It's only when you click through to the journal homepage that you realize the thing is 20+ years old.

This becomes very frustrating when suddenly APL or Rev.Sci.Instr indicates that they have 200 or more new papers! If they wanted to advertise to the world their 1980s physics they should do it with a different RSS feed.

I've emailed AIP and Scitation about this, but didn't get a reply. If you have the same problem I urge you to also email them and complain!

Categories
Research

Aggregating and Filtering Feeds

I'm trying to keep up with the ever increasing volume of scientific publications in my own and related fields. I've been using the Biophysical Journal's email based service for some time, but lately it has been very unreliable - often alerting me about supposedly 'new' papers that have been published in 1994 or so. Another way is to subscribe to the RSS/Atom feeds many journals provide, and I've been doing that also with Google reader, but it easily means wading through 100s of papers per week.

It's clear I need a better solution, something that first aggregates all the new papers into one big feed from the journals I am interested in, and then in a second step filters the big feed down to the few new papers that contain interesting keywords. Yahoo pipes could do that, but the LabVIEW-ish editor doesn't scale very well to a situation where you have 20+ feeds and 20+ keywords you are looking for. There's also google-mashup, but it isn't open for the public yet.

A complex solution would be to set up my own Planet, but it doesn't have web-based setup and administration so requires tinkering with config files etc. which I want to avoid.

So far I've only come up with this Thunderbird-based solution:

On the left I've subscribed to a number of journal feeds and put them in a folder of their own. On the right is a list of filters I am running. Each journal feed needs its own 'dummy filter' which does nothing but moves all the entries into the 'all papers' folder. Then I can run a filter of my own that looks for things in the subject or body of the paper. It's simple, ugly, but seems to work somehow.

Please tell me there is a simpler way to do this in Thunderbird! Or is there already a good web-based service like this around?

My requirements would be:

  • able to read and aggregate: RSS/Atom etc. (whatever the journals provide)
  • set up filters that look for keywords in any field or in only one field (title, author, abstract etc.)
  • output an RSS feed with all papers, and the filtered papers that I can read with Thunderbird or Google-reader.

So far I haven't found anything that would do this in a pain-free way. The aggregation part is handled by most web-based services, but there aren't many that allow searching/filtering and can provide the results as a separate feed.

Something like this is already going on with 'virtual-journals' that aggregate papers across journals in one field (e.g. Virtual Journal of Biological Physics Research or Virtual Journal of Nanoscale Science & Technology). Papers get selected to these 'VJs' by their editors, but I'm thinking my aggregator+filter idea will be able to cover a broader range of journals and look for more specific search terms.

Categories
Computer

Google Reader

I've found a new way to keep up with the latest on the web: Hello Google Reader, bye bye Live Bookmarks!