Python RSS aggregator?

Erik Max Francis max at alcyone.com
Fri Dec 17 04:37:33 EST 2004


Back in 2000 I made a news aggregation site (REALpolitik, 
http://www.realpolitik.com/) since I didn't find anything that fit my 
needs.  (REALpolitik is unfortunately made in Perl; it was my last 
significant project before I started using Python for most of my work.) 
  At the time, RSS had not reached the near-universality that it has 
now, so RP itself uses a combination of custom scraping and whatever 
XML-type feeds that were available.

I've checked and all the feeds _I_ care about :-) are available in RSS 
now, so it would make sense to move to an RSS aggregator if it has the 
same features.  I've looked around at some that are available, both in 
Python and not, and haven't found anything that had the feature set I 
want.  One in Python would obviously be a huge benefit.

I'm not looking for anything all that fancy, but there are a combination 
of some seemingly basic features I just can't seem to find in other 
aggregators.  They are:

- one-page display:  It's awkward going back and forth between multiple 
feeds in a hierarchical format, and so it's much nicer if they're all 
presented on one page available for perusal.

- filtering news items:  Preferably for filtering out as well as 
highlighting, and also being able to selectively pass on, say, the first 
item in an RSS feed, since some popular feeds use that slot as an 
advertisement.

- caching news items:  I read news sporadically throughout the day, so 
one feature I really like is the ability to queue up new items over 
time, as distinguished by unique GUID.  For example, if an RSS feed only 
provided one (unique) item at all in its feed and that was updated once 
a day, letting the system run for several days would collect them all, 
each stored uniquely and available.

- recent items:  When you check for news, it only shows you the news 
items in each category that are new since you last caught up (catching 
up is the equivalent of a "mark all as read" feature).  That way, new 
news accumulates, and it's only news you haven't seen before.

Somewhat surprisingly to me, I can't seem to find an aggregator that 
supports all these features (using Mozilla).  Is it possible it's time 
for another Yet Another-type project?

-- 
Erik Max Francis && max at alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
   If love be good, from whence cometh my woe?
   -- Chaucer



More information about the Python-list mailing list