Fetching a bunch of RDF-files in an asynchron, resource-friendly way ?

[Thomas Weholt]

Ops! Sorry, some deeper dives into the endless ocean of information called
Google gave me a link to EffBots article on parsing RSS-feeds. Seem to be
just what I've been looking for.

http://www.effbot.org/zone/effnews-1.htm

Now, what happens if I merge parts of his code with Mark Pilgrims Ultra
Liberal RSS Parser? :-)

Thomas

"Thomas Weholt" <2002 at weholt.org> wrote in message
news:G%r4a.24068$CG6.394150 at news4.e.nsc.no...
> I got a bunch of RDF-files on several different sites I need to download
> several times a day. What is the most fastest, most effective way of doing
> this, while keeping the bandwidth use to a minimum? The fetched data will
be
> kept in memory between fetches, and if possible I'd like to check
> HTTP-headers for modification times to skip files that are not updated. Is
> there some asynchronous or threaded way this can be done? I have a feeling
> that my approach of having a bunch of threads or doing them in serial is
not
> very smart.
>
> Using something like Twisted is ok, since the code will be used in that
> framework, but a pure python way is preferred, but speed/low resource-cost
> wins if there's more than one solution.
>
> Any clues or hints are appreciated.
>
> Best regards,
> Thomas
>
>