XML document causes pickle to go into infinite recursion

Orest Kozyar orest.kozyar at gmail.com
Thu Nov 1 17:25:22 EDT 2007


> > I'm working on a CGI script that pulls XML data from a public database
>
> Ah, I missed that bit on first read. Consider using something different than
> CGI here if you want to do caching. FCGI would allow you to do in-memory
> caching, for example, as would mod_python and a lot of other solutions.

What I'm aiming for is sort of a "permanent" disk cache/mirror that
adds records as needed.  The main issue is that the database (PubMed)
requests that we limit requests to once every three seconds.  I often
need to access data for hundreds of records, so I've set up a cron job
to cache the needed records to disk overnight.  I was using shelve
before, but the size of the file grew to about 500 mB and I started
having issues with shelve performance.  So, I just stopped using
shelve and am storing each record separately on disk now.

Thanks!
Orest




More information about the Python-list mailing list