XML document causes pickle to go into infinite recursion

[Paul Boddie]

On 1 Nov, 22:25, Orest Kozyar <orest.koz... at gmail.com> wrote:
> > > I'm working on a CGI script that pulls XML data from a public database
>
> > Ah, I missed that bit on first read. Consider using something different than
> > CGI here if you want to do caching. FCGI would allow you to do in-memory
> > caching, for example, as would mod_python and a lot of other solutions.
>
> What I'm aiming for is sort of a "permanent" disk cache/mirror that
> adds records as needed.  The main issue is that the database (PubMed)
> requests that we limit requests to once every three seconds.

Drifting off-topic slightly, it is possible to download archives of
PubMed, but you'll need a lot of disk space; you probably know this
already. Putting the data into something rapidly searchable can demand
even more space, depending on what you use and what you want to be
able to search for. I guess it depends on whether you're retrieving
relatively few documents across the whole of PubMed or whether your
searches are concentrated in particular sections of the whole archive.

Paul