writing to file very slow

[lucio]

Hi,

Moritz Lennert wrote:

>Everything seems to work fine, except for the fact that writing the file
>is very slow (example: 4 minutes for 4 thousand lines).
>
>  
>
The code provided by Jack seems ok, given the PyGreSQL interface (except 
that the map statement should be inside the loop). So if its still not 
enough then there may be some problem somewhere else.

If its so slow it may be processor bound or memory bound. If its memory 
bound, i would advice to write() more often and to try to use an SQL API 
that allows you to query the result set one record at a time. How big is 
each field?

If its processor bound, then surely it is because you have too many 
fields (assuming 4K rows).

This should not take so long. One this you could do is time the differnt 
parts. How much does result.getresults() take? If thats the bottleneck, 
the you should swittch API. If thats not it, the you should find the 
correct mix of buffering and writing for your OS, so that it will 
minimize time.

HTH,

Lucio.