psycopg2 & large result set

[Rob Wolfe]

Jon Clements wrote:
> Hi All.
>
> I'm using psycopg2 to retrieve results from a rather large query (it
> returns 22m records); unsurprisingly this doesn't fit in memory all at
> once. What I'd like to achieve is something similar to a .NET data
> provider I have which allows you to set a 'FetchSize' property; it
> then retrieves 'n' many rows at a time, and fetches the next 'chunk'
> after you read past the end of the current chunk. I suppose I could
> use Python for .NET or IronPython but I'd rather stick with CPython
> 2.5 if possible.

psycopg2 is DB-API 2.0 [1]_ compliant, so you can use
``fetchmany`` method and set ``cursor.arraysize`` accordingly.

[1] .. http://www.python.org/dev/peps/pep-0249/

--
HTH,
Rob