PEP 249 (database api) -- executemany() with iterable?

[Ned Deily]

In article 
<2eeb1c54-83f5-4375-93fb-478bdbd7e041 at j25g2000yqa.googlegroups.com>,
 Jon Clements <joncle at googlemail.com> wrote:

> On 12 Oct, 18:32, Roy Smith <r... at panix.com> wrote:
> > On Oct 12, 1:20 pm, Jon Clements <jon... at googlemail.com> wrote:
> >
> > > On 12 Oct, 16:10, Roy Smith <r... at panix.com> wrote:
> >
> > > > PEP 249 says about executemany():
> >
> > > >         Prepare a database operation (query or command) and then
> > > >         execute it against all parameter sequences or mappings
> > > >         found in the sequence seq_of_parameters.
> >
> > > > are there any plans to update the api to allow an iterable instead of
> > > > a sequence?
> >
> > > I'm not understanding (probably me). Example?
> >
> > I have a dictionary, d, which has a million items in it, and want to
> > do something like:
> >
> >     executemany("update foo set bar = %s where id = %s",
> > d.iteritems())
> >
> > If executemany accepted an iterable, that would work.  But, it only
> > accepts a sequence, so I need to do:
> >
> >     executemany("update foo set bar = %s where id = %s", d.items())
> >
> > which generates a million-item temporary list.  Or am I mis-
> > understanding the PEP?
> 
> Interesting, but here's my guess...
> 
> Replace d.items() with itertools.repeat( ('a', 'b') )
> 
> So, if you have a sequence, which has a length and known size, at
> least you can have an attempt at the DB operations: whether the
> transaction fails or not is another thing...In short, a sequence is
> finite, while an iterable may be infinite.

Also, keep in mind that PEP 249 DB adapter implementations are typically 
wrappers around a lower-level client library for a particular DB 
implementation and that most of those client APIs - written in C - will 
likely require all of the items to be passed in one call.  If so, the DB 
adapter would need to immediately evaluate the iterable and produce a 
list in memory anyway.

-- 
 Ned Deily,
 nad at acm.org