Python3 + sqlite3: Where's the bug?

[Chris Angelico]

On Fri, Dec 21, 2012 at 2:20 AM, Johannes Bauer <dfnsonfsduifb at gmx.de> wrote:
> Hmm, but this:
>
>                 result = cursor.fetchone()
>                         yield result
>
> Works nicely -- only the fetchmany() makes the example break.

Okay, now it's sounding specific to sqlite. I'll bow out. :)

>
>> Would it spoil your performance improvements to do all the fetchmany
>> calls before yielding anything?
>
> Well this would effectively then be a fetchall() call -- this is
> problematic since the source data is LARGE (spekaing of gigabytes of
> data here).

That would be a "yes", then. Scratch that!

>> Alternatively, can you separate the
>> two by opening a separate database connection for the foo-reading (so
>> it isn't affected by the commit)?
>
> At that point in the code I don't actually have a filename anymore,
> merely the connection. But shouldn't the cursor actually be the
> "correct" solution? I.e. in theory, should the example work at all or am
> I thinking wrong?

You say "db.commit()", not "cur2.commit()", so I don't see that a
cursor would un-break what part-way commits is breaking.

> Because if I'm approaching this from the wrong angle, I'll have no
> choice but to change all that code to open separate connections to the
> same file (something that currently are no provisions for).

Is that an sqlite limitation, or just one of your code?

I poked around at the sqlite3 docs, but didn't find any obvious
"clone" option on the connection, nor a way to retrieve the file name.
That would have been fairly convenient. Oh well.

ChrisA