Is there a maximum size to a Python program?

[Paul Hemans]

The reason I started down this path is that the program is grinding to a 
halt (my PC as well) and the only way to get it going again is:
>>>import gc
>>>gc.collect()
2524104

Memory used by the process is around 500MB, I have posted about this problem 
before on this newsgroup but I didn't get a resolution. So I was trying to 
separate the logic into different files and finish each one with 
gc.collect().
Sorry I didn't post the original problem as well but I really just wanted 
the thing to work and not cloud the issue again.
I have implemented your approach and the code is much cleaner (except for 
another problem that I have posted to sqlAlchemy). It doesn't solve my 
original problem with the memory, but I really need SOMETHING to work. So I 
will keep plodding away and then maybe move the XML processing to lxml and 
see if that resolves the memory problem.


"John Machin" <sjmachin at lexicon.net> wrote in message 
news:3215638c-2408-4cf6-9321-f85f560e682d at u39g2000pru.googlegroups.com...
On Apr 27, 3:31 pm, "Paul Hemans" <dar... at nowhere.com> wrote:
> Thanks John, I understand where you are coming from and will try and 
> digest
> it all. One problem though that I didn't mention in my original posting 
> was
> that the replication may only require updating one or more fields, that is 
> a
> problem with a generating a single SQL statement to cover all requests.

OK so you need TWO programs -- one tailored for initial loading of a
table, the other for subsequent updates. Still O(1) :-)

OR you could still do it in one, by using the prepared whole-row SQL
statement if appropriate, or building an SQL statement on the fly if
it's an update.

> I am
> having a look at eval and exec

WRONG WAY
GO BACK

> to see if they will give me a little more
> flexibility in runtime generation of the code.

And another thing that you didn't mention was that you are using
SQLAlchemy -- perhaps you might like to ask your question on that
package's forum ... including a few more facts than heretofore :-)

HTH
John