MemoryError and Pickle

[John Gordon]

In <o0vvtm$1rpo$1 at gioia.aioe.org> Fillmore <fillmore_remove at hotmail.com> writes:


> Question for experts: is there a way to refactor this so that data may 
> be filled/written/released as the scripts go and avoid the problem?
> code below.

That depends on how the data will be read.  Here is one way to do it:

    fileObject = open(filename, "w")
    for line in sys.stdin:
        parts = line.strip().split("\t")
        fileObject.write("ta: %s\n" % parts[0])
        fileObject.write("wa: %s\n" % parts[1])
        fileObject.write("ua: %s\n" % parts[2])
    fileObject.close()

But this doesn't use pickle format, so your reader program would have to
be modified to read this format.  And you'll run into the same problem if
the reader expects to keep all the data in memory.

-- 
John Gordon                   A is for Amy, who fell down the stairs
gordon at panix.com              B is for Basil, assaulted by bears
                                -- Edward Gorey, "The Gashlycrumb Tinies"