[Tutor] memory error files over 100MB

[A.T.Hofkamp]

Kent Johnson wrote:
> On Mon, Mar 16, 2009 at 12:30 PM, A.T.Hofkamp <a.t.hofkamp at tue.nl> wrote:
> 
>> I don't know what code is executed in an assignment exactly, but
>> **possibly**, first the 'read()' is executed (thus loading a very big string
>> into memory), before assigning the value to the variable (which releases the
>> previous value of the variable).
>> That means that just after reading but before assigning, you **may** have
>> two very big strings in memory that cannot be garbage collected.
> 
> No. Python variables are references to values, not containers for
> values. Python assignment is *always* reference assignment, not
> copying of a value. More here;
> http://personalpages.tds.net/~kent37/kk/00012.html

Nice web-page!



I am aware of how variables are treated in Python.
Let me explain my reasoning in more detail. Consider the statement

s = s + "def"

under the assumption that s is now "abc" (or rather, s references the data 
value "abc").

For the assignment, I believe the following happens inside the python 
interpreter (but I am not 100% sure that it is exactly the sequence):

1. get a reference to the current value of s.
2. get a reference to the constant value "def".
3. compute the new value "abcdef", store it in memory, and make a reference to it.
4. drop the old reference of s (thus free-ing "abc")
5. give s a reference to the newly computed value.


The point I was trying to make is that after step 3 and before step 4, the old 
value of s is still referenced by s, and the new value is referenced 
internally (so step 5 can be performed).
In other words, both the old and the new value are in memory at the same time 
after step 3 and before step 4, and both are referenced (that is, they cannot 
be garbage-collected).

Assuming that the above mechanism is used with all assignments, this will also 
  be the case in the sequence of assignments

s = read()
# write s
s = read()
# write s

where in the second assignment statement, the read() is done (creating the new 
value in memory) before dropping the value of s from the first assignment.

You can do the above statements also iteratively of course

for i in ...
   s = read()
   # write s

but since the loop does nothing with either s or read(), this will not change 
how the assignment works.


In the case that you are manipulating large values (as in taking a lot of 
computer memory for each value), the execution of the read() during step 3 may 
fail due to memory being used for the previously assigned value of s.


Sincerely,
Albert