CPython 2.7: Weakset data changing size during internal iteration

[Steven D'Aprano]

On Fri, 01 Jun 2012 20:24:30 -0700, Temia Eszteri wrote:

> On 02 Jun 2012 03:05:01 GMT, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
> 
>>I doubt that very much. If you are using threads, it is more likely your
>>code has a race condition where you are modifying a weak set at the same
>>time another thread is trying to iterate over it (in this case, to
>>determine it's length), and because it's a race condition, it only
>>happens when conditions are *just right*. Since race conditions hitting
>>are usually rare, you only notice it when there's a lot of data.
> 
> Except that the few threads I use don't modify that data at all 
[...]

And should I have known this from your initial post?


[...]
> Don't be so fast to dismiss things when the situation would not have
> made a race condition possible to begin with.

If you have been part of this newsgroup and mailing list as long as I 
have, you should realise that there is no shortage of people who come 
here and make grand claims that they have discovered a bug in Python 
(either the language, or the standard library). Nine times out of ten, 
they have not, and the bug is in their code, or their understanding.

Perhaps you are one of the few who has actually found a bug in the 
standard library rather than one in your own code. But your initial post 
showed no sign that you had done any investigation beyond reading the 
traceback and immediately jumping to the conclusion that it was a bug in 
the standard library.

Frankly, I still doubt that your analysis of the problem is correct:

    [quote]
    Problem is that for certain high-frequency operations, it 
    seems there's too much data going in and out for it to handle
    [end quote]


I still can't see any way for this bug to occur due to "too much data", 
as you suggest, or in the absence of one thread modifying the set while 
another is iterating over it. But I could be wrong.

In any case, it appears that this bug has already been reported and fixed:

http://bugs.python.org/issue14159


Consider updating to the latest bug fix of 2.7.



-- 
Steven