os walk() and threads problems (os.walk are thread safe?)

[Diez B. Roggisch]

Marcus Alves Grando wrote:

> Diez B. Roggisch wrote:
>> Marcus Alves Grando wrote:
>> 
>>> Hello list,
>>>
>>> I have a strange problem with os.walk and threads in python script. I
>>> have one script that create some threads and consume Queue. For every
>>> value in Queue this script run os.walk() and printing root dir. But if i
>>> increase number of threads the result are inconsistent compared with one
>>> thread.
>>>
>>> For example, run this code plus sort with one thread and after run again
>>> with ten threads and see diff(1).
>> 
>> I don't see any difference. I ran it with 1 and 10 workers + sorted the
>> output. No diff whatsoever.
> 
> Do you test in one dir with many subdirs? like /usr or /usr/ports (in
> freebsd) for example?

Yes, over 1000 subdirs/files.
 
>> 
>> And I don't know what you mean by diff(1) - was that supposed to be some
>> output?
> 
> No. One thread produce one result and ten threads produce another result
> with less lines.
> 
> Se example below:
> 
> @@ -13774,8 +13782,6 @@
>   /usr/compat/linux/proc/44
>   /usr/compat/linux/proc/45
>   /usr/compat/linux/proc/45318
> -/usr/compat/linux/proc/45484
> -/usr/compat/linux/proc/45532
>   /usr/compat/linux/proc/45857
>   /usr/compat/linux/proc/45903
>   /usr/compat/linux/proc/46

I'm not sure what that directory is, but to me that looks like the
linux /proc dir, containing process ids. Which incidentially changes
between the two runs, as more threads will have process id aliases.

Try your script on another directory.

Diez