[python-win32] Possilbly PyWin32/IOCP issue

[Oleksii Shevchuk]

Hi!

I wrote a small sample program ("distributed" proxy server) using
asyncoro (https://github.com/pgiri/asyncoro) library. When I try to use
it on Windows (tried with 7/10 32/64) I observe python (2.7.12/2.7.13)
interpreter crash in GC (always in derefs). It doesn't matter is GC disabled (with
gc.disable) or not, crashes always happen after some time in objects
cleanup.

Why I wrote this message here.

AsynCoro uses pywin32 for IOCP bindings.
(https://github.com/pgiri/asyncoro/blob/master/py2/asyncoro/__init__.py#L917)

When I use AsynCoro with python.select backend I don't meet these
problems. So I wrote a bug to the author of the library
(https://github.com/pgiri/asyncoro/issues/10), but looks like he (like
me) a Linux guy, and see and use these APIs first time.

So I write mail here, maybe somebody can give us advice how to resolve
this situation.

A bit more details.
Sample program is here:
https://gist.github.com/alxchk/482055c284dc3289f1b61b843908c2af

To reproduce I use next steps:
1. rpyc_classic on Linux host
2. asyncoroproxy-test.py runned on Windows host
3. Firefox + Socks5 proxy walks via asyncoroproxy from Windows host
4. Open several sites simultaneously (and youtube among them)

It's interesting, that in reverse direction (Socket server on linux,
exit point on windows) I can't get a crash. So a difference may be in
1. Firefox closes sockets in some unusual way (whatever this means)
2. Some issues with AcceptEx
3. Some issues with cancelation WSASend/WSARecv

Crash callstack example:

0:005> k
 # Child-SP          RetAddr           Call Site
00 00000000`03e7efc0 00000000`778a1edf python27!BaseException_dealloc+0x16 [c:\build27\cpython\objects\exceptions.c @ 83]
01 00000000`03e7eff0 00000000`778e84ce python27!tupledealloc+0xcf [c:\build27\cpython\objects\tupleobject.c @ 222]
02 00000000`03e7f020 00000000`778eca11 python27!PyEval_EvalFrameEx+0xa4e [c:\build27\cpython\python\ceval.c @ 2483]
03 00000000`03e7f130 00000000`77877038 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
04 00000000`03e7f1e0 00000000`77842d23 python27!function_call+0x178 [c:\build27\cpython\objects\funcobject.c @ 528]
05 00000000`03e7f260 00000000`778ee8eb python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
06 00000000`03e7f290 00000000`778eb573 python27!ext_do_call+0x2eb [c:\build27\cpython\python\ceval.c @ 4666]
07 00000000`03e7f2f0 00000000`778ee178 python27!PyEval_EvalFrameEx+0x3af3 [c:\build27\cpython\python\ceval.c @ 3031]
08 00000000`03e7f400 00000000`778ee02f python27!fast_function+0xc8 [c:\build27\cpython\python\ceval.c @ 4438]
09 00000000`03e7f470 00000000`778eb49b python27!call_function+0x3af [c:\build27\cpython\python\ceval.c @ 4372]
0a 00000000`03e7f4d0 00000000`778ee178 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
0b 00000000`03e7f5e0 00000000`778ee02f python27!fast_function+0xc8 [c:\build27\cpython\python\ceval.c @ 4438]
0c 00000000`03e7f650 00000000`778eb49b python27!call_function+0x3af [c:\build27\cpython\python\ceval.c @ 4372]
0d 00000000`03e7f6b0 00000000`778eca11 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
0e 00000000`03e7f7c0 00000000`77877038 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
0f 00000000`03e7f870 00000000`77842d23 python27!function_call+0x178 [c:\build27\cpython\objects\funcobject.c @ 528]
10 00000000`03e7f8f0 00000000`778575d9 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
11 00000000`03e7f920 00000000`77842d23 python27!instancemethod_call+0x219 [c:\build27\cpython\objects\classobject.c @ 2603]
12 00000000`03e7fba0 00000000`778edb41 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
13 00000000`03e7fbd0 00000000`7780cb1e python27!PyEval_CallObjectWithKeywords+0x141 [c:\build27\cpython\python\ceval.c @ 4222]
14 00000000`03e7fc00 00000000`77931405 python27!t_bootstrap+0x9e [c:\build27\cpython\modules\threadmodule.c @ 621]
15 00000000`03e7fc30 00000000`77c72fdf python27!bootstrap+0x35 [c:\build27\cpython\python\thread_nt.h @ 124]
16 00000000`03e7fc60 00000000`77c73080 MSVCR90!_callthreadstartex+0x17 [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c @ 348]
17 00000000`03e7fc90 00007ff9`6ebe8364 MSVCR90!_threadstartex+0x84 [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c @ 326]
18 00000000`03e7fcc0 00007ff9`6f1570d1 KERNEL32!BaseThreadInitThunk+0x14
19 00000000`03e7fcf0 00000000`00000000 ntdll!RtlUserThreadStart+0x21
static void
BaseException_dealloc(PyBaseExceptionObject *self)
{
    _PyObject_GC_UNTRACK(self); <<<--- HERE

    BaseException_clear(self);
    Py_TYPE(self)->tp_free((PyObject *)self);
}

This trace was with gc.set_threshold(0)

Without disabled gc, it will die during garbage collection:

0:005> k
 # Child-SP          RetAddr           Call Site
00 00000000`04bfd250 00000000`778a4fc8 python27!visit_decref+0x6 [c:\build27\cpython\modules\gcmodule.c @ 360]
01 00000000`04bfd280 00000000`777dcf08 python27!subtype_traverse+0x78 [c:\build27\cpython\objects\typeobject.c @ 845]
02 00000000`04bfd2d0 00000000`777de2a0 python27!collect+0x1e8 [c:\build27\cpython\modules\gcmodule.c @ 925]
03 00000000`04bfd370 00000000`778a1ab8 python27!_PyObject_GC_Malloc+0x120 [c:\build27\cpython\modules\gcmodule.c @ 1512]
04 00000000`04bfd3a0 00000000`778ee438 python27!PyTuple_New+0x158 [c:\build27\cpython\objects\tupleobject.c @ 90]
05 00000000`04bfd3d0 00000000`778edef9 python27!load_args+0x18 [c:\build27\cpython\python\ceval.c @ 4521]
06 00000000`04bfd400 00000000`778eb49b python27!call_function+0x279 [c:\build27\cpython\python\ceval.c @ 4352]
07 00000000`04bfd460 00000000`778eca11 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
08 00000000`04bfd570 00000000`77877038 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
09 00000000`04bfd620 00000000`77842d23 python27!function_call+0x178 [c:\build27\cpython\objects\funcobject.c @ 528]
0a 00000000`04bfd6a0 00000000`77799d33 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
0b 00000000`04bfd6d0 00000000`77842d23 python27!partial_call+0x73 [c:\build27\cpython\modules\_functoolsmodule.c @ 198]
0c 00000000`04bfd700 00000000`778ee5ab python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
0d 00000000`04bfd730 00000000`778ee045 python27!do_call+0x11b [c:\build27\cpython\python\ceval.c @ 4569]
0e 00000000`04bfd770 00000000`778eb49b python27!call_function+0x3c5 [c:\build27\cpython\python\ceval.c @ 4374]
0f 00000000`04bfd7d0 00000000`778eca11 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
10 00000000`04bfd8e0 00000000`77877038 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
11 00000000`04bfd990 00000000`77842d23 python27!function_call+0x178 [c:\build27\cpython\objects\funcobject.c @ 528]
12 00000000`04bfda10 00000000`778575d9 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
13 00000000`04bfda40 00000000`77842d23 python27!instancemethod_call+0x219 [c:\build27\cpython\objects\classobject.c @ 2603]
14 00000000`04bfdcc0 00000000`778ee5ab python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
15 00000000`04bfdcf0 00000000`778ee045 python27!do_call+0x11b [c:\build27\cpython\python\ceval.c @ 4569]
16 00000000`04bfdd30 00000000`778eb49b python27!call_function+0x3c5 [c:\build27\cpython\python\ceval.c @ 4374]
17 00000000`04bfdd90 00000000`778ee178 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
18 00000000`04bfdea0 00000000`778ee02f python27!fast_function+0xc8 [c:\build27\cpython\python\ceval.c @ 4438]
19 00000000`04bfdf10 00000000`778eb49b python27!call_function+0x3af [c:\build27\cpython\python\ceval.c @ 4372]
1a 00000000`04bfdf70 00000000`778eca11 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
1b 00000000`04bfe080 00000000`77877038 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
1c 00000000`04bfe130 00000000`77842d23 python27!function_call+0x178 [c:\build27\cpython\objects\funcobject.c @ 528]
1d 00000000`04bfe1b0 00000000`778575d9 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
1e 00000000`04bfe1e0 00000000`77842d23 python27!instancemethod_call+0x219 [c:\build27\cpython\objects\classobject.c @ 2603]
1f 00000000`04bfe460 00000000`778ee8eb python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
20 00000000`04bfe490 00000000`778eb573 python27!ext_do_call+0x2eb [c:\build27\cpython\python\ceval.c @ 4666]
21 00000000`04bfe4f0 00000000`778eca11 python27!PyEval_EvalFrameEx+0x3af3 [c:\build27\cpython\python\ceval.c @ 3031]
22 00000000`04bfe600 00000000`77877038 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
23 00000000`04bfe6b0 00000000`77842d23 python27!function_call+0x178 [c:\build27\cpython\objects\funcobject.c @ 528]
24 00000000`04bfe730 00000000`778575d9 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
25 00000000`04bfe760 00000000`77842d23 python27!instancemethod_call+0x219 [c:\build27\cpython\objects\classobject.c @ 2603]
26 00000000`04bfe9e0 00000000`778b0390 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
27 00000000`04bfea10 00000000`778a4d76 python27!slot_tp_init+0xd0 [c:\build27\cpython\objects\typeobject.c @ 5807]
28 00000000`04bfea40 00000000`77842d23 python27!type_call+0xd6 [c:\build27\cpython\objects\typeobject.c @ 765]
29 00000000`04bfea70 00000000`778ee5ab python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
2a 00000000`04bfeaa0 00000000`778ee045 python27!do_call+0x11b [c:\build27\cpython\python\ceval.c @ 4569]
2b 00000000`04bfeae0 00000000`778eb49b python27!call_function+0x3c5 [c:\build27\cpython\python\ceval.c @ 4374]
2c 00000000`04bfeb40 00000000`778eca11 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
2d 00000000`04bfec50 00000000`778ee1e9 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
2e 00000000`04bfed00 00000000`778ee02f python27!fast_function+0x139 [c:\build27\cpython\python\ceval.c @ 4451]
2f 00000000`04bfed70 00000000`778eb49b python27!call_function+0x3af [c:\build27\cpython\python\ceval.c @ 4372]
30 00000000`04bfedd0 00000000`7787773c python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
31 00000000`04bfeee0 00000000`778ede8d python27!gen_send_ex+0x12c [c:\build27\cpython\objects\genobject.c @ 86]
32 00000000`04bfef20 00000000`778eb49b python27!call_function+0x20d [c:\build27\cpython\python\ceval.c @ 4340]
33 00000000`04bfef80 00000000`778eca11 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
34 00000000`04bff090 00000000`77877038 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
35 00000000`04bff140 00000000`77842d23 python27!function_call+0x178 [c:\build27\cpython\objects\funcobject.c @ 528]
36 00000000`04bff1c0 00000000`778ee8eb python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
37 00000000`04bff1f0 00000000`778eb573 python27!ext_do_call+0x2eb [c:\build27\cpython\python\ceval.c @ 4666]
38 00000000`04bff250 00000000`778ee178 python27!PyEval_EvalFrameEx+0x3af3 [c:\build27\cpython\python\ceval.c @ 3031]
39 00000000`04bff360 00000000`778ee02f python27!fast_function+0xc8 [c:\build27\cpython\python\ceval.c @ 4438]
3a 00000000`04bff3d0 00000000`778eb49b python27!call_function+0x3af [c:\build27\cpython\python\ceval.c @ 4372]
3b 00000000`04bff430 00000000`778ee178 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
3c 00000000`04bff540 00000000`778ee02f python27!fast_function+0xc8 [c:\build27\cpython\python\ceval.c @ 4438]
3d 00000000`04bff5b0 00000000`778eb49b python27!call_function+0x3af [c:\build27\cpython\python\ceval.c @ 4372]
3e 00000000`04bff610 00000000`778eca11 python27!PyEval_EvalFrameEx+0x3a1b [c:\build27\cpython\python\ceval.c @ 2993]
3f 00000000`04bff720 00000000`77877038 python27!PyEval_EvalCodeEx+0x911 [c:\build27\cpython\python\ceval.c @ 3584]
40 00000000`04bff7d0 00000000`77842d23 python27!function_call+0x178 [c:\build27\cpython\objects\funcobject.c @ 528]
41 00000000`04bff850 00000000`778575d9 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
42 00000000`04bff880 00000000`77842d23 python27!instancemethod_call+0x219 [c:\build27\cpython\objects\classobject.c @ 2603]
43 00000000`04bffb00 00000000`778edb41 python27!PyObject_Call+0x83 [c:\build27\cpython\objects\abstract.c @ 2548]
44 00000000`04bffb30 00000000`7780cb1e python27!PyEval_CallObjectWithKeywords+0x141 [c:\build27\cpython\python\ceval.c @ 4222]
45 00000000`04bffb60 00000000`77931405 python27!t_bootstrap+0x9e [c:\build27\cpython\modules\threadmodule.c @ 621]
46 00000000`04bffb90 00000000`77c72fdf python27!bootstrap+0x35 [c:\build27\cpython\python\thread_nt.h @ 124]
47 00000000`04bffbc0 00000000`77c73080 MSVCR90!_callthreadstartex+0x17 [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c @ 348]
48 00000000`04bffbf0 00007ff9`6ebe8364 MSVCR90!_threadstartex+0x84 [f:\dd\vctools\crt_bld\self_64_amd64\crt\src\threadex.c @ 326]
49 00000000`04bffc20 00007ff9`6f1570d1 KERNEL32!BaseThreadInitThunk+0x14
4a 00000000`04bffc50 00000000`00000000 ntdll!RtlUserThreadStart+0x21
static int
visit_decref(PyObject *op, void *data)
{
    assert(op != NULL);
    if (PyObject_IS_GC(op)) { <<--- HERE
        PyGC_Head *gc = AS_GC(op);
        /* We're only interested in gc_refs for objects in the
         * generation being collected, which can be recognized
         * because only they have positive gc_refs.
         */
        assert(gc->gc.gc_refs != 0); /* else refcount was too small */
        if (gc->gc.gc_refs > 0)
            gc->gc.gc_refs--;
    }
    return 0;
}
0:005> dv op
             op = 0x00000000`77a9a6c0
0:005> dx -r1 (*((python27!_object *)0x77a9a6c0))
(*((python27!_object *)0x77a9a6c0))                 [Type: _object]
    [+0x000] ob_refcnt        : 61836288 [Type: __int64]
    [+0x008] ob_type          : 0x3ae6378 [Type: _typeobject *]
0:005> dx -r1 (*((python27!_typeobject *)0x3ae6378))
(*((python27!_typeobject *)0x3ae6378))                 [Type: _typeobject]
    [+0x000] ob_refcnt        : 2007606976 [Type: __int64]
    [+0x008] ob_type          : 0x3a28588 [Type: _typeobject *]
    [+0x010] ob_size          : 1 [Type: __int64]
    [+0x018] tp_name          : 0x1 : "--- memory read error at address 0x00000000`00000001 ---" [Type: char *]
    [+0x020] tp_basicsize     : 2007389472 [Type: __int64]
    [+0x028] tp_itemsize      : 1 [Type: __int64]
    [+0x030] tp_dealloc       : 0x2f85b60 [Type: void (__cdecl*)(_object *)]
    [+0x038] tp_print         : 0x3b03250 [Type: int (__cdecl*)(_object *,_iobuf *,int)]
    [+0x040] tp_getattr       : 0x3aec8b0 [Type: _object * (__cdecl*)(_object *,char *)]
    [+0x048] tp_setattr       : 0x1 [Type: int (__cdecl*)(_object *,char *,_object *)]
    [+0x050] tp_compare       : 0x1 [Type: int (__cdecl*)(_object *,_object *)]
    [+0x058] tp_repr          : 0x37c4fe8 [Type: _object * (__cdecl*)(_object *)]
    [+0x060] tp_as_number     : 0x3b03268 [Type: PyNumberMethods *]
    [+0x068] tp_as_sequence   : 0x0 [Type: PySequenceMethods *]
    [+0x070] tp_as_mapping    : 0x3b9a650 [Type: PyMappingMethods *]
    [+0x078] tp_hash          : 0x3ace060 [Type: long (__cdecl*)(_object *)]
    [+0x080] tp_call          : 0x1 [Type: _object * (__cdecl*)(_object *,_object *,_object *)]
    [+0x088] tp_str           : 0x1 [Type: _object * (__cdecl*)(_object *)]
    [+0x090] tp_getattro      : 0x77a65520 [Type: _object * (__cdecl*)(_object *,_object *)]
    [+0x098] tp_setattro      : 0x1 [Type: int (__cdecl*)(_object *,_object *,_object *)]
    [+0x0a0] tp_as_buffer     : 0x3ace078 [Type: PyBufferProcs *]
    [+0x0a8] tp_flags         : 61798576 [Type: long]
    [+0x0b0] tp_doc           : 0x3ab19b0 : " d???" [Type: char *]
    [+0x0b8] tp_traverse      : 0x1 [Type: int (__cdecl*)(_object *,int (__cdecl*)(_object *,void *),void *)]
    [+0x0c0] tp_clear         : 0x1 [Type: int (__cdecl*)(_object *)]
    [+0x0c8] tp_richcompare   : 0x38001f8 [Type: _object * (__cdecl*)(_object *,_object *,int)]
    [+0x0d0] tp_weaklistoffset : 61798600 [Type: __int64]
    [+0x0d8] tp_iter          : 0x0 [Type: _object * (__cdecl*)(_object *)]
    [+0x0e0] tp_iternext      : 0x3aecbe0 [Type: _object * (__cdecl*)(_object *)]
    [+0x0e8] tp_methods       : 0x3ab12f0 [Type: PyMethodDef *]
    [+0x0f0] tp_members       : 0x1 [Type: PyMemberDef *]
    [+0x0f8] tp_getset        : 0x1 [Type: PyGetSetDef *]
    [+0x100] tp_base          : 0x38001f8 [Type: _typeobject *]
    [+0x108] tp_dict          : 0x3aecbf8 [Type: _object *]
    [+0x110] tp_descr_get     : 0x0 [Type: _object * (__cdecl*)(_object *,_object *,_object *)]
    [+0x118] tp_descr_set     : 0x3b03030 [Type: int (__cdecl*)(_object *,_object *,_object *)]
    [+0x120] tp_dictoffset    : 61740688 [Type: __int64]
    [+0x128] tp_init          : 0x1 [Type: int (__cdecl*)(_object *,_object *,_object *)]
    [+0x130] tp_alloc         : 0x1 [Type: _object * (__cdecl*)(_typeobject *,__int64)]
    [+0x138] tp_new           : 0x37c4fe8 [Type: _object * (__cdecl*)(_typeobject *,_object *,_object *)]
    [+0x140] tp_free          : 0x3b03048 [Type: void (__cdecl*)(void *)]
    [+0x148] tp_is_gc         : 0x0 [Type: int (__cdecl*)(_object *)]
    [+0x150] tp_bases         : 0x3af9030 [Type: _object *]
    [+0x158] tp_mro           : 0x3ab3458 [Type: _object *]
    [+0x160] tp_cache         : 0x0 [Type: _object *]
    [+0x168] tp_subclasses    : 0x1 [Type: _object *]
    [+0x170] tp_weaklist      : 0x37ff6e8 [Type: _object *]
    [+0x178] tp_del           : 0x3af9048 [Type: void (__cdecl*)(_object *)]
    [+0x180] tp_version_tag   : 0x0 [Type: unsigned int]

gc log:

gc: collecting generation 0...
gc: objects in each generation: 657 2285 9844
gc: done, 0.0020s elapsed.
gc: collecting generation 0...
gc: objects in each generation: 594 2861 9843
gc: done, 0.0000s elapsed.
gc: collecting generation 0...
gc: objects in each generation: 672 3413 9843
gc: collectable <Callbacks 0000000003A82208>
gc: collectable <dict 0000000003A856A8>
gc: collectable <dict 0000000003A85598>
gc: collectable <cell 0000000003A2EC78>
gc: collectable <cell 0000000003A2EBE8>
gc: collectable <tuple 0000000003A7E1C8>
gc: collectable <function 0000000003A7F6D8>
gc: collectable <instancemethod 0000000003A23090>
gc: collectable <Callbacks 0000000003A822B0>
gc: collectable <dict 0000000003A8CAE8>
gc: collectable <dict 0000000003A8C378>
gc: collectable <cell 0000000003A8F0D8>
gc: collectable <cell 0000000003A8F078>
gc: collectable <tuple 00000000039FD588>
gc: collectable <function 0000000003A7F668>
gc: collectable <instancemethod 00000000032453A8>
gc: collectable <Callbacks 0000000003A82828>
gc: collectable <dict 0000000003A94E18>
gc: collectable <dict 0000000003A94AE8>
gc: collectable <cell 0000000003A8F138>
gc: collectable <cell 0000000003A8F258>
gc: collectable <tuple 0000000003A7E148>
gc: collectable <function 0000000003A7FCF8>
gc: collectable <instancemethod 0000000003A35F78>
gc: collectable <Callbacks 0000000003A827F0>
gc: collectable <dict 0000000003A95598>
gc: collectable <dict 0000000003A95158>
gc: collectable <cell 0000000003A8F588>
gc: collectable <cell 0000000003A8F7C8>
gc: collectable <tuple 00000000039FD3C8>
gc: collectable <function 0000000003A7F978>
gc: collectable <instancemethod 0000000003A93798>
gc: collectable <Callbacks 0000000003A82550>
gc: collectable <dict 0000000003A977B8>
gc: collectable <dict 0000000003A94048>
gc: collectable <cell 0000000003A8FC48>
gc: collectable <cell 0000000003A8FC78>
gc: collectable <tuple 0000000003A981C8>
gc: collectable <function 0000000003A8E048>
gc: collectable <instancemethod 0000000003A35C60>
gc: collectable <Callbacks 0000000003A82630>
gc: collectable <dict 0000000003A9C9D8>
gc: collectable <dict 0000000003A958C8>
gc: collectable <cell 0000000003A9D108>
gc: collectable <cell 0000000003A9D138>
gc: collectable <tuple 0000000003A90A88>
gc: collectable <function 0000000003A8E358>
gc: collectable <instancemethod 0000000003A93FC0>
gc: done, 48 unreachable, 0 uncollectable, 0.0280s elapsed.
gc: collecting generation 0...
gc: objects in each generation: 655 3971 9843
gc: collectable <Callbacks 0000000003A822B0>
gc: collectable <dict 0000000003AA09D8>
gc: collectable <dict 0000000003A95158>
gc: collectable <cell 0000000003A8FC78>
gc: collectable <cell 0000000003A8F0D8>
gc: collectable <tuple 00000000039FD3C8>
gc: collectable <function 0000000003A7F668>
gc: collectable <instancemethod 0000000003A9A8B8>
gc: collectable <Callbacks 0000000003A82A58>
gc: collectable <dict 0000000003AA3488>
gc: collectable <dict 0000000003A85598>
gc: collectable <cell 0000000003A9DC18>
gc: collectable <cell 0000000003A9DC48>
gc: collectable <tuple 0000000003A9FF88>
gc: collectable <function 0000000003A7FCF8>
gc: collectable <instancemethod 0000000003A9A240>
gc: collectable <Callbacks 0000000003A827F0>
gc: collectable <dict 0000000003AA3AE8>
gc: collectable <dict 0000000003A958C8>
gc: collectable <cell 0000000003AA8048>
gc: collectable <cell 0000000003AA8078>
gc: collectable <tuple 0000000003A98748>
gc: collectable <function 0000000003A8EF98>
gc: collectable <instancemethod 0000000003AA20D8>
gc: collectable <Callbacks 0000000003A82DA0>
gc: collectable <dict 0000000003AB2488>
gc: collectable <dict 0000000003AAFAE8>
gc: collectable <cell 0000000003AA8498>
gc: collectable <cell 0000000003AA84F8>
gc: collectable <tuple 0000000003AACEC8>
gc: collectable <function 0000000003AA92E8>
gc: collectable <instancemethod 0000000003AA2DC8>
gc: collectable <Callbacks 0000000003AA7F60>
gc: collectable <dict 0000000003AB2048>
gc: collectable <dict 0000000003AAF378>
gc: collectable <cell 0000000003AA8588>
gc: collectable <cell 0000000003AA8558>
gc: collectable <tuple 0000000003AACD88>
gc: collectable <function 0000000003AA9668>
gc: collectable <instancemethod 0000000003AB4090>
gc: done, 40 unreachable, 0 uncollectable, 0.0280s elapsed.
gc: collecting generation 1...
gc: objects in each generation: 874 4496 9843
gc: collectable <Callbacks 0000000003A82D68>
gc: collectable <dict 0000000003A81158>
gc: collectable <cell 0000000003A9D5B8>
gc: collectable <dict 0000000003AA07B8>
gc: collectable <instancemethod 0000000003A93FC0>
gc: collectable <Callbacks 0000000003AA7B70>
gc: collectable <dict 0000000003AAEE18>
gc: collectable <instancemethod 0000000003AB49D8>
gc: collectable <function 0000000003A7F978>
gc: collectable <dict 0000000003AB6048>
gc: collectable <function 0000000003AA9748>
gc: collectable <tuple 0000000003A90A88>
gc: collectable <tuple 0000000003AAC4C8>
gc: collectable <cell 0000000003A9D678>
gc: collectable <cell 0000000003AA8D08>
gc: collectable <cell 0000000003AA8CD8>
gc: collectable <Callbacks 0000000003A827F0>
gc: collectable <dict 0000000003AB6268>
gc: collectable <dict 0000000003AB2268>
gc: collectable <cell 0000000003A8F0D8>
gc: collectable <cell 0000000003AA8A68>
gc: collectable <tuple 0000000003AACEC8>
gc: collectable <function 0000000003A7FCF8>
gc: collectable <instancemethod 0000000003AA2F78>
gc: collectable <Callbacks 0000000003AA7940>
gc: collectable <dict 0000000003AB69D8>
gc: collectable <dict 0000000003AB6598>
gc: collectable <cell 0000000003AA8DC8>
gc: collectable <cell 0000000003AA8078>
gc: collectable <tuple 0000000003AACC48>
gc: collectable <function 0000000003A8EF98>
gc: collectable <instancemethod 0000000003ACD0D8>
gc: collectable <Callbacks 0000000003AA7780>
gc: collectable <dict 0000000003AD1048>
gc: collectable <dict 0000000003A958C8>
gc: collectable <cell 0000000003ACE168>
gc: collectable <cell 0000000003ACE468>
gc: collectable <tuple 0000000003A90D08>
gc: collectable <function 0000000003AA9F98>
gc: collectable <instancemethod 0000000003ACD750>
gc: collectable <Callbacks 0000000003AA7588>
gc: collectable <dict 0000000003AD16A8>
gc: collectable <dict 0000000003AD1268>
gc: collectable <cell 0000000003ACE888>
gc: collectable <cell 0000000003ACE258>
gc: collectable <tuple 0000000003AA4508>
gc: collectable <function 0000000003AD0588>
gc: collectable <instancemethod 0000000003ACDCA8>
gc: collectable <Callbacks 0000000003AB30F0>
gc: collectable <dict 0000000003AD1E18>
gc: collectable <dict 0000000003AABBF8>
gc: collectable <cell 0000000003ACE588>
gc: collectable <cell 0000000003ACE9D8>
gc: collectable <tuple 0000000003AB12C8>
gc: collectable <function 0000000003AA9F28>
gc: collectable <instancemethod 0000000003ACD948>
gc: collectable <Callbacks 0000000003AA7C18>
gc: collectable <dict 0000000003AD4488>
gc: collectable <dict 0000000003AABE18>
gc: collectable <cell 0000000003ACED08>
gc: collectable <cell 0000000003AD70A8>
gc: collectable <tuple 0000000003AD6108>
gc: collectable <function 0000000003AD0898>
gc: collectable <instancemethod 0000000003AD3798>
gc: done, 64 unreachable, 0 uncollectable, 0.0640s elapsed.
gc: collecting generation 0...
gc: objects in each generation: 839 0 14971
gc: collectable <Callbacks 0000000003AB3908>
gc: collectable <dict 0000000003ADB598>
gc: collectable <dict 0000000003ADB158>
gc: collectable <cell 0000000003AD7978>
gc: collectable <cell 0000000003AD79A8>
gc: collectable <tuple 0000000003ACFDC8>
gc: collectable <function 0000000003A7FCF8>
gc: collectable <instancemethod 0000000003AD9510>
gc: collectable <Callbacks 0000000003AB3940>
gc: collectable <dict 0000000003ADBAE8>
gc: collectable <dict 0000000003AAD378>
gc: collectable <cell 0000000003AD7AF8>
gc: collectable <cell 0000000003AD7D68>
gc: collectable <tuple 0000000003AD6908>
gc: collectable <function 0000000003ADD5F8>
gc: collectable <instancemethod 0000000003AD9630>
gc: collectable <Callbacks 0000000003AB3BA8>
gc: collectable <dict 0000000003AE1268>
gc: collectable <dict 0000000003ADBD08>
gc: collectable <cell 0000000003AD7E28>
gc: collectable <cell 0000000003AD7EB8>
gc: collectable <tuple 0000000003ADF488>
gc: collectable <function 0000000003A7F978>
gc: collectable <instancemethod 0000000003AE3048>
gc: collectable <Callbacks 0000000003AB3F98>
gc: collectable <dict 0000000003AE5268>
gc: collectable <dict 0000000003AE1E18>
gc: collectable <cell 0000000003AE2288>
gc: collectable <cell 0000000003AE22B8>
gc: collectable <tuple 0000000003ACFA88>
gc: collectable <function 0000000003ADD978>
gc: collectable <instancemethod 0000000003AE3510>
gc: done, 32 unreachable, 0 uncollectable, 0.0160s elapsed.
gc: collecting generation 0...
gc: objects in each generation: 824 701 14909
gc: collectable <Callbacks 0000000003AE6908>
gc: collectable <dict 0000000003AF1378>
gc: collectable <dict 0000000003AEFE18>
gc: collectable <cell 0000000003AE2F78>
gc: collectable <cell 0000000003AE2FA8>
gc: collectable <tuple 0000000003ACF588>
gc: collectable <function 0000000003AE9438>
gc: collectable <instancemethod 0000000003AEB360>
gc: collectable <Callbacks 0000000003AE6CF8>
gc: collectable <dict 0000000003AF4488>
gc: collectable <dict 0000000003AF1BF8>
gc: collectable <cell 0000000003AF2078>
gc: collectable <cell 0000000003AF2438>
gc: collectable <tuple 0000000003AF0208>
gc: collectable <function 0000000003AE9518>
gc: collectable <instancemethod 0000000003AEB990>
gc: collectable <Callbacks 0000000003AE6F60>
gc: collectable <dict 0000000003AF77B8>
gc: collectable <dict 0000000003AF4E18>
gc: collectable <cell 0000000003AF2858>
gc: collectable <cell 0000000003AF24C8>
gc: collectable <tuple 0000000003AF0BC8>
gc: collectable <function 0000000003AE9AC8>
gc: collectable <instancemethod 0000000003AEBE58>
gc: collectable <Callbacks 0000000003AF8160>
gc: collectable <dict 0000000003AF96A8>
gc: collectable <dict 0000000003AF9598>
gc: collectable <cell 0000000003AF2558>
gc: collectable <cell 0000000003AF2BE8>
gc: collectable <tuple 0000000003AEA508>
gc: collectable <function 0000000003AE97B8>
gc: collectable <instancemethod 0000000003AEBB88>
gc: done, 32 unreachable, 0 uncollectable, 0.0180s elapsed.
gc: collecting generation 0...
gc: objects in each generation: 1371 916 14774

// Thank you in advance
// Oleksii