[Python-ideas] Threading hooks and disable gc per thread

[Nick Coghlan]

On Mon, May 23, 2011 at 11:40 PM, Christian Heimes <lists at cheimes.de> wrote:
> Am 15.05.2011 13:13, schrieb Nick Coghlan:
>> Composability through inheritance should also be discussed - the hook
>> invocation should probably walk the MRO so it is easy to create Thread
>> subclasses that include class specific hooks without inadvertently
>> skipping the hooks installed on threading.Thread.
>
> Good idea!
>
> Do you think, it's sufficient to have hook methods like
>
> class Thread:
>    _start_hooks = []
>
>    def on_thread_starting(self):
>        for hook, args, kwargs in self._start_hooks:
>            hook(*args, **kwargs)
>
> ? Subclasses of threading.Thread can easily overwrite the hook method
> and call its parent's on_thread_starting().

I was actually thinking of making life even easier for subclasses:

class Thread:
  start_hooks = []

  @classmethod
  def _on_thread_starting(cls):
    # Hooks in parent classes are called before hooks in child classes
    hook_sources = reversed(cls.__mro__)
    for hook_source in hook_sources:
      # Arguable design decision here: only look at Thread subclasses,
not any mixins
      if not issubclass(hook_source, Thread):
        continue
      hooks = hook_src.__dict__.get("start_hooks", ())
      for hook, args, kwargs in hooks:
        hook(*args, **kwargs)

With the parent method explicitly walking the whole MRO in reverse,
any subclass hooks will naturally be invoked after any parent hooks
without any particular effort on the part of the subclass implementor
- the just need to provide and populate a "start_hooks" attribute.

The alternative would mean that overriding "_start_hooks" in a
subclass would block ready access to the main hooks in Thread.

>>> gc.disable_thread(), gc.enable_thread(), gc.isenabled_thread()
>>> --------------------------------------------------------------
>>
>> The default setting for this should go in the interpreter state object
>> rather than in a static variable (subinterpreters can then inherit the
>> state of their parent interpreter when they are first created).
>>
>> Otherwise sounds reasonable. (+0)
>
> A subinterpreter flag isn't enough. All subinterpreters share a common
> GC list. A gc.collect() inside a subinterpreter run affects the entire
> interpreter and not just the one subinterpreter. I've to think about the
> issue of subinterpreters ...
>
> If I understand the code correctly, gc.get_objects() punches a hole in
> the subinterpreter isolation. It returns all tracked objects of the
> current process -- from all subinterpreters. Is this a design issue? The
> fact isn't mentioned in
> http://docs.python.org/c-api/init.html#bugs-and-caveats.

It's quite possible - there's a reason that heavy use of
subinterpreters has a "this may fail in unexpected ways" rider
attached. Still, this is the kind of thing a PEP will hopefully do a
reasonable job of flushing out and resolving.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia