Dealing with non-callable classmethod objects

[Ian Pilcher]

I am trying to figure out a way to gracefully deal with uncallable
classmethod objects.  The class hierarchy below illustrates the issue.
(Unfortunately, I haven't been able to come up with a shorter example.)


import datetime


class DUID(object):

     _subclasses = {}

     def __init_subclass__(cls, **kwargs):
         super().__init_subclass__(**kwargs)
         cls._subclasses[cls.duid_type] = cls

     def __init__(self, d):
         for attr, factory in self._attrs.items():
             setattr(self, attr, factory(d[attr]))

     @classmethod
     def from_dict(cls, d):
         subcls = cls._subclasses[d['duid_type']]
         return subcls(d)


class DuidLL(DUID):

     @staticmethod
     def _parse_l2addr(addr):
         return bytes.fromhex(addr.replace(':', ''))

     duid_type = 'DUID-LL'
     _attrs = { 'layer2_addr': _parse_l2addr }


class DuidLLT(DuidLL):

     @classmethod
     def _parse_l2addr(cls, addr):
         return super()._parse_l2addr(addr)

     duid_type = 'DUID-LLT'
     _attrs = {
             'layer2_addr': _parse_l2addr,
             'time': datetime.datetime.fromisoformat
         }


A bit of context on why I want to do this ...

This is a simplified subset of a larger body of code that parses a
somewhat complex configuration.  The configuration is a YAML document,
that pyyaml parses into a dictionary (which contains other dictionaries,
lists, etc., etc.).  My code then parses that dictionary into an object
graph that represents the configuration.

Rather than embedding parsing logic into each of my object classes, I
have "lifted" it into the parent class (DUID in the example).  A
subclasses need only provide a few attributes that identifies its
required and optional attributes, default values, etc. (simplified to
DuidLL._attrs and DuidLLT._attrs in the example).

The parent class factory function (DUID.from_dict) uses the information
in the subclass's _attrs attribute to control how it parses the
configuration dictionary.  Importantly, a subclass's _attrs attribute
maps attribute names to "factories" that are used to parse the values
into various types of objects.

Thus, DuidLL's 'layer2_addr' attribute is parsed with its
_parse_l2addr() static method, and DuidLLT's 'time' attribute is parsed
with datetime.datetime.fromisoformat().  A factory can be any callable
object that takes a dictionary as its only argument.

This works with static methods (as well as normal functions and object
types that have an appropriate constructor):

>>> duid_ll = DUID.from_dict({ 'duid_type': 'DUID-LL', 'layer2_addr': 'de:ad:be:ef:00:00' })
>>> type(duid_ll)
<class '__main__.DuidLL'>
>>> duid_ll.duid_type
'DUID-LL'
>>> duid_ll.layer2_addr
b'\xde\xad\xbe\xef\x00\x00'

It doesn't work with a class method, such as DuidLLT._parse_l2addr():

>>> duid_llt = DUID.from_dict({ 'duid_type': 'DUID-LLT', 'layer2_addr': 'de:ad:be:ef:00:00', 'time': '2015-09-04T07:53:04-05:00' })
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/home/pilcher/subservient/wtf/wtf.py", line 19, in from_dict
     return subcls(d)
   File "/home/pilcher/subservient/wtf/wtf.py", line 14, in __init__
     setattr(self, attr, factory(d[attr]))
TypeError: 'classmethod' object is not callable

In searching, I've found a few articles that discuss the fact that
classmethod objects aren't callable, but the situation actually seems to
be more complicated.

 >>> type(DuidLLT._parse_l2addr)
<class 'method'>
 >>> callable(DuidLLT._parse_l2addr)
True

The method itself is callable, which makes sense.  The factory function
doesn't access it directly, however, it gets it out of the _attrs
dictionary.

 >>> type(DuidLLT._attrs['layer2_addr'])
<class 'classmethod'>
 >>> callable(DuidLLT._attrs['layer2_addr'])
False

I'm not 100% sure, but I believe that this is happening because the
class (DuidLLT) doesn't exist at the time that its _attrs dictionary is
defined.  Thus, there is no class to which the method can be bound at
that time and the dictionary ends up containing the "unbound version."

Fortunately, I do know the class in the context from which I actually
need to call the method, so I am able to call it with its __func__
attribute.  A modified version of DUID.__init__() appears to work:

     def __init__(self, d):
         for attr, factory in self._attrs.items():
             if callable(factory):  # <============= ???!
                 value = factory(d[attr])
             else:
                 value = factory.__func__(type(self), d[attr])
             setattr(self, attr, value)

A couple of questions (finally!):

* Is my analysis of why this is happening correct?

* Can I improve the 'if callable(factory):' test above?  This treats
   all non-callable objects as classmethods, which is obviously not
   correct.  Ideally, I would check specifically for a classmethod, but
   there doesn't seem to be any literal against which I could check the
   factory's type.

Note:  I am aware that there are any number of workarounds for this
issue.  I just want to make sure that I understand what is going on, and
determine if there's a better way to test for a classmethod object.

Thanks!

-- 
========================================================================
Google                                      Where SkyNet meets Idiocracy
========================================================================