Variable arguments (*args, **kwargs): seeking elegance

[Steven D'Aprano]

On Mon, 07 Oct 2013 09:26:51 -0700, John Ladasky wrote:

> Thanks, everyone, for your replies.  Perhaps I have complicated things
> unnecessarily?  I was just trying to do some error-checking on the
> arguments supplied to the class constructor.  Perhaps Python already
> implements automatically what I am trying to accomplish manually?  I'll
> tinker around with some minimal code, try to provoke some errors, and
> see what I get.

It's really hard to make definitive judgements without actually seeing 
your code and understanding your use-case. I can only suggest that, you 
*may* be complicating things unnecessarily. On the other hand, there's 
always the chance that your requirements are sufficiently unusual that 
you have done exactly what needs to be done.

But I suspect even in this case, there may be a more elegant way to solve 
the problem of "I'm finding it to be a bit clunky", to quote your 
original post. Clunky code can sometimes be smoothed out by refactoring 
the complexity by use of decorators. Can you post an example of your code?

One thought -- often, people turn to subclassing as the only tool in 
their toolbox. Have you considered that it may be easier/better to work 
with delegation and composition instead?


> Here is one more detail which may be relevant.  The base class for the
> family of classes I am developing is a numpy.ndarray.  The numpy.ndarray
> is a C extension type (and if I understand correctly, that means it is
> immutable by ordinary Python methods).  Subclassing ndarray can get a
> bit complicated (see
> http://docs.scipy.org/doc/numpy/user/basics.subclassing.html).  The
> ndarray.__init__ method is inaccessible, instead one overrides
> ndarray.__new__.

Don't forget ndarray.__array_finalize__, __array_wrap__ and 
__array_prepare__.

I am not an expert on numpy, but reading that page just makes me think 
they're doing it all wrong, adding far too much complication. (I've 
written code like that myself, but thank goodness I've had the sense to 
throw it away and start again...). I'm trying to give them the benefit of 
the doubt, but I've never liked the amount of DWIM cleverness in numpy, 
and I think they would have been *much* better off having a clean 
separation between the three ways of creating an array:

- the normal Python __new__ and __init__ mechanism

- creating a view into an array

- templating

instead of conflating the three into a single mechanism. I suspect that 
the fundamental confusion comes about because numpy doesn't have a clean 
distinction between views into an array, and actual arrays. Although I 
must admit I've not done more than dip my toe into numpy, so you should 
take my criticisms with a generous pinch of salt.


> Making further subclasses of a subclassed numpy.ndarray, each of which
> may have their own arguments, is what I am trying to accomplish while
> adhering to the "DRY" principle.

The usual way of doing this is to accept only keyword arguments for any 
additional args:


class Base:
    def __new__(cls, doc, grumpy, happy, sleepy, bashful, sneezy, dopey):
        ...

class Subclass(Base):
    def __new__(cls, *args, **kwargs):
        # grab the additional arguments
        sneaky = kwargs.pop('sneaky', True)  # optional
        grabby = kwargs.pop('grabby')  # mandatory
        touchy = kwargs.pop('touchy')
        feely = kwargs.pop('feely')
        instance = super(Subclass, cls).__new__(cls, *args, **kwargs)
        # process additional arguments
        instance.apply_extras(sneaky, grabby, touchy, feely)
        return instance


# In Python 3, I can do this to make it even cleaner:
class Subclass(Base):
    def __new__(cls, *args, sneaky=True, grabby, touchy, feely, **kwargs):
        instance = super(Subclass, cls).__new__(cls, *args, **kwargs)
        # process additional arguments
        instance.apply_extras(sneaky, grabby, touchy, feely)
        return instance



In general, you should aim to use either __new__ or __init__ but not 
both, although that's not a hard law, just a guideline.

Can you adapt this pattern to ndarray?


-- 
Steven