[Python-ideas] Adding "Typed" collections/iterators to Python

[Nick Coghlan]

On Tue, Dec 20, 2011 at 12:30 AM, Nathan Rice
<nathan.alexander.rice at gmail.com> wrote:
> 1. The "broadcasting" that people seemed to have latched on to is only
> part of what I put forward, and I agree it is something that would
> have to be done *correctly* to be beneficial.  I would have no issues
> with providing a userspace lib to do this if type decorations were
> included in homogeneous collections/iterables, as long as the
> implementation of the decoration didn't suffer from some form of
> "string failure" (string subclasses are basically worthless as methods
> return strings, not an instance of the class).

I deliberately ignored everything else, because the broadcasting
aspect is the only part that doesn't reek of "Python, why u no use
static typing?" and hence the only part I find particularly
interesting.

If you reframe the typing ideas in terms of Abstract Base Classes and
generation of typed proxies, the rest of it may become interesting.

> 4. I wanted to gauge people's feelings before I went through the steps
> involved in writing a PEP.  I believe that is the right thing to do,
> so I don't feel the "hand waving" comment is warranted.  I've already
> learned people view collections that provide child object methods in
> vector form as a very big change even if it is backwards compatible;
> that is fine, I'm willing to shelve that if consensus is that people
> aren't comfortable with it.

The "Hand waving" comment is absolutely warranted, because without
code, we don't really know what you mean.

> This may be the case, thus my request for input.  I agree that a
> thoughtful approach is prudent, however as I have stated, as it
> currently stands the language would not be very supportive of an
> add-on module that does this.  With PyPy, I could probably get my
> hooks in deeply enough that I could make something really useful,
> however given CPython is the current flavor du jour, I doubt it would
> get much traction.  Kind of hard to sell someone on syntactic sugar
> when they have to constantly wrap return values.

Everything the numpy folks have done, they have done with the type
system as it currently stands. It would be perfectly possible to
create proxies for objects that implement the container ABCs that
provide the type constraints that you describe, all without touching
the language core or standard library.

If you take full advantage of the power that descriptors and
metaclasses offer, then you can do a *lot* with this concept as a PyPI
module. As a specific suggestion, I'd advise exploring a
"container.broadcast" descriptor on a mixin type that let you do
things like:

L2 = L1.broadcast(X, *args, **kwds)
L3 = L2.broadcast(Y, *args, **kwds)

L2 = L1.broadcast(X).broadcast(Y).broadcast(Z)
L2 = L1.broadcast_chain(X)(Y)(Z).apply()

L2 = L1.broadcast.X().broadcast.Y().broadcast.Z()
L2 = L1.broadcast_chain.X().Y().Z().apply()

my_string.split("\n").broadcast.capitalize().join_items("\n")

> I agree that changes to syntax and commonly used modules that impact
> how people interface with them should be carefully vetted.  Type
> decorations on homogeneous collections/iterators are effectively
> invisible in that perspective though; the main problem with them as I
> see it is that it involves touching a lot of code to implement, even
> if the actual implementation would be simple.

No, they're not invisible at all - they're a massive conceptual
addition, on a similar scale to Abstract Base Classes themselves.
Syntax matters for readability purposes, but it's the *semantics* that
is at issue here (and also in many other PEPs).

We're talking years of exploration and debate here, not something that
can be resolved in a few weeks or months. There's zero chance of
anything like this making it into 3.3., so think 3.4 at the earliest,
and more likely 3.5 (and that's only if coded explorations of the
concept prove useful in practice - it's entirely possible that the
real outcome will be "never").

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia