[Python-ideas] Operator as first class citizens -- like in scala -- or yet another new operator?

[Yanghao Hua]

On Mon, Jun 3, 2019 at 8:01 PM Andrew Barnert <abarnert at yahoo.com> wrote:
>
> I think it was a mistake to even mention assignment here (much less arbitrary operators). Most of the resistance you’re facing is because of that, and I think you’re missing a fundamental reason behind that resistance.
>
> Overloading variable assignment makes no sense in Python. In a language with lvalue variable semantics like Scala or C++, a variable is a thing with type, address, and identity, and values live in variables. So assignment affects a variable, and variables have types that can overload how that works. In a language with name-binding variable semantics like Python, it’s the value that has type, address, and identity, and it lives somewhere unspecified on the heap; a variable is nothing more than a name that can be bound to a value in a namespace. So assignment doesn’t affect variables, and, even if it did, they don’t have types that could overload how that works. Overloading assignment isn’t impossible just because, it’s impossible because it makes no sense.
>
> But the fact that in Scala, and Verilog, you send values to signals with an assignment-like operator on an lvalue for the signal doesn’t mean it actually is assignment. What you’re doing is an operation on the signal object itself, not on the signal variable. So neither = nor := would have been a good fit in the first place, and the fact that the variable is just a name rather than an lvalue isn’t a limitation at all, and there’s really no good reason not to have a “send a value to a signal” operator other than the (already very high) bar to new operators.

Acknowledged. I do love the dynamic nature of Python, and as many of
you could have experienced that you might be developing a type system
sooner or later. And newer python with type hint makes it even easier
to do that. That being said, I fully agree that we should not mess up
with the = or := assignment operators, that was taken as an example.
And I fully agree with you I probably should not even use the term
"assignment operators", what I needed is an operator that does not
collide with all existing number/matrix operators.

> If you just explained the operation without reference to assignment, then the questions would be about why << isn’t a good enough fit, how widely the new operator would be used, whether there might be uses in other domains (Erlang-style channels?), etc. At that point, PEP 465 is a much better parallel.

Because << is really one of the operator I need for signals ... it is
even very straight forward to be translated into actual hardware
implementation. I looked at every single operator I could have re-used
... the most promising one being @= operation, but then I realized I
really want my signal matrix to use that operator for the sake of
readability, it is really far more beautiful to read.

> As a side note, if you haven’t already looked at other expression tree libraries like SymPy and SQLAnywhere and their parallels in languages like Scala and C++, it’s worth doing so. For many things (like most desirable uses of SymPy) the lack of variable assignment is no problem; for a few things (like implicit futures, or replacing namespaces with dataflow), it’s an insurmountable problem. I think your problem is more like the former.

I do have experience with ORMs (Object-Relational-Mapper) like in
Django and others, and just looked at SymPy and it is a great project.
And indeed I agree with you that this problem is actually easily
fixable ... I can instead use: signal.next = something, or use
signal.assign(something), which is doable and many people are doing
it, and you have many different ways of doing so. And then the choice
of "signal.next/assign" becomes arbitrary ... e.g. you can use
signal._next, signal._assign so on and so forth. But all those for me
are actually implementation details and it is killing me that there is
no way to hide it. Every single other major HDLs in the world uses an
operator to do so, (vhdl/verilog <=, chisel :=). And this is not
actually forcing a HDL way of doing things on Python, it actually
represented a common category of problems in Python: there is no
operator support to mean: drop a value on me (where "me" is already
defined somewhere else, it has a type and identity).

In HDL implementation alone I can see already two significant use cases:
* signal assignment: signal <== signal << 2 (vs. signal.next = signal
<< 2 or signal.assign(signal << 2)), and it allows too something like:
signal_a <== signal_b <== signal_c (vs signal_a.next = signal_b,
signal_b.next = signal_c).
* module integration/connection and pipe line construction: module_a
<== module_b <== module_c (vs. module_a.connect(module_b);
module_b.connect(module_c))

And yet again let me acknowledge: I understood all these problem can
be solved by using a function call instead, but it is not as pretty as
an operator. This is the whole point of PEP 465 ... make it looks
good, as readability matters. So my question: does this looking good
to python community? (You don't care much on hardware design is
probably another thing), I think it looks much better, and around <==
a well defined concept can be established in a particular
domain/project.

> Also, I can imagine a reason why << isn’t acceptable: it’s probably not that uncommon to need actual bitshift operators. But if you could build something that uses << anyway, and works perfectly in today’s Python for examples that don’t need shift, and then show that off—and show an example where not having bitshift is a problem—you’d be in an even better position. (See the PEP 465 section on numerical libraries that used * for matmul. They clearly work, and are useful for many things, but not having * for elementwise multiplication significantly limits a wide range of useful programs, which is why @ was needed for numpy.)

Yep, that is my plan too. But instead of using <<, I have already
modified CPython anyway to support <== and ==> operators and release a
fully open sourced, powered-by-python HDL simulator with all major
features you'd expect in hardware design (including dumping waveforms
to VCD files, automatically building the hierarchy of module instance
and signal instance names etc.). Code is already working but I want to
make sure a few corner cases are well covered and it will be released
alone with a PEP.

I understand that after the assignment expression (:=) PEP572 people
are already not happy with anything that could be slightly related to
assignment overloading, I actually don't understand why people object
PEP572 so hard, its practical use is clear and it makes code shorter
and easier to read. Sometimes it seems the obsession of people in one
idea makes them to even deny the existence of problems ... problems
that having been solved well in other programming languages, and is
well understood by majority of the programmers.