From storchaka at gmail.com  Sat Jun  1 03:06:58 2019
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Sat, 1 Jun 2019 10:06:58 +0300
Subject: [Python-ideas] Implement POSIX ln via shutil.link and
 shutil.symlink
In-Reply-To: <00B06EA5-6CF4-4DB8-99C1-127408EAAED3@barrys-emacs.org>
References: <a12f1d2c-96df-4127-fead-f5f1500226d9@hale.ee>
 <91d164e1-0cd4-3908-ab3e-b285939a81ec@hale.ee>
 <1f2cd4a7-0e2b-d346-4089-aec79e6045c1@hale.ee>
 <16b0421e290.2795.1243720eb903aaa2f74f5d2a8df9f525@hale.ee>
 <20190529154804.GN4221@ando.pearwood.info>
 <00B06EA5-6CF4-4DB8-99C1-127408EAAED3@barrys-emacs.org>
Message-ID: <qct86j$41or$1@blaine.gmane.org>

30.05.19 00:22, Barry ????:
> Serhiy, I think, is conflating two things.
> 1. How to write software robust aginst attack.
> 2. How to replace a symlink atomically.

Why do you need to replace a symlink atomically? This is a solution, 
what problem it solves?


From greg.ewing at canterbury.ac.nz  Sat Jun  1 03:29:34 2019
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 01 Jun 2019 19:29:34 +1200
Subject: [Python-ideas] Implement POSIX ln via shutil.link and
 shutil.symlink
In-Reply-To: <20190529154804.GN4221@ando.pearwood.info>
References: <a12f1d2c-96df-4127-fead-f5f1500226d9@hale.ee>
 <91d164e1-0cd4-3908-ab3e-b285939a81ec@hale.ee>
 <1f2cd4a7-0e2b-d346-4089-aec79e6045c1@hale.ee>
 <16b0421e290.2795.1243720eb903aaa2f74f5d2a8df9f525@hale.ee>
 <20190529154804.GN4221@ando.pearwood.info>
Message-ID: <5CF2295E.1010204@canterbury.ac.nz>

> On Wed, May 29, 2019 at 10:07:38PM +0700, Tom Hale wrote:
> 
>     If
>     somebody can create a file named link_name between unlink and 
>     symlink, he can also remove and create a file named link_name after 
>     symlink.

I tbink there are some corner cases that can give different results
if the symlink is not atomic.

For example:

Process A wants to symlink f1 --> f2, replacing any existing f1.

Process B wants to create f1 if it doesn't already exist, or update
it if it does. If f1 is a symlink, the file it's linked to should
be updated.

The end result should be that f1 exists and is a symlink to f2.

If the symlink is not atomic, this can happen:

1. Process A sees that f1 already exists and deletes it.
2. Process B sees that f1 does not exist and creates a new file
    called f1.
3. Process A tries to symlink f1 to f2, which fails because there
    is now an existing file called f1.

This violates the postcondition, because f1 is not a symlink
to f2.

-- 
Greg


From yanghao.py at gmail.com  Sat Jun  1 14:36:00 2019
From: yanghao.py at gmail.com (Yanghao Hua)
Date: Sat, 1 Jun 2019 20:36:00 +0200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAM+UkHWwvtFY2viBs8CRZQMHMoNx971+U057_sDCNF4eTw7NRw@mail.gmail.com>
 <20190523235937.GQ4221@ando.pearwood.info>
 <CAM+UkHUR5RWnj=NY=Wc6hwcVjSHRD_iPmhUMDLbFvuH1gpa96w@mail.gmail.com>
 <5CE7C780.7090803@canterbury.ac.nz>
 <CAM+UkHU7eyE1-1ix_qS-t1ZH62F5yeAebuLhq4voG5y3aXBsPw@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
Message-ID: <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>

On Fri, May 31, 2019 at 3:48 PM Rhodri James <rhodri at kynesim.co.uk> wrote:
> It really doesn't.  If the end user is going to implement the logic of
> this anyway, implementing signal linkage as a method call or class all
> of its own is not a significant extra burden.

There are very constructive discussions in terms how this could be
handled and which turns out not really elegant. Your justification
could be used to reject features like descriptors, decorators, meta
classes, and you can use that reasoning even just to say "let's all go
java". This is not the pythonic for me at all. the burden, is not only
on the savings of a few chars, it is on how easy/difficult it can be
interpreted and understood.

> I'm pretty much done with this conversation too.  You have repeatedly
> been asked what problem you are trying to solve and repeatedly respond
> by restating your solution, which appears to be to impose HDL sematics
> onto a non-HDL language.  That's never going to be a good idea.

Please let's be constructive and be specific, this kind of conclusion
without any reasoning behind it not only makes it weak, this is not
the healthy type of discussion at all.

From tom at hale.ee  Sun Jun  2 09:08:33 2019
From: tom at hale.ee (Tom Hale)
Date: Sun, 2 Jun 2019 20:08:33 +0700
Subject: [Python-ideas] Implement POSIX ln via shutil.link and
 shutil.symlink
In-Reply-To: <qct86j$41or$1@blaine.gmane.org>
References: <a12f1d2c-96df-4127-fead-f5f1500226d9@hale.ee>
 <91d164e1-0cd4-3908-ab3e-b285939a81ec@hale.ee>
 <1f2cd4a7-0e2b-d346-4089-aec79e6045c1@hale.ee>
 <16b0421e290.2795.1243720eb903aaa2f74f5d2a8df9f525@hale.ee>
 <20190529154804.GN4221@ando.pearwood.info>
 <00B06EA5-6CF4-4DB8-99C1-127408EAAED3@barrys-emacs.org>
 <qct86j$41or$1@blaine.gmane.org>
Message-ID: <4f72c7d8-6d24-69b0-651e-cc81fa00198a@hale.ee>

On 1/6/19 2:06 pm, Serhiy Storchaka wrote:
> 30.05.19 00:22, Barry ????:
>> Serhiy, I think, is conflating two things.
>> 1. How to write software robust aginst attack.
>> 2. How to replace a symlink atomically.
> 
> Why do you need to replace a symlink atomically? This is a solution, 
> what problem it solves?

Note that the issue is not limited to symlinks, the analogous problem 
occurs with hard links.

Atomicity is generally considered to be a Good Thing. Atomicity 
eliminates race conditions.

Race conditions are sometimes difficult to enumerate, but here are three 
  I can think of which would not occur given an atomic replace:

1) Unnecessarily Inconsistent Post-conditions
2) Where the link must always exist
3) Unhandled exception attacks

1) Unnecessarily Inconsistent Post-conditions
==============================================

Atomicity ensures that an operation has a consistent post-condition 
where that is possible. Greg did a great job of explaining this:

On 1/6/19 2:29 pm, Greg Ewing wrote:
 > Process A wants to symlink f1 --> f2, replacing any existing f1.
 >
 > Process B wants to create f1 if it doesn't already exist, or update
 > it if it does. If f1 is a symlink, the file it's linked to should
 > be updated.
 >
 > The end result should be that f1 exists and is a symlink to f2.
 >
 > If the symlink is not atomic, this can happen:
 >
 > 1. Process A sees that f1 already exists and deletes it.
 > 2. Process B sees that f1 does not exist and creates a new file
 >     called f1.
 > 3. Process A tries to symlink f1 to f2, which fails because there
 >     is now an existing file called f1.
 >
 > This violates the postcondition, because f1 is not a symlink
 > to f2.


2) Where the link must always exist
====================================

This is an example of atomicity ensuring that a condition remains valid 
*during* the operation.

I gave an example of this in my initial post:

On 13/5/19 4:38 pm, Tom Hale wrote:
 > It would be tempting to do:
 >
 > while True:
 >      try:
 >          os.symlink(target, link_name)
 >          break
 >      except FileExistsError:
 >          os.remove(link_name)
 >
 > But this has a race condition when replacing a symlink should should
 > *always* exist, eg:
 >
 >      /lib/critical.so -> /lib/critical.so.1.2
 >
 > When upgrading by:
 >
 >      symlink('/lib/critical.so.2.0', '/lib/critical.so')
 >
 > There is a point in time when /lib/critical.so doesn't exist.

The way I know to ensure that the well-known symlink exists at all times 
is to replace it.


3) Unhandled exception attacks
===============================

Most people replacing a link or symlink will naively just unlink the 
existing link if it exists, then create the replacement. Most people 
won't think that an exception could occur if somehow the destination is 
recreated after unlink and before link.

Possible DoS:

Someone who doesn't have permissions to kill a process, but has write 
access to a link or symlink could infinite-loop trying to create a new 
file at the location of the (sym)link. While this would be high CPU 
load, if there was a signal that a link replacement was imminent (eg the 
creation of a lockfile) this would be possible to hide. Alternatively, 
if the link updates were known to occur within a very small time window 
(eg cron job), this attack also could be feasible.


Other objections
=================

On 16/5/19 5:05 pm, Serhiy Storchaka wrote:
 >
 > Somebody can replace tmp_symlink between os.symlink() and os.rename().

I raised this in my initial post:

On 13/5/19 4:38 pm, Tom Hale wrote:
 > One issue I see with my suggested code is that the file at
 > temp_link_name could be changed before target is replaced with it.
 > This is mitigated by the randomness introduced by mktemp().
 >
 > While it is far less likely that a file is accessed with a random and
 > unknown name than with an existing known name, I seek input on a
 > solution if this is an unacceptable risk.

My solution reduces risk greatly. I am still open to suggestions to 
totally eliminate it.


Wrap-up
========

It's easier to program when one doesn't constantly need to be aware of 
edge and corner cases, instead having them handled by the standard 
library.  POSIX dictates a certain behaviour for link() and symlink(), 
but we have the opportunity to make life easier for programmers via shutil.

Atomicity is a venerable problem in computing science. We have a 
solution which massively reduces the risk of race conditions.


The larger picture
===================

Atomicity is only ONE point proposed in this post:

https://code.activestate.com/lists/python-ideas/56054/

The linked posts steps back and proposes to support POSIX's ln 
interface: ie, allowing multiple links to be created in a directory with 
a single invocation. Atomicity is optional in this discussion, but so 
far, there has been no discussion.

In that post I also propose documentation updates (also no response).


Concerning the bug report
==========================
Please note that I have changed the bug:
https://bugs.python.org/issue36656

To no longer refer be classed as "security".  If I could change it to 
reference shutil rather than os, I would.

-- 

Regards,

Tom Hale

From dhilliercode at gmail.com  Mon Jun  3 04:22:33 2019
From: dhilliercode at gmail.com (Daniel Hillier)
Date: Mon, 3 Jun 2019 18:22:33 +1000
Subject: [Python-ideas] zipfile refactor and AES
Message-ID: <CAB6C0nnuBZpqDRrBE4oTEONfAX_fGALZUrQv6CEGb6NiOJgUeQ@mail.gmail.com>

Hi,

I've written a package that can read and write zip files encrypted with
Winzip's AES encryption scheme (https://github.com/danifus/pyzipper/). It is
based on Python's zipfile module which I refactored to facilitate adding the
AES code as subclasses of the classes defined in the zipfile module.

I would like to explore integrating some of the refactoring effort into Python
if they are wanted. While the AES specfic code probably isn't suitable for
integration due to its dependence on a crypto package, my hope is that the
refactor of the zipfile project may be beneficial and, in particular, help
development on other features of the zip spec. I'm happy to rework the
changes for inclusion in Python.

The general goals of the refactor were:
- Keep the AES implementation changes in a separate file so changes to
  zipfile.py could potentially be merged into Python (the cryptically named
  "minimal_pep8" branch contains minimal cosmetic changes to the code
  unless the lines around it were changed. The master branch contains all the
  changes in the minimal_pep8 plus pep8 and a few other changes).
- Add hooks for extending the way zipfile works to enable the addition of AES
  encryption without having to duplicate most of the zipfile module. This
  included adding hooks to:
    - Select and call encrypt and decrypt methods.
    - Read and write new "extra" data records in the central file directory
      and local header records.
    - Provide a mechanism to substitute ZipInfo, ZipExtFile and ZipWriteFile
      classes used in a subclass of ZipFile to ease use of subclassed ZipInfo,
      ZipExtFile or ZipWriteFile. This avoids having to rewrite large parts of
      the zipfile module if we only want to change the behaviour of a small
      part of one of those classes.
- Contain all code that reads the header, contents and tail of a file in the
  archive to within ZipExtFile. Previously reading the header and some other
  things were done in the ZipFile class before handing the rest of the
  processing to ZipExtFile.
- Contain all code that writes the header, contents and tail of a file in the
  archive to within ZipWriteFile. Previously reading the header and some other
  things were done in the ZipFile class before handing the rest of the
  processing to ZipExtFile.
- Move generation of local file header and central directory record content to
  the ZipInfo class to be alongside the data that it is packing.
- Add comments to provide context from the zip spec. Replace explicit numbers
  to variables with explanatory names or adding comments.

Concerns:
- The change set is not small. I've attempted to keep each commit in the
  minimal_pep8 branch focused on a single change to simplify review and kept
  the unrelated changes, like pep8, to a minimum. I'm happy to put in
the work to
  get these changes into a patch for the Python project, if it is deemed useful.
- This could move the internals of zipfile towards a public API (maybe it
  would be?) which brings additional complexity in managing future changes.
- Are the hooks inline with Python's coding style? Is there a different
  approach to extensibility preferred within the Python project?

Let me know any improvements, suggestions or concerns to this refactoring
approach you may have. Happy for feedback even if it isn't about integrating the
code into Python.

Thanks,
Dan

From robertc at robertcollins.net  Mon Jun  3 16:12:25 2019
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 4 Jun 2019 08:12:25 +1200
Subject: [Python-ideas] zipfile refactor and AES
In-Reply-To: <CAB6C0nnuBZpqDRrBE4oTEONfAX_fGALZUrQv6CEGb6NiOJgUeQ@mail.gmail.com>
References: <CAB6C0nnuBZpqDRrBE4oTEONfAX_fGALZUrQv6CEGb6NiOJgUeQ@mail.gmail.com>
Message-ID: <CAJ3HoZ2hFBxPFMOkJVLnUFWfdEHhsQXy7LhmwhTJh+RfGUS45w@mail.gmail.com>

This sounds like a valuable refactoring to me.

Is it API compatible with the current zipfile module docs?

On Mon, 3 Jun 2019, 20:23 Daniel Hillier, <dhilliercode at gmail.com> wrote:

> Hi,
>
> I've written a package that can read and write zip files encrypted with
> Winzip's AES encryption scheme (https://github.com/danifus/pyzipper/). It
> is
> based on Python's zipfile module which I refactored to facilitate adding
> the
> AES code as subclasses of the classes defined in the zipfile module.
>
> I would like to explore integrating some of the refactoring effort into
> Python
> if they are wanted. While the AES specfic code probably isn't suitable for
> integration due to its dependence on a crypto package, my hope is that the
> refactor of the zipfile project may be beneficial and, in particular, help
> development on other features of the zip spec. I'm happy to rework the
> changes for inclusion in Python.
>
> The general goals of the refactor were:
> - Keep the AES implementation changes in a separate file so changes to
>   zipfile.py could potentially be merged into Python (the cryptically named
>   "minimal_pep8" branch contains minimal cosmetic changes to the code
>   unless the lines around it were changed. The master branch contains all
> the
>   changes in the minimal_pep8 plus pep8 and a few other changes).
> - Add hooks for extending the way zipfile works to enable the addition of
> AES
>   encryption without having to duplicate most of the zipfile module. This
>   included adding hooks to:
>     - Select and call encrypt and decrypt methods.
>     - Read and write new "extra" data records in the central file directory
>       and local header records.
>     - Provide a mechanism to substitute ZipInfo, ZipExtFile and
> ZipWriteFile
>       classes used in a subclass of ZipFile to ease use of subclassed
> ZipInfo,
>       ZipExtFile or ZipWriteFile. This avoids having to rewrite large
> parts of
>       the zipfile module if we only want to change the behaviour of a small
>       part of one of those classes.
> - Contain all code that reads the header, contents and tail of a file in
> the
>   archive to within ZipExtFile. Previously reading the header and some
> other
>   things were done in the ZipFile class before handing the rest of the
>   processing to ZipExtFile.
> - Contain all code that writes the header, contents and tail of a file in
> the
>   archive to within ZipWriteFile. Previously reading the header and some
> other
>   things were done in the ZipFile class before handing the rest of the
>   processing to ZipExtFile.
> - Move generation of local file header and central directory record
> content to
>   the ZipInfo class to be alongside the data that it is packing.
> - Add comments to provide context from the zip spec. Replace explicit
> numbers
>   to variables with explanatory names or adding comments.
>
> Concerns:
> - The change set is not small. I've attempted to keep each commit in the
>   minimal_pep8 branch focused on a single change to simplify review and
> kept
>   the unrelated changes, like pep8, to a minimum. I'm happy to put in
> the work to
>   get these changes into a patch for the Python project, if it is deemed
> useful.
> - This could move the internals of zipfile towards a public API (maybe it
>   would be?) which brings additional complexity in managing future changes.
> - Are the hooks inline with Python's coding style? Is there a different
>   approach to extensibility preferred within the Python project?
>
> Let me know any improvements, suggestions or concerns to this refactoring
> approach you may have. Happy for feedback even if it isn't about
> integrating the
> code into Python.
>
> Thanks,
> Dan
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20190604/cf440caa/attachment.html>

From GadgetSteve at live.co.uk  Mon Jun  3 17:24:03 2019
From: GadgetSteve at live.co.uk (Steve Barnes)
Date: Mon, 3 Jun 2019 21:24:03 +0000
Subject: [Python-ideas] zipfile refactor and AES
In-Reply-To: <CAJ3HoZ2hFBxPFMOkJVLnUFWfdEHhsQXy7LhmwhTJh+RfGUS45w@mail.gmail.com>
References: <CAB6C0nnuBZpqDRrBE4oTEONfAX_fGALZUrQv6CEGb6NiOJgUeQ@mail.gmail.com>
 <CAJ3HoZ2hFBxPFMOkJVLnUFWfdEHhsQXy7LhmwhTJh+RfGUS45w@mail.gmail.com>
Message-ID: <VI1PR03MB4479E4D3639D91A64C68068B9B140@VI1PR03MB4479.eurprd03.prod.outlook.com>

One specific pain point with zipfile is that if you zip a directory that contains the target zip file you end up trying to add the target file to itself which leads to a rapidly growing archive.

From: Python-ideas <python-ideas-bounces+gadgetsteve=live.co.uk at python.org> On Behalf Of Robert Collins
Sent: 03 June 2019 21:12
To: Daniel Hillier <dhilliercode at gmail.com>
Cc: Python-Ideas <python-ideas at python.org>
Subject: Re: [Python-ideas] zipfile refactor and AES

This sounds like a valuable refactoring to me.

Is it API compatible with the current zipfile module docs?

On Mon, 3 Jun 2019, 20:23 Daniel Hillier, <dhilliercode at gmail.com<mailto:dhilliercode at gmail.com>> wrote:
Hi,

I've written a package that can read and write zip files encrypted with
Winzip's AES encryption scheme (https://github.com/danifus/pyzipper/). It is
based on Python's zipfile module which I refactored to facilitate adding the
AES code as subclasses of the classes defined in the zipfile module.

I would like to explore integrating some of the refactoring effort into Python
if they are wanted. While the AES specfic code probably isn't suitable for
integration due to its dependence on a crypto package, my hope is that the
refactor of the zipfile project may be beneficial and, in particular, help
development on other features of the zip spec. I'm happy to rework the
changes for inclusion in Python.

The general goals of the refactor were:
- Keep the AES implementation changes in a separate file so changes to
  zipfile.py could potentially be merged into Python (the cryptically named
  "minimal_pep8" branch contains minimal cosmetic changes to the code
  unless the lines around it were changed. The master branch contains all the
  changes in the minimal_pep8 plus pep8 and a few other changes).
- Add hooks for extending the way zipfile works to enable the addition of AES
  encryption without having to duplicate most of the zipfile module. This
  included adding hooks to:
    - Select and call encrypt and decrypt methods.
    - Read and write new "extra" data records in the central file directory
      and local header records.
    - Provide a mechanism to substitute ZipInfo, ZipExtFile and ZipWriteFile
      classes used in a subclass of ZipFile to ease use of subclassed ZipInfo,
      ZipExtFile or ZipWriteFile. This avoids having to rewrite large parts of
      the zipfile module if we only want to change the behaviour of a small
      part of one of those classes.
- Contain all code that reads the header, contents and tail of a file in the
  archive to within ZipExtFile. Previously reading the header and some other
  things were done in the ZipFile class before handing the rest of the
  processing to ZipExtFile.
- Contain all code that writes the header, contents and tail of a file in the
  archive to within ZipWriteFile. Previously reading the header and some other
  things were done in the ZipFile class before handing the rest of the
  processing to ZipExtFile.
- Move generation of local file header and central directory record content to
  the ZipInfo class to be alongside the data that it is packing.
- Add comments to provide context from the zip spec. Replace explicit numbers
  to variables with explanatory names or adding comments.

Concerns:
- The change set is not small. I've attempted to keep each commit in the
  minimal_pep8 branch focused on a single change to simplify review and kept
  the unrelated changes, like pep8, to a minimum. I'm happy to put in
the work to
  get these changes into a patch for the Python project, if it is deemed useful.
- This could move the internals of zipfile towards a public API (maybe it
  would be?) which brings additional complexity in managing future changes.
- Are the hooks inline with Python's coding style? Is there a different
  approach to extensibility preferred within the Python project?

Let me know any improvements, suggestions or concerns to this refactoring
approach you may have. Happy for feedback even if it isn't about integrating the
code into Python.

Thanks,
Dan
_______________________________________________
Python-ideas mailing list
Python-ideas at python.org<mailto:Python-ideas at python.org>
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20190603/f6438178/attachment-0001.html>

From jamtlu at gmail.com  Mon Jun  3 20:57:22 2019
From: jamtlu at gmail.com (James Lu)
Date: Mon, 3 Jun 2019 20:57:22 -0400
Subject: [Python-ideas] `if-unless` expressions in Python
Message-ID: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>

`if-unless` expressions in Python

    if condition1 expr unless condition2

is an expression that roughly reduces to

    expr if condition1 and not condition2 else EMPTY


This definition means that expr is only evaluated if `condition1 and not
condition2` evaluates to true. It also means `not condition2` is only
evaluated if `condition1` is true.

# EMPTY

EMPTY is not actually a real Python value-- it's a value that collapses
into nothing when used inside a statement expression:

    print([
      if False never_called() unless False,
      if False never_called() unless False,
    ]) # => []

    print([
      3,
      if False never_called() unless False,
      if False never_called() unless False,
      2,
      if True 5 unless False,
      4
    ]) # => [3, 2, 5, 4]


EMPTY is neither a constant exposed to the Python runtime nor a symbol.
It's a compiler-internal value.

# Use cases

Assertions.

    assert if condition1 predicate(object) unless condition2

(This would be more readable with assert expressions.)

Macros.

# Equivalent syntax in existing Python

As a statement:

if condition1 and not condition2: predicate(object)

predicate(object) if condition1 and not condition2

# Backward compatibility

The `unless` word is only recognized as special inside `if-unless`
statements. The continued use of the word as a variable name is discouraged.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20190603/8af1f47d/attachment.html>

From python at mrabarnett.plus.com  Mon Jun  3 21:23:23 2019
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 4 Jun 2019 02:23:23 +0100
Subject: [Python-ideas] `if-unless` expressions in Python
In-Reply-To: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
References: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
Message-ID: <bc9866e9-d10c-08e0-aac3-a25f837073a6@mrabarnett.plus.com>

On 2019-06-04 01:57, James Lu wrote:
> `if-unless` expressions in Python
> 
>  ? ? if condition1 expr unless condition2
> 
> is an expression that roughly reduces to
> 
>  ? ? expr if condition1 and not condition2 else EMPTY
> 
> 
> This definition means that expr is only evaluated if `condition1 and not 
> condition2` evaluates to true. It also means `not condition2` is only 
> evaluated if `condition1` is true.
> 
> # EMPTY
> 
> EMPTY is not actually a real Python value-- it's a value that collapses 
> into nothing when used inside a statement expression:
> 
>  ? ? print([
>  ? ? ? if False never_called() unless False,
>  ? ? ? if False never_called() unless False,
>  ? ? ]) # => []
> 
>  ? ? print([
>  ? ? ? 3,
>  ? ? ? if False never_called() unless False,
>  ? ? ? if False never_called() unless False,
>  ? ? ? 2,
>  ? ? ? if True 5 unless False,
>  ? ? ? 4
>  ? ? ]) # => [3, 2, 5, 4]
> 
> 
> EMPTY is neither a constant exposed to the Python runtime nor a symbol. 
> It's a compiler-internal value.
> 
> # Use cases
> 
> Assertions.
> 
>  ? ? assert if condition1 predicate(object) unless condition2
> 
> (This would be more readable with assert expressions.)
> 
> Macros.
> 
> # Equivalent syntax in existing Python
> 
> As a statement:
> 
> if condition1 and not condition2: predicate(object)
> 
> predicate(object) if condition1 and not condition2
> 
> # Backward compatibility
> 
> The `unless` word is only recognized as special inside `if-unless` 
> statements. The continued use of the word as a variable name is discouraged.
> 
-1
I find it very difficult to understand.

For statements we already have the 'if' statement, as above.

Within expressions such as a list comprehension, it would be much 
clearer IMHO to have a ternary 'if' with 'pass':

     expr if condition else pass

No new reserved words needed.

From rosuav at gmail.com  Mon Jun  3 21:24:00 2019
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 4 Jun 2019 11:24:00 +1000
Subject: [Python-ideas] `if-unless` expressions in Python
In-Reply-To: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
References: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
Message-ID: <CAPTjJmoAPF1ruTv_w8yBsNWOzcLdN3sx4kQdHLC6vqz7_0EwmA@mail.gmail.com>

On Tue, Jun 4, 2019 at 10:58 AM James Lu <jamtlu at gmail.com> wrote:
>
> `if-unless` expressions in Python
>
>     if condition1 expr unless condition2
>
> is an expression that roughly reduces to
>
>     expr if condition1 and not condition2 else EMPTY
>
> This definition means that expr is only evaluated if `condition1 and not condition2` evaluates to true. It also means `not condition2` is only evaluated if `condition1` is true.

Why the double condition? The existing definition of "condition1 and
not condition2" already guarantees the short-circuiting, and what
you're effectively creating is two pieces of a single condition.

> # EMPTY
>
> EMPTY is not actually a real Python value-- it's a value that collapses into nothing when used inside a statement expression:
>
>     print([
>       if False never_called() unless False,
>       if False never_called() unless False,
>     ]) # => []
>
>     print([
>       3,
>       if False never_called() unless False,
>       if False never_called() unless False,
>       2,
>       if True 5 unless False,
>       4
>     ]) # => [3, 2, 5, 4]
>
>
> EMPTY is neither a constant exposed to the Python runtime nor a symbol. It's a compiler-internal value.

Okay, now this starts to look somewhat interesting. But what you'd be
creating here would be a feature of specific language constructs, and
probably wouldn't work everywhere.

> # Use cases
>
> Assertions.
>
>     assert if condition1 predicate(object) unless condition2
>
> (This would be more readable with assert expressions.)

Honestly not sure what this does that's better than ordinary
expressions. Can you give some concrete examples of how you would use
this, and what the equivalent code for current Pythons would look
like?

> Macros.
>
> # Equivalent syntax in existing Python
>
> As a statement:
>
> if condition1 and not condition2: predicate(object)
>
> predicate(object) if condition1 and not condition2

Adding "else None" would make this legal Python syntax right now, but
for very good reasons, is not how most people write code.

So, looking just at the "EMPTY" part. Allow me to reword your proposal
into what I think you're saying; is this what you're proposing?

## New "unless" construct for list displays and argument lists ##

Inside a list/dict/set/tuple display, or inside an argument list,
elements can be conditionally omitted by providing a predicate.

lst = [f1(), f3() unless f2(), f4()]

The functions will be called in the order indicated. If f2() returns a
falsey value, f3() will not be evaluated, and the list will have only
two elements in it.

#####

That's without the two-part condition. You can always put "expr unless
x and not y" if you need both parts.

How close is this to what you were thinking of?

ChrisA

From steve at pearwood.info  Mon Jun  3 21:28:27 2019
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Jun 2019 11:28:27 +1000
Subject: [Python-ideas] `if-unless` expressions in Python
In-Reply-To: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
References: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
Message-ID: <20190604012826.GY4221@ando.pearwood.info>

On Mon, Jun 03, 2019 at 08:57:22PM -0400, James Lu wrote:
> `if-unless` expressions in Python
> 
>     if condition1 expr unless condition2
> 
> is an expression that roughly reduces to
> 
>     expr if condition1 and not condition2 else EMPTY

Then the "unless" clause is superfluorous and we can write:

    if condition1 and not condition2 expression

which is another way of saying 

    if condition expression

which has been suggested before in the form:

    expression if condition

and rejected. Please check the archives.


> This definition means that expr is only evaluated if `condition1 and not
> condition2` evaluates to true. It also means `not condition2` is only
> evaluated if `condition1` is true.

Which is precisely how "and" already works.


> # EMPTY
> 
> EMPTY is not actually a real Python value-- it's a value that collapses
> into nothing when used inside a statement expression:
[...]
> EMPTY is neither a constant exposed to the Python runtime nor a symbol.
> It's a compiler-internal value.

I don't even know what you mean by that even after reading your 
examples, sorry.

If I do this:


    var = if False 1 unless False
    print(var)


what happens?


-- 
Steven

From tjreedy at udel.edu  Mon Jun  3 21:54:48 2019
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 3 Jun 2019 21:54:48 -0400
Subject: [Python-ideas] `if-unless` expressions in Python
In-Reply-To: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
References: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
Message-ID: <qd4j19$2qg4$1@blaine.gmane.org>

On 6/3/2019 8:57 PM, James Lu wrote:
> `if-unless` expressions in Python
> 
>  ? ? if condition1 expr unless condition2
> 
> is an expression that roughly reduces to
> 
>  ? ? expr if condition1 and not condition2 else EMPTY

An expression MUST evaluate to a value.  EMPTY is spelled None in 
Python.  No need for an alternate spelling.

The only place that None is implicit rather than explicit is the 
implicit 'return None' *statement* at the end of functions.  That is enough.

-- 
Terry Jan Reedy



From dhilliercode at gmail.com  Tue Jun  4 00:53:07 2019
From: dhilliercode at gmail.com (Daniel Hillier)
Date: Tue, 4 Jun 2019 14:53:07 +1000
Subject: [Python-ideas] zipfile refactor and AES
In-Reply-To: <VI1PR03MB4479E4D3639D91A64C68068B9B140@VI1PR03MB4479.eurprd03.prod.outlook.com>
References: <CAB6C0nnuBZpqDRrBE4oTEONfAX_fGALZUrQv6CEGb6NiOJgUeQ@mail.gmail.com>
 <CAJ3HoZ2hFBxPFMOkJVLnUFWfdEHhsQXy7LhmwhTJh+RfGUS45w@mail.gmail.com>
 <VI1PR03MB4479E4D3639D91A64C68068B9B140@VI1PR03MB4479.eurprd03.prod.outlook.com>
Message-ID: <CAB6C0nkui_W4fWJHf+Q-XD81uNCO28ZzaWKWLAwnJ5vQJoJMiA@mail.gmail.com>

Robert Collins wrote:
> Is it API compatible with the current zipfile module docs?

Yes it should be (that was my intention during the refactor and if I
accidentally introduced an incompatibility, I'll fix that up before I attempt
to integrate the work). I had a look through the docs again and I can't see any
incompatibilities.

The zipfile module's test suite is currently passing against the repo too. I
forked the tests into the repo to use as a basis for this work.


On Tue, Jun 4, 2019 at 7:24 AM Steve Barnes <GadgetSteve at live.co.uk> wrote:
> One specific pain point with zipfile is that if you zip a directory that
> contains the target zip file you end up trying to add the target file to
> itself which leads to a rapidly growing archive.

I think this could be another Python-ideas itself as it could be implemented
without a refactor and there is already a lot going on in the refactor at the
moment.

Cheers,
Dan

From songofacandy at gmail.com  Tue Jun  4 03:25:54 2019
From: songofacandy at gmail.com (Inada Naoki)
Date: Tue, 4 Jun 2019 16:25:54 +0900
Subject: [Python-ideas] Implement POSIX ln via shutil.link and
 shutil.symlink
In-Reply-To: <qct86j$41or$1@blaine.gmane.org>
References: <a12f1d2c-96df-4127-fead-f5f1500226d9@hale.ee>
 <91d164e1-0cd4-3908-ab3e-b285939a81ec@hale.ee>
 <1f2cd4a7-0e2b-d346-4089-aec79e6045c1@hale.ee>
 <16b0421e290.2795.1243720eb903aaa2f74f5d2a8df9f525@hale.ee>
 <20190529154804.GN4221@ando.pearwood.info>
 <00B06EA5-6CF4-4DB8-99C1-127408EAAED3@barrys-emacs.org>
 <qct86j$41or$1@blaine.gmane.org>
Message-ID: <CAEfz+Tw5RRDAkhv3KuK5ApHrAZvGaihA-e8RycLkMGBR=jzv1A@mail.gmail.com>

On Sat, Jun 1, 2019 at 4:10 PM Serhiy Storchaka <storchaka at gmail.com> wrote:
>
> Why do you need to replace a symlink atomically? This is a solution,
> what problem it solves?
>

There is another, more common / realistic usage of atomic symlink
replacing.

When deploy PHP application or static web contents to web servers,
symlink is used for "atomic deployment".  Roughly speaking, it is like this:

```
rsync -avK v2/ web:contents/v2/
ssh web "cd contents && ln -sf v2 current"  # current/ is exposed by web server
```

If "ln -sf" is not atomic and do remove & symlink, web server or php
will return 404 error between remove and symlink.

I feel this use case is more real world application than
"I don't want exception even when other process creates file
at the same time".

These are some links referring about "atomic deployment":

* https://github.com/deployphp/deployer/blob/master/recipe/deploy/symlink.php
* https://hackernoon.com/truly-atomic-deployments-with-nginx-and-php-fpm-aed8a8ac1cd9
* https://blog.forrest79.net/?p=537


Then, should Python support it?  Maybe.

Python is used tools like Ansible.  While "atomic deploy" is widely used
in PHP community, it's welcome that PHPer use Python for tools.

So I think it would be nice if shutil.symlink can do same thing
`ln -sf` can do.

On the other hand, I am not sure we should support "make tmp,
rename it to target, or remove tmp if failed" idiom natively
everywhere in shutil and pathlib.

When writing file or copying file into directly which is exposed by
WEB server, this idiom is to avoid exposing partial files to users.

But implementing this idiom in everywhere in shutil and pathlib
is still controversial.

In case of symlink, I'm weak +1 to implement this idiom directly in stdlib.
tempfile.mkstemp doesn't support symlink.  So people need to
use external command `ln -sf` for now.

Regards,
-- 
Inada Naoki  <songofacandy at gmail.com>

From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Jun  4 04:11:03 2019
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Tue, 4 Jun 2019 17:11:03 +0900
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAM+UkHWwvtFY2viBs8CRZQMHMoNx971+U057_sDCNF4eTw7NRw@mail.gmail.com>
 <20190523235937.GQ4221@ando.pearwood.info>
 <CAM+UkHUR5RWnj=NY=Wc6hwcVjSHRD_iPmhUMDLbFvuH1gpa96w@mail.gmail.com>
 <5CE7C780.7090803@canterbury.ac.nz>
 <CAM+UkHU7eyE1-1ix_qS-t1ZH62F5yeAebuLhq4voG5y3aXBsPw@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
Message-ID: <23798.10135.127055.149191@turnbull.sk.tsukuba.ac.jp>

Yanghao Hua writes:
 > On Fri, May 31, 2019 at 3:48 PM Rhodri James <rhodri at kynesim.co.uk> wrote:

 > > It really doesn't.  If the end user is going to implement the
 > > logic of this anyway, implementing signal linkage as a method
 > > call or class all of its own is not a significant extra burden.

 > There are very constructive discussions in terms how this could be
 > handled and which turns out not really elegant. Your justification
 > could be used to reject features like descriptors, decorators, meta
 > classes,

And has been so used.  The question is always "does the use case
justify increasing complexity for a couple hundred maintainers and a
few hundred million readers?"

As far as I can tell, here we have (a) *one* person *strongly* in
favor of adding just one new operator token, who justifies it based on
*personal* preference over style of expression for *one* specific DSL,
and (b) a few supporting a generic facility for adding new operators.

The former (a) is obviously insufficient, considering past practice in
Python.  One person can probably use MacroPy or a preprocessor.  If
the DSL becomes popular, as with "@", first applied for matrix
multiplication, it might become justification for a new operator
token, but the popularity has to come before more than a tiny fraction
of committers (if any) will sign on.  The other possibility would be
to present a complete implementation of a toy language using existing
Python facilities, and then show the alternative with the new
operator.  Preferably at the same time showing how the preprocessor/
MacroPy strategy falls short.

The latter (b) has been discussed and rejected a number of times, on
the grounds that unfamiliar symbols are *harder* to read than
functions, and that Python is not in the DSL-producing industry.  Some
features are clearly more descriptive than procedural in nature (eg,
comprehensions), but they are justified on the ground that they
increase readability when expressing algorithms.

 > Please let's be constructive and be specific, this kind of
 > conclusion without any reasoning behind it

In Python practice, the conclusion of "reject" needs no reasoning,
unfortunately.  Python adds new features in response to a demonstrated
need that overcomes a general conservatism, a conservatism that
extends far past backward compatibility to a certain amount of forward
compatibility as well.

So you should take your own advice.  We have seen you express your
*preference* for a new operator numerous times, but there is no
*visible* logic behind it.  Show us the best you can do with actual
working hardware description classes (say a NAND gate and the
composition into a half-adder, or perhaps you need more to demonstrate
the relevant difficulties -- I know little about hardware, quite a bit
about how the Python enhancement process works).  Then show us the
syntax you want to use to express the same thing.

It's possible that people with wizard-level Python skills can find a
way around the difficulties you describe with descriptors and other
features that have been suggested in this thread.  If not, your case
for a new operator token would be much stronger (though still
insufficient IMO -- fortunately for you, few committers listen to me
about language features ;-).

Without such a test, or a few tens of thousands of hardware designers
lobbying for a decade or so (which is what happened with the numerical
community, although this time around I bet Python will be
substantially more forward-leaning, say 5 years and 5000 hardware
designers), it's your taste against Python's conservatism.  I'll take
long odds that conservatism wins.

Steve


From steve at pearwood.info  Tue Jun  4 04:32:54 2019
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Jun 2019 18:32:54 +1000
Subject: [Python-ideas] zipfile refactor and AES
In-Reply-To: <VI1PR03MB4479E4D3639D91A64C68068B9B140@VI1PR03MB4479.eurprd03.prod.outlook.com>
References: <CAB6C0nnuBZpqDRrBE4oTEONfAX_fGALZUrQv6CEGb6NiOJgUeQ@mail.gmail.com>
 <CAJ3HoZ2hFBxPFMOkJVLnUFWfdEHhsQXy7LhmwhTJh+RfGUS45w@mail.gmail.com>
 <VI1PR03MB4479E4D3639D91A64C68068B9B140@VI1PR03MB4479.eurprd03.prod.outlook.com>
Message-ID: <20190604083252.GA4221@ando.pearwood.info>

On Mon, Jun 03, 2019 at 09:24:03PM +0000, Steve Barnes wrote:

> One specific pain point with zipfile is that if you zip a directory 
> that contains the target zip file you end up trying to add the target 
> file to itself which leads to a rapidly growing archive.

If that is accurate, it probably should be reported as a bug in zipfile.



-- 
Steven

From yanghao.py at gmail.com  Tue Jun  4 06:06:55 2019
From: yanghao.py at gmail.com (Yanghao Hua)
Date: Tue, 4 Jun 2019 12:06:55 +0200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAM+UkHWwvtFY2viBs8CRZQMHMoNx971+U057_sDCNF4eTw7NRw@mail.gmail.com>
 <20190523235937.GQ4221@ando.pearwood.info>
 <CAM+UkHUR5RWnj=NY=Wc6hwcVjSHRD_iPmhUMDLbFvuH1gpa96w@mail.gmail.com>
 <5CE7C780.7090803@canterbury.ac.nz>
 <CAM+UkHU7eyE1-1ix_qS-t1ZH62F5yeAebuLhq4voG5y3aXBsPw@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
Message-ID: <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>

On Mon, Jun 3, 2019 at 8:01 PM Andrew Barnert <abarnert at yahoo.com> wrote:
>
> I think it was a mistake to even mention assignment here (much less arbitrary operators). Most of the resistance you?re facing is because of that, and I think you?re missing a fundamental reason behind that resistance.
>
> Overloading variable assignment makes no sense in Python. In a language with lvalue variable semantics like Scala or C++, a variable is a thing with type, address, and identity, and values live in variables. So assignment affects a variable, and variables have types that can overload how that works. In a language with name-binding variable semantics like Python, it?s the value that has type, address, and identity, and it lives somewhere unspecified on the heap; a variable is nothing more than a name that can be bound to a value in a namespace. So assignment doesn?t affect variables, and, even if it did, they don?t have types that could overload how that works. Overloading assignment isn?t impossible just because, it?s impossible because it makes no sense.
>
> But the fact that in Scala, and Verilog, you send values to signals with an assignment-like operator on an lvalue for the signal doesn?t mean it actually is assignment. What you?re doing is an operation on the signal object itself, not on the signal variable. So neither = nor := would have been a good fit in the first place, and the fact that the variable is just a name rather than an lvalue isn?t a limitation at all, and there?s really no good reason not to have a ?send a value to a signal? operator other than the (already very high) bar to new operators.

Acknowledged. I do love the dynamic nature of Python, and as many of
you could have experienced that you might be developing a type system
sooner or later. And newer python with type hint makes it even easier
to do that. That being said, I fully agree that we should not mess up
with the = or := assignment operators, that was taken as an example.
And I fully agree with you I probably should not even use the term
"assignment operators", what I needed is an operator that does not
collide with all existing number/matrix operators.

> If you just explained the operation without reference to assignment, then the questions would be about why << isn?t a good enough fit, how widely the new operator would be used, whether there might be uses in other domains (Erlang-style channels?), etc. At that point, PEP 465 is a much better parallel.

Because << is really one of the operator I need for signals ... it is
even very straight forward to be translated into actual hardware
implementation. I looked at every single operator I could have re-used
... the most promising one being @= operation, but then I realized I
really want my signal matrix to use that operator for the sake of
readability, it is really far more beautiful to read.

> As a side note, if you haven?t already looked at other expression tree libraries like SymPy and SQLAnywhere and their parallels in languages like Scala and C++, it?s worth doing so. For many things (like most desirable uses of SymPy) the lack of variable assignment is no problem; for a few things (like implicit futures, or replacing namespaces with dataflow), it?s an insurmountable problem. I think your problem is more like the former.

I do have experience with ORMs (Object-Relational-Mapper) like in
Django and others, and just looked at SymPy and it is a great project.
And indeed I agree with you that this problem is actually easily
fixable ... I can instead use: signal.next = something, or use
signal.assign(something), which is doable and many people are doing
it, and you have many different ways of doing so. And then the choice
of "signal.next/assign" becomes arbitrary ... e.g. you can use
signal._next, signal._assign so on and so forth. But all those for me
are actually implementation details and it is killing me that there is
no way to hide it. Every single other major HDLs in the world uses an
operator to do so, (vhdl/verilog <=, chisel :=). And this is not
actually forcing a HDL way of doing things on Python, it actually
represented a common category of problems in Python: there is no
operator support to mean: drop a value on me (where "me" is already
defined somewhere else, it has a type and identity).

In HDL implementation alone I can see already two significant use cases:
* signal assignment: signal <== signal << 2 (vs. signal.next = signal
<< 2 or signal.assign(signal << 2)), and it allows too something like:
signal_a <== signal_b <== signal_c (vs signal_a.next = signal_b,
signal_b.next = signal_c).
* module integration/connection and pipe line construction: module_a
<== module_b <== module_c (vs. module_a.connect(module_b);
module_b.connect(module_c))

And yet again let me acknowledge: I understood all these problem can
be solved by using a function call instead, but it is not as pretty as
an operator. This is the whole point of PEP 465 ... make it looks
good, as readability matters. So my question: does this looking good
to python community? (You don't care much on hardware design is
probably another thing), I think it looks much better, and around <==
a well defined concept can be established in a particular
domain/project.

> Also, I can imagine a reason why << isn?t acceptable: it?s probably not that uncommon to need actual bitshift operators. But if you could build something that uses << anyway, and works perfectly in today?s Python for examples that don?t need shift, and then show that off?and show an example where not having bitshift is a problem?you?d be in an even better position. (See the PEP 465 section on numerical libraries that used * for matmul. They clearly work, and are useful for many things, but not having * for elementwise multiplication significantly limits a wide range of useful programs, which is why @ was needed for numpy.)

Yep, that is my plan too. But instead of using <<, I have already
modified CPython anyway to support <== and ==> operators and release a
fully open sourced, powered-by-python HDL simulator with all major
features you'd expect in hardware design (including dumping waveforms
to VCD files, automatically building the hierarchy of module instance
and signal instance names etc.). Code is already working but I want to
make sure a few corner cases are well covered and it will be released
alone with a PEP.

I understand that after the assignment expression (:=) PEP572 people
are already not happy with anything that could be slightly related to
assignment overloading, I actually don't understand why people object
PEP572 so hard, its practical use is clear and it makes code shorter
and easier to read. Sometimes it seems the obsession of people in one
idea makes them to even deny the existence of problems ... problems
that having been solved well in other programming languages, and is
well understood by majority of the programmers.

From rosuav at gmail.com  Tue Jun  4 06:17:18 2019
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 4 Jun 2019 20:17:18 +1000
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAM+UkHWwvtFY2viBs8CRZQMHMoNx971+U057_sDCNF4eTw7NRw@mail.gmail.com>
 <20190523235937.GQ4221@ando.pearwood.info>
 <CAM+UkHUR5RWnj=NY=Wc6hwcVjSHRD_iPmhUMDLbFvuH1gpa96w@mail.gmail.com>
 <5CE7C780.7090803@canterbury.ac.nz>
 <CAM+UkHU7eyE1-1ix_qS-t1ZH62F5yeAebuLhq4voG5y3aXBsPw@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
Message-ID: <CAPTjJmoGn7qu1Ov+NJyXGejB5-eXvNm5A5UF5sFNVWVrLXKdAg@mail.gmail.com>

On Tue, Jun 4, 2019 at 8:07 PM Yanghao Hua <yanghao.py at gmail.com> wrote:
> I understand that after the assignment expression (:=) PEP572 people
> are already not happy with anything that could be slightly related to
> assignment overloading, I actually don't understand why people object
> PEP572 so hard, its practical use is clear and it makes code shorter
> and easier to read. Sometimes it seems the obsession of people in one
> idea makes them to even deny the existence of problems ... problems
> that having been solved well in other programming languages, and is
> well understood by majority of the programmers.

Assignment overloading and PEP 572 are completely orthogonal. The :=
operator makes assignment available in an expression context, rather
than only as a statement, but this is nothing to do with allowing the
target operator to redefine assignment.

ChrisA

From J.Demeyer at UGent.be  Tue Jun  4 06:47:30 2019
From: J.Demeyer at UGent.be (Jeroen Demeyer)
Date: Tue, 4 Jun 2019 12:47:30 +0200
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
Message-ID: <5CF64C42.1070609@UGent.be>

I'd like to get rid of all the signal and HDL stuff (whatever that 
means) in this thread, so I think what the original poster really wants 
is an "assign in place" operator. Basically, something like += or *= but 
without the arithmetic.

When you think of it this way, it's not an unreasonable request. There 
would be at least one major use of this operator within CPython, for 
lists. With this proposal, the awkward syntax (there are 219 instances 
of this in the CPython sources)

   L[:] = new_list

would become

   L <== new_list

The implementation would be completely analogous to the existing 
in-place arithmetic operators. For example A <== B would become 
equivalent to A = type(A).__iassign__(A, B).

From steve at pearwood.info  Tue Jun  4 07:29:17 2019
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Jun 2019 21:29:17 +1000
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <5CF64C42.1070609@UGent.be>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <5CF64C42.1070609@UGent.be>
Message-ID: <20190604112917.GC4221@ando.pearwood.info>

On Tue, Jun 04, 2019 at 12:47:30PM +0200, Jeroen Demeyer wrote:

> When you think of it this way, it's not an unreasonable request. There 
> would be at least one major use of this operator within CPython, for 
> lists. With this proposal, the awkward syntax (there are 219 instances 
> of this in the CPython sources)
> 
>   L[:] = new_list

What is so awkward about slice assignment? It is an obvious 
generalisation of item assignment to slices of more than one index, with 
the start and end positions being optional.

If you can use ``L[index] = value`` than slice assignment just follows 
from that.


> would become
> 
>   L <== new_list

Creating new syntax to make it easy to do things which are currently 
impossible or difficult is worth considering; creating new syntax just 
because some people don't like the colour of the bike-shed just creates 
language churn for its own sake.

Introducing <== to give alternate syntax to slice assignment is, I 
think, a non-starter.


> The implementation would be completely analogous to the existing 
> in-place arithmetic operators. For example A <== B would become 
> equivalent to A = type(A).__iassign__(A, B).

As far as I can tell, there is no difference between your proposal 
and the OP's proposal except you have changed the name of the dunder 
from __arrow__ to __iassign__.

__iassign__ is inappropriate because there is no __assign__ dunder:

    x += y __iadd__ is related to x + y __add__
    x -= y __isub__ is related to x - y __sub__
    x *= y __imul__ is related to x * y __mul__
    # etc 
    x <== y __iassign__ is related to x <what?> y __assign__ ?

and it is not a form of *augmented assignment*, it's just a method call.



-- 
Steven

From J.Demeyer at UGent.be  Tue Jun  4 07:45:54 2019
From: J.Demeyer at UGent.be (Jeroen Demeyer)
Date: Tue, 4 Jun 2019 13:45:54 +0200
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <20190604112917.GC4221@ando.pearwood.info>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <5CF64C42.1070609@UGent.be> <20190604112917.GC4221@ando.pearwood.info>
Message-ID: <5CF659F2.5010908@UGent.be>

On 2019-06-04 13:29, Steven D'Aprano wrote:> As far as I can tell, there 
is no difference between your proposal
> and the OP's proposal except you have changed the name of the dunder
> from __arrow__ to __iassign__.

I never claimed that there was a difference. I just tried to clarify 
what the original poster asked and put it in a wider context, because 
the original post was way too much focused on hardware stuff.

From p.f.moore at gmail.com  Tue Jun  4 08:06:56 2019
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 4 Jun 2019 13:06:56 +0100
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <5CF659F2.5010908@UGent.be>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <5CF64C42.1070609@UGent.be> <20190604112917.GC4221@ando.pearwood.info>
 <5CF659F2.5010908@UGent.be>
Message-ID: <CACac1F9H1NR-SFU36vWKR-KGG552AybYAejS3jrrRyR4q0PcDw@mail.gmail.com>

On Tue, 4 Jun 2019 at 12:47, Jeroen Demeyer <J.Demeyer at ugent.be> wrote:
>
> On 2019-06-04 13:29, Steven D'Aprano wrote:> As far as I can tell, there
> is no difference between your proposal
> > and the OP's proposal except you have changed the name of the dunder
> > from __arrow__ to __iassign__.
>
> I never claimed that there was a difference. I just tried to clarify
> what the original poster asked and put it in a wider context, because
> the original post was way too much focused on hardware stuff.

... and I'll confirm that for me at least, rephrasing the request this
way did help me understand the proposal better.

(I'm neutral on whether it's a good idea, though)

Paul

From rhodri at kynesim.co.uk  Tue Jun  4 08:20:14 2019
From: rhodri at kynesim.co.uk (Rhodri James)
Date: Tue, 4 Jun 2019 13:20:14 +0100
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
Message-ID: <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>

On 04/06/2019 11:06, Yanghao Hua wrote:
> [...]  what I needed is an operator that does not
> collide with all existing number/matrix operators.

Why?

That's the question that in all your thousands of words of argument you 
still haven't answered beyond "because I want it."

-- 
Rhodri James *-* Kynesim Ltd

From steve at pearwood.info  Tue Jun  4 08:28:27 2019
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Jun 2019 22:28:27 +1000
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
References: <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
Message-ID: <20190604122826.GD4221@ando.pearwood.info>

On Fri, May 31, 2019 at 02:48:24PM +0100, Rhodri James wrote:
> On 29/05/2019 08:31, Yanghao Hua wrote:

> >Python does not need to know this ... just hand it over to end user
> >who knows how to implement such a thing. Python need to provide the
> >mechanism.
> 
> It really doesn't.  If the end user is going to implement the logic of 
> this anyway, implementing signal linkage as a method call or class all 
> of its own is not a significant extra burden.

That's not really fair: syntax matters, and for people working in a 
domain where certain syntax is expected, asking them to use something 
different is a significant cognitive burden. We don't ask people doing 
arithmetic to write code like this:

x.mul(y.sub(1))  # a * (y - 1)

There have been at least two times that Python has added syntax to the 
language to allow a third-party library to write more-idiomatic code in 
their domain. Both extended slicing seq[a:b:c] and the matrix- 
multiplication operator were added for numpy.

So *in principle* we certainly could add a new arrow operator for 
Yanghao Hua so his libraries and code will be more expressive and 
idiomatic in his domain.

But *in practice*, the hard truth is this:

- Yanghao Hua is one developer interested in HDL in Python;

- numpy was, and still is, one of the most important "killer apps"
  responsible for Python's success.

Adding syntax for the benefit of numpy helps millions of users; adding 
syntax for HDL-like helps... how many people? Twenty? A hundred?

Millions of users will have to learn the syntax. Unless they get some 
benefit, why consider this?

(But having said that... I reckon that if we had left and right arrow 
operators, <== and ==>, I have some DSLs where they would work for me 
too. Maybe.)


> I'm pretty much done with this conversation too.  You have repeatedly 
> been asked what problem you are trying to solve and repeatedly respond 
> by restating your solution, which appears to be to impose HDL sematics 
> onto a non-HDL language.  That's never going to be a good idea.

That's totally unfair to Yanghao Hua on two levels.

(1) He has never asked for the interpreter to support the semantics he 
wants. He's not asking for Python to understand and implement HDL 
semantics in the language: he can do that himself, in the class.

(2) He has explained the problem he is trying to solve: he wants to 
write a DSL using syntax which doesn't look like crap from the 
perspective of people in that domain.

He just wants an operator that looks kinda like assignment that calls a 
dunder. He can't use assignment because the semantics are baked into the 
language (and there's no chance of that changing). He can't use existing 
operators because they're already in use.

This is not an unreasonable request. Python has bowed to similar 
requests at least twice before, and honestly, it's not like a <== 
operator would be weirder than other operators in use in mainstream 
languages. We're not talking APL or J here.

On the other hand... its not clear that, as small as this request is, 
the language would be better. There's lots of other domains where Python 
syntax is sub-optimal: 

- We don't have good syntax for writing XML, or calling SQL. We just
  have to use strings.

- No nice syntax for writing Prolog-style deductive code;

- or concatenative DSLs;

- or natural language DSLs like Hypertalk or Inform.


Python can't be all things to all people. 



-- 
Steven

From yanghao.py at gmail.com  Tue Jun  4 08:32:43 2019
From: yanghao.py at gmail.com (Yanghao Hua)
Date: Tue, 4 Jun 2019 14:32:43 +0200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <23798.10135.127055.149191@turnbull.sk.tsukuba.ac.jp>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAM+UkHWwvtFY2viBs8CRZQMHMoNx971+U057_sDCNF4eTw7NRw@mail.gmail.com>
 <20190523235937.GQ4221@ando.pearwood.info>
 <CAM+UkHUR5RWnj=NY=Wc6hwcVjSHRD_iPmhUMDLbFvuH1gpa96w@mail.gmail.com>
 <5CE7C780.7090803@canterbury.ac.nz>
 <CAM+UkHU7eyE1-1ix_qS-t1ZH62F5yeAebuLhq4voG5y3aXBsPw@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <23798.10135.127055.149191@turnbull.sk.tsukuba.ac.jp>
Message-ID: <CAM+UkHU305xVH2DCDjuk1DSK6tJd8k7t=0wvBvXcJVCG3r3qhg@mail.gmail.com>

On Tue, Jun 4, 2019 at 10:11 AM Stephen J. Turnbull
<turnbull.stephen.fw at u.tsukuba.ac.jp> wrote:
>
> Yanghao Hua writes:
>  > On Fri, May 31, 2019 at 3:48 PM Rhodri James <rhodri at kynesim.co.uk> wrote:
>
>  > > It really doesn't.  If the end user is going to implement the
>  > > logic of this anyway, implementing signal linkage as a method
>  > > call or class all of its own is not a significant extra burden.
>
>  > There are very constructive discussions in terms how this could be
>  > handled and which turns out not really elegant. Your justification
>  > could be used to reject features like descriptors, decorators, meta
>  > classes,
>
> And has been so used.  The question is always "does the use case
> justify increasing complexity for a couple hundred maintainers and a
> few hundred million readers?"

That's a valid question. and let's address the items one by one.

Complexity & hundreds of maintainers. It is nothing complex and it is
almost a direct copy of the implementation of @= operator, all logic
behind <== are the same of @=. With some refactoring @= and <==
implementation might even be able to share 99% of the code, so I think
it is not complex at all, and there is not much additional overhead
for maintenance.

hundred million readers. I can only guess you mean python developers
here? Just like @=, <== is not enforced on anyone, but can be used by
anyone who wants to. As it is today I can choose to give @= a complete
different meaning if I chose to, other uses may chose (and most
importantly, being able to) give <== the meaning they see fit when
they want  to preserve all number operators and matrix operators. It
is merely a new possibility for end user with very limited effort.

I am glad that this question comes, which means at least the python
community is no longer denying there is a problem (a small one by the
way), and it can be improved in terms of allowing end user to write
more readable code in this specific domain.

> As far as I can tell, here we have (a) *one* person *strongly* in
> favor of adding just one new operator token, who justifies it based on
> *personal* preference over style of expression for *one* specific DSL,
> and (b) a few supporting a generic facility for adding new operators.
>
> The former (a) is obviously insufficient, considering past practice in
> Python.  One person can probably use MacroPy or a preprocessor.  If
> the DSL becomes popular, as with "@", first applied for matrix
> multiplication, it might become justification for a new operator
> token, but the popularity has to come before more than a tiny fraction
> of committers (if any) will sign on.  The other possibility would be
> to present a complete implementation of a toy language using existing
> Python facilities, and then show the alternative with the new
> operator.  Preferably at the same time showing how the preprocessor/
> MacroPy strategy falls short.

Will do.

> The latter (b) has been discussed and rejected a number of times, on
> the grounds that unfamiliar symbols are *harder* to read than
> functions, and that Python is not in the DSL-producing industry.  Some
> features are clearly more descriptive than procedural in nature (eg,
> comprehensions), but they are justified on the ground that they
> increase readability when expressing algorithms.

Understood.

>  > Please let's be constructive and be specific, this kind of
>  > conclusion without any reasoning behind it
>
> In Python practice, the conclusion of "reject" needs no reasoning,
> unfortunately.  Python adds new features in response to a demonstrated
> need that overcomes a general conservatism, a conservatism that
> extends far past backward compatibility to a certain amount of forward
> compatibility as well.
>
> So you should take your own advice.  We have seen you express your
> *preference* for a new operator numerous times, but there is no
> *visible* logic behind it.  Show us the best you can do with actual
> working hardware description classes (say a NAND gate and the
> composition into a half-adder, or perhaps you need more to demonstrate
> the relevant difficulties -- I know little about hardware, quite a bit
> about how the Python enhancement process works).  Then show us the
> syntax you want to use to express the same thing.

I will come up with a complete working implementation of HDL-in-Python
with <== to show the differences, how it enhances readability, and how
it compares without it.

> It's possible that people with wizard-level Python skills can find a
> way around the difficulties you describe with descriptors and other
> features that have been suggested in this thread.  If not, your case
> for a new operator token would be much stronger (though still
> insufficient IMO -- fortunately for you, few committers listen to me
> about language features ;-).

I'd like to take any recommendations that can achieve the same logical
integrity and readability.

> Without such a test, or a few tens of thousands of hardware designers
> lobbying for a decade or so (which is what happened with the numerical
> community, although this time around I bet Python will be
> substantially more forward-leaning, say 5 years and 5000 hardware
> designers), it's your taste against Python's conservatism.  I'll take
> long odds that conservatism wins.

Thanks for the advice, I will (I was) working on it and will show you
the ideas behind it. I knew it is going to be difficult and probably I
have failed to use some cool features and I think a completely working
system will help to demonstrate the ideas better. Stay tuned.

From ricky at teachey.org  Tue Jun  4 08:34:01 2019
From: ricky at teachey.org (Ricky Teachey)
Date: Tue, 4 Jun 2019 08:34:01 -0400
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <CACac1F9H1NR-SFU36vWKR-KGG552AybYAejS3jrrRyR4q0PcDw@mail.gmail.com>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <5CF64C42.1070609@UGent.be> <20190604112917.GC4221@ando.pearwood.info>
 <5CF659F2.5010908@UGent.be>
 <CACac1F9H1NR-SFU36vWKR-KGG552AybYAejS3jrrRyR4q0PcDw@mail.gmail.com>
Message-ID: <CAP0nE64HuXScUYgjsA-yACa1MEV6vnQB03adZ0AcLa_Zu_0TZQ@mail.gmail.com>

I agree this needs to be reframed but suggest that assignment in place
isn't the most useful mental model.

Instead, something like "generically apply a value to another" (dunder
apply) or "update an object with another" (dunder update) might have a
prayer of making sense. Perhaps there are other situations where having a
*generic* operator meant to communicate the concept of sending and object
into another makes sense. A few come to mind:

my_dict.update
my_gen.send
my_list.append
my_list.extend
my_stream.write

It's worth considering whether there would be benefit in providing a
generic operator that provides nice syntax, intended for general use
dependent entirely on that the user wants to make it mean. And whether some
of the existing functionality above might benefit from some prettier syntax
as well.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20190604/194091cb/attachment.html>

From yanghao.py at gmail.com  Tue Jun  4 08:36:04 2019
From: yanghao.py at gmail.com (Yanghao Hua)
Date: Tue, 4 Jun 2019 14:36:04 +0200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
Message-ID: <CAM+UkHVi-cfA_3FYE0MZfx=Jaa7JSpV7z_qn3XtpdiWqr_Nm-A@mail.gmail.com>

On Tue, Jun 4, 2019 at 2:20 PM Rhodri James <rhodri at kynesim.co.uk> wrote:
>
> On 04/06/2019 11:06, Yanghao Hua wrote:
> > [...]  what I needed is an operator that does not
> > collide with all existing number/matrix operators.
>
> Why?
>
> That's the question that in all your thousands of words of argument you
> still haven't answered beyond "because I want it."

Rhodri, I don't know how could I be more specific, help me out here:

signal << (signal << 2) --> the first >> means "assign" and the second
means shift? Do you really think this is readable? Or maybe you have a
better idea I am not aware of?

signals are numbers (with arbitrary bit width), all arithmetic ops still holds.

From steve at pearwood.info  Tue Jun  4 08:38:13 2019
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 4 Jun 2019 22:38:13 +1000
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
Message-ID: <20190604123811.GE4221@ando.pearwood.info>

On Tue, Jun 04, 2019 at 01:20:14PM +0100, Rhodri James wrote:
> On 04/06/2019 11:06, Yanghao Hua wrote:
> >[...]  what I needed is an operator that does not
> >collide with all existing number/matrix operators.
> 
> Why?
> 
> That's the question that in all your thousands of words of argument you 
> still haven't answered beyond "because I want it."



(1) Because they're already being used. That rules out operators that 
are supported by numbers at least, since the primitive values in his 
code are numbers.

So once you remove the operators supported by numbers:

    + - * / // & ** ^ & | << >> ~ < > <= >= == != 

(have I missed any?) I think that only leaves @ remaining.



(2) Because things which act different should look different, and things 
which act similar should look similar. Yanghao Hua wants an operator 
which suggests a kind of assignment. Out of the remaining set of 
operators, which one do you think suggests assignment?


-- 
Steven

From J.Demeyer at UGent.be  Tue Jun  4 08:48:19 2019
From: J.Demeyer at UGent.be (Jeroen Demeyer)
Date: Tue, 4 Jun 2019 14:48:19 +0200
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <CAP0nE64HuXScUYgjsA-yACa1MEV6vnQB03adZ0AcLa_Zu_0TZQ@mail.gmail.com>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <5CF64C42.1070609@UGent.be> <20190604112917.GC4221@ando.pearwood.info>
 <5CF659F2.5010908@UGent.be>
 <CACac1F9H1NR-SFU36vWKR-KGG552AybYAejS3jrrRyR4q0PcDw@mail.gmail.com>
 <CAP0nE64HuXScUYgjsA-yACa1MEV6vnQB03adZ0AcLa_Zu_0TZQ@mail.gmail.com>
Message-ID: <5CF66893.4090508@UGent.be>

On 2019-06-04 14:34, Ricky Teachey wrote:
> "update an object with another" (dunder update)

Yes, that's essentially what I meant. To me, "assign an object in place" 
and "update an object with another" mean the same thing.

> A few come to mind:
>
> my_dict.update

This is PEP 584, where += is used

> my_gen.send

Sure, this makes sense to me!

> my_list.append

I disagree because this keeps the contents of the old list.

> my_list.extend

This should just be a generalization of the += operator.

> my_stream.write

I'm not convinced. If you have an operator for writing, you expect an 
operator for reading too. But then, the analogy with += breaks down for me.

From eric at trueblade.com  Tue Jun  4 08:50:09 2019
From: eric at trueblade.com (Eric V. Smith)
Date: Tue, 4 Jun 2019 08:50:09 -0400
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <20190604123811.GE4221@ando.pearwood.info>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <20190604123811.GE4221@ando.pearwood.info>
Message-ID: <c4d35c6b-627f-3e93-dccf-bd61efda02a1@trueblade.com>

On 6/4/2019 8:38 AM, Steven D'Aprano wrote:
> On Tue, Jun 04, 2019 at 01:20:14PM +0100, Rhodri James wrote:
>> On 04/06/2019 11:06, Yanghao Hua wrote:
>>> [...]  what I needed is an operator that does not
>>> collide with all existing number/matrix operators.
>>
>> Why?
>>
>> That's the question that in all your thousands of words of argument you
>> still haven't answered beyond "because I want it."
> 
> 
> 
> (1) Because they're already being used. That rules out operators that
> are supported by numbers at least, since the primitive values in his
> code are numbers.
> 
> So once you remove the operators supported by numbers:
> 
>      + - * / // & ** ^ & | << >> ~ < > <= >= == !=
> 
> (have I missed any?) I think that only leaves @ remaining.

One problem is that this has no end. Say <== is added (for "signals"), 
then someone will say "I want something just like a signal, but with 
this one additional operator". What then?

As sad as it may be to many people (and I'm one of them), Python just 
isn't the right fit for designing DSLs. And if we want to improve it in 
that area, SQL is the first place we should start looking. SQLAlchemy 
has any number of functions or weird use of operators (.in_(), &&, etc.) 
that could be improved. SQLAlchemy chose to just accept that Python is 
what it is, and did they best they could, even though some of the 
constructs are not ideal. And I think it's been pretty successful.

Eric

From yanghao.py at gmail.com  Tue Jun  4 08:52:49 2019
From: yanghao.py at gmail.com (Yanghao Hua)
Date: Tue, 4 Jun 2019 14:52:49 +0200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <20190604122826.GD4221@ando.pearwood.info>
References: <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <20190604122826.GD4221@ando.pearwood.info>
Message-ID: <CAM+UkHXmbFJA-hNPFEJdxW6BTMFxB+ZwnkGhytqQw01LANCvkg@mail.gmail.com>

On Tue, Jun 4, 2019 at 2:28 PM Steven D'Aprano <steve at pearwood.info> wrote:
>
> On Fri, May 31, 2019 at 02:48:24PM +0100, Rhodri James wrote:
> > On 29/05/2019 08:31, Yanghao Hua wrote:
>
> > >Python does not need to know this ... just hand it over to end user
> > >who knows how to implement such a thing. Python need to provide the
> > >mechanism.
> >
> > It really doesn't.  If the end user is going to implement the logic of
> > this anyway, implementing signal linkage as a method call or class all
> > of its own is not a significant extra burden.
>
> That's not really fair: syntax matters, and for people working in a
> domain where certain syntax is expected, asking them to use something
> different is a significant cognitive burden. We don't ask people doing
> arithmetic to write code like this:
>
> x.mul(y.sub(1))  # a * (y - 1)
>
> There have been at least two times that Python has added syntax to the
> language to allow a third-party library to write more-idiomatic code in
> their domain. Both extended slicing seq[a:b:c] and the matrix-
> multiplication operator were added for numpy.
>
> So *in principle* we certainly could add a new arrow operator for
> Yanghao Hua so his libraries and code will be more expressive and
> idiomatic in his domain.
>
> But *in practice*, the hard truth is this:
>
> - Yanghao Hua is one developer interested in HDL in Python;
>
> - numpy was, and still is, one of the most important "killer apps"
>   responsible for Python's success.
>
> Adding syntax for the benefit of numpy helps millions of users; adding
> syntax for HDL-like helps... how many people? Twenty? A hundred?
>
> Millions of users will have to learn the syntax. Unless they get some
> benefit, why consider this?
>
> (But having said that... I reckon that if we had left and right arrow
> operators, <== and ==>, I have some DSLs where they would work for me
> too. Maybe.)

Don't make it like a war between me alone with the rest of community,
like your last paragraph, I was hoping this could be useful for more
than just me, if that is not the case, I can happily use a
self-modified cpython and move on. The truth is, I am not the only one
interested in that ... I have a lot of friends/colleagues who had way
more experience than me in hardware design and the first react of
seeing something like signal.next = thing or signal.assign(thing) is,
can you make it like signal <= thing?

I am really a newbie here in python developer community, all I was
hoping is if someone of you might have shared a similar pain. Well the
first 80 or so responses I got is: "We do not think there is a problem
...". Well, maybe I am dump, but I should improved after using python
for 20 years, and I still doesn't like how descriptor works (I wrote a
meta class to make descriptors working the way I liked, e.g.
universally doesn't matter where you define it ...) ... special cases
that it has to be a class member, and it actually overloaded the
assignment operator, but only in certain cases (e.g. signal.descriptor
= thing, not descriptor = thing) What if we had a way to overload
"assign" *universally* everywhere? For me, conceptually, "<==" is much
easier and straight forward and difficult to get wrong than the entire
descriptor concept.

>
> > I'm pretty much done with this conversation too.  You have repeatedly
> > been asked what problem you are trying to solve and repeatedly respond
> > by restating your solution, which appears to be to impose HDL sematics
> > onto a non-HDL language.  That's never going to be a good idea.
>
> That's totally unfair to Yanghao Hua on two levels.
>
> (1) He has never asked for the interpreter to support the semantics he
> wants. He's not asking for Python to understand and implement HDL
> semantics in the language: he can do that himself, in the class.
>
> (2) He has explained the problem he is trying to solve: he wants to
> write a DSL using syntax which doesn't look like crap from the
> perspective of people in that domain.
>
> He just wants an operator that looks kinda like assignment that calls a
> dunder. He can't use assignment because the semantics are baked into the
> language (and there's no chance of that changing). He can't use existing
> operators because they're already in use.
>
> This is not an unreasonable request. Python has bowed to similar
> requests at least twice before, and honestly, it's not like a <==
> operator would be weirder than other operators in use in mainstream
> languages. We're not talking APL or J here.

Thank you Steven! This means a lot to me, that I know we can discuss
on a solid base.

> On the other hand... its not clear that, as small as this request is,
> the language would be better. There's lots of other domains where Python
> syntax is sub-optimal:
>
> - We don't have good syntax for writing XML, or calling SQL. We just
>   have to use strings.
>
> - No nice syntax for writing Prolog-style deductive code;
>
> - or concatenative DSLs;
>
> - or natural language DSLs like Hypertalk or Inform.
>
>
> Python can't be all things to all people.

Fully agree, and I will come up with a comprehensive paper to explain
what can "<==" and "==>" simplify, not only supporting new DSLs, but
also simplify existing things like how you design an descriptor-like
behavior.

From rhodri at kynesim.co.uk  Tue Jun  4 09:00:34 2019
From: rhodri at kynesim.co.uk (Rhodri James)
Date: Tue, 4 Jun 2019 14:00:34 +0100
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAM+UkHVi-cfA_3FYE0MZfx=Jaa7JSpV7z_qn3XtpdiWqr_Nm-A@mail.gmail.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <CAM+UkHVi-cfA_3FYE0MZfx=Jaa7JSpV7z_qn3XtpdiWqr_Nm-A@mail.gmail.com>
Message-ID: <560749eb-d524-1133-915d-8551c2a6bca1@kynesim.co.uk>

On 04/06/2019 13:36, Yanghao Hua wrote:
> On Tue, Jun 4, 2019 at 2:20 PM Rhodri James<rhodri at kynesim.co.uk>  wrote:
>> On 04/06/2019 11:06, Yanghao Hua wrote:
>>> [...]  what I needed is an operator that does not
>>> collide with all existing number/matrix operators.
>> Why?
>>
>> That's the question that in all your thousands of words of argument you
>> still haven't answered beyond "because I want it."
> Rhodri, I don't know how could I be more specific, help me out here:
> 
> signal << (signal << 2) --> the first >> means "assign" and the second
> means shift? Do you really think this is readable? Or maybe you have a
> better idea I am not aware of?

I'm asking why you want the first "assignment" << *at all*.  What is it 
about the operation you are doing (which, incidentally, I still don't 
get) that makes it *so much* better at expressing what you're doing than 
(say) a method call?  What's wrong with

   signal.suitable_descriptive_verb(signal << 2)

The bar for adding a new operator is intentionally high, and I haven't 
seen enough justification to satisfy me yet.  Everyone will need to be 
able to read this and get your intention, don't forget.

-- 
Rhodri James *-* Kynesim Ltd

From yanghao.py at gmail.com  Tue Jun  4 09:02:07 2019
From: yanghao.py at gmail.com (Yanghao Hua)
Date: Tue, 4 Jun 2019 15:02:07 +0200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <c4d35c6b-627f-3e93-dccf-bd61efda02a1@trueblade.com>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <20190604123811.GE4221@ando.pearwood.info>
 <c4d35c6b-627f-3e93-dccf-bd61efda02a1@trueblade.com>
Message-ID: <CAM+UkHXLu6aTOr=A2sS34TjokrgB1x7qKe0ZYrcGeDOOVU+UqQ@mail.gmail.com>

On Tue, Jun 4, 2019 at 2:50 PM Eric V. Smith <eric at trueblade.com> wrote:
>
> On 6/4/2019 8:38 AM, Steven D'Aprano wrote:
> > On Tue, Jun 04, 2019 at 01:20:14PM +0100, Rhodri James wrote:
> >> On 04/06/2019 11:06, Yanghao Hua wrote:
> >>> [...]  what I needed is an operator that does not
> >>> collide with all existing number/matrix operators.
> >>
> >> Why?
> >>
> >> That's the question that in all your thousands of words of argument you
> >> still haven't answered beyond "because I want it."
> >
> >
> >
> > (1) Because they're already being used. That rules out operators that
> > are supported by numbers at least, since the primitive values in his
> > code are numbers.
> >
> > So once you remove the operators supported by numbers:
> >
> >      + - * / // & ** ^ & | << >> ~ < > <= >= == !=
> >
> > (have I missed any?) I think that only leaves @ remaining.
>
> One problem is that this has no end. Say <== is added (for "signals"),
> then someone will say "I want something just like a signal, but with
> this one additional operator". What then?
>
> As sad as it may be to many people (and I'm one of them), Python just
> isn't the right fit for designing DSLs. And if we want to improve it in
> that area, SQL is the first place we should start looking. SQLAlchemy
> has any number of functions or weird use of operators (.in_(), &&, etc.)
> that could be improved. SQLAlchemy chose to just accept that Python is
> what it is, and did they best they could, even though some of the
> constructs are not ideal. And I think it's been pretty successful.

As though as it is an objection I do like the reasoning a lot more,
and that's exactly why the title for this thread I was choosing is "
... yet another operator?". And Yes SQLAlchemy shows similar scenarios
where Python is not yet good at. And I know the potential huge
challenge to actually change something fundamentally ... but sometimes
hope is the most important thing: is Python visioned in a way to allow
users to define there own operators? I think this will be a huge topic
in the coming future. In a world where people just do general purpose
programming/computing, to a world that every one can design its own
system-on-chip, there will simply be more and more domain-specific
languages. This is where I see scala picks up more and more. And as a
Python fan, I really do hope Python could support this (just like it
picked up async/await from C#) and make it truly the first-choice for
designing a domain specific language.

From rhodri at kynesim.co.uk  Tue Jun  4 09:02:32 2019
From: rhodri at kynesim.co.uk (Rhodri James)
Date: Tue, 4 Jun 2019 14:02:32 +0100
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <20190604123811.GE4221@ando.pearwood.info>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <20190604123811.GE4221@ando.pearwood.info>
Message-ID: <b839e7fd-4235-8ef7-10d5-59c9ed4b5a85@kynesim.co.uk>

On 04/06/2019 13:38, Steven D'Aprano wrote:
> (2) Because things which act different should look different, and things
> which act similar should look similar. Yanghao Hua wants an operator
> which suggests a kind of assignment. Out of the remaining set of
> operators, which one do you think suggests assignment?

You're a step ahead of me, Steven :-)  I still haven't been convinced 
that a new operator is appropriate.

-- 
Rhodri James *-* Kynesim Ltd

From greg.ewing at canterbury.ac.nz  Tue Jun  4 09:11:48 2019
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 05 Jun 2019 01:11:48 +1200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAM+UkHXLu6aTOr=A2sS34TjokrgB1x7qKe0ZYrcGeDOOVU+UqQ@mail.gmail.com>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <20190604123811.GE4221@ando.pearwood.info>
 <c4d35c6b-627f-3e93-dccf-bd61efda02a1@trueblade.com>
 <CAM+UkHXLu6aTOr=A2sS34TjokrgB1x7qKe0ZYrcGeDOOVU+UqQ@mail.gmail.com>
Message-ID: <5CF66E14.4000105@canterbury.ac.nz>

Yanghao Hua wrote:
> is Python visioned in a way to allow
> users to define there own operators?

No, it's not. Whenever the topic has come up, Guido has always said
that he is against having user-defined syntax in Python.

-- 
Greg

From ricky at teachey.org  Tue Jun  4 09:37:13 2019
From: ricky at teachey.org (Ricky Teachey)
Date: Tue, 4 Jun 2019 09:37:13 -0400
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <5CF66893.4090508@UGent.be>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <5CF64C42.1070609@UGent.be> <20190604112917.GC4221@ando.pearwood.info>
 <5CF659F2.5010908@UGent.be>
 <CACac1F9H1NR-SFU36vWKR-KGG552AybYAejS3jrrRyR4q0PcDw@mail.gmail.com>
 <CAP0nE64HuXScUYgjsA-yACa1MEV6vnQB03adZ0AcLa_Zu_0TZQ@mail.gmail.com>
 <5CF66893.4090508@UGent.be>
Message-ID: <CAP0nE66OtaSv8-Q9V=qJMyN73PFVHauUxVR_Hc1J7Sqwn3-SmA@mail.gmail.com>

 Ok agreed on .update and .extend. Two operators (+= and <==) doing the
same thing is dumb. And for .append I agree "this thing is the same, just
add this thing" is a little at odds with "update this thing when i send
this other thing into it".

> my_gen.send
>
> Sure, this makes sense to me!
>

I have to admit, I *kind of* love this one.


> If you have an operator for writing, you expect an
>
operator for reading too. But then, the analogy with += breaks down for me.
>

 For reading, can't you just switch the order?

fstream <== x  # write into existing stream
x <== fstream  # read into existing object (ie, this is NOT an assignment
type of action-- this might be confusing)

Worth pointing out that fstream is a type of iterator, so might writing
just be thought of as a specific case of sending into a generator?
Similarly: reading is yield from the generator. Of course you'd have a
battle to the death over whether the read should be Unicode based (reading
lines) or reading a byte at a time.

On the original topic: I was wondering the other day if the HDL thing could
most conveniently be implemented using a generator-like class that allows
things sent into it. But the .send syntax doesn't help the problem the OP
was trying to solve of ugly syntax.

I'm beginning to wonder of associating an operator with .send makes a lot
sense. I think I would definitely find code more readable with that kind of
syntax.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20190604/ee6a2b01/attachment.html>

From cody.piersall at gmail.com  Tue Jun  4 09:45:53 2019
From: cody.piersall at gmail.com (Cody Piersall)
Date: Tue, 4 Jun 2019 08:45:53 -0500
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
Message-ID: <CAFSbXtN40iLDxawqQ1Kb3UkArTZMS5rEYR0XWdL3FQMXu8OaqQ@mail.gmail.com>

On Tue, Jun 4, 2019 at 7:21 AM Rhodri James <rhodri at kynesim.co.uk> wrote:
>
> On 04/06/2019 11:06, Yanghao Hua wrote:
> > [...]  what I needed is an operator that does not
> > collide with all existing number/matrix operators.
>
> Why?
>
> That's the question that in all your thousands of words of argument you
> still haven't answered beyond "because I want it."

All existing operators are actually already meaningful for HDLs, so
using one for assignment would mean it wouldn't be able to be used for
its "natural" operation.  As Yanghao mentioned, the likeliest to use
would be the in-place matmul operator (@=) but there are use cases
where matrix-multiplication of signals would actually be useful too.

From tjreedy at udel.edu  Tue Jun  4 11:38:34 2019
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 4 Jun 2019 11:38:34 -0400
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <5CF64C42.1070609@UGent.be>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <5CF64C42.1070609@UGent.be>
Message-ID: <qd639r$4t0f$1@blaine.gmane.org>

On 6/4/2019 6:47 AM, Jeroen Demeyer wrote:
> I'd like to get rid of all the signal and HDL stuff (whatever that 
> means) in this thread, so I think what the original poster really wants 
> is an "assign in place" operator. Basically, something like += or *= but 
> without the arithmetic.

I believe that what he wanted, at least initially, was not an in place 
mutation, which is nonsensical for ints, but a delayed binding.


-- 
Terry Jan Reedy


From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Jun  4 13:20:00 2019
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 5 Jun 2019 02:20:00 +0900
Subject: [Python-ideas] Assign-in-place operator
In-Reply-To: <5CF64C42.1070609@UGent.be>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <5CF64C42.1070609@UGent.be>
Message-ID: <23798.43072.509873.391949@turnbull.sk.tsukuba.ac.jp>

Jeroen Demeyer writes:

 > When you think of it this way, it's not an unreasonable request. There 
 > would be at least one major use of this operator within CPython, for 
 > lists. With this proposal, the awkward syntax (there are 219 instances 
 > of this in the CPython sources)
 > 
 >    L[:] = new_list

I'd rather not replace it.  It's a perfectly Pythonic syntax, although
it's a Python-specific idiom.  It's an opportunity to see slice
assignment in operation, which I would guess is relatively unusual in
the general case.

Steve


From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Jun  4 13:22:36 2019
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 5 Jun 2019 02:22:36 +0900
Subject: [Python-ideas] `if-unless` expressions in Python
In-Reply-To: <CAPTjJmoAPF1ruTv_w8yBsNWOzcLdN3sx4kQdHLC6vqz7_0EwmA@mail.gmail.com>
References: <CANQHGB1853-fDmZcSet9CX4s5MiknbQ7ggUvzWSYV+U0j8ffcw@mail.gmail.com>
 <CAPTjJmoAPF1ruTv_w8yBsNWOzcLdN3sx4kQdHLC6vqz7_0EwmA@mail.gmail.com>
Message-ID: <23798.43228.796735.723067@turnbull.sk.tsukuba.ac.jp>

Chris Angelico writes:

 > ## New "unless" construct for list displays and argument lists ##
 > 
 > Inside a list/dict/set/tuple display, or inside an argument list,
 > elements can be conditionally omitted by providing a predicate.
 > 
 > lst = [f1(), f3() unless f2(), f4()]

Not a fan of this in displays, YMMV.  IME the cases where I could use
this are generally well-served by comprehensions with if clauses.
(I'm speaking for myself only, I have no idea if others have "more
than once in a blue moon" use cases.)

OTOH

    foo(f1(), f3() unless f2(), f4())

looks horrible to me, a definite -1.  Yes, I know about varargs
functions, but "positional" means positional to me.  Surely this would
almost always be a runtime error in format() if not f2(), for example.
It would have to be used quite frequently for me to get over the
awkwardness, I think.

Granted,

    foo(*[f1(), f3() unless f2(), f4()])

is horrid, too (and if you wanted to use a tuple for efficiency it
would be unreadable).  But how often would you want to call varargs
with arguments that might not even be there?  Would

    args = (f1(), f3() unless f2(), f4())
    foo(*args)

be so bad for the rare occasion?  (Maybe it's not so rare for others,
but I can't recall ever wanting this, while list displays with
variable desired length do come up.)

Steve

From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Jun  4 13:28:19 2019
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 5 Jun 2019 02:28:19 +0900
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAFSbXtN40iLDxawqQ1Kb3UkArTZMS5rEYR0XWdL3FQMXu8OaqQ@mail.gmail.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <CAFSbXtN40iLDxawqQ1Kb3UkArTZMS5rEYR0XWdL3FQMXu8OaqQ@mail.gmail.com>
Message-ID: <23798.43571.275541.291023@turnbull.sk.tsukuba.ac.jp>

Cody Piersall writes:

 > would be the in-place matmul operator (@=) but there are use cases
 > where matrix-multiplication of signals would actually be useful
 > too.

If I recall correctly, the problem that the numeric community faced
was that there are multiple "multiplication" operations that matrices
"want to" support with operator notation because they're all
frequently used in more or less complex expressions, not that matrix
algebra needs to spell its multiplication operator differently from
"*".

According to the OP, signals are "just integers".  Integers do not
need to support matrix multiplication because they *can't*.  There may
be matrices of signals that do want to support multiplication, but
that will be a different type, and presumably multiplication of signal
matrices will be supported by "*".  Can you say that signal matrices
will have more than one frequently needed "multiplication" operation?


From turnbull.stephen.fw at u.tsukuba.ac.jp  Tue Jun  4 13:32:23 2019
From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 5 Jun 2019 02:32:23 +0900
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAM+UkHU305xVH2DCDjuk1DSK6tJd8k7t=0wvBvXcJVCG3r3qhg@mail.gmail.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAM+UkHWwvtFY2viBs8CRZQMHMoNx971+U057_sDCNF4eTw7NRw@mail.gmail.com>
 <20190523235937.GQ4221@ando.pearwood.info>
 <CAM+UkHUR5RWnj=NY=Wc6hwcVjSHRD_iPmhUMDLbFvuH1gpa96w@mail.gmail.com>
 <5CE7C780.7090803@canterbury.ac.nz>
 <CAM+UkHU7eyE1-1ix_qS-t1ZH62F5yeAebuLhq4voG5y3aXBsPw@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <23798.10135.127055.149191@turnbull.sk.tsukuba.ac.jp>
 <CAM+UkHU305xVH2DCDjuk1DSK6tJd8k7t=0wvBvXcJVCG3r3qhg@mail.gmail.com>
Message-ID: <23798.43815.907924.513680@turnbull.sk.tsukuba.ac.jp>

Yanghao Hua writes:
 > On Tue, Jun 4, 2019 at 10:11 AM Stephen J. Turnbull
 > <turnbull.stephen.fw at u.tsukuba.ac.jp> wrote:

 > > The question is always "does the use case justify increasing
 > > complexity for a couple hundred maintainers and a few hundred
 > > million readers?"
 > 
 > That's a valid question. and let's address the items one by one.
 > 
 > Complexity & hundreds of maintainers. It is nothing complex

I'm sorry, but you seem to misunderstand what I mean by "complex" in
this context.  "Complex" refers to the *entire* Python language, and
what happens to the difficulty of maintaining and learning the
language if *all* proposed features of similar (or better)
benefit/cost tradeoff are added.

Some developers argue that our own bar is clearly too high.  I tend to
disagree, mostly on the grounds that many core developers frequently
confess to being overburdened by the current pace of change and needs
for maintenance.

 > hundred million readers. I can only guess you mean python
 > developers here?

I mean anybody who reads Python code.  Mostly developers who program
in Python, of course.  But it also includes lots of people who aren't
Python developers, and may not be experienced developers at all, but
just want to understand if some code fits their needs so they can
cargo-cult it, or just use it as a plug-in.

 > Just like @=, <== is not enforced on anyone, but can be used by
 > anyone who wants to.

But they *are* enforced.  They are forced on anyone who wants to read
code using them.  In general, Python prioritizes readers over writers
of code.  You can, and we do, argue about whether that tradeoff is
made appropriately in any given case, but I doubt you'll find anyone
to disagree that that is an important principle.  (A few disagree with
the importance it's given, of course, but they'll admit that it is
frequently mentioned in these discussions.)

 > I am glad that this question comes, which means at least the python
 > community is no longer denying there is a problem

That's not fair.  I don't recall anyone denying you have a problem
worth solving.  I did see objections to your proposed solution.  I
also saw strong doubt that you've exhausted the possibilities of the
existing Python language.  Essentially all proposals for new syntax
(including new operators) get the same questioning.

 > > The other possibility would be to present a complete
 > > implementation of a toy language using existing Python
 > > facilities, and then show the alternative with the new operator.
 > > Preferably at the same time showing how the preprocessor/ MacroPy
 > > strategy falls short.
 > 
 > Will do.

Good!  It will be interesting to see if any of the wizards are willing
to weigh in, and what they'll come up with.

 > Stay tuned.

Looking forward to it!

Steve

From yanghao.py at gmail.com  Tue Jun  4 16:56:21 2019
From: yanghao.py at gmail.com (Yanghao Hua)
Date: Tue, 4 Jun 2019 22:56:21 +0200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <5CF66E14.4000105@canterbury.ac.nz>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <20190604123811.GE4221@ando.pearwood.info>
 <c4d35c6b-627f-3e93-dccf-bd61efda02a1@trueblade.com>
 <CAM+UkHXLu6aTOr=A2sS34TjokrgB1x7qKe0ZYrcGeDOOVU+UqQ@mail.gmail.com>
 <5CF66E14.4000105@canterbury.ac.nz>
Message-ID: <CAM+UkHVRjY+h_5wPPZFFpWaaYknGjeKjovZVgiCOnN2avGeNQw@mail.gmail.com>

On Tue, Jun 4, 2019 at 3:24 PM Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> Yanghao Hua wrote:
> > is Python visioned in a way to allow
> > users to define there own operators?
>
> No, it's not. Whenever the topic has come up, Guido has always said
> that he is against having user-defined syntax in Python.

Did Guido say "user defined syntax" or "user defined operator"? I love
python's syntax and I don't see the need to change it. But for me
defining new operator is different and we just had a few more recently
(@, @=, :=). The way scala allows user to define new operator is
really elegant, and this is actually the driving force for designing
new DSL in scala.

From yanghao.py at gmail.com  Tue Jun  4 17:09:29 2019
From: yanghao.py at gmail.com (Yanghao Hua)
Date: Tue, 4 Jun 2019 23:09:29 +0200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <23798.43571.275541.291023@turnbull.sk.tsukuba.ac.jp>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <CAEbHw4aO_h4QYijC-WnCiRpq8A-Y5nPMaEqbS4+frrWcoGjBOg@mail.gmail.com>
 <CAM+UkHUCs9GfxOpQd6J3o-F0WGyQy83puaJc8B3CBFUaicYQdg@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <CAFSbXtN40iLDxawqQ1Kb3UkArTZMS5rEYR0XWdL3FQMXu8OaqQ@mail.gmail.com>
 <23798.43571.275541.291023@turnbull.sk.tsukuba.ac.jp>
Message-ID: <CAM+UkHVJVegQt7mO33WaufyOACbOL58j3ekorRyaCWErORDUZA@mail.gmail.com>

On Tue, Jun 4, 2019 at 7:28 PM Stephen J. Turnbull
<turnbull.stephen.fw at u.tsukuba.ac.jp> wrote:
>
> Cody Piersall writes:
>
>  > would be the in-place matmul operator (@=) but there are use cases
>  > where matrix-multiplication of signals would actually be useful
>  > too.
>
> If I recall correctly, the problem that the numeric community faced
> was that there are multiple "multiplication" operations that matrices
> "want to" support with operator notation because they're all
> frequently used in more or less complex expressions, not that matrix
> algebra needs to spell its multiplication operator differently from
> "*".
>
> According to the OP, signals are "just integers".  Integers do not
> need to support matrix multiplication because they *can't*.  There may
> be matrices of signals that do want to support multiplication, but
> that will be a different type, and presumably multiplication of signal
> matrices will be supported by "*".  Can you say that signal matrices
> will have more than one frequently needed "multiplication" operation?

Your statement about the history is absolutely correct, but please
notice two matrix dot-multiplication 1xN @ Nx1 ==> a single value, so
something like below holds:

signal_result <== [sig sig sig ...] @ [sig, sig, sig, ...] (',' used
to make the second a Nx1 for example). And now imagine you want to mix
up @= with @, in one place @= means signal assignment, in another it
means matrix. What's more, when start to use matrix to drive signal
matrix, I assume a lot of matrix ops including the @= is definitely
going to be very often used to produce the stimuli (e.g. using normal
number matrix ops to generate the desired result matrix to drive on
signal matrix), that's why I am refrained from using @= entirely.

From greg.ewing at canterbury.ac.nz  Tue Jun  4 17:27:59 2019
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 05 Jun 2019 09:27:59 +1200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAFSbXtN40iLDxawqQ1Kb3UkArTZMS5rEYR0XWdL3FQMXu8OaqQ@mail.gmail.com>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <CAFSbXtN40iLDxawqQ1Kb3UkArTZMS5rEYR0XWdL3FQMXu8OaqQ@mail.gmail.com>
Message-ID: <5CF6E25F.8080002@canterbury.ac.nz>

Cody Piersall wrote:
> As Yanghao mentioned, the likeliest to use
> would be the in-place matmul operator (@=) but there are use cases
> where matrix-multiplication of signals would actually be useful too.

My question on that is whether matrix multiplication of signals is
likely to be used so heavily that it would be a hardship to represent
it some other way, such as a function.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Jun  4 18:12:36 2019
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 05 Jun 2019 10:12:36 +1200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <23798.43571.275541.291023@turnbull.sk.tsukuba.ac.jp>
References: <CAM+UkHV3JDwvgaaveOu2X4Xx48Tt5_AD55s+C9xNo-xPCq+K8A@mail.gmail.com>
 <02AD20E2-4232-4FF3-8395-EC4B86D564BA@barrys-emacs.org>
 <CAM+UkHWaYj7zTgBzE_-cLadiUa+L_6dDrLgcOG00Afei4HABvA@mail.gmail.com>
 <f815815a-7458-5f09-826e-8b7207f896ae@kynesim.co.uk>
 <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <CAFSbXtN40iLDxawqQ1Kb3UkArTZMS5rEYR0XWdL3FQMXu8OaqQ@mail.gmail.com>
 <23798.43571.275541.291023@turnbull.sk.tsukuba.ac.jp>
Message-ID: <5CF6ECD4.5060503@canterbury.ac.nz>

Stephen J. Turnbull wrote:
> There may
> be matrices of signals that do want to support multiplication, but
> that will be a different type, and presumably multiplication of signal
> matrices will be supported by "*".

Then you lose the ability for '*' to represent elementwise
multiplication of signal arrays. It's the same problem that
numpy faced.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Jun  4 18:51:54 2019
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 05 Jun 2019 10:51:54 +1200
Subject: [Python-ideas] Operator as first class citizens -- like in
 scala -- or yet another new operator?
In-Reply-To: <CAM+UkHVRjY+h_5wPPZFFpWaaYknGjeKjovZVgiCOnN2avGeNQw@mail.gmail.com>
References: <CAM+UkHW0Ge33wq88B1juJ_GkEbJ4Zp_C2x40dV+QeMmms20Szg@mail.gmail.com>
 <d9577248-3398-da60-732d-66bf0cf387d1@trueblade.com>
 <CAM+UkHXhALJQRTG6C=hCLk2HKRwzB4PhMEBSCQ5Hc86ugwyC=g@mail.gmail.com>
 <3CBDB3A8-5FDA-4C8D-9E3E-B4B970D293FA@trueblade.com>
 <CAM+UkHXKhfz19GvguPKz3Df4tNesmJ8Vvkm91b6xHJOR6pXcUQ@mail.gmail.com>
 <d6fd2ce7-7834-6234-0623-50c78bc13d25@kynesim.co.uk>
 <CAM+UkHVkfG5p91m7H8Tu3sD1pPZY1MoMLZc0K3R+nw_2TXH=Wg@mail.gmail.com>
 <FE1A152C-2BEE-41DB-8448-7FC01834A22C@yahoo.com>
 <CAM+UkHUAGc_Exo4rJG17LcQmtgN_+ptCRHEg7UAkTt0Yz0LH+g@mail.gmail.com>
 <18b5d28d-89c7-6d40-189e-d430e713caaf@kynesim.co.uk>
 <20190604123811.GE4221@ando.pearwood.info>
 <c4d35c6b-627f-3e93-dccf-bd61efda02a1@trueblade.com>
 <CAM+UkHXLu6aTOr=A2sS34TjokrgB1x7qKe0ZYrcGeDOOVU+UqQ@mail.gmail.com>
 <5CF66E14.4000105@canterbury.ac.nz>
 <CAM+UkHVRjY+h_5wPPZFFpWaaYknGjeKjovZVgiCOnN2avGeNQw@mail.gmail.com>
Message-ID: <5CF6F60A.6070601@canterbury.ac.nz>

Yanghao Hua wrote:
> Did Guido say "user defined syntax" or "user defined operator"? 
 > ... for me defining new operator is different

To my mind it's not that much different. A reader encountering an
unfamiliar operator is pretty much faced with a new piece of
syntax to learn. Its spelling gives little to no clue as to its
meaning, it precedence in relation to other operators is not
obvious, etc.

> The way scala allows user to define new operator is
> really elegant,

There are some differences between Python and Scala that present
practical difficulties here. Some kind of declaration would be
needed to define its arity, precedence and corresponding dunder
method. This is no problem in Scala because it analyses imported
code at compile time. But the Python compiler only sees the module
being compiled, so it has no ability to import these kinds of
declarations from another module.

Plus, the way the parser works would probably have to be
completely redesigned.

So anyone suggesting user-defined operators be added to Python
has to both convince the community that it's a good idea in the
first place, and come up with solutions to all the technical
problems.

-- 
Greg