From nulla.epistola at web.de  Mon Jan  1 06:02:14 2024
From: nulla.epistola at web.de (Sibylle Koczian)
Date: Mon, 1 Jan 2024 12:02:14 +0100
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
Message-ID: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>

Am 30.12.2023 um 04:04 schrieb Mike Dewhirst via Python-list:
> 
> I had assumed the OP had installed Python from the Microsoft shop and 
> that's where py.exe must have come from.
> 

In fact I didn't say in my post that I always get Python from 
python.org. When I started to use the language there was no Python from 
any Microsoft shop (I'm not sure there was a Microsoft shop, it was in 
the last millenium, Python 1.5 or 1.6). So I tend to forget that 
possible download source.

But in all this thread I didn't see a single explanation for my current 
situation: one and the same shebang line works on Windows 10 / Python 
3.11 and doesn't work on Windows 11 / Python 3.12. I suspect Windows, 
because a change in the way Python 3.12 uses shebang lines should be 
visible in the documentation.

Happy new year to all!
Sibylle



From barry at barrys-emacs.org  Mon Jan  1 06:50:25 2024
From: barry at barrys-emacs.org (Barry)
Date: Mon, 1 Jan 2024 11:50:25 +0000
Subject: Python 3.12.1,
 Windows 11: shebang line #!/usr/bin/env python3 doesn't work any more
In-Reply-To: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
References: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
Message-ID: <F1CB1FD3-4F43-484A-909D-18C20EE32762@barrys-emacs.org>



> On 1 Jan 2024, at 11:14, Sibylle Koczian via Python-list <python-list at python.org> wrote:
> 
> But in all this thread I didn't see a single explanation for my current situation: one and the same shebang line works on Windows 10 / Python 3.11 and doesn't work on Windows 11 / Python 3.12. I suspect Windows, because a change in the way Python 3.12 uses shebang lines should be visible in the documentation.

See my earlier reply with info on ftype etc output.
What do you see on your windows 10 vs windows 11?

Barry



From list1 at tompassin.net  Mon Jan  1 08:19:20 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 1 Jan 2024 08:19:20 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
 <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
Message-ID: <1d191291-f587-426f-83b7-2baa2564c1dc@tompassin.net>

On 1/1/2024 6:02 AM, Sibylle Koczian via Python-list wrote:
> Am 30.12.2023 um 04:04 schrieb Mike Dewhirst via Python-list:
>>
>> I had assumed the OP had installed Python from the Microsoft shop and 
>> that's where py.exe must have come from.
>>
> 
> In fact I didn't say in my post that I always get Python from 
> python.org. When I started to use the language there was no Python from 
> any Microsoft shop (I'm not sure there was a Microsoft shop, it was in 
> the last millenium, Python 1.5 or 1.6). So I tend to forget that 
> possible download source.
> 
> But in all this thread I didn't see a single explanation for my current 
> situation: one and the same shebang line works on Windows 10 / Python 
> 3.11 and doesn't work on Windows 11 / Python 3.12. I suspect Windows, 
> because a change in the way Python 3.12 uses shebang lines should be 
> visible in the documentation.
> 
> Happy new year to all!
> Sibylle

Happy New Year!

I speculated that the shebang line didn't work on Windows 10 either, but 
you didn't realize it because the file associations were right to launch 
".py" programs with the right version of Python.  When the newer version 
of Python got installed, the default Python program to use, was not 
updated correctly, and the shebang line still has nothing to do with the 
launch failure.  This could happen if other the older install went into 
Program Files, while the newer one went into 
%USERPROFILE%\AppData\Local\Programs\Python.

This was backed up with all of 5 minutes of experimenting on my own 
computer, on which Windows launches ".py" programs with an old install 
of Python 3.9.9, but the py launcher launches Python 3.12 by default.

Since I am avoiding Windows 11, I can't try anything on it, so my 
thoughts above may not be relevant.

The Python docs for 3.12.1 cover shebang lines at

https://docs.python.org/3/using/windows.html

"If the first line of a script file starts with #!, it is known as a 
?shebang? line. Linux and other Unix like operating systems have native 
support for such lines and they are commonly used on such systems to 
indicate how a script should be executed. This launcher allows the same 
facilities to be used with Python scripts on Windows and the examples 
above demonstrate their use.

To allow shebang lines in Python scripts to be portable between Unix and 
Windows, this launcher supports a number of ?virtual? commands to 
specify which interpreter to use. The supported virtual commands are:

/usr/bin/env
/usr/bin/python
/usr/local/bin/python
python

For example, if the first line of your script starts with

#! /usr/bin/python
The default Python will be located and used. As many Python scripts 
written to work on Unix will already have this line, you should find 
these scripts can be used by the launcher without modification. If you 
are writing a new script on Windows which you hope will be useful on 
Unix, you should use one of the shebang lines starting with /usr."

But

"The /usr/bin/env form of shebang line has one further special property. 
Before looking for installed Python interpreters, this form will search 
the executable PATH for a Python executable matching the name provided 
as the first argument. This corresponds to the behaviour of the Unix env 
program, which performs a PATH search. If an executable matching the 
first argument after the env command cannot be found, but the argument 
starts with python, it will be handled as described for the other 
virtual commands. The environment variable PYLAUNCHER_NO_SEARCH_PATH may 
be set (to any value) to skip this search of PATH.

Shebang lines that do not match any of these patterns are looked up in 
the [commands] section of the launcher?s .INI file. This may be used to 
handle certain commands in a way that makes sense for your system. The 
name of the command must be a single argument (no spaces in the shebang 
executable), and the value substituted is the full path to the 
executable (additional arguments specified in the .INI will be quoted as 
part of the filename)."



From list1 at tompassin.net  Mon Jan  1 09:11:38 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 1 Jan 2024 09:11:38 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <1d191291-f587-426f-83b7-2baa2564c1dc@tompassin.net>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
 <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <1d191291-f587-426f-83b7-2baa2564c1dc@tompassin.net>
Message-ID: <784bcbbe-f683-4daf-bf57-11c60aedc5b8@tompassin.net>

On 1/1/2024 8:19 AM, Thomas Passin via Python-list wrote:
> On 1/1/2024 6:02 AM, Sibylle Koczian via Python-list wrote:
>> Am 30.12.2023 um 04:04 schrieb Mike Dewhirst via Python-list:
>>>
>>> I had assumed the OP had installed Python from the Microsoft shop and 
>>> that's where py.exe must have come from.
>>>
>>
>> In fact I didn't say in my post that I always get Python from 
>> python.org. When I started to use the language there was no Python 
>> from any Microsoft shop (I'm not sure there was a Microsoft shop, it 
>> was in the last millenium, Python 1.5 or 1.6). So I tend to forget 
>> that possible download source.
>>
>> But in all this thread I didn't see a single explanation for my 
>> current situation: one and the same shebang line works on Windows 10 / 
>> Python 3.11 and doesn't work on Windows 11 / Python 3.12. I suspect 
>> Windows, because a change in the way Python 3.12 uses shebang lines 
>> should be visible in the documentation.
>>
>> Happy new year to all!
>> Sibylle
> 
> Happy New Year!
> 
> I speculated that the shebang line didn't work on Windows 10 either, but 
> you didn't realize it because the file associations were right to launch 
> ".py" programs with the right version of Python.? When the newer version 
> of Python got installed, the default Python program to use, was not 
> updated correctly, and the shebang line still has nothing to do with the 
> launch failure.? This could happen if other the older install went into 
> Program Files, while the newer one went into 
> %USERPROFILE%\AppData\Local\Programs\Python.
> 
> This was backed up with all of 5 minutes of experimenting on my own 
> computer, on which Windows launches ".py" programs with an old install 
> of Python 3.9.9, but the py launcher launches Python 3.12 by default.
> 
> Since I am avoiding Windows 11, I can't try anything on it, so my 
> thoughts above may not be relevant.
> 
> The Python docs for 3.12.1 cover shebang lines at
> 
> https://docs.python.org/3/using/windows.html
> 
> "If the first line of a script file starts with #!, it is known as a 
> ?shebang? line. Linux and other Unix like operating systems have native 
> support for such lines and they are commonly used on such systems to 
> indicate how a script should be executed. This launcher allows the same 
> facilities to be used with Python scripts on Windows and the examples 
> above demonstrate their use.
> 
> To allow shebang lines in Python scripts to be portable between Unix and 
> Windows, this launcher supports a number of ?virtual? commands to 
> specify which interpreter to use. The supported virtual commands are:
> 
> /usr/bin/env
> /usr/bin/python
> /usr/local/bin/python
> python
> 
> For example, if the first line of your script starts with
> 
> #! /usr/bin/python
> The default Python will be located and used. As many Python scripts 
> written to work on Unix will already have this line, you should find 
> these scripts can be used by the launcher without modification. If you 
> are writing a new script on Windows which you hope will be useful on 
> Unix, you should use one of the shebang lines starting with /usr."
> 
> But
> 
> "The /usr/bin/env form of shebang line has one further special property. 
> Before looking for installed Python interpreters, this form will search 
> the executable PATH for a Python executable matching the name provided 
> as the first argument. This corresponds to the behaviour of the Unix env 
> program, which performs a PATH search. If an executable matching the 
> first argument after the env command cannot be found, but the argument 
> starts with python, it will be handled as described for the other 
> virtual commands. The environment variable PYLAUNCHER_NO_SEARCH_PATH may 
> be set (to any value) to skip this search of PATH.
> 
> Shebang lines that do not match any of these patterns are looked up in 
> the [commands] section of the launcher?s .INI file. This may be used to 
> handle certain commands in a way that makes sense for your system. The 
> name of the command must be a single argument (no spaces in the shebang 
> executable), and the value substituted is the full path to the 
> executable (additional arguments specified in the .INI will be quoted as 
> part of the filename)."
> 

Here's how to find out what program Windows thinks it should use to run 
a ".py" file.  In a console:

C:\Users\tom>assoc .py
.py=Python.File

C:\Users\tom>ftype Python.file
Python.file="C:\Windows\py.exe" "%L" %*

If your ".py" files are associated to the py.exe launcher, as mine are, 
then the launcher may try to use your shebang line and you need to make 
sure there aren't any spaces where there shouldn't be.

If your ".py" files are not associated with py.exe, the shebang line 
probably won't be used for anything.



From mats at wichmann.us  Mon Jan  1 11:38:39 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Mon, 1 Jan 2024 09:38:39 -0700
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
 <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
Message-ID: <0f7b9f8d-9be0-4988-bfe1-517ff84e0903@wichmann.us>

On 1/1/24 04:02, Sibylle Koczian via Python-list wrote:
> Am 30.12.2023 um 04:04 schrieb Mike Dewhirst via Python-list:
>>
>> I had assumed the OP had installed Python from the Microsoft shop and 
>> that's where py.exe must have come from.
>>
> 
> In fact I didn't say in my post that I always get Python from 
> python.org. When I started to use the language there was no Python from 
> any Microsoft shop (I'm not sure there was a Microsoft shop, it was in 
> the last millenium, Python 1.5 or 1.6). So I tend to forget that 
> possible download source.
> 
> But in all this thread I didn't see a single explanation for my current 
> situation: one and the same shebang line works on Windows 10 / Python 
> 3.11 and doesn't work on Windows 11 / Python 3.12. I suspect Windows, 
> because a change in the way Python 3.12 uses shebang lines should be 
> visible in the documentation.

The shebang support in the Python Launcher is documented here:

https://docs.python.org/3/using/windows.html#shebang-lines

That says the line you list originally:

 > My shebang line is usually "#!/usr/bin/env python3"

means look for python3 in PATH.  Do you have one? If you don't have one, 
you'll get one you don't want: the stupid Microsoft shim that, which if 
run interactively, encourages you to install from the Microsoft store. 
You should be able to disable this.

File suffix associations are a different thing - they give me no end of 
headaches on Windows. They start out bound to the shim, and should 
rebind to the launcher when you install, but then things can steal it. 
If you install Visual Studio Code with Python extensions, then it takes 
over the running of .py files - if you click in the explorer, you'll get 
it open in the editor, not run.  I've argued about this, to no avail 
(plays havoc with my testsuite, which in some places tries to execute 
Python scripts as a cli command).

And then I've got this:

C:\Users\mats\SOMEWHERE>py -0
  -V:3.13          Python 3.13 (64-bit)
  -V:3.12 *        Python 3.12 (64-bit)
  -V:3.11          Python 3.11 (64-bit)
  -V:3.10          Python 3.10 (64-bit)
  -V:3.9           Python 3.9 (64-bit)
  -V:3.8           Python 3.8 (64-bit)
  -V:3.7           Python 3.7 (64-bit)
  -V:3.6           Python 3.6 (64-bit)

# Okay, it knows about lots of Python versions, and shows a default of 3.12

C:\Users\mats\SOMEWHERE>py
Python 3.12.1 (tags/v3.12.1:2305ca5, Dec  7 2023, 22:03:25) [MSC v.1937 
64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> ^Z

# Great, that works just as expected

C:\Users\mats\SOMEWHERE>py test.py
Python was not found; run without arguments to install from the 
Microsoft Store, or disable this shortcut from Settings > Manage App 
Execution Aliases.

# wait, what? if "py" worked, why doesn't "py test.py"?



From mats at wichmann.us  Mon Jan  1 12:26:16 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Mon, 1 Jan 2024 10:26:16 -0700
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <784bcbbe-f683-4daf-bf57-11c60aedc5b8@tompassin.net>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
 <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <1d191291-f587-426f-83b7-2baa2564c1dc@tompassin.net>
 <784bcbbe-f683-4daf-bf57-11c60aedc5b8@tompassin.net>
Message-ID: <babe94f6-b622-42fa-b9f7-8d0bfec00015@wichmann.us>

On 1/1/24 07:11, Thomas Passin via Python-list wrote:

> Here's how to find out what program Windows thinks it should use to run 
> a ".py" file.? In a console:
> 
> C:\Users\tom>assoc .py
> .py=Python.File
> 
> C:\Users\tom>ftype Python.file
> Python.file="C:\Windows\py.exe" "%L" %*

That's not enough. There is now (has been for a while) a layered system, 
and this gives you just one layer, there may be other associations that 
win out.

Per somebody who actually knows:

 > The only way to determine the association without reimplmenting the 
shell's search is to simply ask the shell via AssocQueryString. Possibly 
PowerShell can provide this information. ? Eryk Sun



From list1 at tompassin.net  Mon Jan  1 14:53:07 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 1 Jan 2024 14:53:07 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <babe94f6-b622-42fa-b9f7-8d0bfec00015@wichmann.us>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
 <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <1d191291-f587-426f-83b7-2baa2564c1dc@tompassin.net>
 <784bcbbe-f683-4daf-bf57-11c60aedc5b8@tompassin.net>
 <babe94f6-b622-42fa-b9f7-8d0bfec00015@wichmann.us>
Message-ID: <b86acfa6-da7a-483e-9872-0ce8a2643229@tompassin.net>

On 1/1/2024 12:26 PM, Mats Wichmann via Python-list wrote:
> On 1/1/24 07:11, Thomas Passin via Python-list wrote:
> 
>> Here's how to find out what program Windows thinks it should use to 
>> run a ".py" file.? In a console:
>>
>> C:\Users\tom>assoc .py
>> .py=Python.File
>>
>> C:\Users\tom>ftype Python.file
>> Python.file="C:\Windows\py.exe" "%L" %*
> 
> That's not enough. There is now (has been for a while) a layered system, 
> and this gives you just one layer, there may be other associations that 
> win out.
> 
> Per somebody who actually knows:
> 
>  > The only way to determine the association without reimplmenting the 
> shell's search is to simply ask the shell via AssocQueryString. Possibly 
> PowerShell can provide this information. ? Eryk Sun

"Possibly", eh?  In fact, on my system those layers must be in effect, 
since ftype claims that the "py" launcher will be used but in actual 
fact the old Python 3.9.9 is used instead, as I wrote earlier.  This is 
verified by this tiny Python script:

# Optional shebang line here
import sys
print(sys.executable)

Then run it with "py", a proposed shebang line, its plain name on the 
command line, whatever.  That will tell you for sure which Python 
executable gets launched by which technique.

On Windows 10, a shebang line gets ignored in favor of Python 3.9.9 (if 
invoked by the script name alone) or Python 3.12.1 (if invoked by the 
"py" launcher).


From olegsivokon at gmail.com  Mon Jan  1 20:16:14 2024
From: olegsivokon at gmail.com (Left Right)
Date: Tue, 2 Jan 2024 02:16:14 +0100
Subject: What is Install-Paths-To in WHEEL file?
In-Reply-To: <CAHVvXxQksQmdnxKtmmwTqEVzsfgpfW726DadohRXPu3egDievw@mail.gmail.com>
References: <CAJQBtgkFLix4GJGQXbemTetA4Bw=VS_SWktOJE5_T3bDtwWgeg@mail.gmail.com>
 <EAC436D9-002D-4762-BCE4-841BD00B7A64@barrys-emacs.org>
 <CAJQBtg=TDvmq7VMWfwS9tNNfFh1umWNaPUQrsEywahVB+dsobw@mail.gmail.com>
 <CAJQBtgk1Th9kA6KVq4TS1E4ZKs7HmV_pFmz4Xba181fX+Usywg@mail.gmail.com>
 <CAHVvXxR4usiht8G7h1CucyuKm+4os=kzFYijLKzHQTqNz8GdOw@mail.gmail.com>
 <CAJQBtgnOr=8eDBE8PavHo3+Yz_faznrYOAq3+tioHZV+x+zC9A@mail.gmail.com>
 <CAPTjJmp_vmhQ9k2XWQD=XWvtnrVJp8oBc1=G7k4PNg3xscTbsA@mail.gmail.com>
 <CAJQBtgmXfHD3p4pZ5HwR2tnoFQDGbA=p1WpDUbu-2kiZ0Cba+w@mail.gmail.com>
 <CAPTjJmrFunkr18kjJGST-j6FRybPhF8vsKuMVN4c0Hc3sxva4A@mail.gmail.com>
 <CAJQBtgmR1-BtTFVC5ah7weeHXL1ZxB8YAC-Nexh961HRBrgjNA@mail.gmail.com>
 <CAPTjJmq6ZxCe-f8e_Bah+fyD6HczaEZ7xuRZ5r9gH=E2+MDP0A@mail.gmail.com>
 <CAJQBtgnKzn=XR=6nmHiMPpEGzt=ixpaUxZi300S9=bqxqtN7vQ@mail.gmail.com>
 <CAHVvXxTBXU2UooWuYuR_1fW=XB54y19zKnzwGSgzcWT1FzqCiA@mail.gmail.com>
 <CAJQBtg=bt1-NE_a7-5Yq09eTLtZeWU3vpyEJGSQsc4ApYJyHFg@mail.gmail.com>
 <CAHVvXxQksQmdnxKtmmwTqEVzsfgpfW726DadohRXPu3egDievw@mail.gmail.com>
Message-ID: <CAJQBtg=SPew74ytZzcoXmKbaB65cepB39xh7fJkFPLAtKX1O4g@mail.gmail.com>

> others do not and so your notion of what is "accepted"
> is not universally shared.

Why should I or anyone else care about what "others" think?  The
important question is whether what I do is right. And the answer is
"yes". That's why there are rules in the first place instead of
polling.

> if you want to influence anything

Usually, when I interact with representatives of Python community I
have two goals:

1. Typically, I need to show to someone who's paying my salary why
something produced by this community doesn't work. I.e. say, I need to
convince a project manager on a project I'm helping maintain that
deploying using "pip install" is a bad idea. I write an explanation
which I share with the PM and the PyPA people in the bug tracker.
They predictably block me out of fear or frustration.  This gives me a
proof that the thing doesn't work (well), and I'm allowed to do it the
right way. Just like in your previous remark: majority could be a good
initial heuristic, but proof is still a lot better.

2. At this point, I have no hope of convincing the prominent members
of Python community how awful a lot of their decisions are.  There are
plenty of socially constructed obstacles on this way.  The reason I do
this is posterity.  There are plenty of people who aren't influenced
by the internal developments of Python community (outside of it) and
they can see much of its development for what it is: commenting on
this development honestly will help them make an informed choice.
It's also important that those who will come after us will learn about
this contradiction.  Too many bad projects with bad design outlived
their good counterparts due to popularity caused by chance.  And today
those better design and ideas are as good as lost.  For example, Unix
outlived and "overpowered" plenty of better operating systems of its
time. But most programmers today would have no idea what those systems
were and how they were different.  Similarly, x86 ISA.  And plenty
more.

Python changed from its early days of trying to be funny and generally
welcoming of many contradicting ideas and opinions into a Lord of the
Flies community that no longer tolerates differences of opinion.  It's
lost the spirit of "playful cleverness" (as RMS would put it), and
became a "don't think, do as I say" community. I want to make sure
those who come to learn about Python will not miss this aspect of its
history.

From ethan at stoneleaf.us  Tue Jan  2 11:23:46 2024
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 2 Jan 2024 08:23:46 -0800
Subject: [dead thread] Re: What is Install-Paths-To in WHEEL file?
In-Reply-To: <CAJQBtgkFLix4GJGQXbemTetA4Bw=VS_SWktOJE5_T3bDtwWgeg@mail.gmail.com>
References: <CAJQBtgkFLix4GJGQXbemTetA4Bw=VS_SWktOJE5_T3bDtwWgeg@mail.gmail.com>
Message-ID: <7e1da981-52cd-aa9c-8f75-e5743e6bc7b1@stoneleaf.us>

This thread is no longer being useful, and is now closed.

--
~Ethan~
Moderator

From ethan at stoneleaf.us  Tue Jan  2 11:29:31 2024
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 2 Jan 2024 08:29:31 -0800
Subject: CoC Warning [was: What is Install-Paths-To in WHEEL file?]
In-Reply-To: <CAJQBtgk1Th9kA6KVq4TS1E4ZKs7HmV_pFmz4Xba181fX+Usywg@mail.gmail.com>
References: <CAJQBtgkFLix4GJGQXbemTetA4Bw=VS_SWktOJE5_T3bDtwWgeg@mail.gmail.com>
 <EAC436D9-002D-4762-BCE4-841BD00B7A64@barrys-emacs.org>
 <CAJQBtg=TDvmq7VMWfwS9tNNfFh1umWNaPUQrsEywahVB+dsobw@mail.gmail.com>
 <CAJQBtgk1Th9kA6KVq4TS1E4ZKs7HmV_pFmz4Xba181fX+Usywg@mail.gmail.com>
Message-ID: <bc74b76e-5900-935b-1871-11bb32508edf@stoneleaf.us>

On 12/29/23 05:02, Left Right via Python-list wrote:

> Wow. That place turned out to be the toxic pit I didn't expect.
> 
> It's a shame that a public discussion of public goods was entrusted to
> a bunch of gatekeepers with no sense of responsibility for the thing
> they keep the keys to.

Personal attacks are not welcome nor tolerated.  Please be constructive when engaging with the community.

--
~Ethan~

From mats at wichmann.us  Tue Jan  2 11:56:54 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Tue, 2 Jan 2024 09:56:54 -0700
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <b86acfa6-da7a-483e-9872-0ce8a2643229@tompassin.net>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
 <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <1d191291-f587-426f-83b7-2baa2564c1dc@tompassin.net>
 <784bcbbe-f683-4daf-bf57-11c60aedc5b8@tompassin.net>
 <babe94f6-b622-42fa-b9f7-8d0bfec00015@wichmann.us>
 <b86acfa6-da7a-483e-9872-0ce8a2643229@tompassin.net>
Message-ID: <4314850c-8f0a-45f6-85d8-61f8243a47fb@wichmann.us>

On 1/1/24 12:53, Thomas Passin via Python-list wrote:

> On Windows 10, a shebang line gets ignored in favor of Python 3.9.9 (if 
> invoked by the script name alone) or Python 3.12.1 (if invoked by the 
> "py" launcher).

fwiw, you can also create an ini file to define to the launcher py which 
version should be the default, if no version is specified.




From list1 at tompassin.net  Tue Jan  2 12:24:05 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 2 Jan 2024 12:24:05 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <4314850c-8f0a-45f6-85d8-61f8243a47fb@wichmann.us>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
 <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <1d191291-f587-426f-83b7-2baa2564c1dc@tompassin.net>
 <784bcbbe-f683-4daf-bf57-11c60aedc5b8@tompassin.net>
 <babe94f6-b622-42fa-b9f7-8d0bfec00015@wichmann.us>
 <b86acfa6-da7a-483e-9872-0ce8a2643229@tompassin.net>
 <4314850c-8f0a-45f6-85d8-61f8243a47fb@wichmann.us>
Message-ID: <2ecb2263-7b28-49f4-af50-ca25353ed33b@tompassin.net>

On 1/2/2024 11:56 AM, Mats Wichmann via Python-list wrote:
> On 1/1/24 12:53, Thomas Passin via Python-list wrote:
> 
>> On Windows 10, a shebang line gets ignored in favor of Python 3.9.9 
>> (if invoked by the script name alone) or Python 3.12.1 (if invoked by 
>> the "py" launcher).
> 
> fwiw, you can also create an ini file to define to the launcher py which 
> version should be the default, if no version is specified.

You might learn about this if you happen to read and remember the right 
part of the Python docs.  Otherwise you have no idea what py.exe is up 
to nor how it does it.  I would say that most people don't know there's 
an ini file, let alone what it can do.  Of course this situation isn't 
unique to py.exe!


From barry at barrys-emacs.org  Wed Jan  3 11:54:25 2024
From: barry at barrys-emacs.org (Barry Scott)
Date: Wed, 3 Jan 2024 16:54:25 +0000
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <2ecb2263-7b28-49f4-af50-ca25353ed33b@tompassin.net>
References: <f53edfe0-d67b-45a8-8604-a3d89734af58@web.de>
 <b22706a9-7200-446f-8a10-587f988a14c1@tompassin.net>
 <a253ca1e-47b2-0b75-bffe-cd04a97bde05@gmail.com>
 <0fdc10a9-a9c1-4f34-8b78-573c3ef18a7c@tompassin.net>
 <aba514c0-6352-5da1-23be-dd846e2223ef@gmail.com>
 <D450A6EE-5EC3-4CCA-BCEA-C91DAF27D723@dewhirst.com.au>
 <mailman.54.1703479290.2892.python-list@python.org>
 <6e856c31-c8aa-4db6-a14a-0c1d2e4e3d8f@gmail.com>
 <a08186cb-b133-4569-8c85-5429b620b6e5@dewhirst.com.au>
 <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <1d191291-f587-426f-83b7-2baa2564c1dc@tompassin.net>
 <784bcbbe-f683-4daf-bf57-11c60aedc5b8@tompassin.net>
 <babe94f6-b622-42fa-b9f7-8d0bfec00015@wichmann.us>
 <b86acfa6-da7a-483e-9872-0ce8a2643229@tompassin.net>
 <4314850c-8f0a-45f6-85d8-61f8243a47fb@wichmann.us>
 <2ecb2263-7b28-49f4-af50-ca25353ed33b@tompassin.net>
Message-ID: <4276C42C-9289-4FA3-BF0F-1033A2E54CD4@barrys-emacs.org>



> On 2 Jan 2024, at 17:24, Thomas Passin via Python-list <python-list at python.org> wrote:
> 
> You might learn about this if you happen to read and remember the right part of the Python docs.  Otherwise you have no idea what py.exe is up to nor how it does it.  I would say that most people don't know there's an ini file, let alone what it can do.  Of course this situation isn't unique to py.exe!

On discuss.python.org we will share the link to the python windows docs and
often to this specific section: https://docs.python.org/3/using/windows.html#python-launcher-for-windows

To create a py.ini does require a user to understand what %localappdata% means.
Some windows users, reasonable, do not use the CLI and know about that syntax for environment variables.

Barry



From guenther.sohler at gmail.com  Wed Jan  3 17:47:20 2024
From: guenther.sohler at gmail.com (Guenther Sohler)
Date: Wed, 3 Jan 2024 23:47:20 +0100
Subject: Using my routines as functions AND methods
Message-ID: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>

Hi,

In my cpython i have written quite some functions to modify "objects".
and their python syntax is e.g.\

translate(obj, vec). e.g whereas obj is ALWAYS first argument.

on c side this functions looks like:
PyObject *python_translate(PyObject *self, PyObject *args, PyObject *kwargs)

this works great when specifying a list of functions in PyModuleDef
structure.

However, I also want to use these functions as class methods without having
to
write the function , twice. When using the SAME function as a methos, the
args tuple must insert/contain "self" in the first location, so i have
written a function to do that:

PyObject *python_oo_args(PyObject *self, PyObject *args) // returns new
reference,
{
  int i;
  PyObject *item;
  int n = PyTuple_Size(args);
  PyObject *new_args = PyTuple_New(n + 1);
  PyTuple_SetItem(new_args, 0, self);

  for (i = 0; i < PyTuple_Size(args); i++) {
    item = PyTuple_GetItem(args, i);
    PyTuple_SetItem(new_args, i + 1, item);
  }
  return new_args;
}

To fill in method array, i have created a #define like this:

#define OO_METHOD_ENTRY(name,desc) \
  { #name, (PyCFunction) ( [ ] (PyObject *self, PyObject *args) -> PyObject
* { \
  PyObject *new_args = python_oo_args(self, args); \
  PyObject *result = python_##name(self, new_args, NULL); \
  return result;  } ),  METH_VARARGS | METH_KEYWORDS, (desc)},

(this uses a lambda function)

and use this in the array as:

PyMethodDef PyOpenSCADMethods[] = {
  OO_METHOD_ENTRY(translate,"Move Object")
  OO_METHOD_ENTRY(right,"Right Object")

Using this  i can reuse all the functions as methods,
but its not 100% stable/bulletproof and crashes sometimes.
So there is the bug ?
Is there a better approach to reach my goal ?


thank you

From learn2program at gmail.com  Wed Jan  3 20:00:54 2024
From: learn2program at gmail.com (Alan Gauld)
Date: Thu, 4 Jan 2024 01:00:54 +0000
Subject: Using my routines as functions AND methods
In-Reply-To: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
References: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
Message-ID: <62297ce5-c36b-afac-c2f8-6c4f0b412875@yahoo.co.uk>

On 03/01/2024 22:47, Guenther Sohler via Python-list wrote:
> Hi,
> 
> In my cpython i have written quite some functions to modify "objects".
> and their python syntax is e.g.\
> 
> translate(obj, vec). e.g whereas obj is ALWAYS first argument.

> However, I also want to use these functions as class methods without having
> to
> write the function , twice. When using the SAME function as a methos, the
> args tuple must insert/contain "self" in the first location, so i have
> written a function to do that:

I'm probably missing something obvious here but can't you
just assign your function to a class member?

def myFunction(obj, ...): ...

class MyClass:
    myMethod = myFunction


Then you can call it as

myObject = MyClass()
myObject.myMethod()

A naive example seems to work but I haven't tried anything
complex so there is probably a catch. But sometimes the simple
things just work?

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From list1 at tompassin.net  Thu Jan  4 00:11:16 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Thu, 4 Jan 2024 00:11:16 -0500
Subject: Using my routines as functions AND methods
In-Reply-To: <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>
References: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
 <62297ce5-c36b-afac-c2f8-6c4f0b412875@yahoo.co.uk>
 <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>
Message-ID: <a9aa7ec0-be49-4081-80a8-63ef9dee0326@tompassin.net>

On 1/3/2024 11:17 PM, Thomas Passin wrote:
> On 1/3/2024 8:00 PM, Alan Gauld via Python-list wrote:
>> On 03/01/2024 22:47, Guenther Sohler via Python-list wrote:
>>> Hi,
>>>
>>> In my cpython i have written quite some functions to modify "objects".
>>> and their python syntax is e.g.\
>>>
>>> translate(obj, vec). e.g whereas obj is ALWAYS first argument.
>>
>>> However, I also want to use these functions as class methods without 
>>> having
>>> to
>>> write the function , twice. When using the SAME function as a methos, 
>>> the
>>> args tuple must insert/contain "self" in the first location, so i have
>>> written a function to do that:
>>
>> I'm probably missing something obvious here but can't you
>> just assign your function to a class member?
>>
>> def myFunction(obj, ...): ...
>>
>> class MyClass:
>> ???? myMethod = myFunction
>>
>>
>> Then you can call it as
>>
>> myObject = MyClass()
>> myObject.myMethod()
>>
>> A naive example seems to work but I haven't tried anything
>> complex so there is probably a catch. But sometimes the simple
>> things just work?
> 
> That works if you assign the function to a class instance, but not if 
> you assign it to a class.
> 
> def f1(x):
>  ??? print(x)
> f1('The plain function')
> 
> class Class1:
>  ??? pass
> 
> class Class2:
>  ??? pass
> 
> c1 = Class1()
> c1.newfunc = f1
> c1.newfunc('f1 assigned to instance') # Works as intended
> 
> Class2.newfunc = f1
> c2 = Class2()
> c2.newfunc('f1 assigned to class')? # Complains about extra argument

If your requirements are not very tricky, you can write a 
convert-to-method function yourself:

def f1(x):
     print(x)
f1('The plain function')

class Class2:
     pass

def convert_method(f):
     """Assign existing method without a "self" arg
        as a class's method.
     """
     def fnew(instance, *args):
         f(*args)
     return fnew

Class2.newfunc = convert_method(f1)
c2 = Class2()
c2.newfunc('f1 assigned as method of Class2') # Prints the arg

This example does not make f1 aware of the self argument, but you asked 
to convert an existing function, and that function would not be aware of 
the self parameter. It's much like a decorator function, but is not here 
being used as a decorator. If you meant something else, please think out 
what you want and explain that.


From list1 at tompassin.net  Thu Jan  4 00:11:16 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Thu, 4 Jan 2024 00:11:16 -0500
Subject: Using my routines as functions AND methods
In-Reply-To: <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>
References: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
 <62297ce5-c36b-afac-c2f8-6c4f0b412875@yahoo.co.uk>
 <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>
Message-ID: <a9aa7ec0-be49-4081-80a8-63ef9dee0326@tompassin.net>

On 1/3/2024 11:17 PM, Thomas Passin wrote:
> On 1/3/2024 8:00 PM, Alan Gauld via Python-list wrote:
>> On 03/01/2024 22:47, Guenther Sohler via Python-list wrote:
>>> Hi,
>>>
>>> In my cpython i have written quite some functions to modify "objects".
>>> and their python syntax is e.g.\
>>>
>>> translate(obj, vec). e.g whereas obj is ALWAYS first argument.
>>
>>> However, I also want to use these functions as class methods without 
>>> having
>>> to
>>> write the function , twice. When using the SAME function as a methos, 
>>> the
>>> args tuple must insert/contain "self" in the first location, so i have
>>> written a function to do that:
>>
>> I'm probably missing something obvious here but can't you
>> just assign your function to a class member?
>>
>> def myFunction(obj, ...): ...
>>
>> class MyClass:
>> ???? myMethod = myFunction
>>
>>
>> Then you can call it as
>>
>> myObject = MyClass()
>> myObject.myMethod()
>>
>> A naive example seems to work but I haven't tried anything
>> complex so there is probably a catch. But sometimes the simple
>> things just work?
> 
> That works if you assign the function to a class instance, but not if 
> you assign it to a class.
> 
> def f1(x):
>  ??? print(x)
> f1('The plain function')
> 
> class Class1:
>  ??? pass
> 
> class Class2:
>  ??? pass
> 
> c1 = Class1()
> c1.newfunc = f1
> c1.newfunc('f1 assigned to instance') # Works as intended
> 
> Class2.newfunc = f1
> c2 = Class2()
> c2.newfunc('f1 assigned to class')? # Complains about extra argument

If your requirements are not very tricky, you can write a 
convert-to-method function yourself:

def f1(x):
     print(x)
f1('The plain function')

class Class2:
     pass

def convert_method(f):
     """Assign existing method without a "self" arg
        as a class's method.
     """
     def fnew(instance, *args):
         f(*args)
     return fnew

Class2.newfunc = convert_method(f1)
c2 = Class2()
c2.newfunc('f1 assigned as method of Class2') # Prints the arg

This example does not make f1 aware of the self argument, but you asked 
to convert an existing function, and that function would not be aware of 
the self parameter. It's much like a decorator function, but is not here 
being used as a decorator. If you meant something else, please think out 
what you want and explain that.


From guenther.sohler at gmail.com  Thu Jan  4 02:38:28 2024
From: guenther.sohler at gmail.com (Guenther Sohler)
Date: Thu, 4 Jan 2024 08:38:28 +0100
Subject: Using my routines as functions AND methods
In-Reply-To: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
References: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
Message-ID: <CAASeUHow--wWcpSZvB+u=nQ9ZqfOb5sRCTB_wT0un5m9zhL3-w@mail.gmail.com>

Thank you for your answers.
apparently I did not express myself clear enough. Let me rephrase.

I got an embedded  C function like this:

==
PyObject *python_translate(PyObject *self, PyObject *args, PyObject *kwargs)
{
  char *kwlist[] = {"obj", "v", NULL};
  PyObject *v = NULL;
  PyObject *obj = NULL;
  double x = 0, y = 0, z = 0;
  if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO", kwlist,
                                   &obj,
                                   &v
                                   )) {
    PyErr_SetString(PyExc_TypeError, "Error during parsing
translate(object,vec3)");
    return NULL;
  }
...

PyMethodDef PyOpenSCADFunctions[] = {
  {"translate", (PyCFunction) python_translate, METH_VARARGS |
METH_KEYWORDS, "Move  Object."},
 ...

static PyModuleDef OpenSCADModule = {
  PyModuleDef_HEAD_INIT,
  "openscad",
  "OpenSCAD Python Module",
  -1,
  PyOpenSCADFunctions,
...
};

============
>From within python i can successfully use it with

translate(my_object, [2,0,0])

my_object is an instance of embedded class PyOpenSCADType, so alternatively
I want to use my function as method too, like so:

my_object.translate([2,0,0])

WITHOUT duplicating the code for python_translate

So the idea is to call the translate-function from within the translate
method and insert "self" into the args tuple.
However my solution somewhat works but is not very stable and quite prone
to crash.

Which are my alertnatives ?


On Wed, Jan 3, 2024 at 11:47?PM Guenther Sohler <guenther.sohler at gmail.com>
wrote:

> Hi,
>
> In my cpython i have written quite some functions to modify "objects".
> and their python syntax is e.g.\
>
> translate(obj, vec). e.g whereas obj is ALWAYS first argument.
>
> on c side this functions looks like:
> PyObject *python_translate(PyObject *self, PyObject *args, PyObject
> *kwargs)
>
> this works great when specifying a list of functions in PyModuleDef
> structure.
>
> However, I also want to use these functions as class methods without
> having to
> write the function , twice. When using the SAME function as a methos, the
> args tuple must insert/contain "self" in the first location, so i have
> written a function to do that:
>
> PyObject *python_oo_args(PyObject *self, PyObject *args) // returns new
> reference,
> {
>   int i;
>   PyObject *item;
>   int n = PyTuple_Size(args);
>   PyObject *new_args = PyTuple_New(n + 1);
>   PyTuple_SetItem(new_args, 0, self);
>
>   for (i = 0; i < PyTuple_Size(args); i++) {
>     item = PyTuple_GetItem(args, i);
>     PyTuple_SetItem(new_args, i + 1, item);
>   }
>   return new_args;
> }
>
> To fill in method array, i have created a #define like this:
>
> #define OO_METHOD_ENTRY(name,desc) \
>   { #name, (PyCFunction) ( [ ] (PyObject *self, PyObject *args) ->
> PyObject * { \
>   PyObject *new_args = python_oo_args(self, args); \
>   PyObject *result = python_##name(self, new_args, NULL); \
>   return result;  } ),  METH_VARARGS | METH_KEYWORDS, (desc)},
>
> (this uses a lambda function)
>
> and use this in the array as:
>
> PyMethodDef PyOpenSCADMethods[] = {
>   OO_METHOD_ENTRY(translate,"Move Object")
>   OO_METHOD_ENTRY(right,"Right Object")
>
> Using this  i can reuse all the functions as methods,
> but its not 100% stable/bulletproof and crashes sometimes.
> So there is the bug ?
> Is there a better approach to reach my goal ?
>
>
> thank you
>
>

From list1 at tompassin.net  Wed Jan  3 23:17:34 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 3 Jan 2024 23:17:34 -0500
Subject: Using my routines as functions AND methods
In-Reply-To: <62297ce5-c36b-afac-c2f8-6c4f0b412875@yahoo.co.uk>
References: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
 <62297ce5-c36b-afac-c2f8-6c4f0b412875@yahoo.co.uk>
Message-ID: <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>

On 1/3/2024 8:00 PM, Alan Gauld via Python-list wrote:
> On 03/01/2024 22:47, Guenther Sohler via Python-list wrote:
>> Hi,
>>
>> In my cpython i have written quite some functions to modify "objects".
>> and their python syntax is e.g.\
>>
>> translate(obj, vec). e.g whereas obj is ALWAYS first argument.
> 
>> However, I also want to use these functions as class methods without having
>> to
>> write the function , twice. When using the SAME function as a methos, the
>> args tuple must insert/contain "self" in the first location, so i have
>> written a function to do that:
> 
> I'm probably missing something obvious here but can't you
> just assign your function to a class member?
> 
> def myFunction(obj, ...): ...
> 
> class MyClass:
>      myMethod = myFunction
> 
> 
> Then you can call it as
> 
> myObject = MyClass()
> myObject.myMethod()
> 
> A naive example seems to work but I haven't tried anything
> complex so there is probably a catch. But sometimes the simple
> things just work?

That works if you assign the function to a class instance, but not if 
you assign it to a class.

def f1(x):
     print(x)
f1('The plain function')

class Class1:
     pass

class Class2:
     pass

c1 = Class1()
c1.newfunc = f1
c1.newfunc('f1 assigned to instance') # Works as intended

Class2.newfunc = f1
c2 = Class2()
c2.newfunc('f1 assigned to class')  # Complains about extra argument



From learn2program at gmail.com  Thu Jan  4 05:44:26 2024
From: learn2program at gmail.com (Alan Gauld)
Date: Thu, 4 Jan 2024 10:44:26 +0000
Subject: Using my routines as functions AND methods
In-Reply-To: <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>
References: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
 <62297ce5-c36b-afac-c2f8-6c4f0b412875@yahoo.co.uk>
 <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>
Message-ID: <e4b89a09-5bcb-3b66-ce3c-87861b9a55c6@yahoo.co.uk>

On 04/01/2024 04:17, Thomas Passin via Python-list wrote:

>> I'm probably missing something obvious here but can't you
>> just assign your function to a class member?
>>
>> def myFunction(obj, ...): ...
>>
>> class MyClass:
>>      myMethod = myFunction
> 
> That works if you assign the function to a class instance, but not if 
> you assign it to a class.
> 
> def f1(x):
>      print(x)
> f1('The plain function')
> 
> class Class1:
>      pass
> 
> class Class2:
>      pass
> 
> c1 = Class1()
> c1.newfunc = f1
> c1.newfunc('f1 assigned to instance') # Works as intended
> 
> Class2.newfunc = f1
> c2 = Class2()
> c2.newfunc('f1 assigned to class')  # Complains about extra argument

Yes but I did the assignment inside the class definition and
that seemed to work just fine:

>>> def g(obj, st): print(st, obj.v)
...
>>> class D:
...    def __init__(self,v): self.v = v
...    m = g
...
>>> d = D(66)
>>> g(d,'val = ')
val =  66
>>> d.m('v = ')
v =  66


-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos



From guenther.sohler at gmail.com  Thu Jan  4 06:51:48 2024
From: guenther.sohler at gmail.com (Guenther Sohler)
Date: Thu, 4 Jan 2024 12:51:48 +0100
Subject: Using my routines as functions AND methods
In-Reply-To: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
References: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
Message-ID: <CAASeUHqCxg58LrD7sk1c-yMxYEUkApAk2cZfaUYBRTudHMzxuA@mail.gmail.com>

Hi list

The approach with defining the methods from python appears to be a very
good idea and it also works for classes, defined in python side.
However, when I try this one:

def mytrans(self):
    print(self)

c=cube()
cls=c.__class__

cls.trans=mytrans

I get this:

Traceback (most recent call last):

File "<string>", line 8, in <module>

TypeError: cannot set 'trans' attribute of immutable type 'PyOpenSCAD'

The Problem is probably that PyOpenSCAD is a class not defined on python
side but on c++ side,
 I fear that I need to solution in c++ side. ...



On Wed, Jan 3, 2024 at 11:47?PM Guenther Sohler <guenther.sohler at gmail.com>
wrote:

> Hi,
>
> In my cpython i have written quite some functions to modify "objects".
> and their python syntax is e.g.\
>
> translate(obj, vec). e.g whereas obj is ALWAYS first argument.
>
> on c side this functions looks like:
> PyObject *python_translate(PyObject *self, PyObject *args, PyObject
> *kwargs)
>
> this works great when specifying a list of functions in PyModuleDef
> structure.
>
> However, I also want to use these functions as class methods without
> having to
> write the function , twice. When using the SAME function as a methos, the
> args tuple must insert/contain "self" in the first location, so i have
> written a function to do that:
>
> PyObject *python_oo_args(PyObject *self, PyObject *args) // returns new
> reference,
> {
>   int i;
>   PyObject *item;
>   int n = PyTuple_Size(args);
>   PyObject *new_args = PyTuple_New(n + 1);
>   PyTuple_SetItem(new_args, 0, self);
>
>   for (i = 0; i < PyTuple_Size(args); i++) {
>     item = PyTuple_GetItem(args, i);
>     PyTuple_SetItem(new_args, i + 1, item);
>   }
>   return new_args;
> }
>
> To fill in method array, i have created a #define like this:
>
> #define OO_METHOD_ENTRY(name,desc) \
>   { #name, (PyCFunction) ( [ ] (PyObject *self, PyObject *args) ->
> PyObject * { \
>   PyObject *new_args = python_oo_args(self, args); \
>   PyObject *result = python_##name(self, new_args, NULL); \
>   return result;  } ),  METH_VARARGS | METH_KEYWORDS, (desc)},
>
> (this uses a lambda function)
>
> and use this in the array as:
>
> PyMethodDef PyOpenSCADMethods[] = {
>   OO_METHOD_ENTRY(translate,"Move Object")
>   OO_METHOD_ENTRY(right,"Right Object")
>
> Using this  i can reuse all the functions as methods,
> but its not 100% stable/bulletproof and crashes sometimes.
> So there is the bug ?
> Is there a better approach to reach my goal ?
>
>
> thank you
>
>

From hjp-python at hjp.at  Sat Jan  6 11:53:43 2024
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 6 Jan 2024 17:53:43 +0100
Subject: Using my routines as functions AND methods
In-Reply-To: <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>
References: <CAASeUHrrsPPe2sqDFjZRLdYTuWoUg1uWNzKRz5QKAhA=T2E0ew@mail.gmail.com>
 <62297ce5-c36b-afac-c2f8-6c4f0b412875@yahoo.co.uk>
 <0f0b9909-8d94-44ca-bd3f-42cda17dd428@tompassin.net>
Message-ID: <20240106165343.7egadkv2rv6fzlqj@hjp.at>

On 2024-01-03 23:17:34 -0500, Thomas Passin via Python-list wrote:
> On 1/3/2024 8:00 PM, Alan Gauld via Python-list wrote:
> > On 03/01/2024 22:47, Guenther Sohler via Python-list wrote:
> > > Hi,
> > > 
> > > In my cpython i have written quite some functions to modify "objects".
> > > and their python syntax is e.g.\
> > > 
> > > translate(obj, vec). e.g whereas obj is ALWAYS first argument.
                ^^^                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > 
> > > However, I also want to use these functions as class methods without having
> > > to
> > > write the function , twice. When using the SAME function as a methos, the
> > > args tuple must insert/contain "self" in the first location, so i have
> > > written a function to do that:
> > 
> > I'm probably missing something obvious here but can't you
> > just assign your function to a class member?
> > 
> > def myFunction(obj, ...): ...
                   ^^^
> > 
> > class MyClass:
> >      myMethod = myFunction
> > 
> > 
> > Then you can call it as
> > 
> > myObject = MyClass()
> > myObject.myMethod()
> > 
> > A naive example seems to work but I haven't tried anything
> > complex so there is probably a catch. But sometimes the simple
> > things just work?
> 
> That works if you assign the function to a class instance, but not if you
> assign it to a class.
> 
> def f1(x):
>     print(x)

You omitted the first argument (obj).

That should be 

def f1(obj, x):
    print(x)


> f1('The plain function')

> 
> class Class1:
>     pass

o = Class1()
f1(o, 'The plain function')

works for me.


> class Class2:
>     pass
> 
> c1 = Class1()
> c1.newfunc = f1
> c1.newfunc('f1 assigned to instance') # Works as intended

Now this doesn't work any more (but the OP doesn't want that anyway,
AFAICT).


> Class2.newfunc = f1
> c2 = Class2()
> c2.newfunc('f1 assigned to class')  # Complains about extra argument

But this does.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20240106/97b2228c/attachment.sig>

From nulla.epistola at web.de  Sun Jan  7 10:09:35 2024
From: nulla.epistola at web.de (Sibylle Koczian)
Date: Sun, 7 Jan 2024 16:09:35 +0100
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <F1CB1FD3-4F43-484A-909D-18C20EE32762@barrys-emacs.org>
References: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <F1CB1FD3-4F43-484A-909D-18C20EE32762@barrys-emacs.org>
Message-ID: <f9532b2c-dcc6-40ae-a632-b3a39b5beb69@web.de>

Am 01.01.2024 um 12:50 schrieb Barry via Python-list:
> 
> 
>> On 1 Jan 2024, at 11:14, Sibylle Koczian via Python-list <python-list at python.org> wrote:
>>
>> But in all this thread I didn't see a single explanation for my current situation: one and the same shebang line works on Windows 10 / Python 3.11 and doesn't work on Windows 11 / Python 3.12. I suspect Windows, because a change in the way Python 3.12 uses shebang lines should be visible in the documentation.
> 
> See my earlier reply with info on ftype etc output.
> What do you see on your windows 10 vs windows 11?
> 

Oh yes, I forgot to answer, here are my results:

py -0:
1. Windows 10, Python 3.11.5:
  -V:3.11 *        Python 3.11 (64-bit)
  -V:3.6           Python 3.6 (64-bit) possibly part of Visual Studio or 
else Calibre?
2. Windows 11, Python 3.12.1;
  -V:3.12 *        Python 3.12 (64-bit)

assoc .py
1. Windows 10, Python 3.11.5:
.py=Python.File
2. Windows 11, Python 3.12.1:
.py=Python.File

ftype Python.File
1. Windows 10, Python 3.11.5:
Python.File="C:\WINDOWS\py.exe" "%L" %*
2. Windows 11, Python 3.12.1:
Python.file="C:\Windows\py.exe" "%L" %*

Test Script test_shebang.py starts with one of these shebang lines:
a) #! python
b) #! /usr/bin/python
c) #! /usr/bin/env python
import sys
print(sys.executable)

Results:
1. Windows 10, Python 3.11.5:
test_shebang.py, Version a): C:\Program Files\Python311\python.exe
py test_shebang.py, Version a): C:\Program Files\Python311\python.exe

test_shebang.py, Version b): C:\Program Files\Python311\python.exe
py test_shebang.py, Version b): C:\Program Files\Python311\python.exe

test_shebang.py, Version c): C:\Program Files\Python311\python.exe
py test_shebang.py, Version c): C:\Program Files\Python311\python.exe

2. Windows 11, Python 3.12.1:
test_shebang.py, Version a): C:\Program Files\Python312\python.exe
py test_shebang.py, Version a): C:\Program Files\Python312\python.exe

test_shebang.py, Version b): C:\Program Files\Python312\python.exe
py test_shebang.py, Version b): C:\Program Files\Python312\python.exe

test_shebang.py, Version c):
Python konnte nicht gefunden werden. F?hren Sie die Verkn?pfung ohne 
Argumente aus, um sie ?ber den Microsoft Store zu installieren, oder 
deaktivieren Sie diese Verkn?pfung unter

py test_shebang.py, Version c):
Python konnte nicht gefunden werden. F?hren Sie die Verkn?pfung ohne 
Argumente aus, um sie ?ber den Microsoft Store zu installieren, oder 
deaktivieren Sie diese Verkn?pfung unter

It's a german Windows version, first sentence: "Python could not be 
found". I don't really know how to translate the second sentence.

Oh, and the two Windows and Python versions are on two different computers.

Will remove the "/env" from my shebang lines, even if I don't understand 
what's happening.

Thank you for helping,
Sibylle



From barry at barrys-emacs.org  Tue Jan  9 06:36:03 2024
From: barry at barrys-emacs.org (Barry Scott)
Date: Tue, 9 Jan 2024 11:36:03 +0000
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <f9532b2c-dcc6-40ae-a632-b3a39b5beb69@web.de>
References: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <F1CB1FD3-4F43-484A-909D-18C20EE32762@barrys-emacs.org>
 <f9532b2c-dcc6-40ae-a632-b3a39b5beb69@web.de>
Message-ID: <65F119FD-4B27-4EB3-89FA-07B50F809596@barrys-emacs.org>



> On 7 Jan 2024, at 15:09, Sibylle Koczian via Python-list <python-list at python.org> wrote:
> 
> Oh, and the two Windows and Python versions are on two different computers.
> 
> Will remove the "/env" from my shebang lines, even if I don't understand what's happening.

Thanks for the details.

Only thing I can think of is that "python" may be defaulting to mean python 2.
If you use "#!/usr/bin/env python3" it may work on both.

Did you creates a py.ini file to configure py.exe?

See if you have %userappdata%\py.ini on either windows 10 or windows 11.
If so what is its contents?

I've tried with and without a py.ini and cannot duplicate what you see.

Barry


From guenther.sohler at gmail.com  Tue Jan  9 02:14:31 2024
From: guenther.sohler at gmail.com (Guenther Sohler)
Date: Tue, 9 Jan 2024 08:14:31 +0100
Subject: extend behaviour of assignment operator
Message-ID: <CAASeUHopbgPeqdC_bbvQSiVnTb0y=20BJz-2T-82Huakhu7Bdw@mail.gmail.com>

Hi,

when i run this code

a = cube([10,1,1])
b = a

i'd like to extend the behaviour  of the assignment operator
a shall not only contain the cube, but  the cube shall also know which
variable name it
was assigned to, lately. I'd like to use that for improved user interaction.

effective code should be:

a=cube([10,1,1])
a.name='a'

b=a
b.name='b' # i am aware that a.name also changes


can decorators also be used with assignment operators ?

thank you for your hints

From cyber at sysrq.in  Tue Jan  9 02:25:19 2024
From: cyber at sysrq.in (Anna (cybertailor) Vyalkova)
Date: Tue, 09 Jan 2024 12:25:19 +0500
Subject: ANN: repology-client library to access Repology API
Message-ID: <de4130dcc4b1a396fcd81f144c68bab542bcece6.camel@sysrq.in>

Hi newsgroup,

I needed to fetch Repology data for my pet project and now it's a
library:
https://pypi.org/project/repology-client/

It uses aiohttp, if that matters.

Feel free to use and contribute.


From dieter at handshake.de  Wed Jan 10 13:44:31 2024
From: dieter at handshake.de (Dieter Maurer)
Date: Wed, 10 Jan 2024 19:44:31 +0100
Subject: extend behaviour of assignment operator
In-Reply-To: <CAASeUHopbgPeqdC_bbvQSiVnTb0y=20BJz-2T-82Huakhu7Bdw@mail.gmail.com>
References: <CAASeUHopbgPeqdC_bbvQSiVnTb0y=20BJz-2T-82Huakhu7Bdw@mail.gmail.com>
Message-ID: <26014.58767.753936.352783@ixdm.fritz.box>

Guenther Sohler wrote at 2024-1-9 08:14 +0100:
>when i run this code
>
>a = cube([10,1,1])
>b = a
>
>i'd like to extend the behaviour  of the assignment operator
>a shall not only contain the cube, but  the cube shall also know which
>variable name it
>was assigned to, lately. I'd like to use that for improved user interaction.

`Acquisition` (--> `PyPI`) implements something similar.

It does not work for variables -- but for attribute access.
Look at the following code:

```
from Acquisition import Implicit

class AccessAwareContainer(Implicit):
  ...

class AccessAwareContent(Implicit):
 ...

container = AccessAwareContainer()
container.content = AccessAwareContent()
```

When you now assign `content = container.content`, then
`content` knows that it has been accessed via `container`.


If fact `content` is not a true `AccessAwareContent` instance
but a wrapper proxy for it. It mostly behaves like an
`AccessAwareContent` object but has additional information
(e.g. it knows the access parent).


It works via a special `__getattribute__` method, essentially
implemented by:

```
   def __getattribute__(self, k):
     v = super().__getattribute__(k)
     return v.__of__(self) if hasattr(v, "__of__") else v
```

Your use case could be implemented similarly (again not for variables
and all objects, but for special classes (and maybe special objects)).

Your `__getattribute__` could look like:
```
   def __getattribute__(self, k):
     v = super().__getattribute__(k)
     try:
       v.name = k
     except TypeError: pass
     return v
```

From rshepard at appl-ecosys.com  Thu Jan 11 13:08:09 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Thu, 11 Jan 2024 10:08:09 -0800 (PST)
Subject: Extract lines from file, add to new files
Message-ID: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>

It's been several years since I've needed to write a python script so I'm
asking for advice to get me started with a brief script to separate names
and email addresses in one file into two separate files: salutation.txt and
emails.txt.

An example of the input file:

Calvin
calvin at example.com

Hobbs
hobbs at some.com

Nancy
nancy at herown.com

Sluggo
sluggo at another.com

Having extracted salutations and addresses I'll write a bash script using
sed and mailx to associate a message file with each name and email address.

I'm unsure where to start given my lack of recent experience.

TIA,

Rich

From python at mrabarnett.plus.com  Thu Jan 11 13:27:31 2024
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 11 Jan 2024 18:27:31 +0000
Subject: Extract lines from file, add to new files
In-Reply-To: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
Message-ID: <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>

On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
> It's been several years since I've needed to write a python script so I'm
> asking for advice to get me started with a brief script to separate names
> and email addresses in one file into two separate files: salutation.txt and
> emails.txt.
> 
> An example of the input file:
> 
> Calvin
> calvin at example.com
> 
> Hobbs
> hobbs at some.com
> 
> Nancy
> nancy at herown.com
> 
> Sluggo
> sluggo at another.com
> 
> Having extracted salutations and addresses I'll write a bash script using
> sed and mailx to associate a message file with each name and email address.
> 
> I'm unsure where to start given my lack of recent experience.
> 
 From the look of it:

1. If the line is empty, ignore it.

2. If the line contains "@", it's an email address.

3. Otherwise, it's a name.


From rshepard at appl-ecosys.com  Thu Jan 11 13:44:23 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Thu, 11 Jan 2024 10:44:23 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
Message-ID: <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>

On Thu, 11 Jan 2024, MRAB via Python-list wrote:

> From the look of it:
> 1. If the line is empty, ignore it.
> 2. If the line contains "@", it's an email address.
> 3. Otherwise, it's a name.

MRAB,

Thanks. I'll take it from here.

Regards,

Rich

From mats at wichmann.us  Thu Jan 11 13:45:27 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Thu, 11 Jan 2024 11:45:27 -0700
Subject: Extract lines from file, add to new files
In-Reply-To: <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
Message-ID: <268794e7-154a-43c2-8ce1-a119978f4606@wichmann.us>

On 1/11/24 11:27, MRAB via Python-list wrote:
> On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
>> It's been several years since I've needed to write a python script so I'm
>> asking for advice to get me started with a brief script to separate names
>> and email addresses in one file into two separate files: 
>> salutation.txt and
>> emails.txt.
>>
>> An example of the input file:
>>
>> Calvin
>> calvin at example.com
>>
>> Hobbs
>> hobbs at some.com
>>
>> Nancy
>> nancy at herown.com
>>
>> Sluggo
>> sluggo at another.com
>>
>> Having extracted salutations and addresses I'll write a bash script using
>> sed and mailx to associate a message file with each name and email 
>> address.
>>
>> I'm unsure where to start given my lack of recent experience.
>>
>  From the look of it:
> 
> 1. If the line is empty, ignore it.
> 
> 2. If the line contains "@", it's an email address.
> 
> 3. Otherwise, it's a name.
> 

4. Don't assume it's going to be "plain text" if the email info is 
harvested from external sources (like incoming emails) - you'll end up 
stumbling over a ??????? from somewhere.  Process as bytes, or be really 
careful about which encodings you allow - which for email "names" is 
something you can't actually control.


From rshepard at appl-ecosys.com  Thu Jan 11 14:22:50 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Thu, 11 Jan 2024 11:22:50 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <268794e7-154a-43c2-8ce1-a119978f4606@wichmann.us>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <268794e7-154a-43c2-8ce1-a119978f4606@wichmann.us>
Message-ID: <cdf656d7-bd9-a73-2bd2-5f9ee8f6b6fe@appl-ecosys.com>

On Thu, 11 Jan 2024, Mats Wichmann via Python-list wrote:

> 4. Don't assume it's going to be "plain text" if the email info is
> harvested from external sources (like incoming emails) - you'll end up
> stumbling over a ??????? from somewhere. Process as bytes, or be
> really careful about which encodings you allow - which for email "names"
> is something you can't actually control.

Mats,

Not an issue for me.

Regards,

Rich

From piergiorgio.sartor.this.should.not.be.used at nexgo.REMOVETHIS.de  Thu Jan 11 13:23:32 2024
From: piergiorgio.sartor.this.should.not.be.used at nexgo.REMOVETHIS.de (Piergiorgio Sartor)
Date: Thu, 11 Jan 2024 19:23:32 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <mailman.3.1704996908.15798.python-list@python.org>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <mailman.3.1704996908.15798.python-list@python.org>
Message-ID: <4ao57k-bca.ln1@lazy.lzy>

On 11/01/2024 19.08, Rich Shepard wrote:
> It's been several years since I've needed to write a python script so I'm
> asking for advice to get me started with a brief script to separate names
> and email addresses in one file into two separate files: salutation.txt and
> emails.txt.
> 
> An example of the input file:
> 
> Calvin
> calvin at example.com
> 
> Hobbs
> hobbs at some.com
> 
> Nancy
> nancy at herown.com
> 
> Sluggo
> sluggo at another.com
> 
> Having extracted salutations and addresses I'll write a bash script using
> sed and mailx to associate a message file with each name and email address.

Why not to use bash script for all?

bye,

-- 

piergiorgio


From rshepard at appl-ecosys.com  Thu Jan 11 14:53:20 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Thu, 11 Jan 2024 11:53:20 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <4ao57k-bca.ln1@lazy.lzy>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <mailman.3.1704996908.15798.python-list@python.org> <4ao57k-bca.ln1@lazy.lzy>
Message-ID: <e3f12f-516c-3859-8d40-8f91bffea740@appl-ecosys.com>

On Thu, 11 Jan 2024, Piergiorgio Sartor via Python-list wrote:

> Why not to use bash script for all?

Piergiorgio,

That's certainly a possibility, and may well be better than python for this
task.

Thank you,

Rich

From mirkok.lists at googlemail.com  Thu Jan 11 15:25:26 2024
From: mirkok.lists at googlemail.com (Mirko)
Date: Thu, 11 Jan 2024 21:25:26 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <e3f12f-516c-3859-8d40-8f91bffea740@appl-ecosys.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <mailman.3.1704996908.15798.python-list@python.org> <4ao57k-bca.ln1@lazy.lzy>
 <e3f12f-516c-3859-8d40-8f91bffea740@appl-ecosys.com>
Message-ID: <5eb229c2-164e-193a-06a5-3b92e9ad8ead@googlemail.com>

Am 11.01.24 um 20:53 schrieb Rich Shepard via Python-list:
> On Thu, 11 Jan 2024, Piergiorgio Sartor via Python-list wrote:
> 
>> Why not to use bash script for all?
> 
> Piergiorgio,
> 
> That's certainly a possibility, and may well be better than python 
> for this
> task.
> 
> Thank you,
> 
> Rich

awk '/@/ {print >>"emails.txt";next};NF{print >>"salutation.txt"}' 
input.txt


SCNR ;-)

From list1 at tompassin.net  Thu Jan 11 13:58:07 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Thu, 11 Jan 2024 13:58:07 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
Message-ID: <a9577a3f-68d6-433b-b87c-4502e2041696@tompassin.net>

On 1/11/2024 1:27 PM, MRAB via Python-list wrote:
> On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
>> It's been several years since I've needed to write a python script so I'm
>> asking for advice to get me started with a brief script to separate names
>> and email addresses in one file into two separate files: 
>> salutation.txt and
>> emails.txt.
>>
>> An example of the input file:
>>
>> Calvin
>> calvin at example.com
>>
>> Hobbs
>> hobbs at some.com
>>
>> Nancy
>> nancy at herown.com
>>
>> Sluggo
>> sluggo at another.com
>>
>> Having extracted salutations and addresses I'll write a bash script using
>> sed and mailx to associate a message file with each name and email 
>> address.
>>
>> I'm unsure where to start given my lack of recent experience.
>>
>  From the look of it:
> 
> 1. If the line is empty, ignore it.
> 
> 2. If the line contains "@", it's an email address.
> 
> 3. Otherwise, it's a name.

You could think about a single Python script that looks through your 
input file and constructs all the message files without ever writing 
separate salutation and address files at all.  Then you wouldn't need to 
write the sed and mailx scripts.  It shouldn't be much harder than 
peeling out the names and addresses into separate files.

If you haven't written any Python for some years, the preferred way to 
read and write files is using a "with" statement, like this:

with open('email_file.txt', encoding = 'utf-8') as f:
     lines = f.readlines()
     for line in lines:
         if not line.strip():  # Skip blank lines
             continue
         # Do something with this line

You don't need to close the file because when the "with" block ends the 
file will be closed for you.

If the encoding is not utf-8 and you know what it will be, use that 
encoding instead.


From olegsivokon at gmail.com  Thu Jan 11 16:33:27 2024
From: olegsivokon at gmail.com (Left Right)
Date: Thu, 11 Jan 2024 22:33:27 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <a9577a3f-68d6-433b-b87c-4502e2041696@tompassin.net>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <a9577a3f-68d6-433b-b87c-4502e2041696@tompassin.net>
Message-ID: <CAJQBtgnNHtX+16LEyCh-GQATCR0Yp-7zaKJaG3y-gyh7+nZJAg@mail.gmail.com>

By the way, in an attempt to golf this problem, I discovered this,
which seems like a parser problem:

This is what Python tells me about its grammar:

with_stmt:
    | 'with' '(' ','.with_item+ ','? ')' ':' block
    | 'with' ','.with_item+ ':' [TYPE_COMMENT] block
    | ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block
    | ASYNC 'with' ','.with_item+ ':' [TYPE_COMMENT] block

with_item:
    | expression 'as' star_target &(',' | ')' | ':')
    | expression

>From which I figured why not something like this:

with (open('example.txt', 'r'), open('emails.txt', 'w'),
open('salutations.txt', 'w')) as e, m, s:
    for line in e:
        if line.strip():
            (m if '@' in line else s).write(line)

Which, surprise, parsers! But it seems like it's parse is wrong,
because running this I get:

? python ./split_emails.py
Traceback (most recent call last):
  File "/home/?/doodles/python/./split_emails.py", line 1, in <module>
    with (open('example.txt', 'r'), open('emails.txt', 'w'),
open('salutations.txt', 'w')) as e, m, s:
TypeError: 'tuple' object does not support the context manager protocol

It seems to me it shouldn't have been parsed as a tuple. The
parenthesis should've been interpreted just as a decoration.

NB. I'm using 3.11.6.

On Thu, Jan 11, 2024 at 10:20?PM Thomas Passin via Python-list
<python-list at python.org> wrote:
>
> On 1/11/2024 1:27 PM, MRAB via Python-list wrote:
> > On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
> >> It's been several years since I've needed to write a python script so I'm
> >> asking for advice to get me started with a brief script to separate names
> >> and email addresses in one file into two separate files:
> >> salutation.txt and
> >> emails.txt.
> >>
> >> An example of the input file:
> >>
> >> Calvin
> >> calvin at example.com
> >>
> >> Hobbs
> >> hobbs at some.com
> >>
> >> Nancy
> >> nancy at herown.com
> >>
> >> Sluggo
> >> sluggo at another.com
> >>
> >> Having extracted salutations and addresses I'll write a bash script using
> >> sed and mailx to associate a message file with each name and email
> >> address.
> >>
> >> I'm unsure where to start given my lack of recent experience.
> >>
> >  From the look of it:
> >
> > 1. If the line is empty, ignore it.
> >
> > 2. If the line contains "@", it's an email address.
> >
> > 3. Otherwise, it's a name.
>
> You could think about a single Python script that looks through your
> input file and constructs all the message files without ever writing
> separate salutation and address files at all.  Then you wouldn't need to
> write the sed and mailx scripts.  It shouldn't be much harder than
> peeling out the names and addresses into separate files.
>
> If you haven't written any Python for some years, the preferred way to
> read and write files is using a "with" statement, like this:
>
> with open('email_file.txt', encoding = 'utf-8') as f:
>      lines = f.readlines()
>      for line in lines:
>          if not line.strip():  # Skip blank lines
>              continue
>          # Do something with this line
>
> You don't need to close the file because when the "with" block ends the
> file will be closed for you.
>
> If the encoding is not utf-8 and you know what it will be, use that
> encoding instead.
>
> --
> https://mail.python.org/mailman/listinfo/python-list

From olegsivokon at gmail.com  Thu Jan 11 16:35:34 2024
From: olegsivokon at gmail.com (Left Right)
Date: Thu, 11 Jan 2024 22:35:34 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgnNHtX+16LEyCh-GQATCR0Yp-7zaKJaG3y-gyh7+nZJAg@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <a9577a3f-68d6-433b-b87c-4502e2041696@tompassin.net>
 <CAJQBtgnNHtX+16LEyCh-GQATCR0Yp-7zaKJaG3y-gyh7+nZJAg@mail.gmail.com>
Message-ID: <CAJQBtg=bKdJAQPjOot_t-+AJTqkTPOSE2tM3ycO=4YmTyTZkcg@mail.gmail.com>

Ah, nevermind. I need to be more careful, there isn't an "'as'
star_target" after the first rule.

On Thu, Jan 11, 2024 at 10:33?PM Left Right <olegsivokon at gmail.com> wrote:
>
> By the way, in an attempt to golf this problem, I discovered this,
> which seems like a parser problem:
>
> This is what Python tells me about its grammar:
>
> with_stmt:
>     | 'with' '(' ','.with_item+ ','? ')' ':' block
>     | 'with' ','.with_item+ ':' [TYPE_COMMENT] block
>     | ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block
>     | ASYNC 'with' ','.with_item+ ':' [TYPE_COMMENT] block
>
> with_item:
>     | expression 'as' star_target &(',' | ')' | ':')
>     | expression
>
> From which I figured why not something like this:
>
> with (open('example.txt', 'r'), open('emails.txt', 'w'),
> open('salutations.txt', 'w')) as e, m, s:
>     for line in e:
>         if line.strip():
>             (m if '@' in line else s).write(line)
>
> Which, surprise, parsers! But it seems like it's parse is wrong,
> because running this I get:
>
> ? python ./split_emails.py
> Traceback (most recent call last):
>   File "/home/?/doodles/python/./split_emails.py", line 1, in <module>
>     with (open('example.txt', 'r'), open('emails.txt', 'w'),
> open('salutations.txt', 'w')) as e, m, s:
> TypeError: 'tuple' object does not support the context manager protocol
>
> It seems to me it shouldn't have been parsed as a tuple. The
> parenthesis should've been interpreted just as a decoration.
>
> NB. I'm using 3.11.6.
>
> On Thu, Jan 11, 2024 at 10:20?PM Thomas Passin via Python-list
> <python-list at python.org> wrote:
> >
> > On 1/11/2024 1:27 PM, MRAB via Python-list wrote:
> > > On 2024-01-11 18:08, Rich Shepard via Python-list wrote:
> > >> It's been several years since I've needed to write a python script so I'm
> > >> asking for advice to get me started with a brief script to separate names
> > >> and email addresses in one file into two separate files:
> > >> salutation.txt and
> > >> emails.txt.
> > >>
> > >> An example of the input file:
> > >>
> > >> Calvin
> > >> calvin at example.com
> > >>
> > >> Hobbs
> > >> hobbs at some.com
> > >>
> > >> Nancy
> > >> nancy at herown.com
> > >>
> > >> Sluggo
> > >> sluggo at another.com
> > >>
> > >> Having extracted salutations and addresses I'll write a bash script using
> > >> sed and mailx to associate a message file with each name and email
> > >> address.
> > >>
> > >> I'm unsure where to start given my lack of recent experience.
> > >>
> > >  From the look of it:
> > >
> > > 1. If the line is empty, ignore it.
> > >
> > > 2. If the line contains "@", it's an email address.
> > >
> > > 3. Otherwise, it's a name.
> >
> > You could think about a single Python script that looks through your
> > input file and constructs all the message files without ever writing
> > separate salutation and address files at all.  Then you wouldn't need to
> > write the sed and mailx scripts.  It shouldn't be much harder than
> > peeling out the names and addresses into separate files.
> >
> > If you haven't written any Python for some years, the preferred way to
> > read and write files is using a "with" statement, like this:
> >
> > with open('email_file.txt', encoding = 'utf-8') as f:
> >      lines = f.readlines()
> >      for line in lines:
> >          if not line.strip():  # Skip blank lines
> >              continue
> >          # Do something with this line
> >
> > You don't need to close the file because when the "with" block ends the
> > file will be closed for you.
> >
> > If the encoding is not utf-8 and you know what it will be, use that
> > encoding instead.
> >
> > --
> > https://mail.python.org/mailman/listinfo/python-list

From PythonList at DancesWithMice.info  Thu Jan 11 17:20:33 2024
From: PythonList at DancesWithMice.info (dn)
Date: Fri, 12 Jan 2024 11:20:33 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgnNHtX+16LEyCh-GQATCR0Yp-7zaKJaG3y-gyh7+nZJAg@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <a9577a3f-68d6-433b-b87c-4502e2041696@tompassin.net>
 <CAJQBtgnNHtX+16LEyCh-GQATCR0Yp-7zaKJaG3y-gyh7+nZJAg@mail.gmail.com>
Message-ID: <02fee9f5-546d-418c-9def-9d89dda8230c@DancesWithMice.info>

On 12/01/24 10:33, Left Right via Python-list wrote:
> By the way, in an attempt to golf this problem, I discovered this,
> which seems like a parser problem:
> 
> This is what Python tells me about its grammar:
> 
> with_stmt:
>      | 'with' '(' ','.with_item+ ','? ')' ':' block
>      | 'with' ','.with_item+ ':' [TYPE_COMMENT] block
>      | ASYNC 'with' '(' ','.with_item+ ','? ')' ':' block
>      | ASYNC 'with' ','.with_item+ ':' [TYPE_COMMENT] block
> 
> with_item:
>      | expression 'as' star_target &(',' | ')' | ':')
>      | expression
> 
>  From which I figured why not something like this:
> 
> with (open('example.txt', 'r'), open('emails.txt', 'w'),
> open('salutations.txt', 'w')) as e, m, s:
>      for line in e:
>          if line.strip():
>              (m if '@' in line else s).write(line)
> 
> Which, surprise, parsers! But it seems like it's parse is wrong,
> because running this I get:
> 
> ? python ./split_emails.py
> Traceback (most recent call last):
>    File "/home/?/doodles/python/./split_emails.py", line 1, in <module>
>      with (open('example.txt', 'r'), open('emails.txt', 'w'),
> open('salutations.txt', 'w')) as e, m, s:
> TypeError: 'tuple' object does not support the context manager protocol
> 
> It seems to me it shouldn't have been parsed as a tuple. The
> parenthesis should've been interpreted just as a decoration.
> 
> NB. I'm using 3.11.6.
A series of comma-separated items will be parsed as a tuple (some people 
think it is bounding-parentheses which define).

In this case, the issue is 'connecting' the context-manager "expression" 
with its (as) "target". These should be more-closely paired:-

with ( open( 'example.txt', 'r', ) as e,
        open( 'emails.txt', 'w', ) as m,
        open( 'salutations.txt', 'w', ) as s
        ):

(NB code not executed here)


A data-architecture of having related-data in separated serial-files is 
NOT recommendable!

-- 
Regards,
=dn

From rosuav at gmail.com  Thu Jan 11 18:56:37 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 12 Jan 2024 10:56:37 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgnNHtX+16LEyCh-GQATCR0Yp-7zaKJaG3y-gyh7+nZJAg@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <a9577a3f-68d6-433b-b87c-4502e2041696@tompassin.net>
 <CAJQBtgnNHtX+16LEyCh-GQATCR0Yp-7zaKJaG3y-gyh7+nZJAg@mail.gmail.com>
Message-ID: <CAPTjJmqx3aDeSDYXDiT7xSSLtJUYzqzc4zY7f16w9VGJYsXE-g@mail.gmail.com>

On Fri, 12 Jan 2024 at 08:56, Left Right via Python-list
<python-list at python.org> wrote:
>
> By the way, in an attempt to golf this problem, I discovered this,
> which seems like a parser problem:

When you jump immediately to "this is a bug", all you do is make
yourself look like an idiot. Unsurprisingly, this is NOT a bug, this
is simply that you didn't understand what was going on. The grammar
isn't easy to read, and it's usually better to read the documentation
instead.

(Plus, golfing isn't really a goal in Python, and you didn't  shorten
the code by much at all. Good job.)

ChrisA

From PythonList at DancesWithMice.info  Thu Jan 11 23:34:56 2024
From: PythonList at DancesWithMice.info (dn)
Date: Fri, 12 Jan 2024 17:34:56 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmqx3aDeSDYXDiT7xSSLtJUYzqzc4zY7f16w9VGJYsXE-g@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <a9577a3f-68d6-433b-b87c-4502e2041696@tompassin.net>
 <CAJQBtgnNHtX+16LEyCh-GQATCR0Yp-7zaKJaG3y-gyh7+nZJAg@mail.gmail.com>
 <CAPTjJmqx3aDeSDYXDiT7xSSLtJUYzqzc4zY7f16w9VGJYsXE-g@mail.gmail.com>
Message-ID: <75950bec-a404-4de8-94aa-b8ab1eeede04@DancesWithMice.info>

On 12/01/24 12:56, Chris Angelico via Python-list wrote:
> On Fri, 12 Jan 2024 at 08:56, Left Right via Python-list
> <python-list at python.org> wrote:
>>
>> By the way, in an attempt to golf this problem, I discovered this,
>> which seems like a parser problem:
> 
> When you jump immediately to "this is a bug", all you do is make

"seems"!

but yes, it is a (much) less-likely explanation.


> yourself look like an idiot. Unsurprisingly, this is NOT a bug, this
> is simply that you didn't understand what was going on. The grammar
> isn't easy to read, and it's usually better to read the documentation
> instead.

Those of us who studied Computer Science may well have been 
taught/expected to learn how to read [modified] BNF - indeed to have 
worked in that (cf coding in Python).

Accordingly, the English text is likely easier to understand, but 
sometimes the BNF offers finer-detail or can be used to clarify some 
mis- or insufficiently-understood aspect of the text. IMHO/YMMV/etc...


> (Plus, golfing isn't really a goal in Python, and you didn't  shorten
> the code by much at all. Good job.)

I took my hat off to the poster, being prepared to dive-in and do this. 
Accordingly, was more than happy to help set him/her back onto 'the 
straight and narrow'.

(yes it was a BNF-failing - which, credit where credit's due, I think 
was realised at the same time as response was typed)

How many others just want us to do all their thinking for them?
(there's a rude comment about wiping noses - but probably a step too far 
wrt the CoC)

-- 
Regards,
=dn

From RealGrizzlyAdams at vivaldi.net  Fri Jan 12 01:58:50 2024
From: RealGrizzlyAdams at vivaldi.net (Grizzy Adams)
Date: Fri, 12 Jan 2024 06:58:50 -0000
Subject: Extract lines from file, add to new files
In-Reply-To: <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>,
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>,
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
Message-ID: <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>

Thursday, January 11, 2024  at 10:44, Rich Shepard via Python-list wrote:
Re: Extract lines from file, add to (at least in part)

>On Thu, 11 Jan 2024, MRAB via Python-list wrote:

>> From the look of it:
>> 1. If the line is empty, ignore it.
>> 2. If the line contains "@", it's an email address.
>> 3. Otherwise, it's a name.

If that is it all? a simple Grep would do (and save on the blank line)

From antoon.pardon at vub.be  Fri Jan 12 08:23:43 2024
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Fri, 12 Jan 2024 14:23:43 +0100
Subject: mypy question
In-Reply-To: <ZY7faSj-OA-lDqxC@hermes.hilbert.loc>
References: <ZY64YbzKqfGbZcpC@hermes.hilbert.loc>
 <fd6bbad8-2736-4305-982d-eabbf212a920@wichmann.us>
 <ZY7faSj-OA-lDqxC@hermes.hilbert.loc>
Message-ID: <ad016be2-6d57-46d9-943e-1cb398b54e7f@vub.be>

Op 29/12/2023 om 16:02 schreef Karsten Hilbert via Python-list:

> Am Fri, Dec 29, 2023 at 07:49:17AM -0700 schrieb Mats Wichmann via Python-list:
>
>>> I am not sure why mypy thinks this
>>>
>>> gmPG2.py:554: error: Argument "queries" to "run_rw_queries" has incompatible type "List[Dict[str, str]]"; expected
>>> "List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"  [arg-type]
>>>                      rows, idx = run_rw_queries(link_obj = conn, queries = queries, return_data = True)
>>>                                                                            ^~~~~~~
>>>
>>> should be flagged. The intent is for "queries" to be
>>>
>>> a list
>>> 	of dicts
>>> 		with keys of str
>>> 		and values of
>>> 			str OR
>>> 			list of anything OR
>>> 			dict with
>>> 				keys of str
>>> 				and values of anything
>>>
>>> I'd have thunk list[dict[str,str]] matches that ?
>> Dict[str, str] means the key type and value type should both be strings,
> Indeed, I know that much, list[dict[str, str]] is what is getting
> passed in in this particular invocation of run_rw_queries().
>
> For what it's worth here's the signature of that function:
>
> 	def run_rw_queries (
> 		link_obj:_TLnkObj=None,
> 		queries:list[dict[str, str | list | dict[str, Any]]]=None,
> 		end_tx:bool=False,
> 		return_data:bool=None,
> 		get_col_idx:bool=False,
> 		verbose:bool=False
> 	) -> tuple[list[dbapi.extras.DictRow], dict[str, int] | None]:
>
> Given that I would have thought that passing in
> list[dict[str, str]] for "queries" ought to be type safe.
> Mypy indicates otherwise which I am not grokking as to why.
>
>> but in your
>> retelling above you indicate lots of possible value types... actually the mypy guess
>> seems to be a pretty good recreation of your psuedo-code description.
> I agree that mypy's grasp of my intent from
>
> 	queries:list[dict[str, str | list | dict[str, Any]]]=None,
>
> into
>
> 	"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"
>
> seems accurate. I just don't understand why list[dict[str,
> str]] should not pass that construct.

Sorry for the late reaction and may be I am missing something, but I was wondering if
your type hint for queries shouldn't be the following.

queries:list[dict[str,str]|dict[str,list]|dict[str,dict[str, dict[str, Ant]]]

My impression at this moment is that you are write something like: dict[str, str | int] as
as shorthand for dict[str, str] | dict[str, int]. But those two are different types.

-- 
Antoon Pardon.

From avi.e.gross at gmail.com  Fri Jan 12 09:42:18 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Fri, 12 Jan 2024 09:42:18 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>,
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>,
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
Message-ID: <002401da4565$8b477360$a1d65a20$@gmail.com>

If the data in the input file is exactly as described and consists of
alternating lines containing a name and email address, or perhaps an
optional blank line, then many solutions are possible using many tools
including python programs.

But is the solution a good one for some purpose? The two output files may
end up being out of sync for all kinds of reasons. One of many "errors" can
happen if multiple lines in a row do not have an "@" or a person's name
does, for example. What if someone supplied more than one email address with
a comma separator? This may not be expected but could cause problems.

Some of the other tools mentioned would not care and produce garbage. Grep
as an example could be run twice asking for lines with an "@" and then lines
without. In this case, that would be trivial. Blank lines, or ones with just
whitespace, might need another pass to be omitted.

But a real challenge would be to parse the file in a language like Python
and find all VALID stretches in the data and construct a data structure
containing either a valid name or something specific like "ANONYMOUS"
alongside an email address. These may be written out as soon as it is
considered valid, or collected in something like a list. You can do further
processing if you want the results in some order or remove duplicates or bad
email addresses and so on. In that scenario, the two files would be written
out at the end.

Python can do the above while some of the other tools mentioned are not
really designed for it. Further, many of the tools are not generally
available everywhere.

Another question is why it makes sense to produce two output files to
contain the data that may not be linked and would not be easy to edit and
keep synchronized such as to remove or add entries. There are many ways to
save the data that might be more robust for many purposes. It looks like the
application intended is a sort of form letter merge where individual emails
will be sent that contain a personalized greeting. Unless that application
has already been written, there are many other ways that make sense. One
obvious one is to save the data in a databases as columns in a table. Other
ones are to write one file with entries easily parsed out such as:

NAME: name | EMAIL: email

Whatever the exact design, receiving software could parse that out as needed
by the simpler act of reading one line at a time.

And, of course, there are endless storage formats such as a CSV file or
serializing your list of objects to a file so that the next program can load
them in and operate from memory on all the ones it wants. The two file
solution may seem simpler but harks back to how some computing was done in
early days when list of objects might be handled by having multiple arrays
with each containing one aspect of the object and updating required
rememebreing to touch each array the same way.. That can still be a useful
technique when some operations being done in a vectoried manner might be
faster than an array of objects, but is more often a sign of poor code.






-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Grizzy Adams via Python-list
Sent: Friday, January 12, 2024 1:59 AM
To: Rich Shepard via Python-list <python-list at python.org>; Rich Shepard
<rshepard at appl-ecosys.com>
Subject: Re: Extract lines from file, add to new files

Thursday, January 11, 2024  at 10:44, Rich Shepard via Python-list wrote:
Re: Extract lines from file, add to (at least in part)

>On Thu, 11 Jan 2024, MRAB via Python-list wrote:

>> From the look of it:
>> 1. If the line is empty, ignore it.
>> 2. If the line contains "@", it's an email address.
>> 3. Otherwise, it's a name.

If that is it all? a simple Grep would do (and save on the blank line)
-- 
https://mail.python.org/mailman/listinfo/python-list


From rshepard at appl-ecosys.com  Fri Jan 12 11:39:17 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Fri, 12 Jan 2024 08:39:17 -0800 (PST)
Subject: Extract lines from file, add to new files
Message-ID: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>

On Fri, 12 Jan 2024, AVI GROSS via Python-list wrote:

> But is the solution a good one for some purpose? The two output files may
> end up being out of sync for all kinds of reasons. One of many "errors"
> can happen if multiple lines in a row do not have an "@" or a person's
> name does, for example. What if someone supplied more than one email
> address with a comma separator? This may not be expected but could cause
> problems.

Avi,

For my use 1) the salutation and email address (always with an '@') are
sequential and 2) I'm developing the script to extract both from the same
file.

Regards,

Rich

From olegsivokon at gmail.com  Fri Jan 12 06:11:31 2024
From: olegsivokon at gmail.com (Left Right)
Date: Fri, 12 Jan 2024 12:11:31 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
Message-ID: <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>

To people discussing BNF:

The grammar language Python uses is *very* far from BNF. It's more
similar to PEG, but even then it's still quite far.  Python's grammar
is just its own thing, which makes it harder to read, if you are
already familiar with other more popular formats.

I've also found bugs in Python parser before, so had this turned out
to be a real issue, this wouldn't have been the first time.  There are
plenty of weird corners in Python grammar that allow unexpected
programs to parse (and sometimes even run!), and these are very often
connected to assignments, because, in general, assignments in Python
are very elaborate and hard to describe / conceptualize about.  The
most popular example I've even seen used in coding interviews (which I
think is a silly gimmick, but that's kind of the whole point of a lot
of these interviews...) is:

    x = [...]
    for x[i] in x: print(i)

Which is not an assignment by itself, but the "weirdness" results from
the loop syntax sharing definitions with the "destructuring bind"
style of assignment (i.e. where the left-hand side can be an arbitrary
complex expression).

I was surprised, for example, to learn that "as" in "with_stmt" isn't
shared with "as" in "except_block" (so, from the grammar perspective,
these are two different keywords), and that asterisk in "except_block"
isn't shared with "star_target" (also weird, since you'd think these
should be the same thing).  In general, and by and large, if you look
at Python's grammar there are many "weird" choices that it makes to
describe the language which seem counterintuitive to the programmer
who tries to learn the language from examples (i.e. context-depending
meaning of parenthesis, of asterisk, of period etc.)  Having been
exposed to this, you'd start to expect that some of this weirdness
will eventually result in bugs, or at least in unexpected behavior.

----

Anyways. To the OP: I'm sorry to hijack your question. Below is the
complete program:

with (
    open('example.txt', 'r') as e,
    open('emails.txt', 'w') as m,
    open('salutations.txt', 'w') as s,
):
    for line in e:
        if line.strip():
            (m if '@' in line else s).write(line)

it turned out to be not quite the golfing material I was hoping for.
But, perhaps a somewhat interesting aspect of this program you don't
see used a lot in the wild is the parenthesis in the "with" head.  So,
it's not a total write-off from the learning perspective.  I.e. w/o
looking at the grammar, and had I have this code in a coding interview
question, I wouldn't be quite sure whether this code would work or
not: one way to interpret what's going on here is to think that the
expression inside parentheses is a tuple, and since tuples aren't
context managers, it wouldn't have worked (or maybe not even parsed as
"as" wouldn't be allowed inside tuple definition since there's no
"universal as-expression" in Python it's hard to tell what the rules
are).  But, it turns out there's a form of "with" that has parentheses
for decoration purposes, and that's why it parses and works to the
desired effect.

Since it looks like you are doing this for educational reasons, I
think there's a tiny bit of value to my effort.

On Fri, Jan 12, 2024 at 8:08?AM Grizzy Adams via Python-list
<python-list at python.org> wrote:
>
> Thursday, January 11, 2024  at 10:44, Rich Shepard via Python-list wrote:
> Re: Extract lines from file, add to (at least in part)
>
> >On Thu, 11 Jan 2024, MRAB via Python-list wrote:
>
> >> From the look of it:
> >> 1. If the line is empty, ignore it.
> >> 2. If the line contains "@", it's an email address.
> >> 3. Otherwise, it's a name.
>
> If that is it all? a simple Grep would do (and save on the blank line)
> --
> https://mail.python.org/mailman/listinfo/python-list

From greg.ewing at canterbury.ac.nz  Fri Jan 12 18:34:07 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 13 Jan 2024 12:34:07 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <mailman.20.1705101297.15798.python-list@python.org>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
Message-ID: <l0e0jhFef82U1@mid.individual.net>

On 13/01/24 12:11 am, Left Right wrote:
>      x = [...]
>      for x[i] in x: print(i)

I suspect you've misremembered something, because this doesn't
do anything surprising for me:

 >>> x = [1, 2, 3]
 >>> for x[i] in x: print(i)
...
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
NameError: name 'i' is not defined

There's no destructuring going on here, just assignment to a
sequence item.

-- 
Greg

From olegsivokon at gmail.com  Fri Jan 12 19:45:37 2024
From: olegsivokon at gmail.com (Left Right)
Date: Sat, 13 Jan 2024 01:45:37 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <l0e0jhFef82U1@mid.individual.net>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
Message-ID: <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>

> surprising for me:

Surprise is subjective, it's based on personal experience. Very few
languages allow arbitrary complex expressions in the same place they
allow variable introduction. The fact that "i" is not defined is
irrelevant to this example.  Most programmers who haven't memorized
Python grammar by heart, but expect the language to behave similar to
the languages in the same category would be surprised this code is
valid (i.e. can be parsed), whether it results in error or not is of
no consequence.

> There's no destructuring going on here

I use the term "destructuring" in the same way Hyperspec uses it.
It's not a Python term.  I don't know what you call the same thing in
Python.  I'm not sure what you understand from it.

On Sat, Jan 13, 2024 at 12:37?AM Greg Ewing via Python-list
<python-list at python.org> wrote:
>
> On 13/01/24 12:11 am, Left Right wrote:
> >      x = [...]
> >      for x[i] in x: print(i)
>
> I suspect you've misremembered something, because this doesn't
> do anything surprising for me:
>
>  >>> x = [1, 2, 3]
>  >>> for x[i] in x: print(i)
> ...
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> NameError: name 'i' is not defined
>
> There's no destructuring going on here, just assignment to a
> sequence item.
>
> --
> Greg
> --
> https://mail.python.org/mailman/listinfo/python-list

From olegsivokon at gmail.com  Fri Jan 12 20:02:39 2024
From: olegsivokon at gmail.com (Left Right)
Date: Sat, 13 Jan 2024 02:02:39 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
 <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
Message-ID: <CAJQBtg=vjoMGcTw2a5C2FFdzVnmNBwFyf2Pu0gEXhjYvOq3MGw@mail.gmail.com>

Actually, after some Web search.  I think, based on this:
https://docs.python.org/3/reference/simple_stmts.html#grammar-token-python-grammar-augtarget
that in Python you call this "augmented assignment target". The term
isn't in the glossary, but so are many others.

On Sat, Jan 13, 2024 at 1:45?AM Left Right <olegsivokon at gmail.com> wrote:
>
> > surprising for me:
>
> Surprise is subjective, it's based on personal experience. Very few
> languages allow arbitrary complex expressions in the same place they
> allow variable introduction. The fact that "i" is not defined is
> irrelevant to this example.  Most programmers who haven't memorized
> Python grammar by heart, but expect the language to behave similar to
> the languages in the same category would be surprised this code is
> valid (i.e. can be parsed), whether it results in error or not is of
> no consequence.
>
> > There's no destructuring going on here
>
> I use the term "destructuring" in the same way Hyperspec uses it.
> It's not a Python term.  I don't know what you call the same thing in
> Python.  I'm not sure what you understand from it.
>
> On Sat, Jan 13, 2024 at 12:37?AM Greg Ewing via Python-list
> <python-list at python.org> wrote:
> >
> > On 13/01/24 12:11 am, Left Right wrote:
> > >      x = [...]
> > >      for x[i] in x: print(i)
> >
> > I suspect you've misremembered something, because this doesn't
> > do anything surprising for me:
> >
> >  >>> x = [1, 2, 3]
> >  >>> for x[i] in x: print(i)
> > ...
> > Traceback (most recent call last):
> >    File "<stdin>", line 1, in <module>
> > NameError: name 'i' is not defined
> >
> > There's no destructuring going on here, just assignment to a
> > sequence item.
> >
> > --
> > Greg
> > --
> > https://mail.python.org/mailman/listinfo/python-list

From rosuav at gmail.com  Fri Jan 12 21:14:56 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 13 Jan 2024 13:14:56 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
 <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
Message-ID: <CAPTjJmraZ9ffpjLfyurT1gS0=JVY-S8s6+iRtPLDqcn0PV2v9w@mail.gmail.com>

On Sat, 13 Jan 2024 at 13:11, Left Right via Python-list
<python-list at python.org> wrote:
>
>  Very few
> languages allow arbitrary complex expressions in the same place they
> allow variable introduction.

What do you mean by this? Most languages I've worked with allow
variables to be initialized with arbitrary expressions, and a lot of
languages allow narrowly-scoped variables.

ChrisA

From 2QdxY4RzWzUUiLuE at potatochowder.com  Fri Jan 12 21:34:23 2024
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Fri, 12 Jan 2024 21:34:23 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtg=vjoMGcTw2a5C2FFdzVnmNBwFyf2Pu0gEXhjYvOq3MGw@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
 <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
 <CAJQBtg=vjoMGcTw2a5C2FFdzVnmNBwFyf2Pu0gEXhjYvOq3MGw@mail.gmail.com>
Message-ID: <ZaH2rwgbIHpCmJtr@anomaly>

On 2024-01-13 at 02:02:39 +0100,
Left Right via Python-list <python-list at python.org> wrote:

> Actually, after some Web search.  I think, based on this:
> https://docs.python.org/3/reference/simple_stmts.html#grammar-token-python-grammar-augtarget
> that in Python you call this "augmented assignment target". The term
> isn't in the glossary, but so are many others.

The Python term, at least colloquially, is "tuple unpacking."

HTH.

From 2QdxY4RzWzUUiLuE at potatochowder.com  Sat Jan 13 06:58:34 2024
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Sat, 13 Jan 2024 06:58:34 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgmLwSUPy8jngpZGyxjVXEeXkr7Rte1z5_uFXZOG8ECH3g@mail.gmail.com>
References: <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
 <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
 <CAJQBtg=vjoMGcTw2a5C2FFdzVnmNBwFyf2Pu0gEXhjYvOq3MGw@mail.gmail.com>
 <ZaH2rwgbIHpCmJtr@anomaly>
 <CAJQBtgmLwSUPy8jngpZGyxjVXEeXkr7Rte1z5_uFXZOG8ECH3g@mail.gmail.com>
Message-ID: <ZaJ66tFRHJ_6iXsx@anomaly>

On 2024-01-13 at 11:34:29 +0100,
Left Right <olegsivokon at gmail.com> wrote:

> > The Python term, at least colloquially, is "tuple unpacking."

That quote is from me.  Please do preserve attributions.

> Well, why use colloquialism if there's a language specification? Also,
> there weren't any tuples used in my example, at least not explicitly
> (i could've been a tuple, but that wasn't specified).

According to the language specification,? it's a "target list," and
there can be more than one target in that list.

The unpacking isn't really called anything, it's just the way Python
assignment works, all the way back to its earliest stages.?

? https://docs.python.org/3/reference/simple_stmts.html#assignment-statements,
? https://docs.python.org/release/1.4/ref/ref6.html#HDR2

From mk1853387 at gmail.com  Sat Jan 13 07:41:10 2024
From: mk1853387 at gmail.com (marc nicole)
Date: Sat, 13 Jan 2024 13:41:10 +0100
Subject: best tool to extract domain hierarchy from a dimension in an OLAP
 dataset (csv)
Message-ID: <CAGJtH9T=PBQZKH8epaFtmjZqHPSP1dQF9DKTvYuXbg02P1b71A@mail.gmail.com>

Hi all,

I have a csv OLAP dataset that I want to extract the domain hierarchies
from each of its dimensions.

Anybody could recommend a Python tool that could manage this properly?

Thanks

From greg.ewing at canterbury.ac.nz  Sat Jan 13 08:04:41 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 14 Jan 2024 02:04:41 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <mailman.21.1705111793.15798.python-list@python.org>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
 <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
 <mailman.21.1705111793.15798.python-list@python.org>
Message-ID: <l0fg3aFn07jU1@mid.individual.net>

On 13/01/24 1:45 pm, Left Right wrote:

> I use the term "destructuring" in the same way Hyperspec uses it.
> It's not a Python term.  I don't know what you call the same thing in
> Python.  I'm not sure what you understand from it.

I thought you meant what is usually called "unpacking" in Python. I
don't know anything about Hyperspec, so I don't know what it means
there.

The fact that i was being printed inside the loop made me think
that some deeper level of surprise was being intended, such as
the value of i somehow getting changed by the assignment.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Jan 13 08:18:00 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 14 Jan 2024 02:18:00 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <mailman.23.1705112110.15798.python-list@python.org>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
 <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
 <CAPTjJmraZ9ffpjLfyurT1gS0=JVY-S8s6+iRtPLDqcn0PV2v9w@mail.gmail.com>
 <mailman.23.1705112110.15798.python-list@python.org>
Message-ID: <l0fgscFn5ueU1@mid.individual.net>

On 13/01/24 3:14 pm, Chris Angelico wrote:
> On Sat, 13 Jan 2024 at 13:11, Left Right via Python-list
> <python-list at python.org> wrote:
>>
>>   Very few
>> languages allow arbitrary complex expressions in the same place they
>> allow variable introduction.
> 
> What do you mean by this? Most languages I've worked with allow
> variables to be initialized with arbitrary expressions, and a lot of
> languages allow narrowly-scoped variables.

I think he means that in some languages the for-loop target serves as
the declaration of a new variable, and as such has to be a bare name.

Python isn't like that -- the target of a for-statement is treated
exactly the same way as the lhs of an assignment. It's not scoped to the
loop.

BTW, the equivalent thing is valid in C too, so anyone familiar with C
is unlikely to be surprised by this either.

#include <stdio.h>

int x[10];
int i;

int main() {
   i = 5;
   for (x[i] = 0; x[i] < 10; x[i]++)
     printf("%d\n", x[i]);
}

Output:

0
1
2
3
4
5
6
7
8
9

-- 
Greg

From Karsten.Hilbert at gmx.net  Sat Jan 13 15:20:00 2024
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Sat, 13 Jan 2024 21:20:00 +0100
Subject: mypy question
In-Reply-To: <ad016be2-6d57-46d9-943e-1cb398b54e7f@vub.be>
References: <ZY64YbzKqfGbZcpC@hermes.hilbert.loc>
 <fd6bbad8-2736-4305-982d-eabbf212a920@wichmann.us>
 <ZY7faSj-OA-lDqxC@hermes.hilbert.loc>
 <ad016be2-6d57-46d9-943e-1cb398b54e7f@vub.be>
Message-ID: <ZaLwcNHO7UBkoXOm@hermes.hilbert.loc>

Am Fri, Jan 12, 2024 at 02:23:43PM +0100 schrieb Antoon Pardon via Python-list:

> >	queries:list[dict[str, str | list | dict[str, Any]]]=None,
> >
> >into
> >
> >	"List[Dict[str, Union[str, List[Any], Dict[str, Any]]]]"
> >
> >seems accurate. I just don't understand why list[dict[str,
> >str]] should not pass that construct.
>
> Sorry for the late reaction

ne'er mind ya

> I was wondering if
> your type hint for queries shouldn't be the following.
>
> queries:list[dict[str,str]|dict[str,list]|dict[str,dict[str, dict[str, Ant]]]
>
> My impression at this moment is that you are write something like: dict[str, str | int] as
> as shorthand for dict[str, str] | dict[str, int].

I do.

> But those two are different types.

A-ha ! In what way ?

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B

From Karsten.Hilbert at gmx.net  Sat Jan 13 15:49:15 2024
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Sat, 13 Jan 2024 21:49:15 +0100
Subject: mypy question
In-Reply-To: <ZaLwcNHO7UBkoXOm@hermes.hilbert.loc>
References: <ZY64YbzKqfGbZcpC@hermes.hilbert.loc>
 <fd6bbad8-2736-4305-982d-eabbf212a920@wichmann.us>
 <ZY7faSj-OA-lDqxC@hermes.hilbert.loc>
 <ad016be2-6d57-46d9-943e-1cb398b54e7f@vub.be>
 <ZaLwcNHO7UBkoXOm@hermes.hilbert.loc>
Message-ID: <ZaL3SwTUtY1FZNVa@hermes.hilbert.loc>

Am Sat, Jan 13, 2024 at 09:20:00PM +0100 schrieb Karsten Hilbert via Python-list:

> > I was wondering if
> > your type hint for queries shouldn't be the following.
> >
> > queries:list[dict[str,str]|dict[str,list]|dict[str,dict[str, dict[str, Ant]]]

Wait, not really. Let me give an example. Here's three times
the same query (as far as PostgreSQL is concerned, after
having been passed through psycopg2):

queries = [
	{
		'SQL': 'SELECT 1'
	},
	{
		'SQL': 'SELECT %s',
		'args': [1]
	},
	{
		'SQL': 'SELECT %(value)s',
		'args': {'value': 1}
	}
]

The value for key "SQL" will always be str-like.

The value for "args" can be a list or a dict itself.

If "args" is a dict it will be of type [str, Any].

That's what I am trying to tell mypy.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B

From PythonList at DancesWithMice.info  Sat Jan 13 17:01:12 2024
From: PythonList at DancesWithMice.info (dn)
Date: Sun, 14 Jan 2024 11:01:12 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <e3f12f-516c-3859-8d40-8f91bffea740@appl-ecosys.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <mailman.3.1704996908.15798.python-list@python.org> <4ao57k-bca.ln1@lazy.lzy>
 <e3f12f-516c-3859-8d40-8f91bffea740@appl-ecosys.com>
Message-ID: <47a3d257-a97a-4acf-a721-fe2b7fce12bd@DancesWithMice.info>

On 12/01/24 08:53, Rich Shepard via Python-list wrote:
> On Thu, 11 Jan 2024, Piergiorgio Sartor via Python-list wrote:
> 
>> Why not to use bash script for all?
> 
> Piergiorgio,
> 
> That's certainly a possibility, and may well be better than python for this
> task.

(sitting in a meeting with little to occupy my mind, whilst tidying 
email-InBox came back to this conversation)


In the bare-description of the task, I might agree to sticking with 
BASH. The OP did say that the output from this will become input to a 
sed/mailx task!
(we trust, does not involve spamming innocent folk)

However, that task could also be accomplished in Python. So, unless 
there is an existing script (perhaps) quite why one would choose to do 
half in Python and half in BASH (or...) is a question.


Because this is a Python forum, do the whole thing in one mode - our mode!

Previous suggestions involved identifying a line by its content.

Could use a neat state-transition solution.

However, there is no need to consider the input-data as lines because of 
the concept of "white-space", well-utilised by some of Python's built-in 
string-functions. See code-sample, below.

As mentioned before, the idea of splitting the one file (data-items 
related by serial-progression) and creating two quite-separate 
data-constructs (in this case: one holding the person's name in one file 
and the other the person's email-address in another) which are related 
'across', ie line-by-line, is an architectural error?horror. Such would 
be hard to maintain, and over-time impossible to guarantee integrity. 
Assuming this is not a one-off exercise, see elsewhere for advice to 
store the captured data in some more-useful format, eg JSON, CSV, or 
even put into a MongoDB or RDBMS.


****** code

""" PythonExperiments:rich.py
     Demonstrate string extraction.
"""

__author__ = "dn, IT&T Consultant"
__python__ = "3.12"
__created__ = "PyCharm, 14 Jan 2024"
__copyright__ = "Copyright ? 2024~"
__license__ = "GNU General Public License v3.0"

# PSL
import more_itertools as it

DATA_FILE = "rich_data_file"
READ_ONLY = "r"
AS_PAIRS = 2
STRICT_PAIRING = True


if __name__ == "__main__":
     print("\nCommencing execution\n")

     with open( DATA_FILE, READ_ONLY, ) as df:
         data = df.read()

     data_as_list = data.split()
     paired_data = it.chunked( data_as_list, AS_PAIRS, STRICT_PAIRING, )

     for name, email_address in paired_data:
         # replace this with email-function
         # and/or with storage-function
         print( name, email_address, )

     print("\nTerminating")

****** sample output

Calvin calvin at example.com
Hobbs hobbs at some.com
...

******

-- 
Regards,
=dn

From PythonList at DancesWithMice.info  Sat Jan 13 21:51:39 2024
From: PythonList at DancesWithMice.info (dn)
Date: Sun, 14 Jan 2024 15:51:39 +1300
Subject: Mtg: Object-Oriented VacExcHndlrs (UTC+13)
Message-ID: <e2366e71-ced0-436e-af2c-2f45f64418b9@DancesWithMice.info>

Let's meet on Wednesday (17Jan, 1600 NZDT (UTC+13), wearing a head-set) 
to talk about Object-Oriented everything. Is O-O worthwhile, or does is 
it just a load of guys running around and getting no-where?

NB this is not a formal PUG-meeting. It's part of the "Vacation 
Exception Handlers" series 
(https://danceswithmice.info/Python/2024/VacExcHndlrs.html) - 
virtual-gatherings for folk left-behind to keep the wheels turning, 
whilst everyone else swans-off sunning themselves...

(non-Kiwis please remember: it's not just school vacation, but 
summer-time down-under. Wish you were here?)

Caf?-style approach, so there will be no formal presentation. All 
welcome. No presumption of knowledge/skill. This gathering is for 
everyone, from Beginner to Python-Master.


Is Python an Object-Oriented language?
Why does Python use (what appear to be) procedural constructs for so 
many of its core functions, eg len(a_list) rather than a_list.length() 
and sqrt(a_number) rather than a_number.sqrt()?
Why do pythonista say "everything in Python is an object"?
Is it faster to write in an OOP-style and/or does OOP-code run faster? 
If not, why bother?
- insert your question here: What do you want to know? What has been 
bothering you about OOP (or O-O in Python) that you'd like to settle?

To join us (we don't bite!), please RSVP at 
https://www.meetup.com/nzpug-auckland/events/298536620/

-- 
Regards,
=dn

From PythonList at DancesWithMice.info  Sat Jan 13 22:41:44 2024
From: PythonList at DancesWithMice.info (dn)
Date: Sun, 14 Jan 2024 16:41:44 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
Message-ID: <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>

On 13/01/24 00:11, Left Right via Python-list wrote:
> To people discussing BNF:
> 
> The grammar language Python uses is *very* far from BNF. It's more
> similar to PEG, but even then it's still quite far.  Python's grammar
> is just its own thing, which makes it harder to read, if you are
> already familiar with other more popular formats.

Second time to ameliorate wording-dispute in this thread! The original 
phrase was: "[modified] BNF". Some of us have worked with various forms 
and evolutions of BNF since back in the days of COBOL-60 proposals, and 
know it when we see it!

 From the 'book of words': <<<The descriptions of lexical analysis and 
syntax use a modified Backus?Naur form (BNF) grammar notation.>>>
https://docs.python.org/3/reference/introduction.html#notation

Yes it is hard to read - and even harder to learn-from; which is why 
@Chris gave advice about preferring tutorials/text.

Just because there are other (more popular?) formats, doesn't make the 
one used here 'wrong'. In the same way that Python code differs from 
'the same' in other languages.

Putting it another way: if what Python is doing is wrong in your 
opinion, you are in the wrong place (for you).

That is not to say that Python has everything 'right'. One of my own 
bug-bears is very similar - that the string formatting 'mini-language' 
(https://docs.python.org/3/library/string.html#formatspec) does not 
follow the same rules about white-space as everything covered by the PEG 
Parser.

BTW the PEG Parser is relatively new to Python. IIRC there was comment 
at the time of its first application, that some 'other' areas of Python 
might take a while to be converted-over.


> I've also found bugs in Python parser before, so had this turned out

Sorry, didn't recognise your email-handle - not that I'm a Python 
Core-Dev, and pretty much ignore the "Ideas" list these days. Must have 
missed your previous contributions...


> to be a real issue, this wouldn't have been the first time.  There are
> plenty of weird corners in Python grammar that allow unexpected
> programs to parse (and sometimes even run!), and these are very often
> connected to assignments, because, in general, assignments in Python
> are very elaborate and hard to describe / conceptualize about.  The
> most popular example I've even seen used in coding interviews (which I
> think is a silly gimmick, but that's kind of the whole point of a lot
> of these interviews...) is:
> 
>      x = [...]
>      for x[i] in x: print(i)
> 
> Which is not an assignment by itself, but the "weirdness" results from
> the loop syntax sharing definitions with the "destructuring bind"
> style of assignment (i.e. where the left-hand side can be an arbitrary
> complex expression).

You're right. (also about stupid 'interviewing' ideas) If someone asked 
me this, I'd respond by asking if that was the standard of code they 
work towards - and depending upon that answer would either walk-out or 
refer the matter to a more senior manager!


In Python, everything is an object. As long as the LHS is a legal-object 
which  makes sense for the situation, it can be used.

Also, an identifier (whether x, i, or x[ i ]) should not only be 
considered to be its own object, but is best regarded as a pointer to 
some value. This is how we can have an 'immutable' tuple 'containing' a 
mutable list (for example) - such that elements of that list may be 
changed, despite being 'part of' an immutable construct!

Programs are read by people. If something is a "weirdness", then 
chances-are it won't survive a CodeReview/a professional team's 
expected-standard. Not limited to Python-code!


> I was surprised, for example, to learn that "as" in "with_stmt" isn't
> shared with "as" in "except_block" (so, from the grammar perspective,
> these are two different keywords), and that asterisk in "except_block"
> isn't shared with "star_target" (also weird, since you'd think these
> should be the same thing).  In general, and by and large, if you look
> at Python's grammar there are many "weird" choices that it makes to
> describe the language which seem counterintuitive to the programmer
> who tries to learn the language from examples (i.e. context-depending
> meaning of parenthesis, of asterisk, of period etc.)  Having been
> exposed to this, you'd start to expect that some of this weirdness
> will eventually result in bugs, or at least in unexpected behavior.

You're right. It is potentially confusing when the same word/symbol is 
used in different contexts.

I've heard similar questions from learners, but not had anyone trying to 
mis-use something extrapolating from how the 'same' is used elsewhere. YMMV!

It's the context part that's important to remember. If someone calls you 
"mate", that has different connotations depending upon whether you're 
friends, you're on a Navy ship, or in a more intimate situation - indeed 
there are some cultures in which the word "mate" is not used to mean 
'friend' at all. Which is (more) right? Which wrong?

Perhaps you're aiming for, or even used to, a more perfect and 
predictable language?


> ----
> 
> Anyways. To the OP: I'm sorry to hijack your question. Below is the
> complete program:
> 
> with (
>      open('example.txt', 'r') as e,
>      open('emails.txt', 'w') as m,
>      open('salutations.txt', 'w') as s,
> ):
>      for line in e:
>          if line.strip():
>              (m if '@' in line else s).write(line)

Please see responses elsewhere which say why this sort of thing, whilst 
possible and 'easy', is not recommendable.


> it turned out to be not quite the golfing material I was hoping for.
> But, perhaps a somewhat interesting aspect of this program you don't
> see used a lot in the wild is the parenthesis in the "with" head.  So,
> it's not a total write-off from the learning perspective.  I.e. w/o
> looking at the grammar, and had I have this code in a coding interview
> question, I wouldn't be quite sure whether this code would work or
> not: one way to interpret what's going on here is to think that the
> expression inside parentheses is a tuple, and since tuples aren't
> context managers, it wouldn't have worked (or maybe not even parsed as
> "as" wouldn't be allowed inside tuple definition since there's no
> "universal as-expression" in Python it's hard to tell what the rules
> are).  But, it turns out there's a form of "with" that has parentheses
> for decoration purposes, and that's why it parses and works to the
> desired effect.

Again, context! All that is important 'here' is how to 'link' the 
file-descriptor with an identifier. Similarly, whilst we could write:

a, b, c = 1, 2, 3

(and BTW that is legal Python - for anyone seeing such for the first time)
and whilst it is shorter (and I've been known to write such), the need 
to read carefully in order to pair-up the relative positions make it 
less readable than

a = 1; b = 2; c = 3

(and some would argue, quite reasonably, that it would be better were 
they on separate lines)

Similarly, many dev.teams have a 'standard' which suggests that once a 
function/method has three or more arguments, relative-positioning should 
go out-the-window, in favor of named-arguments. This speeds 
comprehension and reduces errors.

In the original mental-model, the difficulty was which file-descriptor 
would be paired with which file (previously described). The multiple 
as-s make it more readable and more comprehensible.


> Since it looks like you are doing this for educational reasons, I
> think there's a tiny bit of value to my effort.

That's what we're (all) here for!
(and not forgetting that the OP described a skill-level well below that 
of most of this post and your question, which enabled (and deserved, 
IMHO) appropriate respect).

-- 
Regards,
=dn

From rosuav at gmail.com  Sat Jan 13 22:48:09 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 14 Jan 2024 14:48:09 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
Message-ID: <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>

On Sun, 14 Jan 2024 at 14:43, dn via Python-list <python-list at python.org> wrote:
> Similarly, whilst we could write:
>
> a, b, c = 1, 2, 3
>

I would only do this when it aligns particularly well with the
algorithm being implemented. For example, you could start a Fibonacci
evaluator with "a, b = 0, 1". Otherwise, there's not all that much
reason to unpack three constants in this way.

(Though I am much more likely to use multiple initialization to set a
bunch of things to the SAME value, lilke "a = b = c = 0".)

From rosuav at gmail.com  Sun Jan 14 07:32:10 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 14 Jan 2024 23:32:10 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
Message-ID: <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>

On Sun, 14 Jan 2024 at 23:28, Left Right <olegsivokon at gmail.com> wrote:
> Having worked with a bunch of different grammar languages, the one
> used for Python isn't a recognizable BNF derivative.

That might possibly be because it isn't? It's not BNF. It's PEG. Or
are you a long way behind the times?

> For example, you may say "functions in Python are
> objects", but you cannot put a function definition in the head of the
> for loop clause.

What do you mean?

for x in lambda: ...:
   ...

Perfectly grammatical.

ChrisA

From nulla.epistola at web.de  Sun Jan 14 07:48:22 2024
From: nulla.epistola at web.de (Sibylle Koczian)
Date: Sun, 14 Jan 2024 13:48:22 +0100
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <65F119FD-4B27-4EB3-89FA-07B50F809596@barrys-emacs.org>
References: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <F1CB1FD3-4F43-484A-909D-18C20EE32762@barrys-emacs.org>
 <f9532b2c-dcc6-40ae-a632-b3a39b5beb69@web.de>
 <65F119FD-4B27-4EB3-89FA-07B50F809596@barrys-emacs.org>
Message-ID: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>

Am 09.01.2024 um 12:36 schrieb Barry Scott via Python-list:
> 
> 
>> On 7 Jan 2024, at 15:09, Sibylle Koczian via Python-list <python-list at python.org> wrote:
>>
>> Oh, and the two Windows and Python versions are on two different computers.
>>
>> Will remove the "/env" from my shebang lines, even if I don't understand what's happening.
> 
> Thanks for the details.
> 
> Only thing I can think of is that "python" may be defaulting to mean python 2.
> If you use "#!/usr/bin/env python3" it may work on both.

No, it doesn't. That's the form I started with. When it didn't work I 
thought "python3" might be too old, because Python 2 is dead for so long.
> 
> Did you creates a py.ini file to configure py.exe?
> 
> See if you have %userappdata%\py.ini on either windows 10 or windows 11.
> If so what is its contents?

No to both.
> 
> I've tried with and without a py.ini and cannot duplicate what you see.
> 

It really seems strange. Only thing I can think of - and I don't really 
believe in that idea: as far as I know in Windows 11 the handling of 
PATH has changed. My Python isn't on the path, perhaps that is it. A 
shebang line without "/env" doesn't check the path, right?

Thank you for helping,
Sibylle


From rosuav at gmail.com  Sun Jan 14 08:34:11 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 15 Jan 2024 00:34:11 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
Message-ID: <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>

On Mon, 15 Jan 2024 at 00:27, Left Right <olegsivokon at gmail.com> wrote:
>
> > What do you mean?
> >
> > for x in lambda: ...:
> >   ...
> >
> > Perfectly grammatical.
>
> 1. You put the lambda definition in the wrong place (it should be in
> the left-hand side, or as Python calls it "star_targets", but you put
> it into "star_expressions", which would be where the right-hand side
> is drawn from).
> 2. You used what Python calls "lambdadef" in place of what Python
> calls "function_def". I.e. lambda definition and function definition
> are two different things, at least as far as grammar is considered.
>
> So, you solved a different problem.

You said function. I made a function. You said "head of a for loop
clause". I put it there. Problem was underspecified.

But if you're trying to tell me that a def statement should be a valid
assignment target, I don't know what you're smoking, but I want you to
keep it a long way away from me. Can you name ANY language in which
that would make the slightest bit of sense?

ChrisA

From list1 at tompassin.net  Sun Jan 14 08:54:17 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 14 Jan 2024 08:54:17 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
References: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <F1CB1FD3-4F43-484A-909D-18C20EE32762@barrys-emacs.org>
 <f9532b2c-dcc6-40ae-a632-b3a39b5beb69@web.de>
 <65F119FD-4B27-4EB3-89FA-07B50F809596@barrys-emacs.org>
 <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
Message-ID: <40c0fb26-6865-4a2f-8f7c-586d0ed4d23d@tompassin.net>

On 1/14/2024 7:48 AM, Sibylle Koczian via Python-list wrote:
> Am 09.01.2024 um 12:36 schrieb Barry Scott via Python-list:
>>
>>
>>> On 7 Jan 2024, at 15:09, Sibylle Koczian via Python-list 
>>> <python-list at python.org> wrote:
>>>
>>> Oh, and the two Windows and Python versions are on two different 
>>> computers.
>>>
>>> Will remove the "/env" from my shebang lines, even if I don't 
>>> understand what's happening.
>>
>> Thanks for the details.
>>
>> Only thing I can think of is that "python" may be defaulting to mean 
>> python 2.
>> If you use "#!/usr/bin/env python3" it may work on both.
> 
> No, it doesn't. That's the form I started with. When it didn't work I 
> thought "python3" might be too old, because Python 2 is dead for so long.
>>
>> Did you creates a py.ini file to configure py.exe?
>>
>> See if you have %userappdata%\py.ini on either windows 10 or windows 11.
>> If so what is its contents?
> 
> No to both.
>>
>> I've tried with and without a py.ini and cannot duplicate what you see.
>>
> 
> It really seems strange. Only thing I can think of - and I don't really 
> believe in that idea: as far as I know in Windows 11 the handling of 
> PATH has changed. My Python isn't on the path, perhaps that is it. A 
> shebang line without "/env" doesn't check the path, right?

 From what I've read recently, if you have a Python program that starts 
with a shebang line with any of four standard unix-like paths, then 
Python (not Windows) will look for a version of Python in standard 
locations - *NOT* in the shebang line locations:

"To allow shebang lines in Python scripts to be portable between Unix 
and Windows, this launcher supports a number of ?virtual? commands to 
specify which interpreter to use. The supported virtual commands are:

/usr/bin/env
/usr/bin/python
/usr/local/bin/python
python
"

Also -
"The /usr/bin/env form of shebang line has one further special property. 
Before looking for installed Python interpreters, this form will search 
the executable PATH for a Python executable matching the name provided 
as the first argument. This corresponds to the behaviour of the Unix env 
program, which performs a PATH search. If an executable matching the 
first argument after the env command cannot be found, but the argument 
starts with python, it will be handled as described for the other 
virtual commands.
"

There are some other complications, too, depending on whether you 
specify bare "python" or some specific version. The form with 
"/usr/bin/env" is the closest to the unix behavior, in that it searches 
the PATH.  And you write that your intended version of Python is not on 
the path.

IOW, these shebang lines don't work the way you seem to think that they do.

See https://docs.python.org/3/using/windows.html for a more complete 
rundown.

From avi.e.gross at gmail.com  Sun Jan 14 10:21:50 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sun, 14 Jan 2024 10:21:50 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
Message-ID: <026f01da46fd$65fc1360$31f43a20$@gmail.com>

It can be worth considering why a language is designed or altered in certain
ways to see if there was a tradeoff that made it seem worthwhile or easier
than some other choice.

Python grew and there was regular pressure to add keywords which might break
existing programs. So, yes, sometimes, a keyword was re-used in a different
context. And, yes, it was not originally conceived in a purely object
oriented context.

If you wanted to start over and built a new language very similar to python,
you might indeed make other choices now that seem more seamlessly to fit
together. You could set aside and reserve hundreds of keywords or some way
to extend keywords by insisting anything staring with "key_" cannot be used
in a variable name. You might design all the main objects supported to all
support a function that provides a length as well as every other method
needed so it looks purely object oriented.

But perhaps that would make it a tad harder to program it using other ways.
As an example, I can ask some sort program to order the results by the
length of items by passing it the function that does lengths as an argument.
If instead all we had was a method, that might be a bit different and
perhaps someone would simply make a tiny function that when called, invoked
the method.

So, we have a hybrid of sorts and have to live with it, warts and all, and
some of the warts may be seen by some as beauty marks.




-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Chris Angelico via Python-list
Sent: Sunday, January 14, 2024 7:32 AM
To: python-list at python.org
Subject: Re: Extract lines from file, add to new files

On Sun, 14 Jan 2024 at 23:28, Left Right <olegsivokon at gmail.com> wrote:
> Having worked with a bunch of different grammar languages, the one
> used for Python isn't a recognizable BNF derivative.

That might possibly be because it isn't? It's not BNF. It's PEG. Or
are you a long way behind the times?

> For example, you may say "functions in Python are
> objects", but you cannot put a function definition in the head of the
> for loop clause.

What do you mean?

for x in lambda: ...:
   ...

Perfectly grammatical.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Sun Jan 14 09:53:58 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 14 Jan 2024 09:53:58 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <40c0fb26-6865-4a2f-8f7c-586d0ed4d23d@tompassin.net>
References: <8808f1aa-d707-44b1-98fa-36331e04a50f@web.de>
 <F1CB1FD3-4F43-484A-909D-18C20EE32762@barrys-emacs.org>
 <f9532b2c-dcc6-40ae-a632-b3a39b5beb69@web.de>
 <65F119FD-4B27-4EB3-89FA-07B50F809596@barrys-emacs.org>
 <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <40c0fb26-6865-4a2f-8f7c-586d0ed4d23d@tompassin.net>
Message-ID: <bd50cbc9-48d8-4d1d-9dee-43323281ceb5@tompassin.net>

On 1/14/2024 8:54 AM, Thomas Passin via Python-list wrote:
> On 1/14/2024 7:48 AM, Sibylle Koczian via Python-list wrote:
>> Am 09.01.2024 um 12:36 schrieb Barry Scott via Python-list:
>>>
>>>
>>>> On 7 Jan 2024, at 15:09, Sibylle Koczian via Python-list 
>>>> <python-list at python.org> wrote:
>>>>
>>>> Oh, and the two Windows and Python versions are on two different 
>>>> computers.
>>>>
>>>> Will remove the "/env" from my shebang lines, even if I don't 
>>>> understand what's happening.
>>>
>>> Thanks for the details.
>>>
>>> Only thing I can think of is that "python" may be defaulting to mean 
>>> python 2.
>>> If you use "#!/usr/bin/env python3" it may work on both.
>>
>> No, it doesn't. That's the form I started with. When it didn't work I 
>> thought "python3" might be too old, because Python 2 is dead for so long.
>>>
>>> Did you creates a py.ini file to configure py.exe?
>>>
>>> See if you have %userappdata%\py.ini on either windows 10 or windows 11.
>>> If so what is its contents?
>>
>> No to both.
>>>
>>> I've tried with and without a py.ini and cannot duplicate what you see.
>>>
>>
>> It really seems strange. Only thing I can think of - and I don't 
>> really believe in that idea: as far as I know in Windows 11 the 
>> handling of PATH has changed. My Python isn't on the path, perhaps 
>> that is it. A shebang line without "/env" doesn't check the path, right?
> 
>  From what I've read recently, if you have a Python program that starts 
> with a shebang line with any of four standard unix-like paths, then 
> Python (not Windows) will look for a version of Python in standard 
> locations - *NOT* in the shebang line locations:

I meant to write "the Python launcher", that is, the "py" program. 
Normal Python installs on Windows install the launcher and Windows will 
run it on ".py" files if no other program has been specified on the 
command line.

> "To allow shebang lines in Python scripts to be portable between Unix 
> and Windows, this launcher supports a number of ?virtual? commands to 
> specify which interpreter to use. The supported virtual commands are:
> 
> /usr/bin/env
> /usr/bin/python
> /usr/local/bin/python
> python
> "
> 
> Also -
> "The /usr/bin/env form of shebang line has one further special property. 
> Before looking for installed Python interpreters, this form will search 
> the executable PATH for a Python executable matching the name provided 
> as the first argument. This corresponds to the behaviour of the Unix env 
> program, which performs a PATH search. If an executable matching the 
> first argument after the env command cannot be found, but the argument 
> starts with python, it will be handled as described for the other 
> virtual commands.
> "
> 
> There are some other complications, too, depending on whether you 
> specify bare "python" or some specific version. The form with 
> "/usr/bin/env" is the closest to the unix behavior, in that it searches 
> the PATH.? And you write that your intended version of Python is not on 
> the path.
> 
> IOW, these shebang lines don't work the way you seem to think that they do.
> 
> See https://docs.python.org/3/using/windows.html for a more complete 
> rundown.


From avi.e.gross at gmail.com  Sun Jan 14 14:06:27 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sun, 14 Jan 2024 14:06:27 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
Message-ID: <02b201da471c$c6bafad0$5430f070$@gmail.com>

Chris,

It gets frustrating when people demand too much when strictly speaking, it
is not really needed or may cause new problems.

How often do you really think anyone out there NEEDS to define a function in
the context mentioned?

You provided a way to create an anonymous function and that was not enough.
I wonder if you could throw in the new := walrus operator to similarly make
a named lambda function in a similar way. 

And, in any case, you can make a function factory that returns a function
and call that function with appropriate arguments that might then create a
wide variety of functions by doing the def within.

Languages are tools and need not be the end-all for all people. You do not
want the language to get so complex that programmers do not have any idea
what valid code does. Consider the mess in multiple inheritance and trying
to figure out which of many classes will be the one where a method is
finally called using some diamond algorithm. It is both extremely powerful
but also silly to overuse such features.

Avi

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Chris Angelico via Python-list
Sent: Sunday, January 14, 2024 8:34 AM
To: python-list at python.org
Subject: Re: Extract lines from file, add to new files

On Mon, 15 Jan 2024 at 00:27, Left Right <olegsivokon at gmail.com> wrote:
>
> > What do you mean?
> >
> > for x in lambda: ...:
> >   ...
> >
> > Perfectly grammatical.
>
> 1. You put the lambda definition in the wrong place (it should be in
> the left-hand side, or as Python calls it "star_targets", but you put
> it into "star_expressions", which would be where the right-hand side
> is drawn from).
> 2. You used what Python calls "lambdadef" in place of what Python
> calls "function_def". I.e. lambda definition and function definition
> are two different things, at least as far as grammar is considered.
>
> So, you solved a different problem.

You said function. I made a function. You said "head of a for loop
clause". I put it there. Problem was underspecified.

But if you're trying to tell me that a def statement should be a valid
assignment target, I don't know what you're smoking, but I want you to
keep it a long way away from me. Can you name ANY language in which
that would make the slightest bit of sense?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


From rosuav at gmail.com  Sun Jan 14 16:23:24 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 15 Jan 2024 08:23:24 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
Message-ID: <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>

On Mon, 15 Jan 2024 at 08:15, Left Right <olegsivokon at gmail.com> wrote:
> Python grammar rules prevent function definition from
> appearing in left-hand side of the head of the for loop.  However, a
> variable declaration, which is also a statement, is allowed there.

What is a "variable declaration" in Python? Please elaborate.

ChrisA

From olegsivokon at gmail.com  Sat Jan 13 05:34:29 2024
From: olegsivokon at gmail.com (Left Right)
Date: Sat, 13 Jan 2024 11:34:29 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <ZaH2rwgbIHpCmJtr@anomaly>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
 <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
 <CAJQBtg=vjoMGcTw2a5C2FFdzVnmNBwFyf2Pu0gEXhjYvOq3MGw@mail.gmail.com>
 <ZaH2rwgbIHpCmJtr@anomaly>
Message-ID: <CAJQBtgmLwSUPy8jngpZGyxjVXEeXkr7Rte1z5_uFXZOG8ECH3g@mail.gmail.com>

> What do you mean by this? Most languages I've worked with allow
> variables to be initialized with arbitrary expressions, and a lot of
> languages allow narrowly-scoped variables.

I'm talking about the *left* hand side of the assignment, not the
right hand side. Initialization with arbitrary expression -- arbitrary
expression is on the right. So, that's beside the point.

Here are examples of languages that don't have a feature analogous to
"augmented assignment target":

* Java
* C
* Shell

Examples of languages with limited use of destructuring:

* Haskell
* JavaScript
* Ruby
* Common Lisp

Examples of languages with a superset of destructuring:

* Prolog family of languages (in Prolog it's called "unification")

What is the problem with Python's "augmented assignment target"? -- It
is used in places where syntactically it is more common to introduce
variables (and in languages with the limited use of destructuring,
it's only possible to introduce variables in this context).  For
example, when destructuring an Array in JavaScript, the left-hand side
is restricted syntactically to a very small subset of the language
that, for example, excludes function application.  Typically, it's not
possible to use already defined variables in the left-hand side of the
variable definition, even if destructuring assignment is possible.
Prolog is the example of the opposite, where already defined variables
are allowed on both sides of unification, but Prolog doesn't have
function application in the same sense Python has, so it's still OK.

In general, in languages that aren't like Prolog, conceptually, it's
possible to either *define* variables (with optional initialization)
or to *reuse* them (in the context of assignment that usually looks
similar to initialization), but not both.  The fact that in Python you
can do both in the same place is surprising, eg. in the context of
loops.  Any language that distinguishes between expressions and
statements would have conceptual difficulties with allowing a mix in
the same context.  Typically, variable introduction is a statement in
such languages (as is the case in Python), so using an expression in
the same place as a variable introduction is strange.  To make this
shorter, Python allows:

for <statement> in ... : ...

and

for <expression> in ... : ...

which is unexpected, especially since the first form is a lot more
popular. Because the limited subset of expressions is desirable in
this context, many languages try to "cram" it into this box.  C, after
some standard iterations caved in and allowed statements in the
initialization component of the for loop (but only variable
declaration statements), for example. Other languages like JavaScript
developed a special subset of language for the purpose of describing
the relationship between multiple components of the object being
assigned as variables.

In every case, from the language development perspective, this looks
clumsy and unnecessary, as it is usually easy to write programs that
are exactly equivalent but don't require such workarounds. But, in
practice, programmers want to save a few keystrokes, and this pushes
the language authors to add such "features".  Python is furthermore
unique in how the workaround creates a lot of opportunities for abuse.

> The Python term, at least colloquially, is "tuple unpacking."

Well, why use colloquialism if there's a language specification? Also,
there weren't any tuples used in my example, at least not explicitly
(i could've been a tuple, but that wasn't specified).

From olegsivokon at gmail.com  Sun Jan 14 07:28:20 2024
From: olegsivokon at gmail.com (Left Right)
Date: Sun, 14 Jan 2024 13:28:20 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
Message-ID: <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>

> Second time to ameliorate wording-dispute in this thread! The original
> phrase was: "[modified] BNF". Some of us have worked with various forms
> and evolutions of BNF since back in the days of COBOL-60 proposals, and
> know it when we see it!

OK, here are the conceptual differences between what Python grammar
language does and what you'd expect from anything that's based on BNF,
modified or not:

Python isn't a context-free language, so the grammar that is used to
describe it doesn't actually describe the language... so, it's a
"pretend grammar" that ignores indentation.  BNF is supposed to be
used to describe the language, it's not a "pretend" or "pseudo"
grammar, in a way we have at least two established grammar for
pseudo-code.

BNF and derivatives don't have an inherent mechanism for tiebreaks.
The mechanism is necessary because BNF rules can be tried in any
order.  Some grammar languages derived from BNF declare ambiguous
grammars invalid, some allow ambiguity, but say that the longest
prefix wins, and if there's still ambiguity after that, then such
grammar is invalid, some have special constructs to define "priority"
etc. My reading of Python grammar is that it works like PEG, where
rules are tried in the order they are defined.  This makes it less
expressive, but easier to work with.  This is, probably, the most
fundamental difference between the BNF family and the PEG family.

BNF and family languages rarely incorporate elements of Perl-like
regular expression parsing in the language (i.e. things like
lookaheads, lookbehinds etc.) This is more typical of the PEG family.

On top of this, the Python grammar language has a bunch of
"inventions" that are unique to it (I've never seen any other grammar
language use '.' in the same way Python uses it).  So, there's that
too.

Having worked with a bunch of different grammar languages, the one
used for Python isn't a recognizable BNF derivative.  I think the
authors used this as a description in the same way as today a lot of
programmers would use the word "IDE" to describe any text editor or
"REST" to describe any kind of server-client protocol over HTTP and so
on.  Or, how we'd use "Xerox" to name a copier machine, even if that
company didn't manufacture it, and even if the tech used for copying
is completely different.  And that's why I wrote that the grammar is
actually more like PEG, adding that it's neither, but seems to fall
more into that later category.

> Yes it is hard to read - and even harder to learn-from;

This wasn't my point.  My point is that it's hard to learn languages
that are "one off" in the group languages that all share a similar set
of rules. The difficulty comes from the surprise caused by the unique
use, not because there's something inherently difficult about reading
grammar languages.  In fact, however you look at Python's grammar
language, in a sense, it's a lot easier to read than Python itself
because it has significantly fewer rules.  Of course, the number of
rules doesn't entirely capture the difficulty, but it's a useful
metric.

> In Python, everything is an object. As long as the LHS is a legal-object
> which  makes sense for the situation, it can be used.

This is a very interesting statement... I don't think you are fully
aware of what it might mean :)  Here are just a few questions for you
to ponder:

* What is Python? Is it only Python 3.12?  Is Python 3.11 not Python?
How far back do you go to draw the line?
* What makes something an "object"? Is it the ability to dispatch on?
Is it the inheritance from "object" type?
* What elements of the language do you consider part of the language
that can be included in your "all" set.  Do types belong in that set?
Do expressions belong in that set?  What about comments?

Depending on how you answer these questions, you'd have some further
problems to deal with.  For example, historically, Python had plenty
of things that didn't inherit from "object" but acted similar to one.
I believe "module" objects were among the last to transition into
object inheritance lane, which might have happened some time around
Python 3.5.  Of course, there are plenty of things that are "in
Python", at least due to its grammar, that are hard to describe as
objects (eg. comments).  So, you'd have to make a special subset of
Python language that (eg. excludes comments) to claim that everything
is an object.

Most importantly, however, regardless of what you understand to be an
object, or how you decide to answer any of those questions: what value
does such a claim possibly have? Especially, given the context...

Furthermore, I'm absolutely convinced that what governs the
restrictions on the left-hand side isn't not whether it's understood
to be an object, but the grammar rules, that are unaware of the
concept of objects.  For example, you may say "functions in Python are
objects", but you cannot put a function definition in the head of the
for loop clause.

From olegsivokon at gmail.com  Sun Jan 14 08:27:15 2024
From: olegsivokon at gmail.com (Left Right)
Date: Sun, 14 Jan 2024 14:27:15 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
Message-ID: <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>

> What do you mean?
>
> for x in lambda: ...:
>   ...
>
> Perfectly grammatical.

1. You put the lambda definition in the wrong place (it should be in
the left-hand side, or as Python calls it "star_targets", but you put
it into "star_expressions", which would be where the right-hand side
is drawn from).
2. You used what Python calls "lambdadef" in place of what Python
calls "function_def". I.e. lambda definition and function definition
are two different things, at least as far as grammar is considered.

So, you solved a different problem.

From olegsivokon at gmail.com  Sun Jan 14 16:15:18 2024
From: olegsivokon at gmail.com (Left Right)
Date: Sun, 14 Jan 2024 22:15:18 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <02b201da471c$c6bafad0$5430f070$@gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
Message-ID: <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>

> You said function. I made a function. You said "head of a for loop
> clause". I put it there. Problem was underspecified.

I also wrote a lot of letters, if you combine them very liberally,
without any regard to the order in which they were written or the
context in which they were used, you may come up with very surprising
findings.

> But if you're trying to tell me that a def statement should be a valid
> assignment target,

Why not just read what I wrote and work from there?  No, I didn't
write anything even remotely similar to this...  I don't want function
definition to be an assignment target.  I was giving an example of how
Python grammar works, how the rules govern what can or cannot be used
in a particular place...

In other words, if you aren't sure you understand the question, why
are you trying to reply to it? Is your goal to learn the meaning of
the question by giving arbitrary replies and hoping that the author of
the question restates it so that you understand it?  If so, I believe,
the better strategy would be to simply ask to restate the question.
Will save you the round-trip.

> You provided a way to create an anonymous function and that was not enough.
> I wonder if you could throw in the new := walrus operator to similarly make
> a named lambda function in a similar way.

The person you are replying to didn't understand the question and has
written something irrelevant.  It's not about being "enough".  I
honestly don't know why they are spending so much energy replying to
my messages :|

> Python grew and there was regular pressure to add keywords which might break
> existing programs. So, yes, sometimes, a keyword was re-used in a different
> context.

Why are keywords relevant to this?

> How often do you really think anyone out there NEEDS to define a function in
> the context mentioned?

This isn't about programmers writing programs that aren't about the
language.  It's about programmers who write language-related tools,
like linters, formatters etc.  I.e. the programmers who need to
consider any possible grammar product. And the reason I mentioned
function definition is, this, again: function definition is a
statement.  Python grammar rules prevent function definition from
appearing in left-hand side of the head of the for loop.  However, a
variable declaration, which is also a statement, is allowed there.
Programmers like grammar rules to be consistent, and it's surprising
if a particular larger context allows both statements and expressions.
I also explained why and how language authors would make a decision to
break this consistency: it saves some keystrokes for the programmers.
I.e. allows for shorter programs, while doesn't add any new abilities
to the language.

From PythonList at DancesWithMice.info  Sun Jan 14 17:29:02 2024
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 15 Jan 2024 11:29:02 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
 <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>
Message-ID: <8d468f66-4a25-4450-972a-54a41e125153@DancesWithMice.info>

On 15/01/24 10:23, Chris Angelico via Python-list wrote:
> On Mon, 15 Jan 2024 at 08:15, Left Right <olegsivokon at gmail.com> wrote:
>> Python grammar rules prevent function definition from
>> appearing in left-hand side of the head of the for loop.  However, a
>> variable declaration, which is also a statement, is allowed there.
> 
> What is a "variable declaration" in Python? Please elaborate.

We may be in danger of disappearing down an unintended 'rabbit hole' 
with this side-discussion (he says, with graceful under-statement).


The basic challenge came from my earlier (and blas?) repetition of the 
Python refrain "everything in Python is an object". Which led to:

<<<
For example, you may say "functions in Python are
objects", but you cannot put a function definition in the head of the
for loop clause.
 >>>

Which is logical - to some degree, and in-isolation.

     for def a_function( etc )... in iterable/iterator:

does not make sense. The 'head' (a more generic name, where Python says 
"target_list", that refines down to 'something which can identify the 
generated-value'.

So, no, there's an "everything" which (might be) an object but which 
cannot be used in that scenario.


Two "howevers":

However, instead of looking at the narrow clause, (third comment about 
wording not being taken as an whole!!!)* the full quotation was:

<<<
In Python, everything is an object. As long as the LHS is a legal-object 
which  makes sense for the situation, it can be used.
 >>>

Context!

However, from the docs: "A function definition defines a user-defined 
function object (see section The standard type hierarchy)". Accordingly, 
is a function-definition an object? No! It defines an object.

That said, does anyone think that something like:

     for a_function( etc ) in iterable/iterator:

is acceptable?
- see both Python definition and (full-)quotation.

I've not come-across a language which does allow such - YMMV/mea culpa; 
and am struggling to see how it could possibly be useful.

In-turn, how this discussion could become profitable...


* Looking at the correspondent's email-address (cf 'handle') - and as an 
unfair stereotype, raises the question of issues related to (English) 
language-skills - which, arrogantly implies/assumes that native 
English-speakers are all highly-capable. (?) A negative-interpretation 
is to note his apparent intelligence, but wonder if failing to represent 
others' comments fairly is deliberate, or carelessness. Is there an 
irony in behaving/failing in such, whilst attempting to hold Python's 
structure to some golden-ideal?


Web.Refs:
https://docs.python.org/3/reference/compound_stmts.html#the-for-statement
https://docs.python.org/3/reference/simple_stmts.html#grammar-token-python-grammar-target_list
https://docs.python.org/3/reference/compound_stmts.html#function-definitions

-- 
Regards,
=dn

From PythonList at DancesWithMice.info  Sun Jan 14 17:30:03 2024
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 15 Jan 2024 11:30:03 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
Message-ID: <cc121807-9518-4bbf-a535-0e0d8ceaa99f@DancesWithMice.info>

On 14/01/24 16:48, Chris Angelico wrote:
> On Sun, 14 Jan 2024 at 14:43, dn via Python-list <python-list at python.org> wrote:
>> Similarly, whilst we could write:
>>
>> a, b, c = 1, 2, 3
>>
> 
> I would only do this when it aligns particularly well with the
> algorithm being implemented. For example, you could start a Fibonacci
> evaluator with "a, b = 0, 1". Otherwise, there's not all that much
> reason to unpack three constants in this way.
> 
> (Though I am much more likely to use multiple initialization to set a
> bunch of things to the SAME value, lilke "a = b = c = 0".)

Neatly stated!

-- 
Regards,
=dn


From avi.e.gross at gmail.com  Sun Jan 14 17:36:11 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sun, 14 Jan 2024 17:36:11 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
Message-ID: <030701da473a$136ee180$3a4ca480$@gmail.com>

Straight Ahead, on average,

I am not sure what your beef is as apparently it is always something else than some others assumed.

If your point is that you want consistency, sure that would be nice. But maybe part of the inconsistency I am not sure you mean is an artifact of the language.

There are programming languages where a keyword like "def" is used to create a function and similarly some where variables are declared some special way like "let" or "var" and "const" or "class" to provide the right hints. Some languages skip that and just let you pop up a variable as in "x=5" both declares and instantiates a variable. Strictly speaking, in a language like python, everything is an object and some of the way of creating them remains bound to other ideas like declaring a class uses key words as does declaring a function. 

But consider a language like R which declares a function similar to anything else:

X <- 5
Times2 <- function(x) { x*2 }

The keyword has been moved as compared to the python:

Def Times2(x):
    return(x*2)

It is not a question of being better or worse, but simply speaking, there is no special keyword to start the process before the name, but rather a keyword later that is really similar to a lambda method. Secondarily, not relying on indentation, may make it easier in some cases to insert the function declaration anywhere a statement might fit as there is no special keyword.

Other languages have their own chosen ways and many will fail various tests you can come up with.

Could python have chosen some other way that would have fit some grammar needs? Probably. But it did not.

If you want to play a common grammar game, you can define a higher-level construct I will simply call A  that is decomposed into two or more subcontexts one of which is B and then populate the tree on each side with more and more specific stuff. Then when talking about what is allowed in context alpha, where all expressions are allowed, say it supports A or anything in the tree below it. In another context, say it only supports B and anything below that.

Think of it a bit like subclassing.

Perhaps you can then build a complex description that can be instantiated by code to define all valid programs from invalid ones. But it would remain a pain to implement all the tools you want to help you, including in an editor or programming environment, where figuring out the context may become very difficult.

Still, making things very loose and general so that your design looks simple has many negative tradeoffs too, including allowing rather nonsensical things.

People who create programming languages have various goals in mind that guide what they choose. Python was not made to be a mathematically perfect object that guided being able to have programs proven to work and so on. It is acknowledged various aspects do not please some people or others and I am not defending it.

I am wondering if what is being discussed is in any way a serious issue.

The original question in this thread really was a minor one and how it became whatever this is, well, I give up! LOL!


-----Original Message-----
From: Left Right <olegsivokon at gmail.com> 
Sent: Sunday, January 14, 2024 4:15 PM
To: avi.e.gross at gmail.com
Cc: Chris Angelico <rosuav at gmail.com>; python-list at python.org
Subject: Re: Extract lines from file, add to new files

> You said function. I made a function. You said "head of a for loop
> clause". I put it there. Problem was underspecified.

I also wrote a lot of letters, if you combine them very liberally,
without any regard to the order in which they were written or the
context in which they were used, you may come up with very surprising
findings.

> But if you're trying to tell me that a def statement should be a valid
> assignment target,

Why not just read what I wrote and work from there?  No, I didn't
write anything even remotely similar to this...  I don't want function
definition to be an assignment target.  I was giving an example of how
Python grammar works, how the rules govern what can or cannot be used
in a particular place...

In other words, if you aren't sure you understand the question, why
are you trying to reply to it? Is your goal to learn the meaning of
the question by giving arbitrary replies and hoping that the author of
the question restates it so that you understand it?  If so, I believe,
the better strategy would be to simply ask to restate the question.
Will save you the round-trip.

> You provided a way to create an anonymous function and that was not enough.
> I wonder if you could throw in the new := walrus operator to similarly make
> a named lambda function in a similar way.

The person you are replying to didn't understand the question and has
written something irrelevant.  It's not about being "enough".  I
honestly don't know why they are spending so much energy replying to
my messages :|

> Python grew and there was regular pressure to add keywords which might break
> existing programs. So, yes, sometimes, a keyword was re-used in a different
> context.

Why are keywords relevant to this?

> How often do you really think anyone out there NEEDS to define a function in
> the context mentioned?

This isn't about programmers writing programs that aren't about the
language.  It's about programmers who write language-related tools,
like linters, formatters etc.  I.e. the programmers who need to
consider any possible grammar product. And the reason I mentioned
function definition is, this, again: function definition is a
statement.  Python grammar rules prevent function definition from
appearing in left-hand side of the head of the for loop.  However, a
variable declaration, which is also a statement, is allowed there.
Programmers like grammar rules to be consistent, and it's surprising
if a particular larger context allows both statements and expressions.
I also explained why and how language authors would make a decision to
break this consistency: it saves some keystrokes for the programmers.
I.e. allows for shorter programs, while doesn't add any new abilities
to the language.


From rosuav at gmail.com  Sun Jan 14 17:47:32 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 15 Jan 2024 09:47:32 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <8d468f66-4a25-4450-972a-54a41e125153@DancesWithMice.info>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
 <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>
 <8d468f66-4a25-4450-972a-54a41e125153@DancesWithMice.info>
Message-ID: <CAPTjJmob-Y8VswZxALnF21Bkf=Y5CJdDfgjTLOhawWY5pA3VTw@mail.gmail.com>

On Mon, 15 Jan 2024 at 09:40, dn via Python-list <python-list at python.org> wrote:
> The basic challenge came from my earlier (and blas?) repetition of the
> Python refrain "everything in Python is an object". Which led to:
>
> <<<
> For example, you may say "functions in Python are
> objects", but you cannot put a function definition in the head of the
> for loop clause.
>  >>>
>
> Which is logical - to some degree, and in-isolation.
>
>      for def a_function( etc )... in iterable/iterator:
>
> does not make sense. The 'head' (a more generic name, where Python says
> "target_list", that refines down to 'something which can identify the
> generated-value'.
>
> So, no, there's an "everything" which (might be) an object but which
> cannot be used in that scenario.

More accurately, every VALUE in Python is an object. This does not
mean that syntax is an object. Very few languages would say that every
single grammatical element is a value.

Yes, it's sloppy to say "everything" is an object, but it's also
rather nonintuitive to claim that, therefore, syntax elements are all
objects. It's like claiming that everything that this dealership sells
is a car (or "everything in this dealership is a car"), and therefore
the dealership's name must itself be a car.

> That said, does anyone think that something like:
>
>      for a_function( etc ) in iterable/iterator:
>
> is acceptable?
> - see both Python definition and (full-)quotation.
>
> I've not come-across a language which does allow such - YMMV/mea culpa;
> and am struggling to see how it could possibly be useful.

You could do something close to that:

for a_function(etc)[0] in iterable: ...

because an assignment target can contain an arbitrary expression
followed by the subscript.

> * Looking at the correspondent's email-address (cf 'handle') - and as an
> unfair stereotype, raises the question of issues related to (English)
> language-skills - which, arrogantly implies/assumes that native
> English-speakers are all highly-capable. (?) A negative-interpretation
> is to note his apparent intelligence, but wonder if failing to represent
> others' comments fairly is deliberate, or carelessness. Is there an
> irony in behaving/failing in such, whilst attempting to hold Python's
> structure to some golden-ideal?

Seems likely.

ChrisA

From avi.e.gross at gmail.com  Sun Jan 14 17:56:03 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sun, 14 Jan 2024 17:56:03 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
Message-ID: <030c01da473c$da1fd4e0$8e5f7ea0$@gmail.com>

Whoa, ???? ????????!

I do not understand any arguments about whether comments are, or are not an
object.

>From one perspective, python comments have even less meaning than whitespace
and simply do not exist. I mean once a naked "#" is seen, the rest of that
line is effectively discarded by the interpreter. The supposed multi-line
comments though, meaning something like a triple set of quotes up to another
set are probably objects in the sense that they create a text literal in a
context where there is no usual pointer to them and are thus largely ignored
and perhaps garbage collected. Some programs look for them in code as part
of documentation and it is possible they are stored in some object that
"holds" aspects of a function.

But I think that talking about some inconsistency in python because comments
are not an object is a bit silly. Why should they be? If I choose to leave
lots of blank lines between my function definitions or statements, do they
need to be made into an object, or ignored as superfluous but legal
whitespace?

I have seen many debates about some form of purity and generally, they get
boring. Nobody in their right mind can defend any computer language as being
100% right for every purpose. Python has some reasonable tradeoffs and is
highly popular and there are other languages with other tradeoffs you can
use instead. At this point, making changes without disrupting things gets
ever harder.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Left Right via Python-list
Sent: Sunday, January 14, 2024 7:28 AM
To: Chris Angelico <rosuav at gmail.com>
Cc: python-list at python.org
Subject: Re: Extract lines from file, add to new files

> Second time to ameliorate wording-dispute in this thread! The original
> phrase was: "[modified] BNF". Some of us have worked with various forms
> and evolutions of BNF since back in the days of COBOL-60 proposals, and
> know it when we see it!

OK, here are the conceptual differences between what Python grammar
language does and what you'd expect from anything that's based on BNF,
modified or not:

Python isn't a context-free language, so the grammar that is used to
describe it doesn't actually describe the language... so, it's a
"pretend grammar" that ignores indentation.  BNF is supposed to be
used to describe the language, it's not a "pretend" or "pseudo"
grammar, in a way we have at least two established grammar for
pseudo-code.

BNF and derivatives don't have an inherent mechanism for tiebreaks.
The mechanism is necessary because BNF rules can be tried in any
order.  Some grammar languages derived from BNF declare ambiguous
grammars invalid, some allow ambiguity, but say that the longest
prefix wins, and if there's still ambiguity after that, then such
grammar is invalid, some have special constructs to define "priority"
etc. My reading of Python grammar is that it works like PEG, where
rules are tried in the order they are defined.  This makes it less
expressive, but easier to work with.  This is, probably, the most
fundamental difference between the BNF family and the PEG family.

BNF and family languages rarely incorporate elements of Perl-like
regular expression parsing in the language (i.e. things like
lookaheads, lookbehinds etc.) This is more typical of the PEG family.

On top of this, the Python grammar language has a bunch of
"inventions" that are unique to it (I've never seen any other grammar
language use '.' in the same way Python uses it).  So, there's that
too.

Having worked with a bunch of different grammar languages, the one
used for Python isn't a recognizable BNF derivative.  I think the
authors used this as a description in the same way as today a lot of
programmers would use the word "IDE" to describe any text editor or
"REST" to describe any kind of server-client protocol over HTTP and so
on.  Or, how we'd use "Xerox" to name a copier machine, even if that
company didn't manufacture it, and even if the tech used for copying
is completely different.  And that's why I wrote that the grammar is
actually more like PEG, adding that it's neither, but seems to fall
more into that later category.

> Yes it is hard to read - and even harder to learn-from;

This wasn't my point.  My point is that it's hard to learn languages
that are "one off" in the group languages that all share a similar set
of rules. The difficulty comes from the surprise caused by the unique
use, not because there's something inherently difficult about reading
grammar languages.  In fact, however you look at Python's grammar
language, in a sense, it's a lot easier to read than Python itself
because it has significantly fewer rules.  Of course, the number of
rules doesn't entirely capture the difficulty, but it's a useful
metric.

> In Python, everything is an object. As long as the LHS is a legal-object
> which  makes sense for the situation, it can be used.

This is a very interesting statement... I don't think you are fully
aware of what it might mean :)  Here are just a few questions for you
to ponder:

* What is Python? Is it only Python 3.12?  Is Python 3.11 not Python?
How far back do you go to draw the line?
* What makes something an "object"? Is it the ability to dispatch on?
Is it the inheritance from "object" type?
* What elements of the language do you consider part of the language
that can be included in your "all" set.  Do types belong in that set?
Do expressions belong in that set?  What about comments?

Depending on how you answer these questions, you'd have some further
problems to deal with.  For example, historically, Python had plenty
of things that didn't inherit from "object" but acted similar to one.
I believe "module" objects were among the last to transition into
object inheritance lane, which might have happened some time around
Python 3.5.  Of course, there are plenty of things that are "in
Python", at least due to its grammar, that are hard to describe as
objects (eg. comments).  So, you'd have to make a special subset of
Python language that (eg. excludes comments) to claim that everything
is an object.

Most importantly, however, regardless of what you understand to be an
object, or how you decide to answer any of those questions: what value
does such a claim possibly have? Especially, given the context...

Furthermore, I'm absolutely convinced that what governs the
restrictions on the left-hand side isn't not whether it's understood
to be an object, but the grammar rules, that are unaware of the
concept of objects.  For example, you may say "functions in Python are
objects", but you cannot put a function definition in the head of the
for loop clause.
-- 
https://mail.python.org/mailman/listinfo/python-list


From miked at dewhirst.com.au  Sun Jan 14 18:46:06 2024
From: miked at dewhirst.com.au (Mike Dewhirst)
Date: Mon, 15 Jan 2024 10:46:06 +1100
Subject: =?US-ASCII?Q?Re:_Python_3.12.1,_Windows_11:_shebang_line_?=
 =?US-ASCII?Q?#!/usr/bin/env_python3=0D__doesn't_work_any_more?=
In-Reply-To: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
Message-ID: <4TCsVz4CzPznVGL@mail.python.org>

In Windows the provided methods for running complex command lines are either a batch file or a shortcut.Someone very kindly pointed out to me in this thread that there is a PEP for py.exe.?I don't use py.exe originally because I didn't trust it believing it was a new-fangled Microsoft trick. I did read that PEP but it has no relevance for my mixed Windows/Linux environments.?On reflection I now believe I won't use py.exe because it introduces an unnecessary layer of indirection.The? bottom line is that you still need to know which Python a particular set of circumstances demands and if you use py.exe you then need to also understand how it chooses and how it interprets shebang lines written for your Linux environment.?And if that isn't your situation I have jumped to the wrong conclusion.I have found no problem in Windows when I use shebang lines in scripts intended for execution in both Linux and Windows. They are ignored unless you use py.exe.My advice is to give up py.exe unless your use case mandates shebang lines in Windows.M--(Unsigned mail from my phone)
-------- Original message --------From: Sibylle Koczian via Python-list <python-list at python.org> Date: 14/1/24  23:59  (GMT+10:00) To: python-list at python.org Subject: Re: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
  doesn't work any more Am 09.01.2024 um 12:36 schrieb Barry Scott via Python-list:> > >> On 7 Jan 2024, at 15:09, Sibylle Koczian via Python-list <python-list at python.org> wrote:>>>> Oh, and the two Windows and Python versions are on two different computers.>>>> Will remove the "/env" from my shebang lines, even if I don't understand what's happening.> > Thanks for the details.> > Only thing I can think of is that "python" may be defaulting to mean python 2.> If you use "#!/usr/bin/env python3" it may work on both.No, it doesn't. That's the form I started with. When it didn't work I thought "python3" might be too old, because Python 2 is dead for so long.> > Did you creates a py.ini file to configure py.exe?> > See if you have %userappdata%\py.ini on either windows 10 or windows 11.> If so what is its contents?No to both.> > I've tried with and without a py.ini and cannot duplicate what you see.> It really seems strange. Only thing I can think of - and I don't really believe in that idea: as far as I know in Windows 11 the handling of PATH has changed. My Python isn't on the path, perhaps that is it. A shebang line without "/env" doesn't check the path, right?Thank you for helping,Sibylle-- https://mail.python.org/mailman/listinfo/python-list

From PythonList at DancesWithMice.info  Sun Jan 14 19:54:16 2024
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 15 Jan 2024 13:54:16 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
Message-ID: <a7323568-a438-4e7c-a6b2-9006e5ca43d9@DancesWithMice.info>

On 15/01/24 01:28, Left Right wrote:
>> Second time to ameliorate wording-dispute in this thread! The original
>> phrase was: "[modified] BNF". Some of us have worked with various forms
>> and evolutions of BNF since back in the days of COBOL-60 proposals, and
>> know it when we see it!
> 
> OK, here are the conceptual differences between what Python grammar
> language does and what you'd expect from anything that's based on BNF,
> modified or not:
> 
> Python isn't a context-free language, so the grammar that is used to
> describe it doesn't actually describe the language... so, it's a
> "pretend grammar" that ignores indentation.  BNF is supposed to be
> used to describe the language, it's not a "pretend" or "pseudo"
> grammar, in a way we have at least two established grammar for
> pseudo-code.
> 
> BNF and derivatives don't have an inherent mechanism for tiebreaks.
> The mechanism is necessary because BNF rules can be tried in any
> order.  Some grammar languages derived from BNF declare ambiguous
> grammars invalid, some allow ambiguity, but say that the longest
> prefix wins, and if there's still ambiguity after that, then such
> grammar is invalid, some have special constructs to define "priority"
> etc. My reading of Python grammar is that it works like PEG, where
> rules are tried in the order they are defined.  This makes it less
> expressive, but easier to work with.  This is, probably, the most
> fundamental difference between the BNF family and the PEG family.
> 
> BNF and family languages rarely incorporate elements of Perl-like
> regular expression parsing in the language (i.e. things like
> lookaheads, lookbehinds etc.) This is more typical of the PEG family.
> 
> On top of this, the Python grammar language has a bunch of
> "inventions" that are unique to it (I've never seen any other grammar
> language use '.' in the same way Python uses it).  So, there's that
> too.
> 
> Having worked with a bunch of different grammar languages, the one
> used for Python isn't a recognizable BNF derivative.  I think the
> authors used this as a description in the same way as today a lot of
> programmers would use the word "IDE" to describe any text editor or
> "REST" to describe any kind of server-client protocol over HTTP and so
> on.  Or, how we'd use "Xerox" to name a copier machine, even if that
> company didn't manufacture it, and even if the tech used for copying
> is completely different.  And that's why I wrote that the grammar is
> actually more like PEG, adding that it's neither, but seems to fall
> more into that later category.

We are in broad-agreement. I'd enjoy the discussion, except that I can't 
see where it is leading or how it will benefit you...

Where we seem to differ is the length of time we've been part of the 
Python-world.

An important point is that languages are often described as 'living', 
because they evolve over time. I'm relieved to note that the youngsters 
in the family have stopped describing 'everything' as "awesome". That 
word, has taken-on aspects of the original meanings of "awful". The 
former being positive-factors, and the negative ascribed to the latter. 
However, we don't have to step far back into history to find that 
"awful" meant 'full of awe' or awe-inspiring - with no connotation of 
'good' or 'bad'. There was no need for a word with opposite-meaning, 
apart from "mundane".

Similarly, spoken-language often lacks precision. My CO, to this day, 
uses phrases like "all done" [with time for me to take a breath, 
followed by "except ...", eg "are we ready to leave for ...?". This 
drives me nuts. (the word "nuts" requires consideration too!) Thus, have 
learned to elicit the pertinent information with phrasing such as "what 
is left to be done?" rather than arguing the illogic between "all" and 
"except".

Python is not different. We're currently looking at version 3.12, which 
implies three major iterations of the language. Accordingly, when it was 
a 'green fields project', what may have been a pure and elegant 
expression of a language, has become "acquisitive". Sometimes these 
additions-and-alterations have required compromise, eg lack of elegance. 
Sometimes, it has been realised that there is a flaw, and things need to 
be improved, even removed.

You've made comments about BNF. Like Python, it is a 'language' (a bit 
of stretch, but...) which has also been pulled, poked, and perhaps 
contorted over time. It had a body of folk who understood its 
expression, and therefore it made sense to stick with some variation of 
it (even warts-and-all) rather than going with something 'new'.


All of which creates the sorts of issues described.

The PEG-parser is a comparatively-recent improvement - other aspects 
previously-mentioned.

I'm not going to touch "Perl".

Is there really some sort of standard-understanding/format for 
"pseudo-code"? (ISO?)

Yes, there's some 'pretense' (not sure I'd choose that exact term). 
However, it is working for the vast-majority. Perhaps after using Python 
for so long, I'm comfortable accepting such 'pretense', in the same way 
that when 'the kids' use phrases like "yeah, nah" which are totally 
devoid of logical-meaning, I can deduce the attitude behind their utterance.

So, again, yes I think you're right about what appears in the docs - why 
change it if we can accept that what was there before can still be 
applied today? Again, a comfort for those who've been-around, but 
perhaps failing to take into account others. (?)

Recently we 'lost' Niklaus Wirth. I recall the days when he was part of 
teams looking at languages. Back then we contemplated improvements to 
COBOL, FORTRAN, even PL/1 (an attempt said to take the 'best' of the 
previous two - but the scuttlebutt suggested its result was the 
opposite), ALGOL68, and the idea of a new language for the world's 
largest user of computers: Ada. We devoured new drafts of future 
standards; but they were thick tomes. Soon after, Wirth simplified 
rather than expanded, and developed Pascal. When I first saw this I was 
stunned, then attracted to its simplicity, but then steered-away once 
realised that it needed 'more' to cope with 'the outside world'. 
Even-so, lessons were learned from Pascal, and its development path, and 
thus Oberon is 'same but different'. Things change. We learn (and 
improve - hopefully). etc, etc...


As mentioned, the value of this discussion would be better realised on 
the Python-Ideas Discussion List or amongst the Core-Devs - and there is 
a Python Docs Team, if you'd prefer to contribute at that level.


Back to the point that has been made, repeatedly: relying upon a single 
source is not likely to be helpful. We've been talking about 'rules', 
let's expand that out to "laws". I am unaware of a jurisdiction which 
relies totally upon written legislation. Every legal system brings those 
into the real-world with interpretation and "case law" or "the law of 
precedent". Often, the latter ends-up contorting or stretching the 
former in ways the original authors did not (even, could not) imagine.

Life, the universe, and everything!


>> Yes it is hard to read - and even harder to learn-from;
> 
> This wasn't my point.  My point is that it's hard to learn languages
> that are "one off" in the group languages that all share a similar set
> of rules. The difficulty comes from the surprise caused by the unique
> use, not because there's something inherently difficult about reading
> grammar languages.  In fact, however you look at Python's grammar
> language, in a sense, it's a lot easier to read than Python itself
> because it has significantly fewer rules.  Of course, the number of
> rules doesn't entirely capture the difficulty, but it's a useful
> metric.

Rules? What rules?

Each (programming) language was originally developed with specific 
objectives in-mind. Usually the history would include determinations to 
avoid certain aspects of other/existing languages.

As long as that objective includes 'pushing the envelope' or 'making 
things better'; any 'rules' must bend or break in-concert.


>> In Python, everything is an object. As long as the LHS is a legal-object
>> which  makes sense for the situation, it can be used.
> 
> This is a very interesting statement... I don't think you are fully
> aware of what it might mean :)  Here are just a few questions for you
> to ponder:
> 
> * What is Python? Is it only Python 3.12?  Is Python 3.11 not Python?
> How far back do you go to draw the line?
> * What makes something an "object"? Is it the ability to dispatch on?
> Is it the inheritance from "object" type?
> * What elements of the language do you consider part of the language
> that can be included in your "all" set.  Do types belong in that set?
> Do expressions belong in that set?  What about comments?

Topic addressed elsewhere.

Yet, here we are discussing a Python 'eco-system', which consists of 
numbers of parts. That those parts do, or do not, integrate as tidily or 
elegantly as we might like, is a fact of life (also previously discussed).

What language might fulfill these exacting criteria?
- and if that exists, what are you doing here?


> Depending on how you answer these questions, you'd have some further
> problems to deal with.  For example, historically, Python had plenty
> of things that didn't inherit from "object" but acted similar to one.
> I believe "module" objects were among the last to transition into
> object inheritance lane, which might have happened some time around
> Python 3.5.  Of course, there are plenty of things that are "in
> Python", at least due to its grammar, that are hard to describe as
> objects (eg. comments).  So, you'd have to make a special subset of
> Python language that (eg. excludes comments) to claim that everything
> is an object.
> 
> Most importantly, however, regardless of what you understand to be an
> object, or how you decide to answer any of those questions: what value
> does such a claim possibly have? Especially, given the context...

Intriguing. I laughed when I first saw Python's docstrings (one type of 
"comment") because it was obvious (to me) that they were being treated 
as strings. The fact that they were not identified ("assigned" or 
"bound", if you prefer) is neither here-nor-there.
(just as, on the RHS, all Python-functions return a value, even if it is 
None - which is not obvious to many, at-first)

Admittedly, a #-comment does not qualify as an object; but then the hash 
is an exclusion signal, which the lexer understands as ending the 
logical line.

Thus, a comment has meaning at 'compile time', but not at 'execution 
time'. Such would be true, strictly-speaking. However, most of us would 
say that a comment 'has no meaning' in terms of the parser, and what it 
delivers.

Shall we change the phrase to "everything in Python, at run-time, is an 
object"? As a phrase it is obiter-dictum, not ratio decidendi! (to use a 
language which has only extended in dubious modes for the last 
couple-of-thousand years - but which still has illogical structure)

Suspect that clumsy exclusion also lacks precision to the standard being 
demanded. Thus return to the suggestion that you seem in the wrong 
place, because Python doesn't meet the required standard. Sorry!


-- 
Regards,
=dn


From PythonList at DancesWithMice.info  Sun Jan 14 20:01:07 2024
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 15 Jan 2024 14:01:07 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <02b201da471c$c6bafad0$5430f070$@gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
Message-ID: <be82181d-2ac6-46c0-abb0-f09e66443894@DancesWithMice.info>

On 15/01/24 08:06, AVI GROSS via Python-list wrote:
...> You provided a way to create an anonymous function and that was not 
enough.
> I wonder if you could throw in the new := walrus operator to similarly make
> a named lambda function in a similar way.

Why would @Chris have anything to do with the 'walrus-operator'?

PS our interlocutor doesn't like colloquialisms such as these - despite 
them being near-and-dear to our hearts!

-- 
Regards,
=dn

From PythonList at DancesWithMice.info  Sun Jan 14 20:10:29 2024
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 15 Jan 2024 14:10:29 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmob-Y8VswZxALnF21Bkf=Y5CJdDfgjTLOhawWY5pA3VTw@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
 <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>
 <8d468f66-4a25-4450-972a-54a41e125153@DancesWithMice.info>
 <CAPTjJmob-Y8VswZxALnF21Bkf=Y5CJdDfgjTLOhawWY5pA3VTw@mail.gmail.com>
Message-ID: <c4ae0113-856c-4f54-a12b-24342b6ad4e1@DancesWithMice.info>

On 15/01/24 11:47, Chris Angelico via Python-list wrote:
> On Mon, 15 Jan 2024 at 09:40, dn via Python-list <python-list at python.org> wrote:
>> The basic challenge came from my earlier (and blas?) repetition of the
>> Python refrain "everything in Python is an object". Which led to:
...

>> So, no, there's an "everything" which (might be) an object but which
>> cannot be used in that scenario.
> 
> More accurately, every VALUE in Python is an object. This does not
> mean that syntax is an object. Very few languages would say that every
> single grammatical element is a value.

+1

Thank you. This old-dog will try to re-learn that term...


> Yes, it's sloppy to say "everything" is an object, but it's also
> rather nonintuitive to claim that, therefore, syntax elements are all
> objects. It's like claiming that everything that this dealership sells
> is a car (or "everything in this dealership is a car"), and therefore
> the dealership's name must itself be a car.

To say nothing of 'extended warranties'!
(and their dubious rationale)


>> That said, does anyone think that something like:
>>
>>       for a_function( etc ) in iterable/iterator:
>>
>> is acceptable?
>> - see both Python definition and (full-)quotation.
>>
>> I've not come-across a language which does allow such - YMMV/mea culpa;
>> and am struggling to see how it could possibly be useful.
> 
> You could do something close to that:
> 
> for a_function(etc)[0] in iterable: ...
> 
> because an assignment target can contain an arbitrary expression
> followed by the subscript.

Here's another witticism I'll often toss at trainees (in many languages, 
and especially in UX): just because we can do it, doesn't make it a good 
idea!


>> * Looking at the correspondent's email-address (cf 'handle') - and as an
>> unfair stereotype, raises the question of issues related to (English)
>> language-skills - which, arrogantly implies/assumes that native
>> English-speakers are all highly-capable. (?) A negative-interpretation
>> is to note his apparent intelligence, but wonder if failing to represent
>> others' comments fairly is deliberate, or carelessness. Is there an
>> irony in behaving/failing in such, whilst attempting to hold Python's
>> structure to some golden-ideal?
> 
> Seems likely.


-- 
Regards,
=dn

From rosuav at gmail.com  Sun Jan 14 20:33:56 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 15 Jan 2024 12:33:56 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <c4ae0113-856c-4f54-a12b-24342b6ad4e1@DancesWithMice.info>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
 <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>
 <8d468f66-4a25-4450-972a-54a41e125153@DancesWithMice.info>
 <CAPTjJmob-Y8VswZxALnF21Bkf=Y5CJdDfgjTLOhawWY5pA3VTw@mail.gmail.com>
 <c4ae0113-856c-4f54-a12b-24342b6ad4e1@DancesWithMice.info>
Message-ID: <CAPTjJmqLu0ZUcb_f1oo1jFDb_p4Vn9by0CGj80YtLrz2=Sofcw@mail.gmail.com>

On Mon, 15 Jan 2024 at 12:12, dn via Python-list <python-list at python.org> wrote:
> Here's another witticism I'll often toss at trainees (in many languages,
> and especially in UX): just because we can do it, doesn't make it a good
> idea!
>

Programming. We were so busy with whether we COULD that we didn't stop
to think if we SHOULD.

I think that's basically my whole life.

ChrisA

From PythonList at DancesWithMice.info  Sun Jan 14 20:41:25 2024
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 15 Jan 2024 14:41:25 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmqLu0ZUcb_f1oo1jFDb_p4Vn9by0CGj80YtLrz2=Sofcw@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
 <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>
 <8d468f66-4a25-4450-972a-54a41e125153@DancesWithMice.info>
 <CAPTjJmob-Y8VswZxALnF21Bkf=Y5CJdDfgjTLOhawWY5pA3VTw@mail.gmail.com>
 <c4ae0113-856c-4f54-a12b-24342b6ad4e1@DancesWithMice.info>
 <CAPTjJmqLu0ZUcb_f1oo1jFDb_p4Vn9by0CGj80YtLrz2=Sofcw@mail.gmail.com>
Message-ID: <fda229a0-8587-4cbf-8244-95705239ef5e@DancesWithMice.info>

On 15/01/24 14:33, Chris Angelico via Python-list wrote:
> On Mon, 15 Jan 2024 at 12:12, dn via Python-list <python-list at python.org> wrote:
>> Here's another witticism I'll often toss at trainees (in many languages,
>> and especially in UX): just because we can do it, doesn't make it a good
>> idea!
>>
> 
> Programming. We were so busy with whether we COULD that we didn't stop
> to think if we SHOULD.
> 
> I think that's basically my whole life.

Don't be too hard on yourself.

My life-advice has always been "don't do anything I wouldn't do"
- which pretty-much gives you carte blanche.

-- 
Regards,
=dn

From rosuav at gmail.com  Sun Jan 14 20:45:47 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 15 Jan 2024 12:45:47 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <fda229a0-8587-4cbf-8244-95705239ef5e@DancesWithMice.info>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
 <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>
 <8d468f66-4a25-4450-972a-54a41e125153@DancesWithMice.info>
 <CAPTjJmob-Y8VswZxALnF21Bkf=Y5CJdDfgjTLOhawWY5pA3VTw@mail.gmail.com>
 <c4ae0113-856c-4f54-a12b-24342b6ad4e1@DancesWithMice.info>
 <CAPTjJmqLu0ZUcb_f1oo1jFDb_p4Vn9by0CGj80YtLrz2=Sofcw@mail.gmail.com>
 <fda229a0-8587-4cbf-8244-95705239ef5e@DancesWithMice.info>
Message-ID: <CAPTjJmrKkhntDqC1A3F3mf1XFJzJapmGAPCsK8FxESpgZB-JbQ@mail.gmail.com>

On Mon, 15 Jan 2024 at 12:42, dn via Python-list <python-list at python.org> wrote:
>
> On 15/01/24 14:33, Chris Angelico via Python-list wrote:
> > On Mon, 15 Jan 2024 at 12:12, dn via Python-list <python-list at python.org> wrote:
> >> Here's another witticism I'll often toss at trainees (in many languages,
> >> and especially in UX): just because we can do it, doesn't make it a good
> >> idea!
> >>
> >
> > Programming. We were so busy with whether we COULD that we didn't stop
> > to think if we SHOULD.
> >
> > I think that's basically my whole life.
>
> Don't be too hard on yourself.
>
> My life-advice has always been "don't do anything I wouldn't do"
> - which pretty-much gives you carte blanche.
>

I would NEVER give that advice to anyone. They would leave the dishes
unwashed, and a number of other problems :)

ChrisA

From PythonList at DancesWithMice.info  Sun Jan 14 20:49:03 2024
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 15 Jan 2024 14:49:03 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <CAPTjJmrKkhntDqC1A3F3mf1XFJzJapmGAPCsK8FxESpgZB-JbQ@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <CAPTjJmrvUxenuBptvTDKiXptnzvLZQRhhYR_nuRTGibE5LaB3Q@mail.gmail.com>
 <CAJQBtgk23N+TZ-Wce2iHwZo4pGX14gL2+k6NDre=TJ4c96wt=A@mail.gmail.com>
 <CAPTjJmqxy7CdqfgX0GQDQB8C4t35rUrrch3yM_wG1jJ6rU0k5w@mail.gmail.com>
 <02b201da471c$c6bafad0$5430f070$@gmail.com>
 <CAJQBtgntgGc6oFqLS0nKuG8tAZT+fSA0d6on3Xw=0rHB_mANBQ@mail.gmail.com>
 <CAPTjJmq2iXXuiRa0CHPddBQHm3kcoxGWW42sVAJKT9TUFxDiJw@mail.gmail.com>
 <8d468f66-4a25-4450-972a-54a41e125153@DancesWithMice.info>
 <CAPTjJmob-Y8VswZxALnF21Bkf=Y5CJdDfgjTLOhawWY5pA3VTw@mail.gmail.com>
 <c4ae0113-856c-4f54-a12b-24342b6ad4e1@DancesWithMice.info>
 <CAPTjJmqLu0ZUcb_f1oo1jFDb_p4Vn9by0CGj80YtLrz2=Sofcw@mail.gmail.com>
 <fda229a0-8587-4cbf-8244-95705239ef5e@DancesWithMice.info>
 <CAPTjJmrKkhntDqC1A3F3mf1XFJzJapmGAPCsK8FxESpgZB-JbQ@mail.gmail.com>
Message-ID: <34f9781f-0869-47c4-8fc5-baf119780ec6@DancesWithMice.info>

On 15/01/24 14:45, Chris Angelico wrote:
> On Mon, 15 Jan 2024 at 12:42, dn via Python-list <python-list at python.org> wrote:
>>
>> On 15/01/24 14:33, Chris Angelico via Python-list wrote:
>>> On Mon, 15 Jan 2024 at 12:12, dn via Python-list <python-list at python.org> wrote:
>>>> Here's another witticism I'll often toss at trainees (in many languages,
>>>> and especially in UX): just because we can do it, doesn't make it a good
>>>> idea!
>>>>
>>>
>>> Programming. We were so busy with whether we COULD that we didn't stop
>>> to think if we SHOULD.
>>>
>>> I think that's basically my whole life.
>>
>> Don't be too hard on yourself.
>>
>> My life-advice has always been "don't do anything I wouldn't do"
>> - which pretty-much gives you carte blanche.
>>
> 
> I would NEVER give that advice to anyone. They would leave the dishes
> unwashed, and a number of other problems :)

I'd soon have you sorted-out - except for one small problem: all the 
dishes have been done!

If you can get here before the rain, the lawn needs mowing...

-- 
Regards,
=dn


From greg.ewing at canterbury.ac.nz  Mon Jan 15 02:37:35 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 15 Jan 2024 20:37:35 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <mailman.41.1705271232.15798.python-list@python.org>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <mailman.20.1705101297.15798.python-list@python.org>
 <l0e0jhFef82U1@mid.individual.net>
 <CAJQBtgkEpBrAgbnN=PXH_sJLdYH4oPWanP5mcyQoKz1sudQ7kQ@mail.gmail.com>
 <CAJQBtg=vjoMGcTw2a5C2FFdzVnmNBwFyf2Pu0gEXhjYvOq3MGw@mail.gmail.com>
 <ZaH2rwgbIHpCmJtr@anomaly>
 <CAJQBtgmLwSUPy8jngpZGyxjVXEeXkr7Rte1z5_uFXZOG8ECH3g@mail.gmail.com>
 <mailman.41.1705271232.15798.python-list@python.org>
Message-ID: <l0k5m1Fif3eU1@mid.individual.net>

On 13/01/24 11:34 pm, Left Right wrote:
> To make this
> shorter, Python allows:
> 
> for <statement> in ... : ...
>

Um, no, it doesn't. An assignment target is not, on its own, a
statement.

It's hard to make sense of what you're saying. You seem to be
surprised by the fact that Python doesn't require variables to
be declared separately from their use. But this is a very common
feature of dynamic languages generally. As language oddities go,
it hardly rates a mention.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Mon Jan 15 02:53:03 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 15 Jan 2024 20:53:03 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <mailman.42.1705271233.15798.python-list@python.org>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <mailman.42.1705271233.15798.python-list@python.org>
Message-ID: <l0k6j0Fijj9U1@mid.individual.net>

On 15/01/24 1:28 am, Left Right wrote:
> Python isn't a context-free language, so the grammar that is used to
> describe it doesn't actually describe the language

Very few languages have a formal grammar that *fully* describes
the set of strings that constitute valid programs, including all
the rules about things having to be declared, types matching up,
etc. The only one I know of which attempted that is Algol 68,
and it seems to be regarded as a technical success but a practical
failure.

> ... so, it's a "pretend grammar" that ignores indentation.

Indentation isn't ignored, it appears in the grammar by means of
INDENT and DEDENT lexical tokens.

It's true that the meaning of these tokens is described informally
elsewhere, but that's true of all the lexical features.

-- 
Greg

From rosuav at gmail.com  Mon Jan 15 03:07:51 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 15 Jan 2024 19:07:51 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <l0k6j0Fijj9U1@mid.individual.net>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <mailman.42.1705271233.15798.python-list@python.org>
 <l0k6j0Fijj9U1@mid.individual.net>
Message-ID: <CAPTjJmqT1EJmeAnPgu1WXyJwXrJQGof7zXrX9jiXugr3BUz=5Q@mail.gmail.com>

On Mon, 15 Jan 2024 at 18:56, Greg Ewing via Python-list
<python-list at python.org> wrote:
>
> On 15/01/24 1:28 am, Left Right wrote:
> > Python isn't a context-free language, so the grammar that is used to
> > describe it doesn't actually describe the language
>
> Very few languages have a formal grammar that *fully* describes
> the set of strings that constitute valid programs, including all
> the rules about things having to be declared, types matching up,
> etc. The only one I know of which attempted that is Algol 68,
> and it seems to be regarded as a technical success but a practical
> failure.
>
> > ... so, it's a "pretend grammar" that ignores indentation.
>
> Indentation isn't ignored, it appears in the grammar by means of
> INDENT and DEDENT lexical tokens.
>
> It's true that the meaning of these tokens is described informally
> elsewhere, but that's true of all the lexical features.
>

I've recently been doing a bit of work with grammar parsers, and to be
quite honest, the grammar is only about one third of the overall
parser.  There are three sections with roughly equal importance:

1. Tokenizer
2. Grammar
3. What to DO with that grammar (actions)

INDENT and DEDENT are being handled at the tokenizer stage, and so are
a lot of other rules like backslashes in quoted strings. On the flip
side, string prefixes (like b"...") seem to be handled in the third
phase, and the grammar actually doesn't concern itself with those
either.

The grammar *can't* specify everything. If it did, it would have to
have rules for combining individual letters into a NAME and individual
characters into a string literal. The grammar would be completely
unreadable. (I tried, and even just building up a decimal literal in
that style was quite a pain.)

ChrisA

From greg.ewing at canterbury.ac.nz  Mon Jan 15 03:13:41 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 15 Jan 2024 21:13:41 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <mailman.51.1705280065.15798.python-list@python.org>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <a7323568-a438-4e7c-a6b2-9006e5ca43d9@DancesWithMice.info>
 <mailman.51.1705280065.15798.python-list@python.org>
Message-ID: <l0k7pmFiqkgU1@mid.individual.net>

On 15/01/24 1:54 pm, dn wrote:
> Soon after, Wirth simplified 
> rather than expanded, and developed Pascal.

Before Pascal there was Algol-W, which Wirth invented as a rebellion
against how complicated Algol 68 was becoming.

> When I first saw this I was 
> stunned, then attracted to its simplicity, but then steered-away once 
> realised that it needed 'more' to cope with 'the outside world'.

Pascal was intended as a teaching language, and as such it was lacking
in practicality in a few spots. But it didn't need much tweaking to
make it a very useful language. UCSD Pascal, Turbo Pascal, Delphi, etc.
enjoyed a lot of popularity. A variant of UCSD was the main language
for Macintosh application development for a number of years.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Mon Jan 15 03:21:44 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 15 Jan 2024 21:21:44 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <mailman.58.1705306086.15798.python-list@python.org>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <mailman.42.1705271233.15798.python-list@python.org>
 <l0k6j0Fijj9U1@mid.individual.net>
 <CAPTjJmqT1EJmeAnPgu1WXyJwXrJQGof7zXrX9jiXugr3BUz=5Q@mail.gmail.com>
 <mailman.58.1705306086.15798.python-list@python.org>
Message-ID: <l0k88pFiv3hU1@mid.individual.net>

On 15/01/24 9:07 pm, Chris Angelico wrote:
> The grammar *can't* specify everything. If it did, it would have to
> have rules for combining individual letters into a NAME and individual
> characters into a string literal.

The lexical level of a grammar can be, and often is, described
formally using regular expressions.

Although some might consider that this doesn't contradict
your statement about readability. :-)

-- 
Greg

From rosuav at gmail.com  Mon Jan 15 03:51:26 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 15 Jan 2024 19:51:26 +1100
Subject: Extract lines from file, add to new files
In-Reply-To: <l0k88pFiv3hU1@mid.individual.net>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <mailman.42.1705271233.15798.python-list@python.org>
 <l0k6j0Fijj9U1@mid.individual.net>
 <CAPTjJmqT1EJmeAnPgu1WXyJwXrJQGof7zXrX9jiXugr3BUz=5Q@mail.gmail.com>
 <mailman.58.1705306086.15798.python-list@python.org>
 <l0k88pFiv3hU1@mid.individual.net>
Message-ID: <CAPTjJmq0Vj1_tYrqPsinhszN93-9khAitPVsx9u2gwy_Xyp2MQ@mail.gmail.com>

On Mon, 15 Jan 2024 at 19:26, Greg Ewing via Python-list
<python-list at python.org> wrote:
>
> On 15/01/24 9:07 pm, Chris Angelico wrote:
> > The grammar *can't* specify everything. If it did, it would have to
> > have rules for combining individual letters into a NAME and individual
> > characters into a string literal.
>
> The lexical level of a grammar can be, and often is, described
> formally using regular expressions.
>
> Although some might consider that this doesn't contradict
> your statement about readability. :-)
>

Not even slightly :)

ChrisA

From PythonList at DancesWithMice.info  Mon Jan 15 04:37:15 2024
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 15 Jan 2024 22:37:15 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <l0k7pmFiqkgU1@mid.individual.net>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
 <a7323568-a438-4e7c-a6b2-9006e5ca43d9@DancesWithMice.info>
 <mailman.51.1705280065.15798.python-list@python.org>
 <l0k7pmFiqkgU1@mid.individual.net>
Message-ID: <812f1414-9914-4c93-a080-75189b40453d@DancesWithMice.info>

On 15/01/24 21:13, Greg Ewing via Python-list wrote:
> On 15/01/24 1:54 pm, dn wrote:
>> Soon after, Wirth simplified rather than expanded, and developed Pascal.
> 
> Before Pascal there was Algol-W, which Wirth invented as a rebellion
> against how complicated Algol 68 was becoming.
> 
>> When I first saw this I was stunned, then attracted to its simplicity, 
>> but then steered-away once realised that it needed 'more' to cope with 
>> 'the outside world'.
> 
> Pascal was intended as a teaching language, and as such it was lacking
> in practicality in a few spots. But it didn't need much tweaking to
> make it a very useful language. UCSD Pascal, Turbo Pascal, Delphi, etc.
> enjoyed a lot of popularity. A variant of UCSD was the main language
> for Macintosh application development for a number of years.


Ironically, I didn't come across Pascal as a teaching-language.

Borland were trying to turn Turbo Pascal into a practical development 
environment, beyond teaching (as with others of their 'Turbo' series). 
As I say, it didn't float my business-world boat.
- not before I won a case a wine from Philippe Kahn's own vineyard for 
solving some 'interview question' and exchanging jokes with him, in 
French, at some Trade Show in London. Two surprises: one, that it 
actually turned-up a few weeks later, and two, that I (suddenly) had so 
many friends!

Ah, the good, old days...


-- 
Regards,
=dn

From antoon.pardon at vub.be  Mon Jan 15 04:52:02 2024
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Mon, 15 Jan 2024 10:52:02 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
References: <5a6d88e-46b1-a3ea-333-d053cbe5654d@appl-ecosys.com>
 <24aeb00e-41fd-4809-ae96-d429645cbc07@mrabarnett.plus.com>
 <4a215b7d-f1af-49d7-1496-96e290255314@appl-ecosys.com>
 <65A0E32A.2458.26DB92@RealGrizzlyAdams.vivaldi.net>
 <CAJQBtgnNs6q_+MrPsir4Kioh8Zra+UepDiwx61c0NXcm3P3xOw@mail.gmail.com>
 <76935eb3-ed67-428e-9aea-a8e527e12914@DancesWithMice.info>
 <CAPTjJmqc6g9oBLMh9Mt6K5ozN+xUJJJ68TjHBHnMXAS62mydew@mail.gmail.com>
 <CAJQBtg=+_46_NDqYcpbf+heO4RjStvpJOuui9wgKmiEDinS9jg@mail.gmail.com>
Message-ID: <1ea42c72-a898-478b-a254-8157cab22b09@vub.be>



Op 14/01/2024 om 13:28 schreef Left Right via Python-list:
> Python isn't a context-free language, so the grammar that is used to
> describe it doesn't actually describe the language... so, it's a
> "pretend grammar" that ignores indentation.

No it doesn't. Here is the definition of a block, it clearly mentions
indentation:

block:
     |  NEWLINE INDENTstatements  DEDENT
     |  simple_stmts But you are correct that python in not a context-free 
language. But so is any programming language. Yet a lot of those non 
context-free language designers thought it helpful to have a 
modified/extended BNF description of a superset of the intended language 
and used other means to further check whether the code in question was 
valid or not. -- Antoon Pardon

From frank at chagford.com  Mon Jan 15 08:51:26 2024
From: frank at chagford.com (Frank Millman)
Date: Mon, 15 Jan 2024 15:51:26 +0200
Subject: Question about garbage collection
Message-ID: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>

Hi all

I have read that one should not have to worry about garbage collection 
in modern versions of Python - it 'just works'.

I don't want to rely on that. My app is a long-running server, with 
multiple clients logging on, doing stuff, and logging off. They can 
create many objects, some of them long-lasting. I want to be sure that 
all objects created are gc'd when the session ends.

I do have several circular references. My experience is that if I do not 
take some action to break the references when closing the session, the 
objects remain alive. Below is a very simple program to illustrate this.

Am I missing something? All comments appreciated.

Frank Millman

==================================================

import gc

class delwatcher:
 ??? # This stores enough information to identify the object being watched.
 ??? # It does not store a reference to the object itself.
 ??? def __init__(self, obj):
 ??????? self.id = (obj.type, obj.name, id(obj))
 ??????? print('***', *self.id, 'created ***')
 ??? def __del__(self):
 ??????? print('***', *self.id, 'deleted ***')

class Parent:
 ??? def __init__(self, name):
 ??????? self.type = 'parent'
 ??????? self.name = name
 ??????? self.children = []
 ??????? self._del = delwatcher(self)

class Child:
 ??? def __init__(self, parent, name):
 ??????? self.type = 'child'
 ??????? self.parent = parent
 ??????? self.name = name
 ??????? parent.children.append(self)
 ??????? self._del = delwatcher(self)

p1 = Parent('P1')
p2 = Parent('P2')

c1_1 = Child(p1, 'C1_1')
c1_2 = Child(p1, 'C1_2')
c2_1 = Child(p2, 'C2_1')
c2_2 = Child(p2, 'C2_2')

input('waiting ...')

# if next 2 lines are included, parent and child can be gc'd
# for ch in p1.children:
#???? ch.parent = None

# if next line is included, child can be gc'd, but not parent
# p1.children = None

del c1_1
del p1
gc.collect()

input('wait some more ...')


From skip.montanaro at gmail.com  Mon Jan 15 09:14:20 2024
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 15 Jan 2024 08:14:20 -0600
Subject: Question about garbage collection
In-Reply-To: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
Message-ID: <CANc-5Uy4MN6_7Fe3n-yXcaVTifkQQZEeOVJ3ab8HGh6=6fK-6A@mail.gmail.com>

> I do have several circular references. My experience is that if I do not
> take some action to break the references when closing the session, the
> objects remain alive. Below is a very simple program to illustrate this.
>
> Am I missing something? All comments appreciated.

Python has normal reference counting, but also has a cyclic garbage
collector. Here's plenty of detail about how it works:

https://devguide.python.org/internals/garbage-collector/index.html

Skip

From nulla.epistola at web.de  Mon Jan 15 11:44:36 2024
From: nulla.epistola at web.de (Sibylle Koczian)
Date: Mon, 15 Jan 2024 17:44:36 +0100
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <4TCsVz4CzPznVGL@mail.python.org>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
Message-ID: <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>

Am 15.01.2024 um 00:46 schrieb Mike Dewhirst via Python-list:
> In Windows the provided methods for running complex command lines are either a batch file or a shortcut.Someone very kindly pointed out to me in this thread that there is a PEP for py.exe.?I don't use py.exe originally because I didn't trust it believing it was a new-fangled Microsoft trick. I did read that PEP but it has no relevance for my mixed Windows/Linux environments.?On reflection I now believe I won't use py.exe because it introduces an unnecessary layer of indirection.The? bottom line is that you still need to know which Python a particular set of circumstances demands and if you use py.exe you then need to also understand how it chooses and how it interprets shebang lines written for your Linux environment.?And if that isn't your situation I have jumped to the wrong conclusion.I have found no problem in Windows when I use shebang lines in scripts intended for execution in both Linux and Windows. They are ignored unless you use py.exe.My advice is to give up py.exe unless your use case mandates shebang lines in Windows.M--(Unsigned mail from my phone)

First and foremost I want to understand why I'm seeing this:

- Python scripts with "/usr/bin/env python3" as shebang line work as 
expected on a computer with Windows 10 and Python 3.11.5. They have 
worked for years on this machine, using either the latest Python or one 
version before (depending on availability of some packages). There is a 
virtual machine with ArchLinux on the same machine and some of the 
scripts are copies from that.

- I've got a second computer with Windows 11 and I installed Python 
3.12.1 on it. After copying some scripts from my first computer I found 
that I couldn't start them: not by entering the script name in a 
console, not using py.exe, not double clicking in the explorer. Entering 
<full_path_to_python.exe>\python <scriptname> probably worked - I think 
I tried that too, but I'm not really sure, because that's really not 
practical.

In the Python documentation for versions 3.11 and 3.12 I found no 
differences regarding py.exe and shebang lines.

Then I removed the "/env" from the shebang lines and could start the 
scripts from the second computer. That certainly is a solution, but why???

Sibylle




From mats at wichmann.us  Mon Jan 15 13:26:16 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Mon, 15 Jan 2024 11:26:16 -0700
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
Message-ID: <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>

On 1/15/24 09:44, Sibylle Koczian via Python-list wrote:

> First and foremost I want to understand why I'm seeing this:
> 
> - Python scripts with "/usr/bin/env python3" as shebang line work as 
> expected on a computer with Windows 10 and Python 3.11.5. They have 
> worked for years on this machine, using either the latest Python or one 
> version before (depending on availability of some packages). There is a 
> virtual machine with ArchLinux on the same machine and some of the 
> scripts are copies from that.
> 
> - I've got a second computer with Windows 11 and I installed Python 
> 3.12.1 on it. After copying some scripts from my first computer I found 
> that I couldn't start them: not by entering the script name in a 
> console, not using py.exe, not double clicking in the explorer. Entering 
> <full_path_to_python.exe>\python <scriptname> probably worked - I think 
> I tried that too, but I'm not really sure, because that's really not 
> practical.
> 
> In the Python documentation for versions 3.11 and 3.12 I found no 
> differences regarding py.exe and shebang lines.
> 
> Then I removed the "/env" from the shebang lines and could start the 
> scripts from the second computer. That certainly is a solution, but why???

It's because of Windows itself.  The default nowadays is that irritating 
little stub that prompts you to go install Python from the WIndows 
store.  When you use the "env" form, it looks for python (or python3 in 
your case) in the PATH *first* and you'll get a hit.   Mine looks like:

C:\Users\mats\AppData\Local\Microsoft\WindwsApps\python.exe and python3.exe

you can check what it's doing for you by using the "where" command in a 
windows shell.

On your older Windows 10 machine you either never had that stub - I 
don't know when it was added, maybe someone from Microsoft listening 
here knows - or it's been superseded by changes to the PATH, or 
something.  On my fairly new Win 11 box the base of that path is early 
in the user portion of PATH, so that must be a default.

py.exe without the "/usr/bin/env" magic doesn't put PATH searching 
first, according to that snip from the docs that's been posted here 
several times., so you shouldn't fall down that particular rathole.


From mats at wichmann.us  Mon Jan 15 13:28:03 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Mon, 15 Jan 2024 11:28:03 -0700
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
Message-ID: <16442885-0187-483c-b136-81f3bc64cb26@wichmann.us>

On 1/15/24 09:44, Sibylle Koczian via Python-list wrote:

> In the Python documentation for versions 3.11 and 3.12 I found no 
> differences regarding py.exe and shebang lines.
> 
> Then I removed the "/env" from the shebang lines and could start the 
> scripts from the second computer. That certainly is a solution, but why???
> 
> Sibylle

also, it looks like you can disable the PATH-searching behavior of the 
/usr/bin/env virtual path:

 > The environment variable PYLAUNCHER_NO_SEARCH_PATH may be set (to any 
value) to skip this search of PATH.


From dieter.maurer at online.de  Mon Jan 15 13:59:06 2024
From: dieter.maurer at online.de (dieter.maurer at online.de)
Date: Mon, 15 Jan 2024 19:59:06 +0100
Subject: Question about garbage collection
In-Reply-To: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
Message-ID: <26021.32890.649779.99770@ixdm.fritz.box>

Frank Millman wrote at 2024-1-15 15:51 +0200:
>I have read that one should not have to worry about garbage collection 
>in modern versions of Python - it 'just works'.

There are still some isolated cases when not all objects
in an unreachable cycle are destroyed
(see e.g. step 2 of
"https://devguide.python.org/internals/garbage-collector/index.html#destroying-unreachable-objects").
But Python's own objects (e.g. traceback cycles)
or instances of classes implemented in Python
should no longer be affected.

Thus, unless you use extensions implemented in C (with "legacy finalizer"s),
garbage collection should not make problems.


On the other hand, your application, too, must avoid memory leaks.
Caches of various forms (with data for several sessions) might introduce them.


From list1 at tompassin.net  Mon Jan 15 14:01:51 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 15 Jan 2024 14:01:51 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
 <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
Message-ID: <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>

On 1/15/2024 1:26 PM, Mats Wichmann via Python-list wrote:
> On 1/15/24 09:44, Sibylle Koczian via Python-list wrote:
> 
>> First and foremost I want to understand why I'm seeing this:
>>
>> - Python scripts with "/usr/bin/env python3" as shebang line work as 
>> expected on a computer with Windows 10 and Python 3.11.5. They have 
>> worked for years on this machine, using either the latest Python or 
>> one version before (depending on availability of some packages). There 
>> is a virtual machine with ArchLinux on the same machine and some of 
>> the scripts are copies from that.
>>
>> - I've got a second computer with Windows 11 and I installed Python 
>> 3.12.1 on it. After copying some scripts from my first computer I 
>> found that I couldn't start them: not by entering the script name in a 
>> console, not using py.exe, not double clicking in the explorer. 
>> Entering <full_path_to_python.exe>\python <scriptname> probably worked 
>> - I think I tried that too, but I'm not really sure, because that's 
>> really not practical.
>>
>> In the Python documentation for versions 3.11 and 3.12 I found no 
>> differences regarding py.exe and shebang lines.
>>
>> Then I removed the "/env" from the shebang lines and could start the 
>> scripts from the second computer. That certainly is a solution, but 
>> why???
> 
> It's because of Windows itself.? The default nowadays is that irritating 
> little stub that prompts you to go install Python from the WIndows 
> store.? When you use the "env" form, it looks for python (or python3 in 
> your case) in the PATH *first* and you'll get a hit.?? Mine looks like:
> 
> C:\Users\mats\AppData\Local\Microsoft\WindwsApps\python.exe and python3.exe
> 
> you can check what it's doing for you by using the "where" command in a 
> windows shell.
> 
> On your older Windows 10 machine you either never had that stub - I 
> don't know when it was added, maybe someone from Microsoft listening 
> here knows - or it's been superseded by changes to the PATH, or 
> something.? On my fairly new Win 11 box the base of that path is early 
> in the user portion of PATH, so that must be a default.
> 
> py.exe without the "/usr/bin/env" magic doesn't put PATH searching 
> first, according to that snip from the docs that's been posted here 
> several times., so you shouldn't fall down that particular rathole.

Python from the App Store is not the same as Python from python.org:

"The Microsoft Store package is a simple installation of Python that is 
suitable for running scripts and packages, and using IDLE or other 
development environments. It requires Windows 10 and above, but can be 
safely installed without corrupting other programs. It also provides 
many convenient commands for launching Python and its tools."

- https://docs.python.org/3/using/windows.html

Also:

"The Windows Store distribution of Python is a sandboxed application ... 
The internal components of Windows Store apps are protected from being 
accessed from other applications, and so the PyXLL add-in cannot use the 
Python DLLs and packages that are installed as part of the Windows Store 
Python app."

 From the PyXLL support site -

https://support.pyxll.com/hc/en-gb/articles/4417634326675-Python-installed-via-the-Windows-Store-cannot-be-used-with-PyXLL

The "py" launcher is installed by the installer from python.org.





From akkana at shallowsky.com  Mon Jan 15 14:21:05 2024
From: akkana at shallowsky.com (Akkana Peck)
Date: Mon, 15 Jan 2024 12:21:05 -0700
Subject: Question about garbage collection
In-Reply-To: <26021.32890.649779.99770@ixdm.fritz.box>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <26021.32890.649779.99770@ixdm.fritz.box>
Message-ID: <ZaWFoQ_PlT-XsNEN@shallowsky.com>

> Frank Millman wrote at 2024-1-15 15:51 +0200:
> >I have read that one should not have to worry about garbage collection 
> >in modern versions of Python - it 'just works'.

Dieter Maurer via Python-list writes:
> There are still some isolated cases when not all objects
> in an unreachable cycle are destroyed
> (see e.g. step 2 of
> "https://devguide.python.org/internals/garbage-collector/index.html#destroying-unreachable-objects").

Also be warned that some modules (particularly if they're based on libraries not written in Python) might not garbage collect, so you may need to use other methods of cleaning up after those objects.

        ...Akkana

From rosuav at gmail.com  Mon Jan 15 16:57:27 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 16 Jan 2024 08:57:27 +1100
Subject: Question about garbage collection
In-Reply-To: <ZaWFoQ_PlT-XsNEN@shallowsky.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <26021.32890.649779.99770@ixdm.fritz.box> <ZaWFoQ_PlT-XsNEN@shallowsky.com>
Message-ID: <CAPTjJmoxVTwD+1tKPWj6YSBdxPG=TDbKz5mUdTwMMVoTsoXBzg@mail.gmail.com>

On Tue, 16 Jan 2024 at 06:32, Akkana Peck via Python-list
<python-list at python.org> wrote:
>
> > Frank Millman wrote at 2024-1-15 15:51 +0200:
> > >I have read that one should not have to worry about garbage collection
> > >in modern versions of Python - it 'just works'.
>
> Dieter Maurer via Python-list writes:
> > There are still some isolated cases when not all objects
> > in an unreachable cycle are destroyed
> > (see e.g. step 2 of
> > "https://devguide.python.org/internals/garbage-collector/index.html#destroying-unreachable-objects").
>
> Also be warned that some modules (particularly if they're based on libraries not written in Python) might not garbage collect, so you may need to use other methods of cleaning up after those objects.
>

Got any examples of that?

ChrisA

From mats at wichmann.us  Mon Jan 15 17:55:44 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Mon, 15 Jan 2024 15:55:44 -0700
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
 <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
 <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>
Message-ID: <23c6ee06-5e7b-42b0-a1f4-d151f8a47a1c@wichmann.us>

On 1/15/24 12:01, Thomas Passin via Python-list wrote:
> On 1/15/2024 1:26 PM, Mats Wichmann via Python-list wrote:
>> On 1/15/24 09:44, Sibylle Koczian via Python-list wrote:
>>
>>> First and foremost I want to understand why I'm seeing this:
>>>
>>> - Python scripts with "/usr/bin/env python3" as shebang line work as 
>>> expected on a computer with Windows 10 and Python 3.11.5. They have 
>>> worked for years on this machine, using either the latest Python or 
>>> one version before (depending on availability of some packages). 
>>> There is a virtual machine with ArchLinux on the same machine and 
>>> some of the scripts are copies from that.
>>>
>>> - I've got a second computer with Windows 11 and I installed Python 
>>> 3.12.1 on it. After copying some scripts from my first computer I 
>>> found that I couldn't start them: not by entering the script name in 
>>> a console, not using py.exe, not double clicking in the explorer. 
>>> Entering <full_path_to_python.exe>\python <scriptname> probably 
>>> worked - I think I tried that too, but I'm not really sure, because 
>>> that's really not practical.
>>>
>>> In the Python documentation for versions 3.11 and 3.12 I found no 
>>> differences regarding py.exe and shebang lines.
>>>
>>> Then I removed the "/env" from the shebang lines and could start the 
>>> scripts from the second computer. That certainly is a solution, but 
>>> why???
>>
>> It's because of Windows itself.? The default nowadays is that 
>> irritating little stub that prompts you to go install Python from the 
>> WIndows store.? When you use the "env" form, it looks for python (or 
>> python3 in your case) in the PATH *first* and you'll get a hit.?? Mine 
>> looks like:
>>
>> C:\Users\mats\AppData\Local\Microsoft\WindwsApps\python.exe and 
>> python3.exe
>>
>> you can check what it's doing for you by using the "where" command in 
>> a windows shell.
>>
>> On your older Windows 10 machine you either never had that stub - I 
>> don't know when it was added, maybe someone from Microsoft listening 
>> here knows - or it's been superseded by changes to the PATH, or 
>> something.? On my fairly new Win 11 box the base of that path is early 
>> in the user portion of PATH, so that must be a default.
>>
>> py.exe without the "/usr/bin/env" magic doesn't put PATH searching 
>> first, according to that snip from the docs that's been posted here 
>> several times., so you shouldn't fall down that particular rathole.
> 
> Python from the App Store is not the same as Python from python.org:

yes. this question is about the python.org distribution. but, Windows 
natively has something called python.exe and python3.exe which is 
interfering here, IF the python.org install isn't directed to put itself 
into the path, AND if the "#!/usr/bin/env python3" form is used, causing 
a search in PATH, which is the setup Sibylle has described, unless I've 
misunderstood details.




From greg.ewing at canterbury.ac.nz  Mon Jan 15 18:27:57 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 16 Jan 2024 12:27:57 +1300
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <mailman.71.1705359353.15798.python-list@python.org>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
 <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
 <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>
 <23c6ee06-5e7b-42b0-a1f4-d151f8a47a1c@wichmann.us>
 <mailman.71.1705359353.15798.python-list@python.org>
Message-ID: <l0ltbtFsqtqU1@mid.individual.net>

On 16/01/24 11:55 am, Mats Wichmann wrote:
> Windows 
> natively has something called python.exe and python3.exe which is 
> interfering here

I'm wondering whether py.exe should be taught to recognise these stubs
and ignore them. This sounds like something that could trip a lot of
people up.

-- 
Greg

From list1 at tompassin.net  Mon Jan 15 19:24:44 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 15 Jan 2024 19:24:44 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <l0ltbtFsqtqU1@mid.individual.net>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
 <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
 <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>
 <23c6ee06-5e7b-42b0-a1f4-d151f8a47a1c@wichmann.us>
 <mailman.71.1705359353.15798.python-list@python.org>
 <l0ltbtFsqtqU1@mid.individual.net>
Message-ID: <f17b9cde-231b-47d5-bb91-04b4b6b86dc8@tompassin.net>

On 1/15/2024 6:27 PM, Greg Ewing via Python-list wrote:
> On 16/01/24 11:55 am, Mats Wichmann wrote:
>> Windows natively has something called python.exe and python3.exe which 
>> is interfering here
> 
> I'm wondering whether py.exe should be taught to recognise these stubs
> and ignore them. This sounds like something that could trip a lot of
> people up.

There are registry entries that say where all the python.org install 
locations are.  I suppose, but don't know, that py.exe checks them.  The 
registry entries are inComputer\HKEY_CURRENT_USER\SOFTWARE\Python\PythonCore


From akkana at shallowsky.com  Mon Jan 15 21:47:32 2024
From: akkana at shallowsky.com (Akkana Peck)
Date: Mon, 15 Jan 2024 19:47:32 -0700
Subject: Question about garbage collection
In-Reply-To: <CAPTjJmoxVTwD+1tKPWj6YSBdxPG=TDbKz5mUdTwMMVoTsoXBzg@mail.gmail.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <26021.32890.649779.99770@ixdm.fritz.box>
 <ZaWFoQ_PlT-XsNEN@shallowsky.com>
 <CAPTjJmoxVTwD+1tKPWj6YSBdxPG=TDbKz5mUdTwMMVoTsoXBzg@mail.gmail.com>
Message-ID: <ZaXuRAa9Hi9df1mS@shallowsky.com>

I wrote:
> > Also be warned that some modules (particularly if they're based on libraries not written in Python) might not garbage collect, so you may need to use other methods of cleaning up after those objects.

Chris Angelico writes:
> Got any examples of that?

The big one for me was gdk-pixbuf, part of GTK. When you do something like gtk.gdk.pixbuf_new_from_file(), there's a Python object that gets created, but there's also the underlying C code that allocates memory for the pixbuf. When the object went out of scope, the Python object was automatically garbage collected, but the pixbuf data leaked. Calling gc.collect() caused the pixbuf data to be garbage collected too.

There used to be a post explaining this on the pygtk mailing list: the link was
http://www.daa.com.au/pipermail/pygtk/2003-December/006499.html
but that page is gone now and I can't seem to find any other archives of that list (it's not on archive.org either). And this was from GTK2; I never checked whether the extra gc.collect() is still necessary in GTK3, but I figure leaving it in doesn't hurt anything. I use pixbufs in a tiled map application, so there are a lot of small pixbufs being repeatedly read and then deallocated.

        ...Akkana

From rosuav at gmail.com  Mon Jan 15 21:57:08 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 16 Jan 2024 13:57:08 +1100
Subject: Question about garbage collection
In-Reply-To: <ZaXuRAa9Hi9df1mS@shallowsky.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <26021.32890.649779.99770@ixdm.fritz.box> <ZaWFoQ_PlT-XsNEN@shallowsky.com>
 <CAPTjJmoxVTwD+1tKPWj6YSBdxPG=TDbKz5mUdTwMMVoTsoXBzg@mail.gmail.com>
 <ZaXuRAa9Hi9df1mS@shallowsky.com>
Message-ID: <CAPTjJmr=6Of-TXxuP_1_rkf8xocDGUPvqBDzjM=Ey_Xbb4xrKQ@mail.gmail.com>

On Tue, 16 Jan 2024 at 13:49, Akkana Peck via Python-list
<python-list at python.org> wrote:
>
> I wrote:
> > > Also be warned that some modules (particularly if they're based on libraries not written in Python) might not garbage collect, so you may need to use other methods of cleaning up after those objects.
>
> Chris Angelico writes:
> > Got any examples of that?
>
> The big one for me was gdk-pixbuf, part of GTK. When you do something like gtk.gdk.pixbuf_new_from_file(), there's a Python object that gets created, but there's also the underlying C code that allocates memory for the pixbuf. When the object went out of scope, the Python object was automatically garbage collected, but the pixbuf data leaked. Calling gc.collect() caused the pixbuf data to be garbage collected too.
>
> There used to be a post explaining this on the pygtk mailing list: the link was
> http://www.daa.com.au/pipermail/pygtk/2003-December/006499.html
> but that page is gone now and I can't seem to find any other archives of that list (it's not on archive.org either). And this was from GTK2; I never checked whether the extra gc.collect() is still necessary in GTK3, but I figure leaving it in doesn't hurt anything. I use pixbufs in a tiled map application, so there are a lot of small pixbufs being repeatedly read and then deallocated.
>

Okay, so to clarify: the Python object will always be garbage
collected correctly, but a buggy third-party module might have
*external* resources (in that case, the pixbuf) that aren't properly
released. Either that, or there is a reference loop, which doesn't
necessarily mean you NEED to call gc.collect(), but it can help if you
want to get rid of them more promptly. (Python will detect such loops
at some point, but not always immediately.) But these are bugs in the
module, particularly the first case, and should be considered as such.
2003 is fully two decades ago now, and I would not expect that a
serious bug like that has been copied into PyGObject (the newer way of
using GTK from Python).

So, Python's garbage collection CAN be assumed to "just work", unless
you find evidence to the contrary.

ChrisA

From list1 at tompassin.net  Mon Jan 15 22:36:26 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 15 Jan 2024 22:36:26 -0500
Subject: Question about garbage collection
In-Reply-To: <ZaXuRAa9Hi9df1mS@shallowsky.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <26021.32890.649779.99770@ixdm.fritz.box> <ZaWFoQ_PlT-XsNEN@shallowsky.com>
 <CAPTjJmoxVTwD+1tKPWj6YSBdxPG=TDbKz5mUdTwMMVoTsoXBzg@mail.gmail.com>
 <ZaXuRAa9Hi9df1mS@shallowsky.com>
Message-ID: <dc90a01b-37b9-432e-9ac8-e2b10102949e@tompassin.net>

On 1/15/2024 9:47 PM, Akkana Peck via Python-list wrote:
> I wrote:
>>> Also be warned that some modules (particularly if they're based on libraries not written in Python) might not garbage collect, so you may need to use other methods of cleaning up after those objects.
> 
> Chris Angelico writes:
>> Got any examples of that?
> 
> The big one for me was gdk-pixbuf, part of GTK. When you do something like gtk.gdk.pixbuf_new_from_file(), there's a Python object that gets created, but there's also the underlying C code that allocates memory for the pixbuf. When the object went out of scope, the Python object was automatically garbage collected, but the pixbuf data leaked.

This kind of thing can happen with PyQt, also.  There are ways to 
minimize it but I don't know if you can ever be sure all Qt C++ objects 
will get deleted. It depends on the type of object and the circumstances.

> Calling gc.collect() caused the pixbuf data to be garbage collected too.
> 
> There used to be a post explaining this on the pygtk mailing list: the link was
> http://www.daa.com.au/pipermail/pygtk/2003-December/006499.html
> but that page is gone now and I can't seem to find any other archives of that list (it's not on archive.org either). And this was from GTK2; I never checked whether the extra gc.collect() is still necessary in GTK3, but I figure leaving it in doesn't hurt anything. I use pixbufs in a tiled map application, so there are a lot of small pixbufs being repeatedly read and then deallocated.
> 
>          ...Akkana


From list1 at tompassin.net  Mon Jan 15 19:28:18 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 15 Jan 2024 19:28:18 -0500
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <f17b9cde-231b-47d5-bb91-04b4b6b86dc8@tompassin.net>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
 <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
 <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>
 <23c6ee06-5e7b-42b0-a1f4-d151f8a47a1c@wichmann.us>
 <mailman.71.1705359353.15798.python-list@python.org>
 <l0ltbtFsqtqU1@mid.individual.net>
 <f17b9cde-231b-47d5-bb91-04b4b6b86dc8@tompassin.net>
Message-ID: <85ecc7a6-3b35-42ff-8a7f-85353d0b7386@tompassin.net>

On 1/15/2024 7:24 PM, Thomas Passin wrote:
> On 1/15/2024 6:27 PM, Greg Ewing via Python-list wrote:
>> On 16/01/24 11:55 am, Mats Wichmann wrote:
>>> Windows natively has something called python.exe and python3.exe 
>>> which is interfering here
>>
>> I'm wondering whether py.exe should be taught to recognise these stubs
>> and ignore them. This sounds like something that could trip a lot of
>> people up.
> 
> There are registry entries that say where all the python.org install 
> locations are.? I suppose, but don't know, that py.exe checks them.? The 
> registry entries are 
> in Computer\HKEY_CURRENT_USER\SOFTWARE\Python\PythonCore

For python.org installs that are installed for all users, the entries are in

Computer\HKEY_CURRENT_USER\SOFTWARE\Python\PythonCore


From barry at barrys-emacs.org  Tue Jan 16 04:17:35 2024
From: barry at barrys-emacs.org (Barry)
Date: Tue, 16 Jan 2024 09:17:35 +0000
Subject: Question about garbage collection
In-Reply-To: <dc90a01b-37b9-432e-9ac8-e2b10102949e@tompassin.net>
References: <dc90a01b-37b9-432e-9ac8-e2b10102949e@tompassin.net>
Message-ID: <229031E9-18FE-41FB-953C-19C5B320B1C8@barrys-emacs.org>



> On 16 Jan 2024, at 03:49, Thomas Passin via Python-list <python-list at python.org> wrote:
> 
> This kind of thing can happen with PyQt, also.  There are ways to minimize it but I don't know if you can ever be sure all Qt C++ objects will get deleted. It depends on the type of object and the circumstances.

When this has been seen in the past it has been promptly fixed by the maintainer.

Barry





From frank at chagford.com  Tue Jan 16 07:01:48 2024
From: frank at chagford.com (Frank Millman)
Date: Tue, 16 Jan 2024 14:01:48 +0200
Subject: Question about garbage collection
In-Reply-To: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
Message-ID: <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>

On 2024-01-15 3:51 PM, Frank Millman via Python-list wrote:
> Hi all
> 
> I have read that one should not have to worry about garbage collection 
> in modern versions of Python - it 'just works'.
> 
> I don't want to rely on that. My app is a long-running server, with 
> multiple clients logging on, doing stuff, and logging off. They can 
> create many objects, some of them long-lasting. I want to be sure that 
> all objects created are gc'd when the session ends.
> 

I did not explain myself very well. Sorry about that.

My problem is that my app is quite complex, and it is easy to leave a 
reference dangling somewhere which prevents an object from being gc'd.

This can create (at least) two problems. The obvious one is a memory 
leak. The second is that I sometimes need to keep a reference from a 
transient object to a more permanent structure in my app. To save myself 
the extra step of removing all these references when the transient 
object is deleted, I make them weak references. This works, unless the 
transient object is kept alive by mistake and the weak ref is never removed.

I feel it is important to find these dangling references and fix them, 
rather than wait for problems to appear in production. The only method I 
can come up with is to use the 'delwatcher' class that I used in my toy 
program in my original post.

I am surprised that this issue does not crop up more often. Does nobody 
else have these problems?

Frank




From rosuav at gmail.com  Tue Jan 16 07:15:32 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 16 Jan 2024 23:15:32 +1100
Subject: Question about garbage collection
In-Reply-To: <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
Message-ID: <CAPTjJmqJpaY7-VQ1kjpqPOhtQDwgBNiAhqthY43A84eZq6gErg@mail.gmail.com>

On Tue, 16 Jan 2024 at 23:08, Frank Millman via Python-list
<python-list at python.org> wrote:
>
> On 2024-01-15 3:51 PM, Frank Millman via Python-list wrote:
> > Hi all
> >
> > I have read that one should not have to worry about garbage collection
> > in modern versions of Python - it 'just works'.
> >
> > I don't want to rely on that. My app is a long-running server, with
> > multiple clients logging on, doing stuff, and logging off. They can
> > create many objects, some of them long-lasting. I want to be sure that
> > all objects created are gc'd when the session ends.
> >
>
> I did not explain myself very well. Sorry about that.
>
> My problem is that my app is quite complex, and it is easy to leave a
> reference dangling somewhere which prevents an object from being gc'd.
>
> This can create (at least) two problems. The obvious one is a memory
> leak. The second is that I sometimes need to keep a reference from a
> transient object to a more permanent structure in my app. To save myself
> the extra step of removing all these references when the transient
> object is deleted, I make them weak references. This works, unless the
> transient object is kept alive by mistake and the weak ref is never removed.
>
> I feel it is important to find these dangling references and fix them,
> rather than wait for problems to appear in production. The only method I
> can come up with is to use the 'delwatcher' class that I used in my toy
> program in my original post.
>
> I am surprised that this issue does not crop up more often. Does nobody
> else have these problems?
>

It really depends on how big those dangling objects are. My personal
habit is to not worry about a few loose objects, by virtue of ensuring
that everything either has its reference loops deliberately broken at
some point in time, or by keeping things small.

An example of deliberately breaking a refloop would be when I track
websockets. Usually I'll tag the socket object itself with some kind
of back-reference to my own state, but I also need to be able to
iterate over all of my own state objects (let's say they're
dictionaries for simplicity) and send a message to each socket. So
there'll be a reference loop between the socket and the state. But at
some point, I will be notified that the socket has been disconnected,
and that's when I go to its state object and wipe out its
back-reference. It can then be disposed of promptly, since there's no
loop.

It takes a bit of care, but in general, large state objects won't have
these kinds of loops, and dangling references haven't caused me any
sort of major issues in production.

Where do you tend to "leave a reference dangling somewhere"? How is
this occurring? Is it a result of an incomplete transaction (like an
HTTP request that never finishes), or a regular part of the operation
of the server?

ChrisA

From list1 at tompassin.net  Tue Jan 16 07:47:40 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 16 Jan 2024 07:47:40 -0500
Subject: Question about garbage collection
In-Reply-To: <229031E9-18FE-41FB-953C-19C5B320B1C8@barrys-emacs.org>
References: <dc90a01b-37b9-432e-9ac8-e2b10102949e@tompassin.net>
 <229031E9-18FE-41FB-953C-19C5B320B1C8@barrys-emacs.org>
Message-ID: <ad85711c-e71d-44ac-941e-55272e175811@tompassin.net>

On 1/16/2024 4:17 AM, Barry wrote:
> 
> 
>> On 16 Jan 2024, at 03:49, Thomas Passin via Python-list <python-list at python.org> wrote:
>>
>> This kind of thing can happen with PyQt, also.  There are ways to minimize it but I don't know if you can ever be sure all Qt C++ objects will get deleted. It depends on the type of object and the circumstances.
> 
> When this has been seen in the past it has been promptly fixed by the maintainer.

The usual advice is to call deleteLater() on objects derived from PyQt 
classes.  I don't know enough about PyQt to know if this takes care of 
all dangling reference problems, though.


From frank at chagford.com  Tue Jan 16 09:43:38 2024
From: frank at chagford.com (Frank Millman)
Date: Tue, 16 Jan 2024 16:43:38 +0200
Subject: Question about garbage collection
In-Reply-To: <CAPTjJmqJpaY7-VQ1kjpqPOhtQDwgBNiAhqthY43A84eZq6gErg@mail.gmail.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
 <CAPTjJmqJpaY7-VQ1kjpqPOhtQDwgBNiAhqthY43A84eZq6gErg@mail.gmail.com>
Message-ID: <048f182a-5573-416f-8266-0956c2491d7f@chagford.com>

On 2024-01-16 2:15 PM, Chris Angelico via Python-list wrote:
> 
> Where do you tend to "leave a reference dangling somewhere"? How is
> this occurring? Is it a result of an incomplete transaction (like an
> HTTP request that never finishes), or a regular part of the operation
> of the server?
> 

I have a class that represents a database table, and another class that 
represents a database column. There is a one-to-many relationship and 
they maintain references to each other.

In another part of the app, there is a class that represents a form, and 
another class that represents the gui elements on the form. Again there 
is a one-to-many relationship.

A gui element that represents a piece of data has to maintain a link to 
its database column object. There can be a many-to-one relationship, as 
there could be more than one gui element referring to the same column.

There are added complications which I won't go into here. The bottom 
line is that on some occasions a form which has been closed does not get 
gc'd.

I have been trying to reproduce the problem in my toy app, but I cannot 
get it to fail. There is a clue there! I think I have just 
over-complicated things.

I will start with a fresh approach tomorrow. If you don't hear from me 
again, you will know that I have solved it!

Thanks for the input, it definitely helped.

Frank



From rosuav at gmail.com  Tue Jan 16 10:00:15 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 17 Jan 2024 02:00:15 +1100
Subject: Question about garbage collection
In-Reply-To: <048f182a-5573-416f-8266-0956c2491d7f@chagford.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
 <CAPTjJmqJpaY7-VQ1kjpqPOhtQDwgBNiAhqthY43A84eZq6gErg@mail.gmail.com>
 <048f182a-5573-416f-8266-0956c2491d7f@chagford.com>
Message-ID: <CAPTjJmrUedFjLNW6Uqo35cGSwkdAW+10t_-DP1wWdbvHL8wxvA@mail.gmail.com>

On Wed, 17 Jan 2024 at 01:45, Frank Millman via Python-list
<python-list at python.org> wrote:
>
> On 2024-01-16 2:15 PM, Chris Angelico via Python-list wrote:
> >
> > Where do you tend to "leave a reference dangling somewhere"? How is
> > this occurring? Is it a result of an incomplete transaction (like an
> > HTTP request that never finishes), or a regular part of the operation
> > of the server?
> >
>
> I have a class that represents a database table, and another class that
> represents a database column. There is a one-to-many relationship and
> they maintain references to each other.
>
> In another part of the app, there is a class that represents a form, and
> another class that represents the gui elements on the form. Again there
> is a one-to-many relationship.

I don't know when you'd be "done" with the table, so I won't try to
give an example, but I'll try this one and maybe it'll give some ideas
that could apply to both.

When you open the form, you initialize it, display it, etc, etc. This
presumably includes something broadly like this:

class Form:
    def __init__(self):
        self.elements = []

class Element:
    def __init__(self, form):
        self.form = form
        form.elements.append(self)

frm = Form(...)
Element(frm, ...) # as many as needed
frm.show() # present it to the user

This is a pretty classic refloop. I don't know exactly what your setup
is, but most likely it's going to look something like this. Feel free
to correct me if it doesn't.

The solution here would be to trap the "form is no longer being
displayed" moment. That'll be some sort of GUI event like a "close" or
"delete" signal. When that comes through (and maybe after doing other
processing), you no longer need the form, and can dispose of it. The
simplest solution here is: Empty out frm.elements. That immediately
leaves the form itself as a leaf (no references to anything relevant),
and the elements still refer back to it, but once nothing ELSE refers
to the form, everything can be disposed of.

> A gui element that represents a piece of data has to maintain a link to
> its database column object. There can be a many-to-one relationship, as
> there could be more than one gui element referring to the same column.

Okay, so the Element also refers to the corresponding Column. If the
Form and Element aren't in a refloop, this shouldn't be a problem.
However, if this is the same Table and Column that you referred to
above, that might be the answer to my question. Are you "done" with
the Table at the same time that the form is no longer visible? If so,
you would probably have something similar where the Form refers to the
Table, and the Table and Columns refer to each other... so the same
solution hopefully should work: wipe out the Table's list of columns.

> There are added complications which I won't go into here. The bottom
> line is that on some occasions a form which has been closed does not get
> gc'd.
>
> I have been trying to reproduce the problem in my toy app, but I cannot
> get it to fail. There is a clue there! I think I have just
> over-complicated things.

Definitely possible.

> I will start with a fresh approach tomorrow. If you don't hear from me
> again, you will know that I have solved it!
>
> Thanks for the input, it definitely helped.

Cool cool, happy to help.

ChrisA

From nulla.epistola at web.de  Tue Jan 16 12:00:33 2024
From: nulla.epistola at web.de (Sibylle Koczian)
Date: Tue, 16 Jan 2024 18:00:33 +0100
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <23c6ee06-5e7b-42b0-a1f4-d151f8a47a1c@wichmann.us>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
 <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
 <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>
 <23c6ee06-5e7b-42b0-a1f4-d151f8a47a1c@wichmann.us>
Message-ID: <d59f5407-0bfd-497e-86ed-66f751a4295e@web.de>

Am 15.01.2024 um 23:55 schrieb Mats Wichmann via Python-list:
> On 1/15/24 12:01, Thomas Passin via Python-list wrote:
>> On 1/15/2024 1:26 PM, Mats Wichmann via Python-list wrote:
>> Python from the App Store is not the same as Python from python.org:
> 
> yes. this question is about the python.org distribution. but, Windows 
> natively has something called python.exe and python3.exe which is 
> interfering here, IF the python.org install isn't directed to put itself 
> into the path, AND if the "#!/usr/bin/env python3" form is used, causing 
> a search in PATH, which is the setup Sibylle has described, unless I've 
> misunderstood details.
> 

No, you didn't misunderstand any detail. It's exactly right. My Windows 
10 box doesn't find anything for "where python", "where python3", while 
the new Windows 11 machine finds the Microsoft stub. "Irritating" is a 
very friendly attribute for that thing. Why must it be called 
"python.exe" and not something else like the installation files from 
python.org?

I'll stop using "/env" - hopefully that won't create problems with the 
scripts I use in the Linux VM. But in that case I'll know what's up.

Thank you very much!
Sibylle



From barry at barrys-emacs.org  Tue Jan 16 17:31:44 2024
From: barry at barrys-emacs.org (Barry)
Date: Tue, 16 Jan 2024 22:31:44 +0000
Subject: Question about garbage collection
In-Reply-To: <ad85711c-e71d-44ac-941e-55272e175811@tompassin.net>
References: <ad85711c-e71d-44ac-941e-55272e175811@tompassin.net>
Message-ID: <C0DC0B92-15EC-4D3F-BA3F-4DFC99AA319B@barrys-emacs.org>



> On 16 Jan 2024, at 13:17, Thomas Passin via Python-list <python-list at python.org> wrote:
> 
> The usual advice is to call deleteLater() on objects derived from PyQt classes.  I don't know enough about PyQt to know if this takes care of all dangling reference problems, though.

It works well and robustly.

Barry



From barry at barrys-emacs.org  Tue Jan 16 17:37:05 2024
From: barry at barrys-emacs.org (Barry)
Date: Tue, 16 Jan 2024 22:37:05 +0000
Subject: Question about garbage collection
In-Reply-To: <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
References: <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
Message-ID: <D0F6FC52-8A46-47F4-9ABD-B769A0F61E24@barrys-emacs.org>



> On 16 Jan 2024, at 12:10, Frank Millman via Python-list <python-list at python.org> wrote:
> 
> My problem is that my app is quite complex, and it is easy to leave a reference dangling somewhere which prevents an object from being gc'd.

What I do to track these problems down is use gc.get_objects() then summerize the number of each type. Part 2 is to print the delta after an interval of a 2nd summary.
Leaks of objects show up as the count of a type increasing every time you sample.


Barry



From barry at barrys-emacs.org  Tue Jan 16 17:44:41 2024
From: barry at barrys-emacs.org (Barry)
Date: Tue, 16 Jan 2024 22:44:41 +0000
Subject: Python 3.12.1,
 Windows 11: shebang line #!/usr/bin/env python3 doesn't work any more
In-Reply-To: <d59f5407-0bfd-497e-86ed-66f751a4295e@web.de>
References: <d59f5407-0bfd-497e-86ed-66f751a4295e@web.de>
Message-ID: <825F8526-3074-4CD0-AED6-CCBE284840B8@barrys-emacs.org>



> On 16 Jan 2024, at 17:11, Sibylle Koczian via Python-list <python-list at python.org> wrote:
> 
> while the new Windows 11 machine finds the Microsoft stub

You can turn off the stub in windows settings. The magic windows jargon is
?App Execution Aliases?. Once you find it in settings you can turn off
the python and python3 aliases.

Barry





From greg.ewing at canterbury.ac.nz  Tue Jan 16 20:01:49 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 17 Jan 2024 14:01:49 +1300
Subject: Question about garbage collection
In-Reply-To: <mailman.82.1705417230.15798.python-list@python.org>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
 <CAPTjJmqJpaY7-VQ1kjpqPOhtQDwgBNiAhqthY43A84eZq6gErg@mail.gmail.com>
 <048f182a-5573-416f-8266-0956c2491d7f@chagford.com>
 <CAPTjJmrUedFjLNW6Uqo35cGSwkdAW+10t_-DP1wWdbvHL8wxvA@mail.gmail.com>
 <mailman.82.1705417230.15798.python-list@python.org>
Message-ID: <l0on7tFfctvU1@mid.individual.net>

On 17/01/24 4:00 am, Chris Angelico wrote:
> class Form:
>      def __init__(self):
>          self.elements = []
> 
> class Element:
>      def __init__(self, form):
>          self.form = form
>          form.elements.append(self)

If you make the reference from Element to Form a weak reference,
it won't keep the Form alive after it's been closed.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Jan 16 20:01:54 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 17 Jan 2024 14:01:54 +1300
Subject: Question about garbage collection
In-Reply-To: <mailman.78.1705406835.15798.python-list@python.org>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
 <mailman.78.1705406835.15798.python-list@python.org>
Message-ID: <l0on83FfctvU2@mid.individual.net>

On 17/01/24 1:01 am, Frank Millman wrote:
> I sometimes need to keep a reference from a 
> transient object to a more permanent structure in my app. To save myself 
> the extra step of removing all these references when the transient 
> object is deleted, I make them weak references.

I don't see how weak references help here at all. If the transient
object goes away, all references from it to the permanent objects also
go away.

A weak reference would only be of use if the reference went the other
way, i.e. from the permanent object to the transient object.

-- 
Greg

From frank at chagford.com  Wed Jan 17 01:42:17 2024
From: frank at chagford.com (Frank Millman)
Date: Wed, 17 Jan 2024 08:42:17 +0200
Subject: Question about garbage collection
In-Reply-To: <l0on83FfctvU2@mid.individual.net>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
 <mailman.78.1705406835.15798.python-list@python.org>
 <l0on83FfctvU2@mid.individual.net>
Message-ID: <a0f9ebdd-eec7-4bd0-a228-f806dd8766fd@chagford.com>

On 2024-01-17 3:01 AM, Greg Ewing via Python-list wrote:
> On 17/01/24 1:01 am, Frank Millman wrote:
>> I sometimes need to keep a reference from a transient object to a more 
>> permanent structure in my app. To save myself the extra step of 
>> removing all these references when the transient object is deleted, I 
>> make them weak references.
> 
> I don't see how weak references help here at all. If the transient
> object goes away, all references from it to the permanent objects also
> go away.
> 
> A weak reference would only be of use if the reference went the other
> way, i.e. from the permanent object to the transient object.
> 

You are right. I got my description above back-to-front. It is a pub/sub 
scenario. A transient object makes a request to the permanent object to 
be notified of any changes. The permanent object stores a reference to 
the transient object and executes a callback on each change. When the 
transient object goes away, the reference must be removed.

Frank


From azaharia at live.com  Wed Jan 17 11:48:32 2024
From: azaharia at live.com (Alan Zaharia)
Date: Wed, 17 Jan 2024 16:48:32 +0000
Subject: PyTorch
Message-ID: <IA0PR20MB5635DE0185D643F52271942EAA722@IA0PR20MB5635.namprd20.prod.outlook.com>

Hello Python

I Need help. it could not be found for PyTorch. It said in the Command Prompt ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch,  Can you help me?

Thank You,

Best,
Alan.

From olegsivokon at gmail.com  Wed Jan 17 09:33:14 2024
From: olegsivokon at gmail.com (Left Right)
Date: Wed, 17 Jan 2024 15:33:14 +0100
Subject: Question about garbage collection
In-Reply-To: <a0f9ebdd-eec7-4bd0-a228-f806dd8766fd@chagford.com>
References: <35f5aba8-047e-477f-b1d5-6892d77aa9d8@chagford.com>
 <36df3ecc-7970-453d-8507-3ff8316c28ca@chagford.com>
 <mailman.78.1705406835.15798.python-list@python.org>
 <l0on83FfctvU2@mid.individual.net>
 <a0f9ebdd-eec7-4bd0-a228-f806dd8766fd@chagford.com>
Message-ID: <CAJQBtgmZyKm=yS5Ab=rWyhCzp+L0SOQ3arsm3XJ4608+j459FQ@mail.gmail.com>

So, here's some info about how to see what's going on with Python's
memory allocation: https://docs.python.org/3/library/tracemalloc.html
. I haven't looked into this in a long time, but it used to be the
case that you needed to compile native modules (and probably Python
itself?) so that instrumentation is possible (I think incref / decref
macros should give you a hint, because they would have to naturally
report some of that info).

Anyways.  The problem of tracing memory allocation / deallocation in
Python can be roughly split into these categories:

1. Memory legitimately claimed by objects created through Python
runtime, but not reclaimed due to programmer error. I.e. the
programmer wrote a program that keeps references to objects which it
will never use again.
2. Memory claimed through native objects obtained by means of
interacting with Python's allocator.  When working with Python C API
it's best to interface with Python allocator to deal with dynamic
memory allocation and release.  However, it's somewhat cumbersome, and
some module authors simply might not know about it, or wouldn't want
to use it because they prefer a different allocator.  Sometimes
library authors don't implement memory deallocation well.  Which
brings us to:
3. Memory claimed by any user-space code that is associated with the
Python process. This can be for example shared libraries loaded by
means of Python bindings, that is on top of the situation described
above.
4. System memory associated with the process.  Some system calls need
to allocate memory on the system side.  Typical examples are opening
files, creating sockets etc.  Typically, the system will limit the
number of such objects, and the user program will hit the numerical
limit before it hits the memory limit, but it can also happen that
this will manifest as a memory problem (one example I ran into was
trying to run conda-build and it would fail due to enormous amounts of
memory it requested, but the specifics of the failure were due to it
trying to create new sub-processes -- another system resource that
requires memory allocation).

There isn't a universal strategy to cover all these cases.  But, if
you have reasons to suspect (4), for example, you'd probably start by
using strace utility (on Linux) to see what system calls are executed.

For something like the (3), you could try to utilize Valgrind (but
it's a lot of work to set it up).  It's also possible to use jemalloc
to profile a program, but you would have to build Python with its
allocator modified to use jemalloc (I've seen an issue in the Python
bug tracker where someone wrote a script to do that, so it should be
possible).  Both of these are quite labor intensive and not trivial to
set up.

(2) could be often diagnosed with tracemalloc Python module and (1) is
something that can be helped with Python's gc module.

It's always better though to have an actual error and work from there.
Or, at least, have some monitoring data that suggests that your
application memory use increases over time.  Otherwise you could be
spending a lot of time chasing problems you don't have.

From mats at wichmann.us  Wed Jan 17 13:44:49 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Wed, 17 Jan 2024 11:44:49 -0700
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <d59f5407-0bfd-497e-86ed-66f751a4295e@web.de>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
 <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
 <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>
 <23c6ee06-5e7b-42b0-a1f4-d151f8a47a1c@wichmann.us>
 <d59f5407-0bfd-497e-86ed-66f751a4295e@web.de>
Message-ID: <12804cfa-e36f-4831-9408-0990a6cce221@wichmann.us>

On 1/16/24 10:00, Sibylle Koczian via Python-list wrote:
> Am 15.01.2024 um 23:55 schrieb Mats Wichmann via Python-list:
>> On 1/15/24 12:01, Thomas Passin via Python-list wrote:
>>> On 1/15/2024 1:26 PM, Mats Wichmann via Python-list wrote:
>>> Python from the App Store is not the same as Python from python.org:
>>
>> yes. this question is about the python.org distribution. but, Windows 
>> natively has something called python.exe and python3.exe which is 
>> interfering here, IF the python.org install isn't directed to put 
>> itself into the path, AND if the "#!/usr/bin/env python3" form is 
>> used, causing a search in PATH, which is the setup Sibylle has 
>> described, unless I've misunderstood details.
>>
> 
> No, you didn't misunderstand any detail. It's exactly right. My Windows 
> 10 box doesn't find anything for "where python", "where python3",

Be interesting to know if your WIndows 10 has those files in place, and 
it's just a missing path entry (a good thing, perhaps) that's causing it 
not to be found there.

> while the new Windows 11 machine finds the Microsoft stub. "Irritating" is a 
> very friendly attribute for that thing. Why must it be called 
> "python.exe" and not something else like the installation files from 
> python.org?

it will be replaced by the real "python.exe" from the Microsoft Store 
install, if you go ahead and install that - I guess that's why that name 
was chosen.

> I'll stop using "/env" - hopefully that won't create problems with the 
> scripts I use in the Linux VM. But in that case I'll know what's up.
> 
> Thank you very much!
> Sibylle
> 
> 


From mats at wichmann.us  Wed Jan 17 13:51:53 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Wed, 17 Jan 2024 11:51:53 -0700
Subject: PyTorch
In-Reply-To: <IA0PR20MB5635DE0185D643F52271942EAA722@IA0PR20MB5635.namprd20.prod.outlook.com>
References: <IA0PR20MB5635DE0185D643F52271942EAA722@IA0PR20MB5635.namprd20.prod.outlook.com>
Message-ID: <8a1670a1-7a3e-4363-b422-77dc197338db@wichmann.us>

On 1/17/24 09:48, Alan Zaharia via Python-list wrote:
> Hello Python
> 
> I Need help. it could not be found for PyTorch. It said in the Command Prompt ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
> ERROR: No matching distribution found for torch,  Can you help me?

Use Python 3.11.


Or follow here:

https://github.com/pytorch/pytorch/issues/110436


From thomas at python.org  Wed Jan 17 19:32:26 2024
From: thomas at python.org (Thomas Wouters)
Date: Thu, 18 Jan 2024 01:32:26 +0100
Subject: Python 3.13.0a3 is now available.
Message-ID: <CAPdQG2p+PX0_XLVZPkxED61ayVRuMqLZJystcHCa9ftOqK6MaQ@mail.gmail.com>

We silently skipped releasing in December (it was too close to the
holidays, a lot of people were away) so by date you may have been expecting
alpha 4, but instead it?s alpha 3:
https://www.python.org/downloads/release/python-3130a3/

*This is an early developer preview of Python 3.13*
<https://discuss.python.org/t/python-3-13-alpha-3/43437#major-new-features-of-the-313-series-compared-to-312-1>Major
new features of the 3.13 series, compared to 3.12

Python 3.13 is still in development. This release, 3.13.0a3, is the third
of six planned alpha releases.

Alpha releases are intended to make it easier to test the current state of
new features and bug fixes and to test the release process.

During the alpha phase, features may be added up until the start of the
beta phase (2024-05-07) and, if necessary, may be modified or deleted up
until the release candidate phase (2024-07-30). Please keep in mind that
this is a preview release and its use is *not* recommended for production
environments.

Many new features for Python 3.13 are still being planned and written. Work
continues apace on both the work to remove the Global Interpeter Lock
<https://peps.python.org/pep-0703/>, and to improve Python performance. The
most notable changes so far:

   - In the interactive interpreter, exception tracebacks are now colorized
   by default
   <https://docs.python.org/dev/whatsnew/3.13.html#improved-error-messages>.
   - Docstrings now have their leading indentation stripped
   <https://docs.python.org/dev/whatsnew/3.13.html#other-language-changes>,
   reducing memory use and the size of .pyc files. (Most tools handling
   docstrings already strip leading indentation.)
   - PEP 594 (Removing dead batteries from the standard library)
   <https://peps.python.org/pep-0594/> scheduled removals of many
   deprecated modules: aifc, audioop, chunk, cgi, cgitb, crypt, imghdr,
   mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau,
   telnetlib, uu, xdrlib, lib2to3.
   - Many other removals
   <https://docs.python.org/dev/whatsnew/3.13.html#removed> of deprecated
   classes, functions and methods in various standard library modules.
   - New deprecations
   <https://docs.python.org/dev/whatsnew/3.13.html#deprecated>, most of
   which are scheduled for removal from Python 3.15 or 3.16.
   - C API removals <https://docs.python.org/dev/whatsnew/3.13.html#id10>
   and deprecations <https://docs.python.org/dev/whatsnew/3.13.html#id9>.
   (Some removals present in alpha 1 have been reverted in alpha 2, as the
   removals were deemed too disruptive at this time.)

(Hey, *fellow core developer,* if a feature you find important is missing
from this list, let Thomas know <thomas at python.org>.)

The next pre-release of Python 3.13 will be 3.13.0a4, currently scheduled
for 2023-02-13.
<https://discuss.python.org/t/python-3-13-alpha-3/43437#more-resources-2>More
resources

   - Online Documentation <https://docs.python.org/3.13/>
   - PEP 719 <https://www.python.org/dev/peps/pep-0719/>, 3.13 Release
   Schedule
   - Report bugs at Issues ? python/cpython ? GitHub
   <https://github.com/python/cpython/issues>.
   - Help fund Python and its community
   <https://www.python.org/psf/donations/>.

<https://discuss.python.org/t/python-3-13-alpha-3/43437#enjoy-the-new-releases-3>Enjoy
the new releases

Thanks to all of the many volunteers who help make Python Development and
these releases possible! Please consider supporting our efforts by
volunteering yourself or through organization contributions to the Python
Software Foundation.

Regards from snowy Amsterdam,

Your release team,
Thomas Wouters
Ned Deily
Steve Dower
?ukasz Langa
-- 
Thomas Wouters <thomas at python.org>

From avi.e.gross at gmail.com  Fri Jan 19 19:32:25 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Fri, 19 Jan 2024 19:32:25 -0500
Subject: extend behaviour of assignment operator
In-Reply-To: <CAASeUHopbgPeqdC_bbvQSiVnTb0y=20BJz-2T-82Huakhu7Bdw@mail.gmail.com>
References: <CAASeUHopbgPeqdC_bbvQSiVnTb0y=20BJz-2T-82Huakhu7Bdw@mail.gmail.com>
Message-ID: <036201da4b38$2475e990$6d61bcb0$@gmail.com>

Guenther,

It is best not to suggest a drastic fix for a more limited problem.

As a general rule, many programming languages only have a pointer concept
even vaguely along the lines you want for garbage collection purposes. An
area of memory may have stored alongside it how many other things point at
it but not which ones. As long as it is decremented when a pointer leaves,
it works.

If you want to design objects that can store additional info when invoked
properly, go for it. No change to python would be needed. In your example,
you could create an object initialized by cube([10,1,1], "a") which now
might remember that something called "a" once pointed at it. But you then
have to figure out how to ensure than when "a" is deleted or reset or goes
out of the current environment, that things are properly updated.

I am not so sure how easy it would be to change the language so it pays
attention to what it is giving a pointer too and then goes and tells ...



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Guenther Sohler via Python-list
Sent: Tuesday, January 9, 2024 2:15 AM
To: python-list at python.org
Subject: extend behaviour of assignment operator

Hi,

when i run this code

a = cube([10,1,1])
b = a

i'd like to extend the behaviour  of the assignment operator
a shall not only contain the cube, but  the cube shall also know which
variable name it
was assigned to, lately. I'd like to use that for improved user interaction.

effective code should be:

a=cube([10,1,1])
a.name='a'

b=a
b.name='b' # i am aware that a.name also changes


can decorators also be used with assignment operators ?

thank you for your hints
-- 
https://mail.python.org/mailman/listinfo/python-list


From mk1853387 at gmail.com  Sun Jan 21 07:37:43 2024
From: mk1853387 at gmail.com (marc nicole)
Date: Sun, 21 Jan 2024 13:37:43 +0100
Subject: How to replace a cell value with each of its contour cells and yield
 the corresponding datasets seperately in a list according to a
 Pandas-way?
Message-ID: <CAGJtH9RvLNeHFQSVk4b-TuYKKLgVbvAnoiQvC37vw5HGm2YjHg@mail.gmail.com>

Hello,

I have an initial dataframe with a random list of target cells (each cell
being identified with a couple (x,y)).
I want to yield four different dataframes each containing the value of one
of the contour (surrounding) cells of each specified target cell.

the surrounding cells to consider for a specific target cell are : (x-1,y),
(x,y-1),(x+1,y);(x,y+1), specifically I randomly choose 1 to 4 cells from
these and consider for replacement to the target cell.

I want to do that through a pandas-specific approach without having to
define the contour cells separately and then apply the changes on the
dataframe (but rather using an all in one approach):
for now I have written this example which I think is not Pandas specific:























































*def select_target_values(dataframe, number_of_target_values):
target_cells = []    for _ in range(number_of_target_values):        row_x
= random.randint(0, len(dataframe.columns) - 1)        col_y =
random.randint(0, len(dataframe) - 1)        target_cells.append((row_x,
col_y))    return target_cellsdef select_contours(target_cells):
contour_coordinates = [(0, 1), (1, 0), (0, -1), (-1, 0)]    contour_cells =
[]    for target_cell in target_cells:        # random contour count for
each cell        contour_cells_count = random.randint(1, 4)        try:
        contour_cells.append(                [tuple(map(lambda i, j: i + j,
(target_cell[0], target_cell[1]), contour_coordinates[iteration_]))
         for iteration_ in range(contour_cells_count)])        except
IndexError:            continue    return contour_cellsdef
apply_contours(target_cells, contour_cells):    target_cells_with_contour =
[]    # create one single list of cells    for idx, target_cell in
enumerate(target_cells):        target_cell_with_contour = [target_cell]
    target_cell_with_contour.extend(contour_cells[idx])
target_cells_with_contour.append(target_cell_with_contour)return
target_cells_with_contourdef create_possible_datasets(dataframe,
target_cells_with_contour):    all_datasets_final = []
dataframe_original = dataframe.copy()    #check for nans
list_tuples_idx_cells_all_datasets = list(filter(lambda x:
utils_tuple_list_not_contain_nan(x),
                [list(tuples) for tuples in list(itertools.product(

*target_cells_with_contour))]))    target_original_cells_coordinates =
list(map(lambda x: x[0],
[target_and_contour_cell for target_and_contour_cell in
                                 target_cells_with_contour]))    for
dataset_index_values in list_tuples_idx_cells_all_datasets:
all_datasets = []        for idx_cell in range(len(dataset_index_values)):
          dataframe_cpy = dataframe.copy()            dataframe_cpy.iat[
            target_original_cells_coordinates[idx_cell][1],
target_original_cells_coordinates[idx_cell][                    0]] =
dataframe_original.iloc[dataset_index_values[idx_cell][1],
dataset_index_values[idx_cell][0]]
all_datasets.append(dataframe_cpy)
all_datasets_final.append(all_datasets)    return all_datasets_final*


If you have a better Pandas approach (unifying all these methods into one
that make use of dataframe methods only) please let me know.

thanks!

From list1 at tompassin.net  Sun Jan 21 09:14:45 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 21 Jan 2024 09:14:45 -0500
Subject: How to replace a cell value with each of its contour cells and
 yield the corresponding datasets seperately in a list according to a
 Pandas-way?
In-Reply-To: <CAGJtH9RvLNeHFQSVk4b-TuYKKLgVbvAnoiQvC37vw5HGm2YjHg@mail.gmail.com>
References: <CAGJtH9RvLNeHFQSVk4b-TuYKKLgVbvAnoiQvC37vw5HGm2YjHg@mail.gmail.com>
Message-ID: <a9aa6809-2842-4e7d-9024-fb2549c2eb65@tompassin.net>

On 1/21/2024 7:37 AM, marc nicole via Python-list wrote:
> Hello,
> 
> I have an initial dataframe with a random list of target cells (each cell
> being identified with a couple (x,y)).
> I want to yield four different dataframes each containing the value of one
> of the contour (surrounding) cells of each specified target cell.
> 
> the surrounding cells to consider for a specific target cell are : (x-1,y),
> (x,y-1),(x+1,y);(x,y+1), specifically I randomly choose 1 to 4 cells from
> these and consider for replacement to the target cell.
> 
> I want to do that through a pandas-specific approach without having to
> define the contour cells separately and then apply the changes on the
> dataframe 

1. Why do you want a Pandas-specific approach?  Many people would rather 
keep code independent of special libraries if possible;

2. How big can these collections of target cells be, roughly speaking? 
The size could make a big difference in picking a design;

3. You really should work on formatting code for this list.  Your code 
below is very complex and would take a lot of work to reformat to the 
point where it is readable, especially with the nearly impenetrable 
arguments in some places.  Probably all that is needed is to replace all 
tabs by (say) three spaces, and to make sure you intentionally break 
lines well before they might get word-wrapped.  Here is one example I 
have reformatted (I hope I got this right):

list_tuples_idx_cells_all_datasets = list(filter(
    lambda x: utils_tuple_list_not_contain_nan(x),
    [list(tuples) for tuples in list(
          itertools.product(*target_cells_with_contour))
    ]))

4. As an aside, it doesn't look like you need to convert all those 
sequences and iterators to lists all over the place;


> (but rather using an all in one approach):
> for now I have written this example which I think is not Pandas specific:
[snip]


From mk1853387 at gmail.com  Sun Jan 21 11:54:09 2024
From: mk1853387 at gmail.com (marc nicole)
Date: Sun, 21 Jan 2024 17:54:09 +0100
Subject: How to replace a cell value with each of its contour cells and
 yield the corresponding datasets seperately in a list according to a
 Pandas-way?
In-Reply-To: <a9aa6809-2842-4e7d-9024-fb2549c2eb65@tompassin.net>
References: <CAGJtH9RvLNeHFQSVk4b-TuYKKLgVbvAnoiQvC37vw5HGm2YjHg@mail.gmail.com>
 <a9aa6809-2842-4e7d-9024-fb2549c2eb65@tompassin.net>
Message-ID: <CAGJtH9Svh5Q5B68aB+WK8JaYzV=EY-pfobi=cSoNjrTxBxv5_Q@mail.gmail.com>

Thanks for the reply,

I think using a Pandas (or a Numpy) approach would optimize the execution
of the program.

Target cells could be up to 10% the size of the dataset, a good example to
start with would have from 10 to 100 values.

Let me know your thoughts, here's a reproducible example which I formatted:



from numpy import random
import pandas as pd
import numpy as np
import operator
import math
from collections import deque
from queue import *
from queue import Queue
from itertools import product


def select_target_values(dataframe, number_of_target_values):
    target_cells = []
    for _ in range(number_of_target_values):
        row_x = random.randint(0, len(dataframe.columns) - 1)
        col_y = random.randint(0, len(dataframe) - 1)
        target_cells.append((row_x, col_y))
    return target_cells


def select_contours(target_cells):
    contour_coordinates = [(0, 1), (1, 0), (0, -1), (-1, 0)]
    contour_cells = []
    for target_cell in target_cells:
        # random contour count for each cell
        contour_cells_count = random.randint(1, 4)
        try:
            contour_cells.append(
                [
                    tuple(
                        map(
                            lambda i, j: i + j,
                            (target_cell[0], target_cell[1]),
                            contour_coordinates[iteration_],
                        )
                    )
                    for iteration_ in range(contour_cells_count)
                ]
            )
        except IndexError:
            continue
    return contour_cells


def create_zipf_distribution():
    zipf_dist = random.zipf(2, size=(50, 5)).reshape((50, 5))

    zipf_distribution_dataset = pd.DataFrame(zipf_dist).round(3)

    return zipf_distribution_dataset


def apply_contours(target_cells, contour_cells):
    target_cells_with_contour = []
    # create one single list of cells
    for idx, target_cell in enumerate(target_cells):
        target_cell_with_contour = [target_cell]
        target_cell_with_contour.extend(contour_cells[idx])
        target_cells_with_contour.append(target_cell_with_contour)
    return target_cells_with_contour


def create_possible_datasets(dataframe, target_cells_with_contour):
    all_datasets_final = []
    dataframe_original = dataframe.copy()

    list_tuples_idx_cells_all_datasets = list(
        filter(
            lambda x: x,
            [list(tuples) for tuples in
list(product(*target_cells_with_contour))],
        )
    )
    target_original_cells_coordinates = list(
        map(
            lambda x: x[0],
            [
                target_and_contour_cell
                for target_and_contour_cell in target_cells_with_contour
            ],
        )
    )
    for dataset_index_values in list_tuples_idx_cells_all_datasets:
        all_datasets = []
        for idx_cell in range(len(dataset_index_values)):
            dataframe_cpy = dataframe.copy()
            dataframe_cpy.iat[
                target_original_cells_coordinates[idx_cell][1],
                target_original_cells_coordinates[idx_cell][0],
            ] = dataframe_original.iloc[
                dataset_index_values[idx_cell][1],
dataset_index_values[idx_cell][0]
            ]
            all_datasets.append(dataframe_cpy)
        all_datasets_final.append(all_datasets)
    return all_datasets_final


def main():
    zipf_dataset = create_zipf_distribution()

    target_cells = select_target_values(zipf_dataset, 5)
    print(target_cells)
    contour_cells = select_contours(target_cells)
    print(contour_cells)
    target_cells_with_contour = apply_contours(target_cells, contour_cells)
    datasets = create_possible_datasets(zipf_dataset,
target_cells_with_contour)
    print(datasets)


main()

Le dim. 21 janv. 2024 ? 16:33, Thomas Passin via Python-list <
python-list at python.org> a ?crit :

> On 1/21/2024 7:37 AM, marc nicole via Python-list wrote:
> > Hello,
> >
> > I have an initial dataframe with a random list of target cells (each cell
> > being identified with a couple (x,y)).
> > I want to yield four different dataframes each containing the value of
> one
> > of the contour (surrounding) cells of each specified target cell.
> >
> > the surrounding cells to consider for a specific target cell are :
> (x-1,y),
> > (x,y-1),(x+1,y);(x,y+1), specifically I randomly choose 1 to 4 cells from
> > these and consider for replacement to the target cell.
> >
> > I want to do that through a pandas-specific approach without having to
> > define the contour cells separately and then apply the changes on the
> > dataframe
>
> 1. Why do you want a Pandas-specific approach?  Many people would rather
> keep code independent of special libraries if possible;
>
> 2. How big can these collections of target cells be, roughly speaking?
> The size could make a big difference in picking a design;
>
> 3. You really should work on formatting code for this list.  Your code
> below is very complex and would take a lot of work to reformat to the
> point where it is readable, especially with the nearly impenetrable
> arguments in some places.  Probably all that is needed is to replace all
> tabs by (say) three spaces, and to make sure you intentionally break
> lines well before they might get word-wrapped.  Here is one example I
> have reformatted (I hope I got this right):
>
> list_tuples_idx_cells_all_datasets = list(filter(
>     lambda x: utils_tuple_list_not_contain_nan(x),
>     [list(tuples) for tuples in list(
>           itertools.product(*target_cells_with_contour))
>     ]))
>
> 4. As an aside, it doesn't look like you need to convert all those
> sequences and iterators to lists all over the place;
>
>
> > (but rather using an all in one approach):
> > for now I have written this example which I think is not Pandas specific:
> [snip]
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From list1 at tompassin.net  Sun Jan 21 12:18:31 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 21 Jan 2024 12:18:31 -0500
Subject: How to replace a cell value with each of its contour cells and
 yield the corresponding datasets seperately in a list according to a
 Pandas-way?
In-Reply-To: <CAGJtH9Svh5Q5B68aB+WK8JaYzV=EY-pfobi=cSoNjrTxBxv5_Q@mail.gmail.com>
References: <CAGJtH9RvLNeHFQSVk4b-TuYKKLgVbvAnoiQvC37vw5HGm2YjHg@mail.gmail.com>
 <a9aa6809-2842-4e7d-9024-fb2549c2eb65@tompassin.net>
 <CAGJtH9Svh5Q5B68aB+WK8JaYzV=EY-pfobi=cSoNjrTxBxv5_Q@mail.gmail.com>
Message-ID: <86081560-08ff-42bc-ae6d-dfb03d8b2a7d@tompassin.net>

On 1/21/2024 11:54 AM, marc nicole wrote:
> Thanks for the reply,
> 
> I think using a Pandas (or a Numpy) approach would optimize the 
> execution of the program.
> 
> Target cells could be up to 10% the size of the dataset, a good example 
> to start with would have from 10 to 100 values.

Thanks for the reformatted code.  It's much easier to read and think about.

For say 100 points, it doesn't seem that "optimization" would be much of 
an issue.  On my laptop machine and Python 3.12, your example takes 
around 5 seconds to run and print().  OTOH if you think you will go to 
much larger datasets, certainly execution time could become a factor.

I would think that NumPy arrays and/or matrices would have good potential.

Is this some kind of a cellular automaton, or an image filtering process?

> Let me know your thoughts, here's a reproducible example which I formatted:
> 
> 
> 
> from numpy import random
> import pandas as pd
> import numpy as np
> import operator
> import math
> from collections import deque
> from queue import *
> from queue import Queue
> from itertools import product
> 
> 
> def select_target_values(dataframe, number_of_target_values):
>  ? ? target_cells = []
>  ? ? for _ in range(number_of_target_values):
>  ? ? ? ? row_x = random.randint(0, len(dataframe.columns) - 1)
>  ? ? ? ? col_y = random.randint(0, len(dataframe) - 1)
>  ? ? ? ? target_cells.append((row_x, col_y))
>  ? ? return target_cells
> 
> 
> def select_contours(target_cells):
>  ? ? contour_coordinates = [(0, 1), (1, 0), (0, -1), (-1, 0)]
>  ? ? contour_cells = []
>  ? ? for target_cell in target_cells:
>  ? ? ? ? # random contour count for each cell
>  ? ? ? ? contour_cells_count = random.randint(1, 4)
>  ? ? ? ? try:
>  ? ? ? ? ? ? contour_cells.append(
>  ? ? ? ? ? ? ? ? [
>  ? ? ? ? ? ? ? ? ? ? tuple(
>  ? ? ? ? ? ? ? ? ? ? ? ? map(
>  ? ? ? ? ? ? ? ? ? ? ? ? ? ? lambda i, j: i + j,
>  ? ? ? ? ? ? ? ? ? ? ? ? ? ? (target_cell[0], target_cell[1]),
>  ? ? ? ? ? ? ? ? ? ? ? ? ? ? contour_coordinates[iteration_],
>  ? ? ? ? ? ? ? ? ? ? ? ? )
>  ? ? ? ? ? ? ? ? ? ? )
>  ? ? ? ? ? ? ? ? ? ? for iteration_ in range(contour_cells_count)
>  ? ? ? ? ? ? ? ? ]
>  ? ? ? ? ? ? )
>  ? ? ? ? except IndexError:
>  ? ? ? ? ? ? continue
>  ? ? return contour_cells
> 
> 
> def create_zipf_distribution():
>  ? ? zipf_dist = random.zipf(2, size=(50, 5)).reshape((50, 5))
> 
>  ? ? zipf_distribution_dataset = pd.DataFrame(zipf_dist).round(3)
> 
>  ? ? return zipf_distribution_dataset
> 
> 
> def apply_contours(target_cells, contour_cells):
>  ? ? target_cells_with_contour = []
>  ? ? # create one single list of cells
>  ? ? for idx, target_cell in enumerate(target_cells):
>  ? ? ? ? target_cell_with_contour = [target_cell]
>  ? ? ? ? target_cell_with_contour.extend(contour_cells[idx])
>  ? ? ? ? target_cells_with_contour.append(target_cell_with_contour)
>  ? ? return target_cells_with_contour
> 
> 
> def create_possible_datasets(dataframe, target_cells_with_contour):
>  ? ? all_datasets_final = []
>  ? ? dataframe_original = dataframe.copy()
> 
>  ? ? list_tuples_idx_cells_all_datasets = list(
>  ? ? ? ? filter(
>  ? ? ? ? ? ? lambda x: x,
>  ? ? ? ? ? ? [list(tuples) for tuples in 
> list(product(*target_cells_with_contour))],
>  ? ? ? ? )
>  ? ? )
>  ? ? target_original_cells_coordinates = list(
>  ? ? ? ? map(
>  ? ? ? ? ? ? lambda x: x[0],
>  ? ? ? ? ? ? [
>  ? ? ? ? ? ? ? ? target_and_contour_cell
>  ? ? ? ? ? ? ? ? for target_and_contour_cell in target_cells_with_contour
>  ? ? ? ? ? ? ],
>  ? ? ? ? )
>  ? ? )
>  ? ? for dataset_index_values in list_tuples_idx_cells_all_datasets:
>  ? ? ? ? all_datasets = []
>  ? ? ? ? for idx_cell in range(len(dataset_index_values)):
>  ? ? ? ? ? ? dataframe_cpy = dataframe.copy()
>  ? ? ? ? ? ? dataframe_cpy.iat[
>  ? ? ? ? ? ? ? ? target_original_cells_coordinates[idx_cell][1],
>  ? ? ? ? ? ? ? ? target_original_cells_coordinates[idx_cell][0],
>  ? ? ? ? ? ? ] = dataframe_original.iloc[
>  ? ? ? ? ? ? ? ? dataset_index_values[idx_cell][1], 
> dataset_index_values[idx_cell][0]
>  ? ? ? ? ? ? ]
>  ? ? ? ? ? ? all_datasets.append(dataframe_cpy)
>  ? ? ? ? all_datasets_final.append(all_datasets)
>  ? ? return all_datasets_final
> 
> 
> def main():
>  ? ? zipf_dataset = create_zipf_distribution()
> 
>  ? ? target_cells = select_target_values(zipf_dataset, 5)
>  ? ? print(target_cells)
>  ? ? contour_cells = select_contours(target_cells)
>  ? ? print(contour_cells)
>  ? ? target_cells_with_contour = apply_contours(target_cells, contour_cells)
>  ? ? datasets = create_possible_datasets(zipf_dataset, 
> target_cells_with_contour)
>  ? ? print(datasets)
> 
> 
> main()
> 
> Le?dim. 21 janv. 2024 ??16:33, Thomas Passin via Python-list 
> <python-list at python.org <mailto:python-list at python.org>> a ?crit?:
> 
>     On 1/21/2024 7:37 AM, marc nicole via Python-list wrote:
>      > Hello,
>      >
>      > I have an initial dataframe with a random list of target cells
>     (each cell
>      > being identified with a couple (x,y)).
>      > I want to yield four different dataframes each containing the
>     value of one
>      > of the contour (surrounding) cells of each specified target cell.
>      >
>      > the surrounding cells to consider for a specific target cell are
>     : (x-1,y),
>      > (x,y-1),(x+1,y);(x,y+1), specifically I randomly choose 1 to 4
>     cells from
>      > these and consider for replacement to the target cell.
>      >
>      > I want to do that through a pandas-specific approach without
>     having to
>      > define the contour cells separately and then apply the changes on the
>      > dataframe
> 
>     1. Why do you want a Pandas-specific approach?? Many people would
>     rather
>     keep code independent of special libraries if possible;
> 
>     2. How big can these collections of target cells be, roughly speaking?
>     The size could make a big difference in picking a design;
> 
>     3. You really should work on formatting code for this list.? Your code
>     below is very complex and would take a lot of work to reformat to the
>     point where it is readable, especially with the nearly impenetrable
>     arguments in some places.? Probably all that is needed is to replace
>     all
>     tabs by (say) three spaces, and to make sure you intentionally break
>     lines well before they might get word-wrapped.? Here is one example I
>     have reformatted (I hope I got this right):
> 
>     list_tuples_idx_cells_all_datasets = list(filter(
>      ? ? lambda x: utils_tuple_list_not_contain_nan(x),
>      ? ? [list(tuples) for tuples in list(
>      ? ? ? ? ? itertools.product(*target_cells_with_contour))
>      ? ? ]))
> 
>     4. As an aside, it doesn't look like you need to convert all those
>     sequences and iterators to lists all over the place;
> 
> 
>      > (but rather using an all in one approach):
>      > for now I have written this example which I think is not Pandas
>     specific:
>     [snip]
> 
>     -- 
>     https://mail.python.org/mailman/listinfo/python-list
>     <https://mail.python.org/mailman/listinfo/python-list>
> 


From mk1853387 at gmail.com  Sun Jan 21 13:25:17 2024
From: mk1853387 at gmail.com (marc nicole)
Date: Sun, 21 Jan 2024 19:25:17 +0100
Subject: How to replace a cell value with each of its contour cells and
 yield the corresponding datasets seperately in a list according to a
 Pandas-way?
In-Reply-To: <86081560-08ff-42bc-ae6d-dfb03d8b2a7d@tompassin.net>
References: <CAGJtH9RvLNeHFQSVk4b-TuYKKLgVbvAnoiQvC37vw5HGm2YjHg@mail.gmail.com>
 <a9aa6809-2842-4e7d-9024-fb2549c2eb65@tompassin.net>
 <CAGJtH9Svh5Q5B68aB+WK8JaYzV=EY-pfobi=cSoNjrTxBxv5_Q@mail.gmail.com>
 <86081560-08ff-42bc-ae6d-dfb03d8b2a7d@tompassin.net>
Message-ID: <CAGJtH9Qcqd_fBB-h7ED9QxnA0aen6HiavM7+pY00T349MzkT8A@mail.gmail.com>

It is part of a larger project aiming at processing data according to a
given algorithm
Do you have any comments or any enhancing recommendations on the code?

Thanks.

Le dim. 21 janv. 2024 ? 18:28, Thomas Passin via Python-list <
python-list at python.org> a ?crit :

> On 1/21/2024 11:54 AM, marc nicole wrote:
> > Thanks for the reply,
> >
> > I think using a Pandas (or a Numpy) approach would optimize the
> > execution of the program.
> >
> > Target cells could be up to 10% the size of the dataset, a good example
> > to start with would have from 10 to 100 values.
>
> Thanks for the reformatted code.  It's much easier to read and think about.
>
> For say 100 points, it doesn't seem that "optimization" would be much of
> an issue.  On my laptop machine and Python 3.12, your example takes
> around 5 seconds to run and print().  OTOH if you think you will go to
> much larger datasets, certainly execution time could become a factor.
>
> I would think that NumPy arrays and/or matrices would have good potential.
>
> Is this some kind of a cellular automaton, or an image filtering process?
>
> > Let me know your thoughts, here's a reproducible example which I
> formatted:
> >
> >
> >
> > from numpy import random
> > import pandas as pd
> > import numpy as np
> > import operator
> > import math
> > from collections import deque
> > from queue import *
> > from queue import Queue
> > from itertools import product
> >
> >
> > def select_target_values(dataframe, number_of_target_values):
> >      target_cells = []
> >      for _ in range(number_of_target_values):
> >          row_x = random.randint(0, len(dataframe.columns) - 1)
> >          col_y = random.randint(0, len(dataframe) - 1)
> >          target_cells.append((row_x, col_y))
> >      return target_cells
> >
> >
> > def select_contours(target_cells):
> >      contour_coordinates = [(0, 1), (1, 0), (0, -1), (-1, 0)]
> >      contour_cells = []
> >      for target_cell in target_cells:
> >          # random contour count for each cell
> >          contour_cells_count = random.randint(1, 4)
> >          try:
> >              contour_cells.append(
> >                  [
> >                      tuple(
> >                          map(
> >                              lambda i, j: i + j,
> >                              (target_cell[0], target_cell[1]),
> >                              contour_coordinates[iteration_],
> >                          )
> >                      )
> >                      for iteration_ in range(contour_cells_count)
> >                  ]
> >              )
> >          except IndexError:
> >              continue
> >      return contour_cells
> >
> >
> > def create_zipf_distribution():
> >      zipf_dist = random.zipf(2, size=(50, 5)).reshape((50, 5))
> >
> >      zipf_distribution_dataset = pd.DataFrame(zipf_dist).round(3)
> >
> >      return zipf_distribution_dataset
> >
> >
> > def apply_contours(target_cells, contour_cells):
> >      target_cells_with_contour = []
> >      # create one single list of cells
> >      for idx, target_cell in enumerate(target_cells):
> >          target_cell_with_contour = [target_cell]
> >          target_cell_with_contour.extend(contour_cells[idx])
> >          target_cells_with_contour.append(target_cell_with_contour)
> >      return target_cells_with_contour
> >
> >
> > def create_possible_datasets(dataframe, target_cells_with_contour):
> >      all_datasets_final = []
> >      dataframe_original = dataframe.copy()
> >
> >      list_tuples_idx_cells_all_datasets = list(
> >          filter(
> >              lambda x: x,
> >              [list(tuples) for tuples in
> > list(product(*target_cells_with_contour))],
> >          )
> >      )
> >      target_original_cells_coordinates = list(
> >          map(
> >              lambda x: x[0],
> >              [
> >                  target_and_contour_cell
> >                  for target_and_contour_cell in target_cells_with_contour
> >              ],
> >          )
> >      )
> >      for dataset_index_values in list_tuples_idx_cells_all_datasets:
> >          all_datasets = []
> >          for idx_cell in range(len(dataset_index_values)):
> >              dataframe_cpy = dataframe.copy()
> >              dataframe_cpy.iat[
> >                  target_original_cells_coordinates[idx_cell][1],
> >                  target_original_cells_coordinates[idx_cell][0],
> >              ] = dataframe_original.iloc[
> >                  dataset_index_values[idx_cell][1],
> > dataset_index_values[idx_cell][0]
> >              ]
> >              all_datasets.append(dataframe_cpy)
> >          all_datasets_final.append(all_datasets)
> >      return all_datasets_final
> >
> >
> > def main():
> >      zipf_dataset = create_zipf_distribution()
> >
> >      target_cells = select_target_values(zipf_dataset, 5)
> >      print(target_cells)
> >      contour_cells = select_contours(target_cells)
> >      print(contour_cells)
> >      target_cells_with_contour = apply_contours(target_cells,
> contour_cells)
> >      datasets = create_possible_datasets(zipf_dataset,
> > target_cells_with_contour)
> >      print(datasets)
> >
> >
> > main()
> >
> > Le dim. 21 janv. 2024 ? 16:33, Thomas Passin via Python-list
> > <python-list at python.org <mailto:python-list at python.org>> a ?crit :
> >
> >     On 1/21/2024 7:37 AM, marc nicole via Python-list wrote:
> >      > Hello,
> >      >
> >      > I have an initial dataframe with a random list of target cells
> >     (each cell
> >      > being identified with a couple (x,y)).
> >      > I want to yield four different dataframes each containing the
> >     value of one
> >      > of the contour (surrounding) cells of each specified target cell.
> >      >
> >      > the surrounding cells to consider for a specific target cell are
> >     : (x-1,y),
> >      > (x,y-1),(x+1,y);(x,y+1), specifically I randomly choose 1 to 4
> >     cells from
> >      > these and consider for replacement to the target cell.
> >      >
> >      > I want to do that through a pandas-specific approach without
> >     having to
> >      > define the contour cells separately and then apply the changes on
> the
> >      > dataframe
> >
> >     1. Why do you want a Pandas-specific approach?  Many people would
> >     rather
> >     keep code independent of special libraries if possible;
> >
> >     2. How big can these collections of target cells be, roughly
> speaking?
> >     The size could make a big difference in picking a design;
> >
> >     3. You really should work on formatting code for this list.  Your
> code
> >     below is very complex and would take a lot of work to reformat to the
> >     point where it is readable, especially with the nearly impenetrable
> >     arguments in some places.  Probably all that is needed is to replace
> >     all
> >     tabs by (say) three spaces, and to make sure you intentionally break
> >     lines well before they might get word-wrapped.  Here is one example I
> >     have reformatted (I hope I got this right):
> >
> >     list_tuples_idx_cells_all_datasets = list(filter(
> >          lambda x: utils_tuple_list_not_contain_nan(x),
> >          [list(tuples) for tuples in list(
> >                itertools.product(*target_cells_with_contour))
> >          ]))
> >
> >     4. As an aside, it doesn't look like you need to convert all those
> >     sequences and iterators to lists all over the place;
> >
> >
> >      > (but rather using an all in one approach):
> >      > for now I have written this example which I think is not Pandas
> >     specific:
> >     [snip]
> >
> >     --
> >     https://mail.python.org/mailman/listinfo/python-list
> >     <https://mail.python.org/mailman/listinfo/python-list>
> >
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From bagratte at live.com  Sun Jan 21 14:39:56 2024
From: bagratte at live.com (bagratte at live.com)
Date: Sun, 21 Jan 2024 23:39:56 +0400
Subject: How would you name this dictionary?
Message-ID: <AM9P191MB15876A35121C06FF56E2D0F3B1762@AM9P191MB1587.EURP191.PROD.OUTLOOK.COM>

class NameMe(dict):
    def __missing__(self, key):
        return key


From cs at cskk.id.au  Sun Jan 21 19:09:45 2024
From: cs at cskk.id.au (Cameron Simpson)
Date: Mon, 22 Jan 2024 11:09:45 +1100
Subject: How would you name this dictionary?
In-Reply-To: <AM9P191MB15876A35121C06FF56E2D0F3B1762@AM9P191MB1587.EURP191.PROD.OUTLOOK.COM>
References: <AM9P191MB15876A35121C06FF56E2D0F3B1762@AM9P191MB1587.EURP191.PROD.OUTLOOK.COM>
Message-ID: <Za2ySXa9Dy4TFejy@cskk.homeip.net>

On 21Jan2024 23:39, bagratte at live.com <bagratte at live.com> wrote:
>class NameMe(dict):
>    def __missing__(self, key):
>        return key

I would need to know more about what it might be used for. What larger 
problem led you to writing a `dict` subclass with this particular 
`__missing__` implementation?

From list1 at tompassin.net  Sun Jan 21 22:57:02 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 21 Jan 2024 22:57:02 -0500
Subject: How to replace a cell value with each of its contour cells and
 yield the corresponding datasets seperately in a list according to a
 Pandas-way?
In-Reply-To: <CAGJtH9Qcqd_fBB-h7ED9QxnA0aen6HiavM7+pY00T349MzkT8A@mail.gmail.com>
References: <CAGJtH9RvLNeHFQSVk4b-TuYKKLgVbvAnoiQvC37vw5HGm2YjHg@mail.gmail.com>
 <a9aa6809-2842-4e7d-9024-fb2549c2eb65@tompassin.net>
 <CAGJtH9Svh5Q5B68aB+WK8JaYzV=EY-pfobi=cSoNjrTxBxv5_Q@mail.gmail.com>
 <86081560-08ff-42bc-ae6d-dfb03d8b2a7d@tompassin.net>
 <CAGJtH9Qcqd_fBB-h7ED9QxnA0aen6HiavM7+pY00T349MzkT8A@mail.gmail.com>
Message-ID: <055533d2-0600-4089-990a-6e25c172a45f@tompassin.net>

On 1/21/2024 1:25 PM, marc nicole wrote:
> It is part of a larger project aiming at processing data according to a 
> given algorithm
> Do you have any comments or any enhancing recommendations on the code?

I'm not knowledgeable enough about either pandas or numpy, I'm afraid, 
just very basic usage.  Someone else will probably pitch in.

> Thanks.
> 
> Le?dim. 21 janv. 2024 ??18:28, Thomas Passin via Python-list 
> <python-list at python.org <mailto:python-list at python.org>> a ?crit?:
> 
>     On 1/21/2024 11:54 AM, marc nicole wrote:
>      > Thanks for the reply,
>      >
>      > I think using a Pandas (or a Numpy) approach would optimize the
>      > execution of the program.
>      >
>      > Target cells could be up to 10% the size of the dataset, a good
>     example
>      > to start with would have from 10 to 100 values.
> 
>     Thanks for the reformatted code.? It's much easier to read and think
>     about.
> 
>     For say 100 points, it doesn't seem that "optimization" would be
>     much of
>     an issue.? On my laptop machine and Python 3.12, your example takes
>     around 5 seconds to run and print().? OTOH if you think you will go to
>     much larger datasets, certainly execution time could become a factor.
> 
>     I would think that NumPy arrays and/or matrices would have good
>     potential.
> 
>     Is this some kind of a cellular automaton, or an image filtering
>     process?
> 
>      > Let me know your thoughts, here's a reproducible example which I
>     formatted:
>      >
>      >
>      >
>      > from numpy import random
>      > import pandas as pd
>      > import numpy as np
>      > import operator
>      > import math
>      > from collections import deque
>      > from queue import *
>      > from queue import Queue
>      > from itertools import product
>      >
>      >
>      > def select_target_values(dataframe, number_of_target_values):
>      >? ? ? target_cells = []
>      >? ? ? for _ in range(number_of_target_values):
>      >? ? ? ? ? row_x = random.randint(0, len(dataframe.columns) - 1)
>      >? ? ? ? ? col_y = random.randint(0, len(dataframe) - 1)
>      >? ? ? ? ? target_cells.append((row_x, col_y))
>      >? ? ? return target_cells
>      >
>      >
>      > def select_contours(target_cells):
>      >? ? ? contour_coordinates = [(0, 1), (1, 0), (0, -1), (-1, 0)]
>      >? ? ? contour_cells = []
>      >? ? ? for target_cell in target_cells:
>      >? ? ? ? ? # random contour count for each cell
>      >? ? ? ? ? contour_cells_count = random.randint(1, 4)
>      >? ? ? ? ? try:
>      >? ? ? ? ? ? ? contour_cells.append(
>      >? ? ? ? ? ? ? ? ? [
>      >? ? ? ? ? ? ? ? ? ? ? tuple(
>      >? ? ? ? ? ? ? ? ? ? ? ? ? map(
>      >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? lambda i, j: i + j,
>      >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? (target_cell[0], target_cell[1]),
>      >? ? ? ? ? ? ? ? ? ? ? ? ? ? ? contour_coordinates[iteration_],
>      >? ? ? ? ? ? ? ? ? ? ? ? ? )
>      >? ? ? ? ? ? ? ? ? ? ? )
>      >? ? ? ? ? ? ? ? ? ? ? for iteration_ in range(contour_cells_count)
>      >? ? ? ? ? ? ? ? ? ]
>      >? ? ? ? ? ? ? )
>      >? ? ? ? ? except IndexError:
>      >? ? ? ? ? ? ? continue
>      >? ? ? return contour_cells
>      >
>      >
>      > def create_zipf_distribution():
>      >? ? ? zipf_dist = random.zipf(2, size=(50, 5)).reshape((50, 5))
>      >
>      >? ? ? zipf_distribution_dataset = pd.DataFrame(zipf_dist).round(3)
>      >
>      >? ? ? return zipf_distribution_dataset
>      >
>      >
>      > def apply_contours(target_cells, contour_cells):
>      >? ? ? target_cells_with_contour = []
>      >? ? ? # create one single list of cells
>      >? ? ? for idx, target_cell in enumerate(target_cells):
>      >? ? ? ? ? target_cell_with_contour = [target_cell]
>      >? ? ? ? ? target_cell_with_contour.extend(contour_cells[idx])
>      >? ? ? ? ? target_cells_with_contour.append(target_cell_with_contour)
>      >? ? ? return target_cells_with_contour
>      >
>      >
>      > def create_possible_datasets(dataframe, target_cells_with_contour):
>      >? ? ? all_datasets_final = []
>      >? ? ? dataframe_original = dataframe.copy()
>      >
>      >? ? ? list_tuples_idx_cells_all_datasets = list(
>      >? ? ? ? ? filter(
>      >? ? ? ? ? ? ? lambda x: x,
>      >? ? ? ? ? ? ? [list(tuples) for tuples in
>      > list(product(*target_cells_with_contour))],
>      >? ? ? ? ? )
>      >? ? ? )
>      >? ? ? target_original_cells_coordinates = list(
>      >? ? ? ? ? map(
>      >? ? ? ? ? ? ? lambda x: x[0],
>      >? ? ? ? ? ? ? [
>      >? ? ? ? ? ? ? ? ? target_and_contour_cell
>      >? ? ? ? ? ? ? ? ? for target_and_contour_cell in
>     target_cells_with_contour
>      >? ? ? ? ? ? ? ],
>      >? ? ? ? ? )
>      >? ? ? )
>      >? ? ? for dataset_index_values in list_tuples_idx_cells_all_datasets:
>      >? ? ? ? ? all_datasets = []
>      >? ? ? ? ? for idx_cell in range(len(dataset_index_values)):
>      >? ? ? ? ? ? ? dataframe_cpy = dataframe.copy()
>      >? ? ? ? ? ? ? dataframe_cpy.iat[
>      >? ? ? ? ? ? ? ? ? target_original_cells_coordinates[idx_cell][1],
>      >? ? ? ? ? ? ? ? ? target_original_cells_coordinates[idx_cell][0],
>      >? ? ? ? ? ? ? ] = dataframe_original.iloc[
>      >? ? ? ? ? ? ? ? ? dataset_index_values[idx_cell][1],
>      > dataset_index_values[idx_cell][0]
>      >? ? ? ? ? ? ? ]
>      >? ? ? ? ? ? ? all_datasets.append(dataframe_cpy)
>      >? ? ? ? ? all_datasets_final.append(all_datasets)
>      >? ? ? return all_datasets_final
>      >
>      >
>      > def main():
>      >? ? ? zipf_dataset = create_zipf_distribution()
>      >
>      >? ? ? target_cells = select_target_values(zipf_dataset, 5)
>      >? ? ? print(target_cells)
>      >? ? ? contour_cells = select_contours(target_cells)
>      >? ? ? print(contour_cells)
>      >? ? ? target_cells_with_contour = apply_contours(target_cells,
>     contour_cells)
>      >? ? ? datasets = create_possible_datasets(zipf_dataset,
>      > target_cells_with_contour)
>      >? ? ? print(datasets)
>      >
>      >
>      > main()
>      >
>      > Le?dim. 21 janv. 2024 ??16:33, Thomas Passin via Python-list
>      > <python-list at python.org <mailto:python-list at python.org>
>     <mailto:python-list at python.org <mailto:python-list at python.org>>> a
>     ?crit?:
>      >
>      >? ? ?On 1/21/2024 7:37 AM, marc nicole via Python-list wrote:
>      >? ? ? > Hello,
>      >? ? ? >
>      >? ? ? > I have an initial dataframe with a random list of target cells
>      >? ? ?(each cell
>      >? ? ? > being identified with a couple (x,y)).
>      >? ? ? > I want to yield four different dataframes each containing the
>      >? ? ?value of one
>      >? ? ? > of the contour (surrounding) cells of each specified
>     target cell.
>      >? ? ? >
>      >? ? ? > the surrounding cells to consider for a specific target
>     cell are
>      >? ? ?: (x-1,y),
>      >? ? ? > (x,y-1),(x+1,y);(x,y+1), specifically I randomly choose 1 to 4
>      >? ? ?cells from
>      >? ? ? > these and consider for replacement to the target cell.
>      >? ? ? >
>      >? ? ? > I want to do that through a pandas-specific approach without
>      >? ? ?having to
>      >? ? ? > define the contour cells separately and then apply the
>     changes on the
>      >? ? ? > dataframe
>      >
>      >? ? ?1. Why do you want a Pandas-specific approach?? Many people would
>      >? ? ?rather
>      >? ? ?keep code independent of special libraries if possible;
>      >
>      >? ? ?2. How big can these collections of target cells be, roughly
>     speaking?
>      >? ? ?The size could make a big difference in picking a design;
>      >
>      >? ? ?3. You really should work on formatting code for this list. 
>     Your code
>      >? ? ?below is very complex and would take a lot of work to
>     reformat to the
>      >? ? ?point where it is readable, especially with the nearly
>     impenetrable
>      >? ? ?arguments in some places.? Probably all that is needed is to
>     replace
>      >? ? ?all
>      >? ? ?tabs by (say) three spaces, and to make sure you
>     intentionally break
>      >? ? ?lines well before they might get word-wrapped.? Here is one
>     example I
>      >? ? ?have reformatted (I hope I got this right):
>      >
>      >? ? ?list_tuples_idx_cells_all_datasets = list(filter(
>      >? ? ? ? ? lambda x: utils_tuple_list_not_contain_nan(x),
>      >? ? ? ? ? [list(tuples) for tuples in list(
>      >? ? ? ? ? ? ? ? itertools.product(*target_cells_with_contour))
>      >? ? ? ? ? ]))
>      >
>      >? ? ?4. As an aside, it doesn't look like you need to convert all
>     those
>      >? ? ?sequences and iterators to lists all over the place;
>      >
>      >
>      >? ? ? > (but rather using an all in one approach):
>      >? ? ? > for now I have written this example which I think is not
>     Pandas
>      >? ? ?specific:
>      >? ? ?[snip]
>      >
>      >? ? ?--
>      > https://mail.python.org/mailman/listinfo/python-list
>     <https://mail.python.org/mailman/listinfo/python-list>
>      >? ? ?<https://mail.python.org/mailman/listinfo/python-list
>     <https://mail.python.org/mailman/listinfo/python-list>>
>      >
> 
>     -- 
>     https://mail.python.org/mailman/listinfo/python-list
>     <https://mail.python.org/mailman/listinfo/python-list>
> 


From nulla.epistola at web.de  Tue Jan 23 04:49:53 2024
From: nulla.epistola at web.de (Sibylle Koczian)
Date: Tue, 23 Jan 2024 10:49:53 +0100
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <825F8526-3074-4CD0-AED6-CCBE284840B8@barrys-emacs.org>
References: <d59f5407-0bfd-497e-86ed-66f751a4295e@web.de>
 <825F8526-3074-4CD0-AED6-CCBE284840B8@barrys-emacs.org>
Message-ID: <7d49686c-d17b-4993-a9b7-feed59cefa20@web.de>

Am 16.01.2024 um 23:44 schrieb Barry via Python-list:
> 
> 
>> On 16 Jan 2024, at 17:11, Sibylle Koczian via Python-list <python-list at python.org> wrote:
>>
>> while the new Windows 11 machine finds the Microsoft stub
> 
> You can turn off the stub in windows settings. The magic windows jargon is
> ?App Execution Aliases?. Once you find it in settings you can turn off
> the python and python3 aliases.
> 

Thank you! That did it. These two settings were off in the older Windows 
10 installation and on in the new Windows 11. Changed there and the 
shebang lines with "/env" work as expected.

I didn't remember ever seeing these settings on my "old" computer, but 
it seems I turned them off six years ago and completely forgot about 
them. Or could that have been an older default?

"python" or "python3" in the shebang line doesn't seem to make a 
difference, at least not on machines that never saw Python 2.

Very glad to know what was the matter here,
Sibylle


From nulla.epistola at web.de  Tue Jan 23 04:54:07 2024
From: nulla.epistola at web.de (Sibylle Koczian)
Date: Tue, 23 Jan 2024 10:54:07 +0100
Subject: Python 3.12.1, Windows 11: shebang line #!/usr/bin/env python3
 doesn't work any more
In-Reply-To: <12804cfa-e36f-4831-9408-0990a6cce221@wichmann.us>
References: <8dcf3464-e32f-457b-8938-0ff013ed4262@web.de>
 <4TCsVz4CzPznVGL@mail.python.org>
 <f63eac91-be2b-4e13-823f-1b99a43b709d@web.de>
 <b9235356-db47-49fd-b14f-8e9d9fa20149@wichmann.us>
 <fa3a05b8-1aee-4574-8e27-080b84f91de4@tompassin.net>
 <23c6ee06-5e7b-42b0-a1f4-d151f8a47a1c@wichmann.us>
 <d59f5407-0bfd-497e-86ed-66f751a4295e@web.de>
 <12804cfa-e36f-4831-9408-0990a6cce221@wichmann.us>
Message-ID: <f7e019bb-194f-4608-a5b9-0b4178dc682d@web.de>

Am 17.01.2024 um 19:44 schrieb Mats Wichmann via Python-list:
> 
> Be interesting to know if your WIndows 10 has those files in place, and 
> it's just a missing path entry (a good thing, perhaps) that's causing it 
> not to be found there.
> 
Yes. Python is not on the Path - by design.

>> while the new Windows 11 machine finds the Microsoft stub. 

Not any more - see my reply to Barry.

Thanks to all who answered!
Sibylle



From info at egenix.com  Wed Jan 24 04:18:51 2024
From: info at egenix.com (eGenix Team)
Date: Wed, 24 Jan 2024 10:18:51 +0100
Subject: ANN: eGenix Antispam Bot for Telegram 0.6.0
Message-ID: <4d498514-f876-4365-9be0-92667813f98a@egenix.com>

*ANNOUNCING*


    eGenix Antispam Bot for Telegram

Version 0.6.0

A simple, yet effective bot implementation
to address Telegram signup spam.

This announcement is also available on our web-site for online reading:
https://www.egenix.com/company/news/eGenix-Antispam-Bot-for-Telegram-0.6.0-GA.html

------------------------------------------------------------------------
*INTRODUCTION*

eGenix <https://egenix.com/> has long been running a local Python user 
group meeting in D?sseldorf called /Python Meeting D?sseldorf 
<https://pyddf.de/>/ and we are using a Telegram group for most of our 
communication.

In the early days, the group worked well and we only had few spammers 
joining it, which we could well handle manually.

More recently, this has changed dramatically. We are seeing between 2-5 
spam signups per day, often at night. Furthermore, the signups accounts 
are not always easy to spot as spammers, since they often come with 
profile images, descriptions, etc.

With the bot, we now have a more flexible way of dealing with the problem.

Please see our project page for details and download links:

https://www.egenix.com/library/telegram-antispam-bot/

------------------------------------------------------------------------
*FEATURES*

  * Low impact mode of operation: the bot tries to keep noise in the
    group to a minimum
  * Several challenge mechanisms to choose from, more can be added as needed
  * Flexible and easy to use configuration
  * Only needs a few MB of RAM, so can easily be put into a container or
    run on a Raspberry Pi
  * Can handle quite a bit of load due to the async implementation
  * Works with Python 3.9+
  * MIT open source licensed


------------------------------------------------------------------------
*NEWS*

The 0.6.0 release fixes a few bugs and adds more features:

  * Upgraded to pyrogram 2.0.106, which fixes a weird error we have been
    getting recently with the old version 1.4.16 (see
    pyrogram/pyrogram#1347
    <https://github.com/pyrogram/pyrogram/issues/1347>)
  * Catch weird error from Telegram when deleting conversations; this
    seems to sometimes fail, probably due to a glitch on their side
  * Made the math and char entry challenges a little harder
  * Added new DictItemChallenge

It has been battle-tested in production for?several years already 
already and is proving to be a really useful tool to help with Telegram 
group administration.


Enjoy,

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 24 2024)
>>> Python Projects, Coaching and Support ...    https://www.egenix.com/
>>> Python Product Development ...        https://consulting.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::
     
    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
            Registered at Amtsgericht Duesseldorf: HRB 46611
                https://www.egenix.com/company/contact/
                      https://www.malemburg.com/

From k at kerrickstaley.com  Wed Jan 24 12:11:25 2024
From: k at kerrickstaley.com (Kerrick Staley)
Date: Wed, 24 Jan 2024 12:11:25 -0500
Subject: Feature proposal: unittest.mock.NAN
Message-ID: <CANaWP3x_5uQrWWR1HfYMmbLChx=H_uhpUO6C5c0w2GzsOd8_jg@mail.gmail.com>

I think we should define a unittest.mock.NAN constant that can be used with
Mock.assert_called_with() to assert that an argument passed to a Mock was
NaN. NaNs are special in that math.nan != math.nan, so you can't just do
assert_called_with(math.nan). The naming is meant to parallel
unittest.mock.ANY.

Here is a reference implementation:

class _EqNaN:
    def __eq__(self, other):
        return math.isnan(other)

NAN = _EqNaN()

The alternative is that users can just define this EqNaN class themselves
as needed in test code. I encountered the need to test for a NaN argument
today and was surprised to find that (as far as I can tell) there is no
pre-built solution to this in unittest or pytest. It feels like it should
be included in some standard library.

- Kerrick

From barry at barrys-emacs.org  Wed Jan 24 16:08:12 2024
From: barry at barrys-emacs.org (Barry Scott)
Date: Wed, 24 Jan 2024 21:08:12 +0000
Subject: Feature proposal: unittest.mock.NAN
In-Reply-To: <CANaWP3x_5uQrWWR1HfYMmbLChx=H_uhpUO6C5c0w2GzsOd8_jg@mail.gmail.com>
References: <CANaWP3x_5uQrWWR1HfYMmbLChx=H_uhpUO6C5c0w2GzsOd8_jg@mail.gmail.com>
Message-ID: <552AECBB-CD25-4867-9770-7695362A2A26@barrys-emacs.org>

Python ideas are discussed here these days: https://discuss.python.org/
Suggest you raise this there in the Ideas category.

Barry


> On 24 Jan 2024, at 17:11, Kerrick Staley via Python-list <python-list at python.org> wrote:
> 
> I think we should define a unittest.mock.NAN constant that can be used with
> Mock.assert_called_with() to assert that an argument passed to a Mock was
> NaN. NaNs are special in that math.nan != math.nan, so you can't just do
> assert_called_with(math.nan). The naming is meant to parallel
> unittest.mock.ANY.
> 
> Here is a reference implementation:
> 
> class _EqNaN:
>    def __eq__(self, other):
>        return math.isnan(other)
> 
> NAN = _EqNaN()
> 
> The alternative is that users can just define this EqNaN class themselves
> as needed in test code. I encountered the need to test for a NaN argument
> today and was surprised to find that (as far as I can tell) there is no
> pre-built solution to this in unittest or pytest. It feels like it should
> be included in some standard library.
> 
> - Kerrick
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From greg.ewing at canterbury.ac.nz  Fri Jan 26 18:59:39 2024
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 27 Jan 2024 12:59:39 +1300
Subject: Await expressions (Posting On Python-List Prohibited)
In-Reply-To: <specification-20240126223958@ram.dialup.fu-berlin.de>
References: <await-20240126193156@ram.dialup.fu-berlin.de>
 <up16lv$313qt$1@dont-email.me>
 <specification-20240126223958@ram.dialup.fu-berlin.de>
Message-ID: <l1ivbdFovj6U1@mid.individual.net>

On 27/01/24 10:46 am, Stefan Ram wrote:
>    But your explanation seems to have no mention of the "something" /
>    "the awaitable object" part following the preposition "on". Shouldn't
>    this awaitable object play a r?le in the explanation of what happens?

If it helps at all, you can think of an async function as being
very similar to a generator, and "await" as being very similar to
"yield from". In the current implementation they're almost exactly
the same thing underneath.

-- 
Greg


From rosuav at gmail.com  Fri Jan 26 19:04:48 2024
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 27 Jan 2024 11:04:48 +1100
Subject: Await expressions (Posting On Python-List Prohibited)
In-Reply-To: <l1ivbdFovj6U1@mid.individual.net>
References: <await-20240126193156@ram.dialup.fu-berlin.de>
 <up16lv$313qt$1@dont-email.me>
 <specification-20240126223958@ram.dialup.fu-berlin.de>
 <l1ivbdFovj6U1@mid.individual.net>
Message-ID: <CAPTjJmqcgf166WmFKG4_QYFON4cFep-hAGdnAj87wSV4CkCXSQ@mail.gmail.com>

On Sat, 27 Jan 2024 at 11:01, Greg Ewing via Python-list
<python-list at python.org> wrote:
>
> If it helps at all, you can think of an async function as being
> very similar to a generator, and "await" as being very similar to
> "yield from". In the current implementation they're almost exactly
> the same thing underneath.
>

Don't bother responding to people who say that they don't want
responses on a public list.

ChrisA

From dieter.maurer at online.de  Sat Jan 27 12:33:29 2024
From: dieter.maurer at online.de (dieter.maurer at online.de)
Date: Sat, 27 Jan 2024 18:33:29 +0100
Subject: Await expressions (Posting On Python-List Prohibited)
In-Reply-To: <l1ivbdFovj6U1@mid.individual.net>
References: <await-20240126193156@ram.dialup.fu-berlin.de>
 <up16lv$313qt$1@dont-email.me>
 <specification-20240126223958@ram.dialup.fu-berlin.de>
 <l1ivbdFovj6U1@mid.individual.net>
Message-ID: <26037.15977.126265.855727@ixdm.fritz.box>

>On 27/01/24 10:46 am, Stefan Ram wrote:
>>    But your explanation seems to have no mention of the "something" /
>>    "the awaitable object" part following the preposition "on". Shouldn't
>>    this awaitable object play a r?le in the explanation of what happens?

You can explain a function call without saying much about the called function.
Similarly, you can explain "await <expr>" without saying much about
"<expr>".

Important is only: "<expr>" evaluates to an "awaitable".
An "awaitable" (usually an `asyncio.Future`, `asyncio.Task` or
call of an `async` function) eventuelly produces
a value and `await <expr>` waits until
this happens and then returns this value.

Not everything which eventually returns a value is an
"awaitable" -- e.g. a call of `time.sleep` is not an "awaitable".
Special provisions are necessary to be able to wait for a value
(and meanwhile do other things).
`asyncio.sleep` has e.g. this provisions and a call of it is an "awaitable".


From janburse at fastmail.fm  Fri Jan 26 15:08:33 2024
From: janburse at fastmail.fm (Mild Shock)
Date: Fri, 26 Jan 2024 21:08:33 +0100
Subject: Await expressions
In-Reply-To: <await-20240126193156@ram.dialup.fu-berlin.de>
References: <await-20240126193156@ram.dialup.fu-berlin.de>
Message-ID: <up13g0$v8cp$2@solani.org>


We say that an object is an awaitable object if it can be used in an 
await expression. Many asyncio APIs are designed to accept awaitables.

There are three main types of awaitable objects:
coroutines, Tasks, and Futures.

Stefan Ram schrieb:
>    In "The Python Language Reference, Release 3.13.0a0",
>    there is this section:
> 
> |6.4 Await expression
> |
> |Suspend the execution of coroutine on an awaitable object.
> |Can only be used inside a coroutine function.
> |
> |await_expr ::= "await" primary
> |
> |New in version 3.5.
> 
>    . And this is the whole section.
> 
>    What I do not understand:
> 
>    - Which coroutine is suspended?
>    - Which object is the object mentioned?
>    - For what purpose is the value of the primary expression used?
>    - What does it mean to "suspend something on something"?
> 


From janburse at fastmail.fm  Fri Jan 26 15:14:09 2024
From: janburse at fastmail.fm (Mild Shock)
Date: Fri, 26 Jan 2024 21:14:09 +0100
Subject: Await expressions
In-Reply-To: <up13g0$v8cp$2@solani.org>
References: <await-20240126193156@ram.dialup.fu-berlin.de>
 <up13g0$v8cp$2@solani.org>
Message-ID: <up13qh$v8m3$1@solani.org>

Maybe consult:

PEP 492 ? Coroutines with async and await syntax
Created: 09-Apr-2015
Python-Version: 3.5
https://peps.python.org/pep-0492/

Mild Shock schrieb:
> 
> We say that an object is an awaitable object if it can be used in an 
> await expression. Many asyncio APIs are designed to accept awaitables.
> 
> There are three main types of awaitable objects:
> coroutines, Tasks, and Futures.
> 
> Stefan Ram schrieb:
>> ?? In "The Python Language Reference, Release 3.13.0a0",
>> ?? there is this section:
>>
>> |6.4 Await expression
>> |
>> |Suspend the execution of coroutine on an awaitable object.
>> |Can only be used inside a coroutine function.
>> |
>> |await_expr ::= "await" primary
>> |
>> |New in version 3.5.
>>
>> ?? . And this is the whole section.
>>
>> ?? What I do not understand:
>>
>> ?? - Which coroutine is suspended?
>> ?? - Which object is the object mentioned?
>> ?? - For what purpose is the value of the primary expression used?
>> ?? - What does it mean to "suspend something on something"?
>>
> 


From mk1853387 at gmail.com  Sun Jan 28 13:16:10 2024
From: mk1853387 at gmail.com (marc nicole)
Date: Sun, 28 Jan 2024 19:16:10 +0100
Subject: How to create a binary tree hierarchy given a list of elements as its
 leaves
Message-ID: <CAGJtH9TMrH7CJpe1pUyrVnYV65k9u08r7byUn0A9zy0huN5Gng@mail.gmail.com>

So I am trying to build a binary tree hierarchy given numerical elements
serving for its leaves (last level of the tree to build). From the leaves I
want to randomly create a name for the higher level of the hierarchy and
assign it to the children elements. For example: if the elements inputted
are `0,1,2,3` then I would like to create firstly 4 elements (say by random
giving them a label composed of a letter and a number) then for the second
level (iteration) I assign each of 0,1 to a random name label (e.g. `b1`)
and `2,3` to another label (`b2`) then for the third level I assign a
parent label to each of `b1` and `b2` as `c1`.

An illustation of the example is the following tree:


[image: tree_exp.PNG]

For this I use numpy's `array_split()` to get the chunks of arrays based on
the iteration needs.
for example to get the first iteration arrays I use `np.array_split(input,
(input.size // k))` where `k` is an even number. In order to assign a
parent node to the children the array range should enclose the children's.
For example to assign the parent node with label `a1` to children `b1` and
`b2` with range respectively [0,1] and [2,3], the parent should have the
range [0,3].

All is fine until a certain iteration (k=4) returns parent with range [0,8]
which is overlapping to children ranges and therefore cannot be their
parent.

My question is how to evenly partition such arrays in a binary way and
create such binary tree so that to obtain for k=4 the first range to be
[0,7] instead of [0,8]?

My code is the following:

#!/usr/bin/python
# -*- coding: utf-8 -*-
import string
import random
import numpy as np


def generate_numbers_list_until_number(stop_number):
    if str(stop_number).isnumeric():
        return np.arange(stop_number)
    else:
        raise TypeError('Input should be a number!')


def generate_node_label():
    return random.choice(string.ascii_lowercase) \
        + str(random.randint(0, 10))


def main():
    data = generate_numbers_list_until_number(100)
    k = 1
    hierarchies = []
    cells_arrays = np.array_split(data, data.size // k)
    print cells_arrays
    used_node_hierarchy_name = []
    node_hierarchy_name = [generate_node_label() for _ in range(0,
                           len(cells_arrays))]
    used_node_hierarchy_name.extend(node_hierarchy_name)
    while len(node_hierarchy_name) > 1:
        k = k * 2

# bug here in the following line

        cells_arrays = list(map(lambda x: [x[0], x[-1]],
                            np.array_split(data, data.size // k)))
        print cells_arrays
        node_hierarchy_name = []

# node hierarchy names should not be redundant in another level

        for _ in range(0, len(cells_arrays)):
            node_name = generate_node_label()
            while node_name in used_node_hierarchy_name:
                node_name = generate_node_label()
            node_hierarchy_name.append(node_name)
            used_node_hierarchy_name.extend(node_hierarchy_name)
        print used_node_hierarchy_name
        hierarchies.append(list(zip(node_hierarchy_name, cells_arrays)))

From python at mrabarnett.plus.com  Sun Jan 28 14:23:59 2024
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 28 Jan 2024 19:23:59 +0000
Subject: How to create a binary tree hierarchy given a list of elements as
 its leaves
In-Reply-To: <CAGJtH9TMrH7CJpe1pUyrVnYV65k9u08r7byUn0A9zy0huN5Gng@mail.gmail.com>
References: <CAGJtH9TMrH7CJpe1pUyrVnYV65k9u08r7byUn0A9zy0huN5Gng@mail.gmail.com>
Message-ID: <99f5cf21-8f22-4900-889f-b2bbbe8356f5@mrabarnett.plus.com>

On 2024-01-28 18:16, marc nicole via Python-list wrote:
> So I am trying to build a binary tree hierarchy given numerical elements
> serving for its leaves (last level of the tree to build). From the leaves I
> want to randomly create a name for the higher level of the hierarchy and
> assign it to the children elements. For example: if the elements inputted
> are `0,1,2,3` then I would like to create firstly 4 elements (say by random
> giving them a label composed of a letter and a number) then for the second
> level (iteration) I assign each of 0,1 to a random name label (e.g. `b1`)
> and `2,3` to another label (`b2`) then for the third level I assign a
> parent label to each of `b1` and `b2` as `c1`.
> 
> An illustation of the example is the following tree:
> 
> 
> [image: tree_exp.PNG]
> 
This list strips images, and discussion has mostly moved to 
https://discuss.python.org.


From syedhamoodahmed234 at gmail.com  Mon Jan 29 07:19:39 2024
From: syedhamoodahmed234 at gmail.com (Syed Hamood)
Date: Mon, 29 Jan 2024 17:19:39 +0500
Subject: Assistance Needed: Corrupted Python Installation Uninstallation Issue
Message-ID: <CAB00WQ9m-b9L6cinVLMEoQehkwKHX6jkcK=SrhUCCzxe60R9vw@mail.gmail.com>

Dear Python.org Support Team,

I hope this email finds you well. I am writing to seek assistance with an
issue I'm encountering while attempting to uninstall a corrupted Python
installation on my system.

Details of my system:

   - Operating System: Windows 10
   - Python Version: 3.11.3(64-bit)
   - Installation Method: installer from Python.org

Description of the issue: [Provide a brief description of the problem
you're facing, any error messages received, or specific steps you've taken
so far.]

I have already tried the following:

   - Deleting python. removing corrupted files from command prompt with
   administrative privileges.

However, despite my efforts, I have been unable to successfully uninstall
the corrupted Python installation.

I would greatly appreciate any guidance or assistance you could provide to
help me resolve this matter and successfully uninstall Python from my
system. If there are specific steps or commands I should follow, or if
there are additional tools or resources I should utilize, please let me
know.

Thank you very much for your time and support. I look forward to hearing
from you soon.

Best regards, Syed Hamood Email: syedhamoodahmed234 at gmail.com

From mats at wichmann.us  Mon Jan 29 11:07:49 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Mon, 29 Jan 2024 09:07:49 -0700
Subject: Assistance Needed: Corrupted Python Installation Uninstallation
 Issue
In-Reply-To: <CAB00WQ9m-b9L6cinVLMEoQehkwKHX6jkcK=SrhUCCzxe60R9vw@mail.gmail.com>
References: <CAB00WQ9m-b9L6cinVLMEoQehkwKHX6jkcK=SrhUCCzxe60R9vw@mail.gmail.com>
Message-ID: <868d0fb6-d1ee-455d-bee9-6595d3ce168a@wichmann.us>

On 1/29/24 05:19, Syed Hamood via Python-list wrote:
> Dear Python.org Support Team,
> 
> I hope this email finds you well. I am writing to seek assistance with an
> issue I'm encountering while attempting to uninstall a corrupted Python
> installation on my system.
> 
> Details of my system:
> 
>     - Operating System: Windows 10
>     - Python Version: 3.11.3(64-bit)
>     - Installation Method: installer from Python.org
> 
> Description of the issue: [Provide a brief description of the problem
> you're facing, any error messages received, or specific steps you've taken
> so far.]
> 
> I have already tried the following:
> 
>     - Deleting python. removing corrupted files from command prompt with
>     administrative privileges.
> 
> However, despite my efforts, I have been unable to successfully uninstall
> the corrupted Python installation.

The more stuff you remove by hand the harder it is for the Windows 
installer to act to do an uninstall.

This tool usually helps if things are badly messed up:

https://support.microsoft.com/en-us/topic/fix-problems-that-block-programs-from-being-installed-or-removed-cca7d1b6-65a9-3d98-426b-e9f927e1eb4d

Haven't used it for a while, but after it tries basic overall repairs to 
the installation subsystem (which is probably okay), there are prompts 
you can follow to point to a specific program that doesn't want to 
uninstall.



From rshepard at appl-ecosys.com  Mon Jan 29 11:15:44 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Mon, 29 Jan 2024 08:15:44 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
Message-ID: <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>

On Fri, 12 Jan 2024, Rich Shepard via Python-list wrote:

> For my use 1) the salutation and email address (always with an '@') are
> sequential and 2) I'm developing the script to extract both from the same
> file.

I've looked at my Python books "Python Crash Course," "Effective Python,"
and "Python Tricks The Book" as well as web pages in my searches without
finding the answer to what may be a simple question: how to specify a
variable in one file that has its values in another file.

Specifically, how to I designate the salutation holder in the message file
and pass it the name value from the name/email address file?

If this explanation is not sufficiently clear I'll re-write it. :-)

TIA,

Rich

From rshepard at appl-ecosys.com  Mon Jan 29 12:27:08 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Mon, 29 Jan 2024 09:27:08 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <26039.56939.70073.816806@ixdm.fritz.box>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
Message-ID: <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>

On Mon, 29 Jan 2024, dieter.maurer at online.de wrote:

> Have you read "https://docs.python.org/3/library/io.html#module-io"?

Dieter,

No, I hadn't ... but I am reading it now.

Many thanks,

Rich

From dieter.maurer at online.de  Mon Jan 29 12:20:43 2024
From: dieter.maurer at online.de (dieter.maurer at online.de)
Date: Mon, 29 Jan 2024 18:20:43 +0100
Subject: Extract lines from file, add to new files
In-Reply-To: <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
Message-ID: <26039.56939.70073.816806@ixdm.fritz.box>

Rich Shepard wrote at 2024-1-29 08:15 -0800:
> ...
>If this explanation is not sufficiently clear I'll re-write it. :-)

Have you read "https://docs.python.org/3/library/io.html#module-io"?


From rshepard at appl-ecosys.com  Mon Jan 29 12:37:55 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Mon, 29 Jan 2024 09:37:55 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
 <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
Message-ID: <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>

On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:

> No, I hadn't ... but I am reading it now.

Perhaps I missed the answer to my question when reading the io module. It
explains how to open/write/read files of text and binary data, not passing
a variable's value from one file to a place-keeper in another file.

I'll keep searching for a solution.

Rich

From rshepard at appl-ecosys.com  Mon Jan 29 12:55:57 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Mon, 29 Jan 2024 09:55:57 -0800 (PST)
Subject: Extract lines from file, add to new files [RESOLVED]
In-Reply-To: <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
 <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
 <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
Message-ID: <40a3d1e-61ec-6292-e24-92a1231c3c5a@appl-ecosys.com>

On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:

> I'll keep searching for a solution.

IIRC, someone here pointed me to <https://realpython.com/python-send-email/>
and I forgot about it ... until now.

Regards,

Rich

From avi.e.gross at gmail.com  Mon Jan 29 12:57:37 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 29 Jan 2024 12:57:37 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
 <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
 <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
Message-ID: <006301da52dc$a52f8a90$ef8e9fb0$@gmail.com>

Rich,

You got an overly general reply to a question many of us may not understand.

You have not hinted at how the two files are organized, perhaps with an
example.

There are several general solutions that may apply. Some involve reading in
both files into data structures and perhaps linking them together in some
way such as a data.frame or binary tree. You can then process individual
request in memory/

The second should be straightforward as long as text is text. If the first
file tells you to search for XYZ then you search the second file for XYZ and
read in whatever is associated with it and do your thing.

Without a bit more specific detail, you may not get more than a suggestion
as to how to read in files.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Rich Shepard via Python-list
Sent: Monday, January 29, 2024 12:38 PM
To: python-list at python.org
Subject: RE: Extract lines from file, add to new files

On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:

> No, I hadn't ... but I am reading it now.

Perhaps I missed the answer to my question when reading the io module. It
explains how to open/write/read files of text and binary data, not passing
a variable's value from one file to a place-keeper in another file.

I'll keep searching for a solution.

Rich
-- 
https://mail.python.org/mailman/listinfo/python-list


From rshepard at appl-ecosys.com  Mon Jan 29 13:05:25 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Mon, 29 Jan 2024 10:05:25 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <006301da52dc$a52f8a90$ef8e9fb0$@gmail.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
 <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
 <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
 <006301da52dc$a52f8a90$ef8e9fb0$@gmail.com>
Message-ID: <5d771ad6-da46-f060-8e47-402c44326161@appl-ecosys.com>

On Mon, 29 Jan 2024, avi.e.gross at gmail.com wrote:

> There are several general solutions that may apply. Some involve reading
> in both files into data structures and perhaps linking them together in
> some way such as a data.frame or binary tree. You can then process
> individual request in memory/

Avi,

I found several web pages describing how to use the python email library and
tools to send individual or multiple email messages. I'll learn how to do
this based on a couple of detailed examples.

Thanks,

Rich

From PythonList at DancesWithMice.info  Mon Jan 29 14:10:48 2024
From: PythonList at DancesWithMice.info (dn)
Date: Tue, 30 Jan 2024 08:10:48 +1300
Subject: Extract lines from file, add to new files
In-Reply-To: <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
Message-ID: <324c36f6-d44b-4023-90d2-698ace9cdfc8@DancesWithMice.info>

On 30/01/24 05:15, Rich Shepard via Python-list wrote:
> On Fri, 12 Jan 2024, Rich Shepard via Python-list wrote:
> 
>> For my use 1) the salutation and email address (always with an '@') are
>> sequential and 2) I'm developing the script to extract both from the same
>> file.
> 
> I've looked at my Python books "Python Crash Course," "Effective Python,"
> and "Python Tricks The Book" as well as web pages in my searches without
> finding the answer to what may be a simple question: how to specify a
> variable in one file that has its values in another file.
> 
> Specifically, how to I designate the salutation holder in the message file
> and pass it the name value from the name/email address file?
> 
> If this explanation is not sufficiently clear I'll re-write it. :-)


It seems clear - but maybe we (all) misunderstand?

The books don't cover it - but is there a reason why they don't?


(Thunderbird did not thread this message with earlier ones, and it seems 
some time has passed/memory is fading - but hopefully have remembered 
content)


As said previously, the idea of two physical-files containing 
logically-related data (without any form of cross-reference between) is 
bad design.

In the current file, there is one physical structure and each person's 
details are logically-related by proximity. A long way from ideal, but 
workable (as you've described).

However, once split into two files, there is no way to guarantee that 
the two logically-related data-items (name and address) will continue to 
be related by their physical position in the respective files. Worse: 
whilst it would seem apparent that "Alice" from the names file might be 
related to the address "alice at domain.tld", how could one know if "Bob" 
actually corresponds to "list-member at domain.tld"?

This is why dicts, databases, etc, offer keys (as labels for 
data-items/dependent components)!


After a quick look at Eric's Crash Course, yes, his files-intro example 
(digits of pi) is unlikely to have any practical reality (and I work 
with statisticians and quants!). However, at the end of that chapter 
(10), there is mention of JSON files. A JSON version of the existing 
single-file structure will provide human-readable labeling of 
data-items, give better separation between individuals' data, and show 
how name and address are linked. Recommend solving 'the problem' that 
way! (as previously discussed by others 'here', IIRC)

Slatkin's Effective Python doesn't seem to discuss the basics of files 
at all (presumes such knowledge of the reader). It dives into important, 
but rather technical discussions, comparing strings and bytes - somewhat 
beyond the complexity-level of this discussion. That book does however 
include discussions such as "Prefer Multiple Assignment Unpacking over 
Indexing" (Item 6 - also points to Item 19) where relative-positioning 
(indexing in other words) is advised-against.


If you wish to persist with this two-file structure, please see earlier 
responses (again, IIRC) and discussion of file-merge operations. As 
these date back to mainframes and the days of storing files on mag-tape, 
I'd be surprised to see them discussed in 'modern' texts. However, the 
principle is: read a record from each file, do-the-business, read the 
next 'pair' of physically-related records, rinse-and-repeat.

If you require further assistance: how about showing a couple of 
relevant lines of the data-file(s) and the pertinent parts of the code, 
along with a description of what's going-wrong or not making sense to you?

-- 
Regards,
=dn

From jon+usenet at unequivocal.eu  Mon Jan 29 12:51:13 2024
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Mon, 29 Jan 2024 17:51:13 -0000 (UTC)
Subject: Extract lines from file, add to new files
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
 <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
 <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
 <mailman.9.1706549879.3227.python-list@python.org>
Message-ID: <slrnurfpch.62k.jon+usenet@raven.unequivocal.eu>

On 2024-01-29, Rich Shepard <rshepard at appl-ecosys.com> wrote:
> On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:
>> No, I hadn't ... but I am reading it now.
>
> Perhaps I missed the answer to my question when reading the io module. It
> explains how to open/write/read files of text and binary data, not passing
> a variable's value from one file to a place-keeper in another file.

Why would it contain a recipe for performing some specific task that
appears unique to you?

If you want any meaningful help you'll need to explain more precisely
what it is that you're trying to achieve and why.

From grant.b.edwards at gmail.com  Mon Jan 29 17:54:07 2024
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 29 Jan 2024 17:54:07 -0500 (EST)
Subject: Extract lines from file, add to new files
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
 <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
 <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
Message-ID: <4TP3Wg4yldznVG5@mail.python.org>

On 2024-01-29, Rich Shepard via Python-list <python-list at python.org> wrote:
> On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:
>
>> No, I hadn't ... but I am reading it now.
>
> Perhaps I missed the answer to my question when reading the io module. It
> explains how to open/write/read files of text and binary data, not passing
> a variable's value from one file to a place-keeper in another file.

It's not at all clear (to me) what you're asking about.  When you talk
about "files" are you referring to data files? Python modules within a
single program? Seperate Python programs?  Something else?

The phrase "place-keeper in another file" sounds a bit like you're
trying to do templating. There are many, many ways to do templating in
Python -- ranging from literal 'f-strings' to powerful templating
engines that are used to construct entire web sites:

  https://www.google.com/search?q=python+templating

  https://docs.python.org/3/tutorial/inputoutput.html#tut-f-strings

  https://en.wikipedia.org/wiki/Jinja_(template_engine)


From list1 at tompassin.net  Mon Jan 29 19:47:15 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 29 Jan 2024 19:47:15 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
Message-ID: <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>

On 1/29/2024 11:15 AM, Rich Shepard via Python-list wrote:
>> For my use 1) the salutation and email address (always with an '@') are
>> sequential and 2) I'm developing the script to extract both from the same
>> file.
> 
> I've looked at my Python books "Python Crash Course," "Effective Python,"
> and "Python Tricks The Book" as well as web pages in my searches without
> finding the answer to what may be a simple question: how to specify a
> variable in one file that has its values in another file.
> 
> Specifically, how to I designate the salutation holder in the message file
> and pass it the name value from the name/email address file?
> 
> If this explanation is not sufficiently clear I'll re-write it. :-)
> 
> TIA,
> 
> Rich

I'm assuming this is a continuation of a previous thread about working 
with alternate lines with salutation and address, and I assume you've 
got that worked out.

If you aren't going to use one or another existing template system, 
perhaps the easiest is to use unique strings in the message file.  For 
example:

Dear __##so-and-so##__:
    Please don't write this message off as mere spam.
    Respectfully, Rich

Then you just do a replace of the unique string by the salutation. Don't 
change the original (i.e., template), make the changes to a copy that 
you will output.


From avi.e.gross at gmail.com  Tue Jan 30 01:12:51 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 30 Jan 2024 01:12:51 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <4TP3Wg4yldznVG5@mail.python.org>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
 <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
 <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
 <4TP3Wg4yldznVG5@mail.python.org>
Message-ID: <013c01da5343$5b5c97c0$1215c740$@gmail.com>

It can be quite frustrating figuring out what someone wants, Grant,
especially when they just change it.

It is worse when instead of starting a new thread with an appropriate
subject line, it continues and old one that was also frustrating to
understand.

It sounds though like another attempt to do something perhaps a different
way. Both attempts seem to be to use some form of storage of a set of email
addresses plus other info like a name that can be used to make a customized
email.

Frankly, this should have been fairly easy to do without so much back and
forth. I don't care how the email is actually sent, but the rest could have
been done any number of ways such as storing the data as rows in a CSV file
or saved using JSON format and so on. It was never made clear why two files
were needed and then somehow linked and searched. 

If the goal is to be able to search for something like a name and THEN find
an email address, that seems quite trivial if they are I the same file in
some format. If the number of items is small, reading it all in should not
be a big deal and you can use a regular expression or other method to locate
the entry you want and extract the additional info. If you have lots of
data, reading line after line may be less useful than just using a database
and a query.

One way to stop feeling frustrated is to stop reading the thread.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Grant Edwards via Python-list
Sent: Monday, January 29, 2024 5:54 PM
To: python-list at python.org
Subject: Re: Extract lines from file, add to new files

On 2024-01-29, Rich Shepard via Python-list <python-list at python.org> wrote:
> On Mon, 29 Jan 2024, Rich Shepard via Python-list wrote:
>
>> No, I hadn't ... but I am reading it now.
>
> Perhaps I missed the answer to my question when reading the io module. It
> explains how to open/write/read files of text and binary data, not passing
> a variable's value from one file to a place-keeper in another file.

It's not at all clear (to me) what you're asking about.  When you talk
about "files" are you referring to data files? Python modules within a
single program? Seperate Python programs?  Something else?

The phrase "place-keeper in another file" sounds a bit like you're
trying to do templating. There are many, many ways to do templating in
Python -- ranging from literal 'f-strings' to powerful templating
engines that are used to construct entire web sites:

  https://www.google.com/search?q=python+templating

  https://docs.python.org/3/tutorial/inputoutput.html#tut-f-strings

  https://en.wikipedia.org/wiki/Jinja_(template_engine)

-- 
https://mail.python.org/mailman/listinfo/python-list


From rshepard at appl-ecosys.com  Tue Jan 30 08:37:35 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Tue, 30 Jan 2024 05:37:35 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
Message-ID: <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>

On Mon, 29 Jan 2024, Thomas Passin via Python-list wrote:

> If you aren't going to use one or another existing template system,
> perhaps the easiest is to use unique strings in the message file. For
> example:
>
> Dear __##so-and-so##__:
>   Please don't write this message off as mere spam.
>   Respectfully, Rich
>
> Then you just do a replace of the unique string by the salutation. Don't
> change the original (i.e., template), make the changes to a copy that you
> will output.

My script is not a web application, but an emailer that allows me to contact
clients and prospective clients. From the command line on a linux host.
Using the python smtplib and mail modules.

Rich

From larry.martell at gmail.com  Tue Jan 30 09:15:53 2024
From: larry.martell at gmail.com (Larry Martell)
Date: Tue, 30 Jan 2024 09:15:53 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <013c01da5343$5b5c97c0$1215c740$@gmail.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <26039.56939.70073.816806@ixdm.fritz.box>
 <e19178ea-dca9-2669-5ef2-27c58d7c69e1@appl-ecosys.com>
 <9f227581-b3ee-6b8f-9791-f8f6841fbb11@appl-ecosys.com>
 <4TP3Wg4yldznVG5@mail.python.org> <013c01da5343$5b5c97c0$1215c740$@gmail.com>
Message-ID: <CACwCsY43r4yy5zM6v-KgLVhU2=WTW3yF=75uVAb5oXupvUoVVA@mail.gmail.com>

On Tue, Jan 30, 2024 at 1:13?AM AVI GROSS via Python-list
<python-list at python.org> wrote:
>
> It can be quite frustrating figuring out what someone wants, Grant,
> especially when they just change it.
>
> It is worse when instead of starting a new thread with an appropriate
> subject line, it continues and old one that was also frustrating to
> understand.

Is it worse than top posting?

From jon+usenet at unequivocal.eu  Tue Jan 30 09:03:39 2024
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Tue, 30 Jan 2024 14:03:39 -0000 (UTC)
Subject: Extract lines from file, add to new files
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <mailman.17.1706621875.3227.python-list@python.org>
Message-ID: <slrnuri0dr.62k.jon+usenet@raven.unequivocal.eu>

On 2024-01-30, Rich Shepard <rshepard at appl-ecosys.com> wrote:
> On Mon, 29 Jan 2024, Thomas Passin via Python-list wrote:
>
>> If you aren't going to use one or another existing template system,
>> perhaps the easiest is to use unique strings in the message file. For
>> example:
>>
>> Dear __##so-and-so##__:
>>   Please don't write this message off as mere spam.
>>   Respectfully, Rich
>>
>> Then you just do a replace of the unique string by the salutation. Don't
>> change the original (i.e., template), make the changes to a copy that you
>> will output.
>
> My script is not a web application, but an emailer that allows me to contact
> clients and prospective clients. From the command line on a linux host.
> Using the python smtplib and mail modules.

lol good luck with that.

From list1 at tompassin.net  Tue Jan 30 09:07:52 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 30 Jan 2024 09:07:52 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
Message-ID: <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>

On 1/30/2024 8:37 AM, Rich Shepard via Python-list wrote:
> On Mon, 29 Jan 2024, Thomas Passin via Python-list wrote:
> 
>> If you aren't going to use one or another existing template system,
>> perhaps the easiest is to use unique strings in the message file. For
>> example:
>>
>> Dear __##so-and-so##__:
>> ? Please don't write this message off as mere spam.
>> ? Respectfully, Rich
>>
>> Then you just do a replace of the unique string by the salutation. Don't
>> change the original (i.e., template), make the changes to a copy that you
>> will output.
> 
> My script is not a web application, but an emailer that allows me to 
> contact
> clients and prospective clients. From the command line on a linux host.
> Using the python smtplib and mail modules.
> 
> Rich

Fine, my toy example will still be applicable.  But, you know, you 
haven't told us enough to give you help.  Do you want to replace text 
from values in a file?  That's been covered. Do you want to send the 
messages using those libraries?  You haven't said what you don't know 
how to do.  Something else? What is it that you want to do that you 
don't know how?


From avi.e.gross at gmail.com  Tue Jan 30 11:35:43 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 30 Jan 2024 11:35:43 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <slrnuri0dr.62k.jon+usenet@raven.unequivocal.eu>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <mailman.17.1706621875.3227.python-list@python.org>
 <slrnuri0dr.62k.jon+usenet@raven.unequivocal.eu>
Message-ID: <002601da539a$5ebd1550$1c373ff0$@gmail.com>

I deleted the contents of the message so I can avoid both of  the deadly
sins of top posting and bottom posting and chance committing  the sin of
replying without any context.

Of course, I am only replying to Jon wishing a real or feigned good luck to
the OP.

But seriously, the OP, AKA Rich, is making clear that he is making a tool
for his own use. It sounds like he wants to maintain a data repository of
his own with some info about his clients and then have the ability to
specify a name and pop up an email directed to them, or something along
those lines.

Without further info, this sounds like what quite a few popular and even
free mailers already do to some extent as you have an associated address
book. Some start doing possible matches as you type. I hazard Rich is not
using something this simple as he does seem to have some form-letter merge
or similar functionality in mind, such as automating the seemingly mandatory
"Dear XXX" salutation.

But if he is the creator and maintainer of his client data and chooses not
to use one of many available applications, then it seems he already has
reasons he wants a particular design for the data and simply wants us to
help him use it the way he wants. That could be communicated a bit more
clearly but after many messages back and forth, I have not exactly
understood it.

In my opinion, having created all kinds of mailers over the years, sometimes
the hard way, This strikes me as not really being about what mailing
functionality exists at all. As a general rule, you first create supporting
parts for your mail then pass them along to functionality that assembles the
mail as a set of headers and a body and dispatches it.

You as the programmer need to supply a body, tell it who to put on TO/CC/BB
lines, perhaps provide a subject, perhaps specify attachments, and off you
go. The exact methods will differ.

But what Rich presumably needs to do is have his program interact with him
in specifying who he wants to mail to and then looking it up in whatever
file arrangement it contains. If he also wants to use other parts such as a
human name or address inside the body of the text, his program needs to
merge the text he supplies along with extracted parts of the data.

Or is that not what he wants? The above could be quite straightforward and I
recall doing things like this with a simple shell script in UNIX.
Specifically, I created a text file where I recorded info for each person in
some format like

NAME|PHONE|EMAIL|COMMENT

Then to search for someone, you could use something like grep to find a name
by anchoring to the beginning or ending with the "|" so it does not match
the text in another field such as email or address, and use other utilities
ranging from cut to awk and getting the parts you want into variables and
then interpolate them into a message template and so on.

Of course, doing it in python is a good way to go too and should not be hard
once it is decided how to store the data. But again, this is re-inventing
things that others have already done. 

The python modules include many ways to store modular data. The books I have
read mention them all the time. Pick one. And, yes, you can choose to
maintain two files if that design works for you. Consider some storage
method that stores data in sections like:

[NAME one]
First: whatever
Email: whatever

[NAME two]
First: ...
Email: ...

There are specific formats along these lines and you can get python modules
that you ask for "NAME one" and it reads the file until it finds a section
as "[NAME one]" or not. If found, it returns the variables/values right
below it.  So your two step algorithm may consist of two files with one file
containing just names, perhaps to use a grep functionality on. Some of those
names will have a matching section like the above somewhere in the other
file. 

So if you want to send mail to "Jo" then your program may search for all
names starting with "Jo" and offer you "John Smith"  and perhaps also
"Joachim Martillo". The program takes whichever full name(s) you then select
and calls a function that uses that full name to search the second file to
find an exact match and returns what it finds there such as an email
address.

But unless you have lots of contacts, as already discussed, there are far
easier ways to do things in a more brute force way. Take a one-line per
entry format such as a CSV or TSV and extract whatever column contains the
name as needed to do the first search. Yes, this tends to mean reading the
entire file.

And, for the record, I am not a fan of hiding replies at the bottom except
for short messages. I prefer to use some combination of in-line if
addressing many points in the original and mainly the top with perhaps a
preface explaining what is being addressed. The reader is usually capable of
digging below if they want to know more. But this is a more religious war
having nothing to do with python specifically.

My frustration is that I often want to help someone and wish the problem was
stated in a way that made that doable. I do sympathize with Rich as figuring
out which details to focus on or to omit is not trivial. I may well be
wrong, but it sounds like his request could be focused on how to use
supporting files for an email application, or if stated clearly, as he
tried, about how to store data in two files and be able to extract what you
need as if it was all in one file. 

But what may not be easy to get through is the experience of years from when
you often stored data in multiple arrays and had to be careful to update
them all at once, to more modern methods using structs or classes to hold
more or all aspects of a related object as a unit. In this context, some may
simply read in all the data from a single file into something like a list of
objects (or a numpy/pandas variant) and have search and manipulation
abilities to find and change the right ones. The only diddling with files
that is needed is once to read them into the structure (such as JSON) and
perhaps once to write it back out with changes.

Good luck, once you decide on a particular method and flesh that out.





From rshepard at appl-ecosys.com  Tue Jan 30 12:21:51 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Tue, 30 Jan 2024 09:21:51 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
Message-ID: <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>

On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:

> Fine, my toy example will still be applicable. But, you know, you haven't
> told us enough to give you help. Do you want to replace text from values
> in a file? That's been covered. Do you want to send the messages using
> those libraries? You haven't said what you don't know how to do. Something
> else? What is it that you want to do that you don't know how?

Thomas,

For 30 years I've used a bash script using mailx to send messages to a list
of recipients. They have no salutation to personalize each one. Since I want
to add that personalized salutation I decided to write a python script to
replace the bash script.

I have collected 11 docs explaining the smtplib and email modules and
providing example scripts to apply them to send multiple individual messages
with salutations and attachments.

Today I'm going to be reading these. They each recommend using .csv input
files for names and addresses. My first search is learning whether I can
write a single .csv file such as:
"name1","address1"
"mane2","address2"
which I believe will work; and by inserting at the top of the message block
Hi, {yourname}
the name in the .csv file will replace the bracketed place holder.

Still much to learn and the batch of downloaded PDF files should educate me.

Regards,

Rich

From rshepard at appl-ecosys.com  Tue Jan 30 12:25:48 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Tue, 30 Jan 2024 09:25:48 -0800 (PST)
Subject: Extract lines from file, add to new files
In-Reply-To: <002601da539a$5ebd1550$1c373ff0$@gmail.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <mailman.17.1706621875.3227.python-list@python.org>
 <slrnuri0dr.62k.jon+usenet@raven.unequivocal.eu>
 <002601da539a$5ebd1550$1c373ff0$@gmail.com>
Message-ID: <3e61d01f-fbd-1778-f25e-62d1ce2c3b2@appl-ecosys.com>

On Tue, 30 Jan 2024, AVI GROSS via Python-list wrote:

> But seriously, the OP, AKA Rich, is making clear that he is making a tool
> for his own use. It sounds like he wants to maintain a data repository of
> his own with some info about his clients and then have the ability to
> specify a name and pop up an email directed to them, or something along
> those lines.

Close, Avi.

I have no issues sending messages to single individuals or mailing lists. I
want to send the same message to several individuals at one time, which I've
done -- without individual salutations -- for 30 years using a bash script
and mailx.

As I replied to Thomas on the list, I've downloaded 11 PDF docs from the Web
(and a useful book on the Python3 standard library) and will start reading
and learning from them today. I expect to find answers to my few remaining
questions in these docs.

Regards,

Rich

From Karsten.Hilbert at gmx.net  Tue Jan 30 12:34:01 2024
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Tue, 30 Jan 2024 18:34:01 +0100
Subject: Aw: Re: Extract lines from file, add to new files
In-Reply-To: <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
Message-ID: <trinity-484e78fe-5eb9-41ef-a262-f6fc63796acf-1706636041715@3c-app-gmx-bap17>

> For 30 years I've used a bash script using mailx to send messages to a list
> of recipients. They have no salutation to personalize each one. Since I want
> to add that personalized salutation I decided to write a python script to
> replace the bash script.

Why not foxus on just the part you think you are better off using python, namely
personalization ?

Create personalized files and send them with your trusted mailx solution ?

That'll take out wrestling with smptlib et al.

Karsten


From rshepard at appl-ecosys.com  Tue Jan 30 12:53:24 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Tue, 30 Jan 2024 09:53:24 -0800 (PST)
Subject: Aw: Re: Extract lines from file, add to new files
In-Reply-To: <trinity-484e78fe-5eb9-41ef-a262-f6fc63796acf-1706636041715@3c-app-gmx-bap17>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
 <trinity-484e78fe-5eb9-41ef-a262-f6fc63796acf-1706636041715@3c-app-gmx-bap17>
Message-ID: <75d91fb4-30db-c3a7-a78f-34eb4e20363a@appl-ecosys.com>

On Tue, 30 Jan 2024, Karsten Hilbert wrote:

> Why not foxus on just the part you think you are better off using python,
> namely personalization ?
>
> Create personalized files and send them with your trusted mailx solution ?

Karsten,

Too much time. And while mailx accepts the '-a' option for attachments but
has none for individual salutations.

Regards,

Rich

From Karsten.Hilbert at gmx.net  Tue Jan 30 13:01:23 2024
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Tue, 30 Jan 2024 19:01:23 +0100
Subject: Aw: Re:  Re: Extract lines from file, add to new files
In-Reply-To: <75d91fb4-30db-c3a7-a78f-34eb4e20363a@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
 <trinity-484e78fe-5eb9-41ef-a262-f6fc63796acf-1706636041715@3c-app-gmx-bap17>
 <75d91fb4-30db-c3a7-a78f-34eb4e20363a@appl-ecosys.com>
Message-ID: <trinity-fc704946-1c79-4066-aaeb-738b47dd73bb-1706637683462@3c-app-gmx-bap17>


> > Why not foxus on just the part you think you are better off using python,
> > namely personalization ?
> >
> > Create personalized files and send them with your trusted mailx solution ?
>
> Karsten,
>
> Too much time. And while mailx accepts the '-a' option for attachments but
> has none for individual salutations.

It doesn't need to. It just sends the (pre-personalized-by-Python) mail files.

Karsten

From rshepard at appl-ecosys.com  Tue Jan 30 13:02:34 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Tue, 30 Jan 2024 10:02:34 -0800 (PST)
Subject: Aw: Re:  Re: Extract lines from file, add to new files
In-Reply-To: <trinity-fc704946-1c79-4066-aaeb-738b47dd73bb-1706637683462@3c-app-gmx-bap17>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
 <trinity-484e78fe-5eb9-41ef-a262-f6fc63796acf-1706636041715@3c-app-gmx-bap17>
 <75d91fb4-30db-c3a7-a78f-34eb4e20363a@appl-ecosys.com>
 <trinity-fc704946-1c79-4066-aaeb-738b47dd73bb-1706637683462@3c-app-gmx-bap17>
Message-ID: <9544ff9-1412-47c8-ba49-26d61a9d4d2@appl-ecosys.com>

On Tue, 30 Jan 2024, Karsten Hilbert wrote:

> It doesn't need to. It just sends the (pre-personalized-by-Python) mail files.

Karsten,

In which case, I might as well have Python format and send the messages. :-)

Regards,

Rich

From Karsten.Hilbert at gmx.net  Tue Jan 30 13:42:29 2024
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Tue, 30 Jan 2024 19:42:29 +0100
Subject: Aw: Re:  Re:  Re: Extract lines from file, add to new files
In-Reply-To: <9544ff9-1412-47c8-ba49-26d61a9d4d2@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
 <trinity-484e78fe-5eb9-41ef-a262-f6fc63796acf-1706636041715@3c-app-gmx-bap17>
 <75d91fb4-30db-c3a7-a78f-34eb4e20363a@appl-ecosys.com>
 <trinity-fc704946-1c79-4066-aaeb-738b47dd73bb-1706637683462@3c-app-gmx-bap17>
 <9544ff9-1412-47c8-ba49-26d61a9d4d2@appl-ecosys.com>
Message-ID: <trinity-a7db4cfd-ec4d-489e-951a-b60f8b5731ce-1706640149213@3c-app-gmx-bap17>

> On Tue, 30 Jan 2024, Karsten Hilbert wrote:
>
> > It doesn't need to. It just sends the (pre-personalized-by-Python) mail files.
>
> Karsten,
>
> In which case, I might as well have Python format and send the messages. :-)

Certainly. But it seems you are wrestling with Python. Might as well reduce the attack surface.

Karsten

From avi.e.gross at gmail.com  Tue Jan 30 16:46:13 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 30 Jan 2024 16:46:13 -0500
Subject: Aw: Re: Extract lines from file, add to new files
In-Reply-To: <75d91fb4-30db-c3a7-a78f-34eb4e20363a@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
 <trinity-484e78fe-5eb9-41ef-a262-f6fc63796acf-1706636041715@3c-app-gmx-bap17>
 <75d91fb4-30db-c3a7-a78f-34eb4e20363a@appl-ecosys.com>
Message-ID: <001801da53c5$bf21b150$3d6513f0$@gmail.com>

Rich,

You may want to broaden your perspective a bit when people make suggestions.

Karsten did not spell out a full design and should not need to.

But consider this as a scenario.

You want to send (almost) the same message to one or more recipients.

So call a program, perhaps some variant on a shell script, that does some
prep work such as maybe creating a temporary or working directory/folder.
Had one copy of your message ready in a file somewhere, Have a way to get a
list of recipients intended and the file or files containing enough info to
link email addresses to human names and anything else such as their
preferred pronoun  or address.

Now have the script call your super-duper python program with enough info so
it can find the folder to put amended COPIES of your letter into as well as.
Perhaps the email address intended in the filename or whatever works for
you. 

Your program will then simply identify each email recipient you want and
look up the other info and prepend the customized salutation, or make
substitutions in the template and write out a new file in the designated
folder with perhaps the email address as the filename.

When your loop ends, exit the python program with success, or perhaps report
some failure.

The shell script now resumes by checking the exit status and if OK,
continuing to enter the folder and loop on all file contests and invoke the
functionality to send each copied/enhanced file to the intended recipient.
If you also need to support attachments, you can figure out how to attach
the same ones to each as I assume those are not changed for each recipient. 

It may keep track of how many worked or failed and eventually clear out the
files and perhaps the folder and you are done.

This is NOT a required way to do it but for what sounds like a limited
personal project, it should work well enough and have you do the limited
amount of work you need in Python.

Having said that, you can likely also easily do everything without python
and I have written some huge shell scripts in my time to do way more complex
things. But learning how to do things like this well in python can be time
well spent as long as you don't tackle too much at a time and get
overwhelmed.




-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Rich Shepard via Python-list
Sent: Tuesday, January 30, 2024 12:53 PM
To: python-list at python.org
Subject: Re: Aw: Re: Extract lines from file, add to new files

On Tue, 30 Jan 2024, Karsten Hilbert wrote:

> Why not foxus on just the part you think you are better off using python,
> namely personalization ?
>
> Create personalized files and send them with your trusted mailx solution ?

Karsten,

Too much time. And while mailx accepts the '-a' option for attachments but
has none for individual salutations.

Regards,

Rich
-- 
https://mail.python.org/mailman/listinfo/python-list


From mats at wichmann.us  Tue Jan 30 17:18:20 2024
From: mats at wichmann.us (Mats Wichmann)
Date: Tue, 30 Jan 2024 15:18:20 -0700
Subject: Aw: Re: Extract lines from file, add to new files
In-Reply-To: <001801da53c5$bf21b150$3d6513f0$@gmail.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
 <trinity-484e78fe-5eb9-41ef-a262-f6fc63796acf-1706636041715@3c-app-gmx-bap17>
 <75d91fb4-30db-c3a7-a78f-34eb4e20363a@appl-ecosys.com>
 <001801da53c5$bf21b150$3d6513f0$@gmail.com>
Message-ID: <59409de4-b146-4434-90b0-2331f070c2b8@wichmann.us>

On 1/30/24 14:46, AVI GROSS via Python-list wrote:
> Rich,
> 
> You may want to broaden your perspective a bit when people make suggestions.
> 
> Karsten did not spell out a full design and should not need to.
> 
> But consider this as a scenario.
> 
> You want to send (almost) the same message to one or more recipients.
> 
> So call a program, perhaps some variant on a shell script, that does some
> prep work such as maybe creating a temporary or working directory/folder.
> Had one copy of your message ready in a file somewhere, Have a way to get a
> list of recipients intended and the file or files containing enough info to
> link email addresses to human names and anything else such as their
> preferred pronoun  or address.

I'd say based on the bits of the problem description I *have* absorbed, 
which almost certainly isn't all of them, there's a fairly basic 
capability, not terribly often used in my experience, that might be of 
some use:

https://docs.python.org/3/library/string.html#template-strings



From list1 at tompassin.net  Tue Jan 30 22:36:49 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 30 Jan 2024 22:36:49 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
Message-ID: <e0c1c61d-2941-452a-8977-ef5c5bfb8bdb@tompassin.net>

On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
> 
>> Fine, my toy example will still be applicable. But, you know, you haven't
>> told us enough to give you help. Do you want to replace text from values
>> in a file? That's been covered. Do you want to send the messages using
>> those libraries? You haven't said what you don't know how to do. 
>> Something
>> else? What is it that you want to do that you don't know how?
> 
> Thomas,
> 
> For 30 years I've used a bash script using mailx to send messages to a list
> of recipients. They have no salutation to personalize each one. Since I 
> want
> to add that personalized salutation I decided to write a python script to
> replace the bash script.
> 
> I have collected 11 docs explaining the smtplib and email modules and
> providing example scripts to apply them to send multiple individual 
> messages
> with salutations and attachments.

If I had a script that's been working for 30 years, I'd probably just 
use Python to do the personalizing and let the rest of the bash script 
do the rest, like it always has.  The Python program would pipe or send 
the personalized messages to the rest of the bash program. Something in 
that ballpark, anyway.

> Today I'm going to be reading these. They each recommend using .csv input
> files for names and addresses. My first search is learning whether I can
> write a single .csv file such as:
> "name1","address1"
> "mane2","address2"
> which I believe will work; and by inserting at the top of the message block
> Hi, {yourname}
> the name in the .csv file will replace the bracketed place holder
If the file contents are going to be people's names and email addresses, 
I would just tab separate them and split each line on the tab.  Names 
aren't going to include tabs so that would be safe.  Email addresses 
might theoretically include a tab inside a quoted name but that would be 
extremely obscure and unlikely.  No need for CSV, it would just add 
complexity.

data = f.readlines()
for d in data:
     name, addr = line.split('\t') if line.strip() else ('', '')

> Still much to learn and the batch of downloaded PDF files should educate 
> me.
> 
> Regards,
> 
> Rich


From avi.e.gross at gmail.com  Tue Jan 30 23:25:50 2024
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 30 Jan 2024 23:25:50 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <e0c1c61d-2941-452a-8977-ef5c5bfb8bdb@tompassin.net>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
 <e0c1c61d-2941-452a-8977-ef5c5bfb8bdb@tompassin.net>
Message-ID: <004c01da53fd$9273b9e0$b75b2da0$@gmail.com>

Thomas, on some points we may see it differently.

Some formats can be done simply but are maybe better done in somewhat
standard ways.

Some of what the OP has is already tables in a database and that can
trivially be exported into a CSV file or other formats like your TSV file
and more. They can also import from there. As I mentioned, many spreadsheets
and all kinds of statistical programs tend to support some formats making it
quite flexible.

Python has all kinds of functionality, such as in the pandas module, to read
in a CSV or write it out. And once you have the data structure in memory, al
kinds of queries and changes can be made fairly straightforwardly. As one
example, Rich has mentioned wanting finer control in selecting who gets some
version of the email based on concepts like market segmentation. He already
may have info like the STATE (as in Arizona) in his database. He might at
some point enlarge his schema so each entry is placed in one or more
categories and thus his CSV, once imported, can do the usual tasks of
selecting various rows and columns or doing joins or whatever.

Mind you, another architecture could place quite a bit of work completely on
the back end and he could send SQL queries to the database from python and
get back his results into python which would then make the email messages
and pass them on to other functionality to deliver. This would remove any
need for files and just rely on the DB.

There as as usual, too many choices and not necessarily one best answer. Of
course if this was a major product that would be heavily used, sure, you
could tweak and optimize. As it is, Rich is getting a chance to improve his
python skills no matter which way he goes.



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Thomas Passin via Python-list
Sent: Tuesday, January 30, 2024 10:37 PM
To: python-list at python.org
Subject: Re: Extract lines from file, add to new files

On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
> 
>> Fine, my toy example will still be applicable. But, you know, you haven't
>> told us enough to give you help. Do you want to replace text from values
>> in a file? That's been covered. Do you want to send the messages using
>> those libraries? You haven't said what you don't know how to do. 
>> Something
>> else? What is it that you want to do that you don't know how?
> 
> Thomas,
> 
> For 30 years I've used a bash script using mailx to send messages to a
list
> of recipients. They have no salutation to personalize each one. Since I 
> want
> to add that personalized salutation I decided to write a python script to
> replace the bash script.
> 
> I have collected 11 docs explaining the smtplib and email modules and
> providing example scripts to apply them to send multiple individual 
> messages
> with salutations and attachments.

If I had a script that's been working for 30 years, I'd probably just 
use Python to do the personalizing and let the rest of the bash script 
do the rest, like it always has.  The Python program would pipe or send 
the personalized messages to the rest of the bash program. Something in 
that ballpark, anyway.

> Today I'm going to be reading these. They each recommend using .csv input
> files for names and addresses. My first search is learning whether I can
> write a single .csv file such as:
> "name1","address1"
> "mane2","address2"
> which I believe will work; and by inserting at the top of the message
block
> Hi, {yourname}
> the name in the .csv file will replace the bracketed place holder
If the file contents are going to be people's names and email addresses, 
I would just tab separate them and split each line on the tab.  Names 
aren't going to include tabs so that would be safe.  Email addresses 
might theoretically include a tab inside a quoted name but that would be 
extremely obscure and unlikely.  No need for CSV, it would just add 
complexity.

data = f.readlines()
for d in data:
     name, addr = line.split('\t') if line.strip() else ('', '')

> Still much to learn and the batch of downloaded PDF files should educate 
> me.
> 
> Regards,
> 
> Rich

-- 
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Wed Jan 31 07:25:12 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 31 Jan 2024 07:25:12 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <004c01da53fd$9273b9e0$b75b2da0$@gmail.com>
References: <c9cf741b-9781-8dd6-96df-d270d0ac2@appl-ecosys.com>
 <2a5eef2d-3d66-8cea-64eb-602ea5aff946@appl-ecosys.com>
 <f1784a5b-662a-47fc-ada0-cc09ea2cfc24@tompassin.net>
 <9028bd96-dbc8-fbc1-8584-a965ef7def5d@appl-ecosys.com>
 <5cea1ab2-1848-41bd-8e5b-323fe55ba8c9@tompassin.net>
 <478e3bbe-db95-9533-595b-7a19a5e2cf@appl-ecosys.com>
 <e0c1c61d-2941-452a-8977-ef5c5bfb8bdb@tompassin.net>
 <004c01da53fd$9273b9e0$b75b2da0$@gmail.com>
Message-ID: <17769063-0f0d-49ef-9599-88157690cefd@tompassin.net>

On 1/30/2024 11:25 PM, avi.e.gross at gmail.com wrote:
> Thomas, on some points we may see it differently.

I'm mostly going by what the OP originally asked for back on Jan 11. 
He's been too stingy with information since then to be worth spending 
much time on, IMHO.

> Some formats can be done simply but are maybe better done in somewhat
> standard ways.
> 
> Some of what the OP has is already tables in a database and that can
> trivially be exported into a CSV file or other formats like your TSV file
> and more. They can also import from there. As I mentioned, many spreadsheets
> and all kinds of statistical programs tend to support some formats making it
> quite flexible.
> 
> Python has all kinds of functionality, such as in the pandas module, to read
> in a CSV or write it out. And once you have the data structure in memory, al
> kinds of queries and changes can be made fairly straightforwardly. As one
> example, Rich has mentioned wanting finer control in selecting who gets some
> version of the email based on concepts like market segmentation. He already
> may have info like the STATE (as in Arizona) in his database. He might at
> some point enlarge his schema so each entry is placed in one or more
> categories and thus his CSV, once imported, can do the usual tasks of
> selecting various rows and columns or doing joins or whatever.
> 
> Mind you, another architecture could place quite a bit of work completely on
> the back end and he could send SQL queries to the database from python and
> get back his results into python which would then make the email messages
> and pass them on to other functionality to deliver. This would remove any
> need for files and just rely on the DB.
> 
> There as as usual, too many choices and not necessarily one best answer. Of
> course if this was a major product that would be heavily used, sure, you
> could tweak and optimize. As it is, Rich is getting a chance to improve his
> python skills no matter which way he goes.
> 
> 
> 
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
> Behalf Of Thomas Passin via Python-list
> Sent: Tuesday, January 30, 2024 10:37 PM
> To: python-list at python.org
> Subject: Re: Extract lines from file, add to new files
> 
> On 1/30/2024 12:21 PM, Rich Shepard via Python-list wrote:
>> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
>>
>>> Fine, my toy example will still be applicable. But, you know, you haven't
>>> told us enough to give you help. Do you want to replace text from values
>>> in a file? That's been covered. Do you want to send the messages using
>>> those libraries? You haven't said what you don't know how to do.
>>> Something
>>> else? What is it that you want to do that you don't know how?
>>
>> Thomas,
>>
>> For 30 years I've used a bash script using mailx to send messages to a
> list
>> of recipients. They have no salutation to personalize each one. Since I
>> want
>> to add that personalized salutation I decided to write a python script to
>> replace the bash script.
>>
>> I have collected 11 docs explaining the smtplib and email modules and
>> providing example scripts to apply them to send multiple individual
>> messages
>> with salutations and attachments.
> 
> If I had a script that's been working for 30 years, I'd probably just
> use Python to do the personalizing and let the rest of the bash script
> do the rest, like it always has.  The Python program would pipe or send
> the personalized messages to the rest of the bash program. Something in
> that ballpark, anyway.
> 
>> Today I'm going to be reading these. They each recommend using .csv input
>> files for names and addresses. My first search is learning whether I can
>> write a single .csv file such as:
>> "name1","address1"
>> "mane2","address2"
>> which I believe will work; and by inserting at the top of the message
> block
>> Hi, {yourname}
>> the name in the .csv file will replace the bracketed place holder
> If the file contents are going to be people's names and email addresses,
> I would just tab separate them and split each line on the tab.  Names
> aren't going to include tabs so that would be safe.  Email addresses
> might theoretically include a tab inside a quoted name but that would be
> extremely obscure and unlikely.  No need for CSV, it would just add
> complexity.
> 
> data = f.readlines()
> for d in data:
>       name, addr = line.split('\t') if line.strip() else ('', '')
> 
>> Still much to learn and the batch of downloaded PDF files should educate
>> me.
>>
>> Regards,
>>
>> Rich
> 


From rshepard at appl-ecosys.com  Wed Jan 31 09:05:33 2024
From: rshepard at appl-ecosys.com (Rich Shepard)
Date: Wed, 31 Jan 2024 06:05:33 -0800 (PST)
Subject: Extract lines from file, add to new files
Message-ID: <3acb2c7a-54a9-191d-b721-8f7180f32810@appl-ecosys.com>

On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:

> If I had a script that's been working for 30 years, I'd probably just use
> Python to do the personalizing and let the rest of the bash script do the
> rest, like it always has. The Python program would pipe or send the
> personalized messages to the rest of the bash program. Something in that
> ballpark, anyway.

Thomas,

A bash shell script looks easier for me and more promising. Using a while
loop (one for the name file the other for the address file), and sed for
putting the name at the head of the message replacing a generic placeholder
should work with the existing for loop script.

Thanks,

Rich

From list1 at tompassin.net  Wed Jan 31 10:36:24 2024
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 31 Jan 2024 10:36:24 -0500
Subject: Extract lines from file, add to new files
In-Reply-To: <3acb2c7a-54a9-191d-b721-8f7180f32810@appl-ecosys.com>
References: <3acb2c7a-54a9-191d-b721-8f7180f32810@appl-ecosys.com>
Message-ID: <146ad2c4-b243-47ed-87d5-c3f48fa2d3af@tompassin.net>

On 1/31/2024 9:05 AM, Rich Shepard via Python-list wrote:
> On Tue, 30 Jan 2024, Thomas Passin via Python-list wrote:
> 
>> If I had a script that's been working for 30 years, I'd probably just use
>> Python to do the personalizing and let the rest of the bash script do the
>> rest, like it always has. The Python program would pipe or send the
>> personalized messages to the rest of the bash program. Something in that
>> ballpark, anyway.
> 
> Thomas,
> 
> A bash shell script looks easier for me and more promising. Using a while
> loop (one for the name file the other for the address file), and sed for
> putting the name at the head of the message replacing a generic placeholder
> should work with the existing for loop script.

Sounds good.  I'd still be a bit worried about the two files getting out 
of sync, as others have mentioned.


From meejah at meejah.ca  Wed Jan 31 05:09:05 2024
From: meejah at meejah.ca (meejah)
Date: Wed, 31 Jan 2024 03:09:05 -0700
Subject: magic-folder 24.1.0
Message-ID: <5e0f4386-1b99-43c7-8bfe-0d52eb57ccb0@app.fastmail.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Greetings,

We are pleased to announce version 23.6.0 of magic-folder.

Magic Folder synchronizes local data to and from a Tahoe-LAFS Grid,
keeping data private with the end-to-end encrypted "Capabilities" of
Tahoe-LAFS. One or more magic-folder clients can join the same Folder,
adding and synchronizing data across multiple devices. Written in
Python, Magic Folder supports Linux, MacOS and Windows. Python3 and
PyPy are supported.

By itself, this project requires familiarity with the command-line and
long-running processes. The Gridsync project provides a cross-
platform GUI experience using the localhost HTTP API.

Changes in this release include:

- - security: Bump dependencies, including security-relevant cryptography library (#716)

- - feature: Added a description of the datamodel to the documentation (#702)
- - feature: Conflict files now named after the Participant (not Author) (#711)

- - bugfix: "magic-folder status" properly parses scan/poll events (#717)
- - bugfix: Handle updates to conflicted files more robustly (#719)

You may download the release from PyPI:

   https://pypi.org/project/magic-folder/23.6.0/#files

Currently the localhost HTTP API used for integration is not 100%
stable, although we do not expect large changes.

thanks,

meejah
on behalf of all contributors


-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEnVor1WiOy4id680/wmAoAxKAaacFAmSZNRIACgkQwmAoAxKA
aaeMBggAhijgTKRlYSBbIO+Y/mms4drY2ArEptF+Q5secHo9of68xlL1tQoTGgeF
et/xxgrjmn7WmnQYrC6k0f+xH+6mlW6sNb6wCafUmP/oh3p5bjAGv2tJDmfs81e+
iW7IUWBklx2mLdquwU4nWouJKZjeidqOyBvaBxx30hqOQe3yMwL5mxcliZg9ueZ6
M2HnQZmhA0yKblZrQ8H26dGTSvDLSOOCuHJKab04XYrmOJSJRqYK0DFXogjpNZbB
Ep5wiMpxyEsfPc1bHpxdIJmgJZq1iIgx6dFnHiMNe7yQLykLc9kjPJcxvWRLVsk4
OzN5lHzsiYKsS0g3NVSj+2PKxY3d3w==
=abqD
-----END PGP SIGNATURE-----

From PythonList at DancesWithMice.info  Wed Jan 31 15:34:34 2024
From: PythonList at DancesWithMice.info (dn)
Date: Thu, 1 Feb 2024 09:34:34 +1300
Subject: MTG: pytest (NZPUG, Auckland, VacExcHndlrs)
Message-ID: <232586bd-f3dd-4e56-ab07-e500cd8d3b46@DancesWithMice.info>

Wed 7 Feb (evening NZDT) will be the last virtual gathering in the 
current Vacation Exception Handlers (VacExcHndlrs) series 
(https://danceswithmice.info/Python/2024/VacExcHndlrs.html).

You are cordially-invited to join us to investigate the pytest Python 
testing framework.

"The pytest framework makes it easy to write small, readable tests, and 
can scale to support complex functional testing for applications and 
libraries."


Pre-requisites:

1 Head-set to ask questions and make contributions
2 Lab-/Log-/Note-book
3 Python 3.8+ installed
4 Ability to write intro-level Python program[me]s (at least)
5 Credentials to install from PyPi ("The Cheese Shop")
6 Preparedness to research, experiment, work with others...
As before, we will follow a caf? style of meeting. So, come prepared 
with a list of objectives you would like to achieve and a list of 
achievements you will be ready to share.


Possible (but far too many) topics:

- installing pytest (individual, system-wide, PyCharm, VSCodium, ...)
- TDD's (Test-Driven Development) red-green refactoring
- the Python assert-statement
- the project directory-tree and auto-discovery
- adding simple-tests
- running pytest
- reporting
- structuring the code-base to facilitate testing
- advantages/disadvantages of automated testing
- using a GAI to suggest/generate tests
- fixtures and dependencies (set-up and/or tear-down)
- scopes (function, class, module, session)
- parametrizing
- monkey-patching
- CI/CD chaining you've built
- plug-ins you're finding helpful
- coverage
- testing strategies
- other testing frameworks and aids
(open-ended - what would you like to add?)


Come to participate, learn-from, and help others!
Please RSVP at https://www.meetup.com/nzpug-auckland/events/298901851/


-- 
Regards,
=dn