From uri at speedy.net  Wed Feb  1 11:31:29 2023
From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=)
Date: Wed, 1 Feb 2023 18:31:29 +0200
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
Message-ID: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>

Hi,

I have a server with Ubuntu 22.04.1 LTS and the Python version there
is Python 3.10.6. Is there a safe way to upgrade to the latest version of
Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the Python
version remained the same.

Thanks,
????
uri at speedy.net

From grant.b.edwards at gmail.com  Wed Feb  1 11:42:57 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Wed, 01 Feb 2023 08:42:57 -0800 (PST)
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
Message-ID: <63da9691.020a0220.7a118.d42e@mx.google.com>

On 2023-02-01, ???? <uri at speedy.net> wrote:

> I have a server with Ubuntu 22.04.1 LTS and the Python version there
> is Python 3.10.6. Is there a safe way to upgrade to the latest version of
> Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the Python
> version remained the same.

This is an Ubuntu question, not a Python question.

But, the answer is that you generally shouldn't mess with the Python
version installed by apt on Ubuntu. There are too many other packages
that use it.

It may be safest to install mutliple versions:

https://ubuntuhandbook.org/index.php/2022/10/python-3-11-released-how-install-ubuntu/

You can also use Anaconda to do something similar:

https://askubuntu.com/questions/1413421/how-to-install-older-version-of-python-in-ubuntu-22-04






From skip.montanaro at gmail.com  Wed Feb  1 12:11:05 2023
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Wed, 1 Feb 2023 11:11:05 -0600
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
Message-ID: <CANc-5UzEdwqTiqLMpU+T3p8J6gn2YvWoPA9k_RWEr5fO6E71cg@mail.gmail.com>

>
>
> I have a server with Ubuntu 22.04.1 LTS and the Python version there
> is Python 3.10.6. Is there a safe way to upgrade to the latest version of
> Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the Python
> version remained the same.
>
> I have an Ubuntu 22.04 system. The installed Python 3.10 version is 3.10.7:

$ apt search python3 | egrep '^python3.10.*installed'


WARNING: apt does not have a stable CLI interface. Use with caution in
scripts.


python3.10/now 3.10.7-1+focal1 amd64 [installed,local]
python3.10-distutils/now 3.10.7-1+focal1 all [installed,local]
python3.10-doc/jammy-updates,jammy-security,now 3.10.6-1~22.04.2 all
[installed]
python3.10-gdbm/now 3.10.7-1+focal1 amd64 [installed,local]
python3.10-lib2to3/now 3.10.7-1+focal1 all [installed,local]
python3.10-minimal/now 3.10.7-1+focal1 amd64 [installed,local]
python3.10-venv/now 3.10.7-1+focal1 amd64 [installed,local]


Off the top of my head, I can't recall if it's LTS or not. If you want to
go beyond 3.10.6, it should be possible. As Grant indicated though,
upgrading packages on an Ubuntu system (of any flavor) is the province of
the Ubuntu community.

Skip

From rosuav at gmail.com  Wed Feb  1 12:14:16 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 2 Feb 2023 04:14:16 +1100
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
Message-ID: <CAPTjJmpCyTXpVOLdGataJbN2VM=iCNBP54y_75vk6ss-nD37rw@mail.gmail.com>

?On Thu, 2 Feb 2023 at 03:33, ??????? <uri at speedy.net> wrote:?
>
> Hi,
>
> I have a server with Ubuntu 22.04.1 LTS and the Python version there
> is Python 3.10.6. Is there a safe way to upgrade to the latest version of
> Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the Python
> version remained the same.

First, ask: Do you need to? Often patches get backported, but the
version number isn't changed. Find out if there's something fixed in
3.10.9 that still isn't fixed in the version Ubuntu is shipping.

ChrisA

From Muttley at dastardlyhq.com  Wed Feb  1 03:56:40 2023
From: Muttley at dastardlyhq.com (Muttley at dastardlyhq.com)
Date: Wed, 1 Feb 2023 08:56:40 -0000 (UTC)
Subject: evaluation question
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <trbvi6$3vgh9$1@dont-email.me>
Message-ID: <trd9g8$92t2$1@dont-email.me>

On Tue, 31 Jan 2023 21:00:53 +0000
Mark Bourne <nntp.mbourne at spamgourmet.com> wrote:
>Greg Ewing wrote:
>> On 30/01/23 10:41 pm, Muttley at dastardlyhq.com wrote:
>>> What was the point of the upheaval of converting
>>> the print command in python 2 into a function in python 3 if as a 
>>> function
>>> print() doesn't return anything useful?
>> 
>> It was made a function because there's no good reason for it
>> to have special syntax in the language.
>
>I think I saw somewhere that making print a function also had something 
>to do with being able to add extra keyword arguments like sep and end. 
>The syntax for printing to a specific file already seemed a bit odd with 
>the print statement, and adding extra arguments would have made it even 
>more clunky (yeah, I know ">>" is similar to C++ streams, but it looks 
>out of place in Python).

Why couldn't they just keep "print" and call the function , oh I dunno,
"printf" ? :)



From Muttley at dastardlyhq.com  Wed Feb  1 03:59:17 2023
From: Muttley at dastardlyhq.com (Muttley at dastardlyhq.com)
Date: Wed, 1 Feb 2023 08:59:17 -0000 (UTC)
Subject: evaluation question
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
Message-ID: <trd9l4$93l4$1@dont-email.me>

On Wed, 1 Feb 2023 11:59:25 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>On 31/01/23 10:24 pm, Muttley at dastardlyhq.com wrote:
>> All languages have their ugly corners due to initial design mistakes and/or
>> constraints. Eg: java with the special behaviour of its string class, C++
>> with "=0" pure virtual declaration. But they don't dump them and make all old
>
>> code suddenly cease to execute.
>
>No, but it was decided that Python 3 would have to be backwards
>incompatible, mainly to sort out the Unicode mess. Given that,
>the opportunity was taken to clean up some other mistakes as well.

Unicode is just a string of bytes. C supports it with a few extra library
functions to get unicode length vs byte length and similar. Its really
not that hard. Rewriting an entire language just to support that sounds a
bit absurd to me but hey ho...



From Muttley at dastardlyhq.com  Wed Feb  1 04:00:39 2023
From: Muttley at dastardlyhq.com (Muttley at dastardlyhq.com)
Date: Wed, 1 Feb 2023 09:00:39 -0000 (UTC)
Subject: evaluation question
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
Message-ID: <trd9nn$94gv$1@dont-email.me>

On Wed, 1 Feb 2023 13:17:33 +1300
dn <PythonList at DancesWithMice.info> wrote:
>On 01/02/2023 11.59, Greg Ewing wrote:
>> On 31/01/23 10:24 pm, Muttley at dastardlyhq.com wrote:
>>> All languages have their ugly corners due to initial design mistakes 
>>> and/or
>>> constraints. Eg: java with the special behaviour of its string class, C++
>>> with "=0" pure virtual declaration. But they don't dump them and make 
>>> all old
>>> code suddenly cease to execute.
>> 
>> No, but it was decided that Python 3 would have to be backwards
>> incompatible, mainly to sort out the Unicode mess. Given that,
>> the opportunity was taken to clean up some other mistakes as well.
>
>+1
>and the move to Unicode has opened-up the Python community beyond the 
>US, to embrace 'the world' - a proposition (still) not well-recognised 
>by (only) English-speakers/writers/readers.
>
>
>Even though the proposition has a troll-bait smell to it:-
>
>1 nothing "ceased to execute" and Python 2 was maintained and developed 
>for quite some time and in-parallel to many Python 3 releases.

MacOS only comes with python3 now. If you have a whole load of python2 code
you want to run you now have to manually install python2 yourself.

>2 the only constant in this business is 'change'. I'd rather cope with 
>an evolution in this language (which we know and love), than one day 

Its not evolution, its revolution. Evolution retains old functionality.


From bowman at montana.com  Wed Feb  1 11:21:20 2023
From: bowman at montana.com (rbowman)
Date: 1 Feb 2023 16:21:20 GMT
Subject: evaluation question
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <trbvi6$3vgh9$1@dont-email.me> <trd9g8$92t2$1@dont-email.me>
Message-ID: <k3vhs0Fq2i8U8@mid.individual.net>

On Wed, 1 Feb 2023 08:56:40 -0000 (UTC), Muttley wrote:


> Why couldn't they just keep "print" and call the function , oh I dunno,
> "printf" ?

<Venting> Why does every language have to invent their own function to 
print to the console that is very similar but not the same as the rest of 
the herd?</Venting>


From tdldev at gmail.com  Wed Feb  1 11:48:15 2023
From: tdldev at gmail.com (Jack Dangler)
Date: Wed, 1 Feb 2023 11:48:15 -0500
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
Message-ID: <00c6a188-23ad-f8fe-e8b9-ccb392c4127b@gmail.com>

Ari

Have you tried running a snap update?

Regards

???????

On 2/1/23 11:31, ???? wrote:
> Hi,
>
> I have a server with Ubuntu 22.04.1 LTS and the Python version there
> is Python 3.10.6. Is there a safe way to upgrade to the latest version of
> Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the Python
> version remained the same.
>
> Thanks,
> ????
> uri at speedy.net

From uri at speedy.net  Wed Feb  1 12:36:48 2023
From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=)
Date: Wed, 1 Feb 2023 19:36:48 +0200
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <00c6a188-23ad-f8fe-e8b9-ccb392c4127b@gmail.com>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
 <00c6a188-23ad-f8fe-e8b9-ccb392c4127b@gmail.com>
Message-ID: <CABD5YeHB1NSi-9UCx3afHVUsTCD+oLO8S0Okooe6TGKy=gTHFA@mail.gmail.com>

Thank you all.

I'm not familiar with snap update but I did `sudo apt update` & `sudo apt
upgrade`, but about one or two months ago.

????
uri at speedy.net


On Wed, Feb 1, 2023 at 7:33 PM Jack Dangler <tdldev at gmail.com> wrote:

> Ari
>
> Have you tried running a snap update?
>
> Regards
>
> ???????
>
> On 2/1/23 11:31, ???? wrote:
> > Hi,
> >
> > I have a server with Ubuntu 22.04.1 LTS and the Python version there
> > is Python 3.10.6. Is there a safe way to upgrade to the latest version of
> > Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the
> Python
> > version remained the same.
> >
> > Thanks,
> > ????
> > uri at speedy.net
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From hjp-python at hjp.at  Wed Feb  1 12:28:04 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Wed, 1 Feb 2023 18:28:04 +0100
Subject: evaluation question
In-Reply-To: <trd9nn$94gv$1@dont-email.me>
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me>
Message-ID: <20230201172804.whx4lqlngs4qnkic@hjp.at>

On 2023-02-01 09:00:39 -0000, Muttley at dastardlyhq.com wrote:
> Its not evolution, its revolution. Evolution retains old functionality.

Tell a penguin that it can fly :-)

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230201/45833835/attachment.sig>

From grant.b.edwards at gmail.com  Wed Feb  1 12:46:29 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Wed, 01 Feb 2023 09:46:29 -0800 (PST)
Subject: evaluation question
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <trd9l4$93l4$1@dont-email.me>
Message-ID: <63daa575.050a0220.761e6.3eb5@mx.google.com>

On 2023-02-01, Muttley at dastardlyhq.com <Muttley at dastardlyhq.com> wrote:
>
>>No, but it was decided that Python 3 would have to be backwards
>>incompatible, mainly to sort out the Unicode mess. Given that,
>>the opportunity was taken to clean up some other mistakes as well.
>
> Unicode is just a string of bytes.

No it isn't. Certain _encodings_ of Unicode are strings of bytes
(UTF-8, for example).

> C supports it with a few extra library functions to get unicode
> length vs byte length and similar. Its really not that
> hard.

It is, actually.

C (the language) doesn't support Unicode at all. There are, however,
libraries that can be used to deal with it.

> Rewriting an entire language just to support that sounds a bit
> absurd to me but hey ho...

Feel free to maintain your own fork of 2.7 :)

The 2.7->3 incompatibility that created the most work for me was
bytes.

From grant.b.edwards at gmail.com  Wed Feb  1 12:47:12 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Wed, 01 Feb 2023 09:47:12 -0800 (PST)
Subject: evaluation question
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me> <20230201172804.whx4lqlngs4qnkic@hjp.at>
Message-ID: <63daa5a0.020a0220.95e5.b705@mx.google.com>

On 2023-02-01, Peter J. Holzer <hjp-python at hjp.at> wrote:
> On 2023-02-01 09:00:39 -0000, Muttley at dastardlyhq.com wrote:
>> Its not evolution, its revolution. Evolution retains old functionality.
>
> Tell a penguin that it can fly :-)

Penguins can fly. They just do it underwater...



From rosuav at gmail.com  Wed Feb  1 12:57:23 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 2 Feb 2023 04:57:23 +1100
Subject: evaluation question
In-Reply-To: <trd9l4$93l4$1@dont-email.me>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
 <k3r3rgF5l1hU1@mid.individual.net> <tramnp$3ohlk$1@dont-email.me>
 <k3tkqhFhgf7U1@mid.individual.net> <trd9l4$93l4$1@dont-email.me>
Message-ID: <CAPTjJmpVX4y0gmd3S-NAkWZia7BZD5Mn+zvvEYx=9umwRyd+Sg@mail.gmail.com>

On Thu, 2 Feb 2023 at 04:29, <Muttley at dastardlyhq.com> wrote:
>
> On Wed, 1 Feb 2023 11:59:25 +1300
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> >On 31/01/23 10:24 pm, Muttley at dastardlyhq.com wrote:
> >> All languages have their ugly corners due to initial design mistakes and/or
> >> constraints. Eg: java with the special behaviour of its string class, C++
> >> with "=0" pure virtual declaration. But they don't dump them and make all old
> >
> >> code suddenly cease to execute.
> >
> >No, but it was decided that Python 3 would have to be backwards
> >incompatible, mainly to sort out the Unicode mess. Given that,
> >the opportunity was taken to clean up some other mistakes as well.
>
> Unicode is just a string of bytes. C supports it with a few extra library
> functions to get unicode length vs byte length and similar. Its really
> not that hard. Rewriting an entire language just to support that sounds a
> bit absurd to me but hey ho...
>

No, Unicode is NOT a string of bytes. UTF-8 is a string of bytes, but
Unicode is not.

If you disagree with the way Python has been developed, you're welcome
to fork Python 2.7 and make your own language (but not called Python).
Meanwhile, the rest of us really appreciate the fact that Python
supports Unicode properly, not just as "a string of bytes". Also, be
sure to deal with the technical debt of refusing to ever remove any
feature. I'm curious how many dev hours that costs you.

Incidentally, the bytes->unicode transformation wasn't Python 3's
biggest reason for being. See https://peps.python.org/pep-3100/ for
details.

ChrisA

From list1 at tompassin.net  Wed Feb  1 13:13:06 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 1 Feb 2023 13:13:06 -0500
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <CABD5YeHB1NSi-9UCx3afHVUsTCD+oLO8S0Okooe6TGKy=gTHFA@mail.gmail.com>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
 <00c6a188-23ad-f8fe-e8b9-ccb392c4127b@gmail.com>
 <CABD5YeHB1NSi-9UCx3afHVUsTCD+oLO8S0Okooe6TGKy=gTHFA@mail.gmail.com>
Message-ID: <9f81c190-33a4-8147-1fdc-10b1633e4a22@tompassin.net>

On 2/1/2023 12:36 PM, ???? wrote:
> Thank you all.
> 
> I'm not familiar with snap update but I did `sudo apt update` & `sudo apt
> upgrade`, but about one or two months ago.
> 
> ????
> uri at speedy.net

You can probably install it from the deadsnakes repository:

sudo add-apt-repository ppa:deadsnakes/ppa

More than likely it will have your package.  See

https://github.com/deadsnakes



> On Wed, Feb 1, 2023 at 7:33 PM Jack Dangler <tdldev at gmail.com> wrote:
> 
>> Ari
>>
>> Have you tried running a snap update?
>>
>> Regards
>>
>> ???????
>>
>> On 2/1/23 11:31, ???? wrote:
>>> Hi,
>>>
>>> I have a server with Ubuntu 22.04.1 LTS and the Python version there
>>> is Python 3.10.6. Is there a safe way to upgrade to the latest version of
>>> Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the
>> Python
>>> version remained the same.
>>>
>>> Thanks,
>>> ????
>>> uri at speedy.net
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>


From tdldev at gmail.com  Wed Feb  1 13:37:48 2023
From: tdldev at gmail.com (Jack Dangler)
Date: Wed, 1 Feb 2023 13:37:48 -0500
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <9f81c190-33a4-8147-1fdc-10b1633e4a22@tompassin.net>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
 <00c6a188-23ad-f8fe-e8b9-ccb392c4127b@gmail.com>
 <CABD5YeHB1NSi-9UCx3afHVUsTCD+oLO8S0Okooe6TGKy=gTHFA@mail.gmail.com>
 <9f81c190-33a4-8147-1fdc-10b1633e4a22@tompassin.net>
Message-ID: <fc90c57c-dc19-8b90-3399-09990b1540a9@gmail.com>

If you're not familiar with snap, lookup 'snap' 'ubuntu' ...

On 2/1/23 13:13, Thomas Passin wrote:
> On 2/1/2023 12:36 PM, ???? wrote:
>> Thank you all.
>>
>> I'm not familiar with snap update but I did `sudo apt update` & `sudo 
>> apt
>> upgrade`, but about one or two months ago.
>>
>> ????
>> uri at speedy.net
>
> You can probably install it from the deadsnakes repository:
>
> sudo add-apt-repository ppa:deadsnakes/ppa
>
> More than likely it will have your package.? See
>
> https://github.com/deadsnakes
>
>
>
>> On Wed, Feb 1, 2023 at 7:33 PM Jack Dangler <tdldev at gmail.com> wrote:
>>
>>> Ari
>>>
>>> Have you tried running a snap update?
>>>
>>> Regards
>>>
>>> ???????
>>>
>>> On 2/1/23 11:31, ???? wrote:
>>>> Hi,
>>>>
>>>> I have a server with Ubuntu 22.04.1 LTS and the Python version there
>>>> is Python 3.10.6. Is there a safe way to upgrade to the latest 
>>>> version of
>>>> Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the
>>> Python
>>>> version remained the same.
>>>>
>>>> Thanks,
>>>> ????
>>>> uri at speedy.net
>>> -- 
>>> https://mail.python.org/mailman/listinfo/python-list
>>>
>

From rosuav at gmail.com  Wed Feb  1 13:22:41 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 2 Feb 2023 05:22:41 +1100
Subject: evaluation question
In-Reply-To: <trd9nn$94gv$1@dont-email.me>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
 <k3r3rgF5l1hU1@mid.individual.net> <tramnp$3ohlk$1@dont-email.me>
 <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me>
Message-ID: <CAPTjJmomPGF1wk8YP5up6eao2ii+5NHULxts9J02qYd4inC2Mg@mail.gmail.com>

On Thu, 2 Feb 2023 at 04:26, <Muttley at dastardlyhq.com> wrote:
>
> Its not evolution, its revolution. Evolution retains old functionality.
>

By the way, I'd like to see your opinions on eternal retention of old
functionality. Which of these features are you willing to put effort
into supporting?

1) Long integer constants eg 1234L which are the long type rather than
the int type
2) dict.has_key method, doing the same job as "key in dict"
3) The input() function, which automatically evals what was entered at
the keyboard
4) `x` which does the same as repr(x)
5) "from module import *" inside a function

Retaining old functionality is all well and good, but there are
limits, especially when the old functionality is downright wrong
(input vs raw_input). Before you complain about other people's
decisions, find out how many hours of YOUR time you're willing to
invest into a project.

Or alternatively, how many dollars you would spend on it. Let's
pretend that you can pay for as many core Python developers as you
like for USD 150,000 a year each. (See job posting
https://jobs.pyfound.org/apply/TwgMP1b4OV/Security-Developer-In-Residence
for where I got that figure from.) How many are you personally willing
to hire in order to have these features maintained?

It's easy to whine. Much harder to actually do something about it.

ChrisA

From list1 at tompassin.net  Wed Feb  1 13:52:46 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 1 Feb 2023 13:52:46 -0500
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <fc90c57c-dc19-8b90-3399-09990b1540a9@gmail.com>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
 <00c6a188-23ad-f8fe-e8b9-ccb392c4127b@gmail.com>
 <CABD5YeHB1NSi-9UCx3afHVUsTCD+oLO8S0Okooe6TGKy=gTHFA@mail.gmail.com>
 <9f81c190-33a4-8147-1fdc-10b1633e4a22@tompassin.net>
 <fc90c57c-dc19-8b90-3399-09990b1540a9@gmail.com>
Message-ID: <aa8ea6ac-914f-d392-5e03-87c9c6268755@tompassin.net>

On 2/1/2023 1:37 PM, Jack Dangler wrote:
> If you're not familiar with snap, lookup 'snap' 'ubuntu' ...

Many people would rather avoid snap if possible ...

> On 2/1/23 13:13, Thomas Passin wrote:
>> On 2/1/2023 12:36 PM, ???? wrote:
>>> Thank you all.
>>>
>>> I'm not familiar with snap update but I did `sudo apt update` & `sudo 
>>> apt
>>> upgrade`, but about one or two months ago.
>>>
>>> ????
>>> uri at speedy.net
>>
>> You can probably install it from the deadsnakes repository:
>>
>> sudo add-apt-repository ppa:deadsnakes/ppa
>>
>> More than likely it will have your package.? See
>>
>> https://github.com/deadsnakes
>>
>>
>>
>>> On Wed, Feb 1, 2023 at 7:33 PM Jack Dangler <tdldev at gmail.com> wrote:
>>>
>>>> Ari
>>>>
>>>> Have you tried running a snap update?
>>>>
>>>> Regards
>>>>
>>>> ???????
>>>>
>>>> On 2/1/23 11:31, ???? wrote:
>>>>> Hi,
>>>>>
>>>>> I have a server with Ubuntu 22.04.1 LTS and the Python version there
>>>>> is Python 3.10.6. Is there a safe way to upgrade to the latest 
>>>>> version of
>>>>> Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the
>>>> Python
>>>>> version remained the same.
>>>>>
>>>>> Thanks,
>>>>> ????
>>>>> uri at speedy.net
>>>> -- 
>>>> https://mail.python.org/mailman/listinfo/python-list
>>>>
>>


From Richard at Damon-Family.org  Wed Feb  1 20:21:07 2023
From: Richard at Damon-Family.org (Richard Damon)
Date: Wed, 1 Feb 2023 20:21:07 -0500
Subject: evaluation question
In-Reply-To: <trd9l4$93l4$1@dont-email.me>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <trd9l4$93l4$1@dont-email.me>
Message-ID: <e57176e6-5dca-6613-bacc-dcc3f03b7458@Damon-Family.org>

On 2/1/23 3:59 AM, Muttley at dastardlyhq.com wrote:
> On Wed, 1 Feb 2023 11:59:25 +1300
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> On 31/01/23 10:24 pm, Muttley at dastardlyhq.com wrote:
>>> All languages have their ugly corners due to initial design mistakes and/or
>>> constraints. Eg: java with the special behaviour of its string class, C++
>>> with "=0" pure virtual declaration. But they don't dump them and make all old
>>> code suddenly cease to execute.
>> No, but it was decided that Python 3 would have to be backwards
>> incompatible, mainly to sort out the Unicode mess. Given that,
>> the opportunity was taken to clean up some other mistakes as well.
> Unicode is just a string of bytes. C supports it with a few extra library
> functions to get unicode length vs byte length and similar. Its really
> not that hard. Rewriting an entire language just to support that sounds a
> bit absurd to me but hey ho...
>
No, Unicode is a string of 21 bit characters. UTF-8 is a representation 
that uses bytes, but isn't itself "Unicode".

The key fact is that a "String" variable is indexed not by bytes of 
UTF-8 encoding, but by actual characters.

Python3 will store a string as either a sequence of Bytes if the data is 
all Latin-1, as a sequence of 16-bit words if the data all fits on th 
BMP, and a sequence of 32 bit words if it has a value outside the BMP.

-- 
Richard Damon


From Richard at Damon-Family.org  Wed Feb  1 20:32:58 2023
From: Richard at Damon-Family.org (Richard Damon)
Date: Wed, 1 Feb 2023 20:32:58 -0500
Subject: evaluation question
In-Reply-To: <63daa575.050a0220.761e6.3eb5@mx.google.com>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <trd9l4$93l4$1@dont-email.me> <63daa575.050a0220.761e6.3eb5@mx.google.com>
Message-ID: <4a6f95f5-b105-383a-f4e7-9b6e0e7e34cd@Damon-Family.org>

On 2/1/23 12:46 PM, Grant Edwards wrote:
> C (the language) doesn't support Unicode at all. There are, however,
> libraries that can be used to deal with it.

No, it does, but only optionally.

<uchar.h> provides functions that manipulate Unicode "Characters"

The type char32_t will hold Unicode Code Points, and you can define 
string literals of that type with

U"string" notation.


-- 
Richard Damon


From bowman at montana.com  Wed Feb  1 22:33:24 2023
From: bowman at montana.com (rbowman)
Date: 2 Feb 2023 03:33:24 GMT
Subject: evaluation question
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <trbvi6$3vgh9$1@dont-email.me> <trd9g8$92t2$1@dont-email.me>
 <k3vhs0Fq2i8U8@mid.individual.net>
 <languages-20230201183040@ram.dialup.fu-berlin.de>
Message-ID: <k40p84F1nniU1@mid.individual.net>

On 1 Feb 2023 17:31:02 GMT, Stefan Ram wrote:

> rbowman <bowman at montana.com> writes:
>><Venting> Why does every language have to invent their own function to
>>print to the console that is very similar but not the same as the rest
>>of the herd?</Venting>
> 
>   Why do there have to be different languages at all?

https://homepages.cwi.nl/~steven/abc/programmers/introduction.html

"Why ABC?
The answer to the question 'Why a new language?' is the same as the answer 
to the question 'Why new computers?': because they can help you do the job 
better. With the choice between a language where it will take a week to 
write a program, and a language where it will take an afternoon, most 
people will choose the latter."

That leads to the question of when Van Rossum was looking for a hobby 
project, why not extend ABC?  Or Pike?

https://pike.lysator.liu.se/about/history/

Then there is the question of how a new language becomes popular. When 
Matsumoto developed Ruby it was almost 4 years before there was any 
coherent English documentation. How did it get traction?

How about Go? Thompson and Pike hate C++ (with cause) so they went back to 
C and reworked it. Then there is C++ itself, which was released before its 
time. 

There are many more obscure languages when someone saw a need. Then there 
are features the propagate like lambdas. Everyone came down with lambda 
envy and shoehorned them into the language one way or the other.


From Muttley at dastardlyhq.com  Thu Feb  2 04:31:46 2023
From: Muttley at dastardlyhq.com (Muttley at dastardlyhq.com)
Date: Thu, 2 Feb 2023 09:31:46 -0000 (UTC)
Subject: evaluation question
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me>
 <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
Message-ID: <trfvu2$qhcd$1@dont-email.me>

On Wed, 1 Feb 2023 18:28:04 +0100
"Peter J. Holzer" <hjp-python at hjp.at> wrote:
>--b2nljkb3mdefsdhx
>Content-Type: text/plain; charset=us-ascii
>Content-Disposition: inline
>Content-Transfer-Encoding: quoted-printable
>
>On 2023-02-01 09:00:39 -0000, Muttley at dastardlyhq.com wrote:
>> Its not evolution, its revolution. Evolution retains old functionality.
>
>Tell a penguin that it can fly :-)

Yeah ok :) But the ancestors of penguins didn't wake up one morning, flap
their wings and fall out the tree, it happened gradually. Python2 syntax
could have been retained for X versions of 3 just as C++ keeps old stuff
until its eventually deprecated them removed.


From bowman at montana.com  Thu Feb  2 10:41:07 2023
From: bowman at montana.com (rbowman)
Date: 2 Feb 2023 15:41:07 GMT
Subject: evaluation question
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me> <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me>
Message-ID: <k423siF7s1jU1@mid.individual.net>

On Thu, 2 Feb 2023 09:31:46 -0000 (UTC), Muttley wrote:


> Yeah ok  But the ancestors of penguins didn't wake up one morning, flap
> their wings and fall out the tree, it happened gradually. Python2 syntax
> could have been retained for X versions of 3 just as C++ keeps old stuff
> until its eventually deprecated them removed.

Isn't that prolonging the agony? I had some 2.7 scripts I had to move to 
3. It wasn't that painful and I learned the new syntax. Being lazy if they 
still worked I would have kept using 2.7 syntax until someday it really 
went away.

MS did it big time with VB .NET. I'm sure there still are people 
maintaining and extending old-style VB until it ceases to work altogether. 
Then they'll be faced with the same learning curve most people suffered 
through 20 years ago.

From jon+usenet at unequivocal.eu  Thu Feb  2 12:38:56 2023
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Thu, 2 Feb 2023 17:38:56 -0000 (UTC)
Subject: Licensing?
References: <license-20230202182456@ram.dialup.fu-berlin.de>
Message-ID: <slrnttnt9g.84eh.jon+usenet@raven.unequivocal.eu>

On 2023-02-02, Stefan Ram <ram at zedat.fu-berlin.de> wrote:
>   Many licenses in the Python world are like: "You can make
>   changes, but have to leave in my Copyright notice.".
>
>   Would it be possible that the original author could not
>   claim a Copyright anymore when code has been changed?

No. If you change someone else's code then you have created a derived
work, which requires permission from both the original author and you
to copy. (Unless you change it so much that nothing remains of the
original author's code, of course.)

From rosuav at gmail.com  Thu Feb  2 13:12:31 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 3 Feb 2023 05:12:31 +1100
Subject: evaluation question
In-Reply-To: <trfvu2$qhcd$1@dont-email.me>
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
 <k3r3rgF5l1hU1@mid.individual.net> <tramnp$3ohlk$1@dont-email.me>
 <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me>
 <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me>
Message-ID: <CAPTjJmr4Lm5zSM7TiGBkjnKSN+Wyy8Lc7bnRZSogPWKCjeSWZg@mail.gmail.com>

On Fri, 3 Feb 2023 at 04:48, <Muttley at dastardlyhq.com> wrote:
> Yeah ok :) But the ancestors of penguins didn't wake up one morning, flap
> their wings and fall out the tree, it happened gradually. Python2 syntax
> could have been retained for X versions of 3 just as C++ keeps old stuff
> until its eventually deprecated them removed.

What, you mean Python 2.7 could have continued to be supported until
Python 3.8 (yes, that's not a typo) was released?

It was.

ChrisA

From uri at speedy.net  Thu Feb  2 13:56:30 2023
From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=)
Date: Thu, 2 Feb 2023 20:56:30 +0200
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <9f81c190-33a4-8147-1fdc-10b1633e4a22@tompassin.net>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
 <00c6a188-23ad-f8fe-e8b9-ccb392c4127b@gmail.com>
 <CABD5YeHB1NSi-9UCx3afHVUsTCD+oLO8S0Okooe6TGKy=gTHFA@mail.gmail.com>
 <9f81c190-33a4-8147-1fdc-10b1633e4a22@tompassin.net>
Message-ID: <CABD5YeEivsr52PONL_1tBQNtz=VAN9BtOi35aBvYb9s62f2EZw@mail.gmail.com>

deadsnakes ppa don't have Python 3.10 for Ubuntu 22.04.

https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa
????
uri at speedy.net


On Wed, Feb 1, 2023 at 8:14 PM Thomas Passin <list1 at tompassin.net> wrote:

> On 2/1/2023 12:36 PM, ???? wrote:
> > Thank you all.
> >
> > I'm not familiar with snap update but I did `sudo apt update` & `sudo apt
> > upgrade`, but about one or two months ago.
> >
> > ????
> > uri at speedy.net
>
> You can probably install it from the deadsnakes repository:
>
> sudo add-apt-repository ppa:deadsnakes/ppa
>
> More than likely it will have your package.  See
>
> https://github.com/deadsnakes
>
>
>
> > On Wed, Feb 1, 2023 at 7:33 PM Jack Dangler <tdldev at gmail.com> wrote:
> >
> >> Ari
> >>
> >> Have you tried running a snap update?
> >>
> >> Regards
> >>
> >> ???????
> >>
> >> On 2/1/23 11:31, ???? wrote:
> >>> Hi,
> >>>
> >>> I have a server with Ubuntu 22.04.1 LTS and the Python version there
> >>> is Python 3.10.6. Is there a safe way to upgrade to the latest version
> of
> >>> Python 3.10 (3.10.9)? I tried with the OS update and upgrade but the
> >> Python
> >>> version remained the same.
> >>>
> >>> Thanks,
> >>> ????
> >>> uri at speedy.net
> >> --
> >> https://mail.python.org/mailman/listinfo/python-list
> >>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From list1 at tompassin.net  Thu Feb  2 15:12:56 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Thu, 2 Feb 2023 15:12:56 -0500
Subject: Upgrading Python on Ubuntu 22.04.1 LTS
In-Reply-To: <CABD5YeEivsr52PONL_1tBQNtz=VAN9BtOi35aBvYb9s62f2EZw@mail.gmail.com>
References: <CABD5YeER=dX4MYu=81d+3_HWQKv8UKOaHo7d3+816BDeN8OVfA@mail.gmail.com>
 <00c6a188-23ad-f8fe-e8b9-ccb392c4127b@gmail.com>
 <CABD5YeHB1NSi-9UCx3afHVUsTCD+oLO8S0Okooe6TGKy=gTHFA@mail.gmail.com>
 <9f81c190-33a4-8147-1fdc-10b1633e4a22@tompassin.net>
 <CABD5YeEivsr52PONL_1tBQNtz=VAN9BtOi35aBvYb9s62f2EZw@mail.gmail.com>
Message-ID: <4c4b5d99-eadf-e19b-f92c-b693946169d6@tompassin.net>

On 2/2/2023 1:56 PM, ???? wrote:
> deadsnakes ppa don't have Python 3.10 for Ubuntu 22.04.
> 
> https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa 
> <https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa>
> ????
> uri at speedy.net <mailto:uri at speedy.net>

I just last night installed 3.10.9 on Debian Buster following the 
directions at

https://computingforgeeks.com/how-to-install-python-latest-debian/

I only changed the file name of the Python tarball. However, if you want 
to use tkinter, the Tk development libraries need to be on your machine 
before you run ./configure.  The package name may change, but for me it was

sudo apt-get install tk-dev

(see 
https://stackoverflow.com/questions/5459444/tkinter-python-may-not-be-configured-for-tk)

It does not replace the Debian Python3 link to 3.7.3 so it should not 
interfere with normal Debian system services.

(The highest Debian-supported version of Python on Buster is still 3.7+).

> On Wed, Feb 1, 2023 at 8:14 PM Thomas Passin <list1 at tompassin.net 
> <mailto:list1 at tompassin.net>> wrote:
> 
>     On 2/1/2023 12:36 PM, ???? wrote:
>      > Thank you all.
>      >
>      > I'm not familiar with snap update but I did `sudo apt update` &
>     `sudo apt
>      > upgrade`, but about one or two months ago.
>      >
>      > ????
>      > uri at speedy.net <mailto:uri at speedy.net>
> 
>     You can probably install it from the deadsnakes repository:
> 
>     sudo add-apt-repository ppa:deadsnakes/ppa
> 
>     More than likely it will have your package.? See
> 
>     https://github.com/deadsnakes <https://github.com/deadsnakes>
> 
> 
> 
>      > On Wed, Feb 1, 2023 at 7:33 PM Jack Dangler <tdldev at gmail.com
>     <mailto:tdldev at gmail.com>> wrote:
>      >
>      >> Ari
>      >>
>      >> Have you tried running a snap update?
>      >>
>      >> Regards
>      >>
>      >> ???????
>      >>
>      >> On 2/1/23 11:31, ???? wrote:
>      >>> Hi,
>      >>>
>      >>> I have a server with Ubuntu 22.04.1 LTS and the Python version
>     there
>      >>> is Python 3.10.6. Is there a safe way to upgrade to the latest
>     version of
>      >>> Python 3.10 (3.10.9)? I tried with the OS update and upgrade
>     but the
>      >> Python
>      >>> version remained the same.
>      >>>
>      >>> Thanks,
>      >>> ????
>      >>> uri at speedy.net <mailto:uri at speedy.net>
>      >> --
>      >> https://mail.python.org/mailman/listinfo/python-list
>     <https://mail.python.org/mailman/listinfo/python-list>
>      >>
> 
>     -- 
>     https://mail.python.org/mailman/listinfo/python-list
>     <https://mail.python.org/mailman/listinfo/python-list>
> 


From nntp.mbourne at spamgourmet.com  Thu Feb  2 16:15:34 2023
From: nntp.mbourne at spamgourmet.com (Mark Bourne)
Date: Thu, 2 Feb 2023 21:15:34 +0000
Subject: evaluation question
In-Reply-To: <trfvu2$qhcd$1@dont-email.me>
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me> <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me>
Message-ID: <trh95n$1247f$1@dont-email.me>

Muttley at dastardlyhq.com wrote:
> On Wed, 1 Feb 2023 18:28:04 +0100
> "Peter J. Holzer" <hjp-python at hjp.at> wrote:
>> --b2nljkb3mdefsdhx
>> Content-Type: text/plain; charset=us-ascii
>> Content-Disposition: inline
>> Content-Transfer-Encoding: quoted-printable
>>
>> On 2023-02-01 09:00:39 -0000, Muttley at dastardlyhq.com wrote:
>>> Its not evolution, its revolution. Evolution retains old functionality.
>>
>> Tell a penguin that it can fly :-)
> 
> Yeah ok :) But the ancestors of penguins didn't wake up one morning, flap
> their wings and fall out the tree, it happened gradually. Python2 syntax
> could have been retained for X versions of 3 just as C++ keeps old stuff
> until its eventually deprecated them removed.

Python 2 *was* retained for X versions of Python 3.  From a quick check, 
Python 3.0 was released in December 2008 and Python 2 support ended in 
January 2020 - by which time Python 3 was up to 3.8 as ChrisA mentioned. 
  That's about an 11 year transition period, which is hardly sudden! 
Python 3 *was* the point at which the features deprecated in Python 2 
were removed.

The problem is, a lot seemed to ignore Python 3 for the first 12 years 
and then suddenly panic because Python 2 support had ended.

-- 
Mark.

From greg.ewing at canterbury.ac.nz  Thu Feb  2 19:11:23 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 3 Feb 2023 13:11:23 +1300
Subject: Licensing?
In-Reply-To: <slrnttnt9g.84eh.jon+usenet@raven.unequivocal.eu>
References: <license-20230202182456@ram.dialup.fu-berlin.de>
 <slrnttnt9g.84eh.jon+usenet@raven.unequivocal.eu>
Message-ID: <k431pfFcf2kU1@mid.individual.net>

On 3/02/23 6:38 am, Jon Ribbens wrote:
> If you change someone else's code then you have created a derived
> work, which requires permission from both the original author and you
> to copy. (Unless you change it so much that nothing remains of the
> original author's code, of course.)

"Nothing" is probably a bit extreme; somewhere between "exactly the
same" and "completely different" there will be a borderline case,
although exactly where the border lies would require a court case
to determine.

When in doubt, the sensible and courteous thing would be to include
the original copyright notice as requested, maybe with a "based on
work by..." attribution.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Feb  2 19:22:36 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 3 Feb 2023 13:22:36 +1300
Subject: evaluation question
In-Reply-To: <trgn74$v0sn$1@dont-email.me>
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me> <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me> <k423siF7s1jU1@mid.individual.net>
 <trgn74$v0sn$1@dont-email.me>
Message-ID: <k432ehFcisjU1@mid.individual.net>

On 3/02/23 5:09 am, Muttley at dastardlyhq.com wrote:
> What if its 10s of thousands of lines of
> core production code? If the company it belongs to wants to add new Python 3
> features it can't just plug them into the code because it won't run under
> Python 3, they have to do a full overhaul or even complete rewrite and that
> costs a lot of time and money.

A possible strategy in that case would have been to incrementally
rewrite it in such a way that the code would run in both 2.7 and 3.x
(various features were added to 2.7 to make that possible).

When that point is reached, you can then switch to running it with
Python 3 and start using the new features.

Also, if you're a company whose business is totally reliant on some
piece of code, it would be prudent to plan ahead and budget for
rewriting or replacing it at some point.

People seem to think that because code doesn't wear out like
hardware, you don't have to budget for replacing it. But you can't
expect third party software to be maintained forever -- particularly
when, as with Python, the maintenance is mainly being done by
*volunteers*.

-- 
Greg

From jon+usenet at unequivocal.eu  Thu Feb  2 19:39:22 2023
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Fri, 3 Feb 2023 00:39:22 -0000 (UTC)
Subject: Licensing?
References: <license-20230202182456@ram.dialup.fu-berlin.de>
 <slrnttnt9g.84eh.jon+usenet@raven.unequivocal.eu>
 <k431pfFcf2kU1@mid.individual.net>
Message-ID: <slrnttoltq.84eh.jon+usenet@raven.unequivocal.eu>

On 2023-02-03, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> On 3/02/23 6:38 am, Jon Ribbens wrote:
>> If you change someone else's code then you have created a derived
>> work, which requires permission from both the original author and you
>> to copy. (Unless you change it so much that nothing remains of the
>> original author's code, of course.)
>
> "Nothing" is probably a bit extreme; somewhere between "exactly the
> same" and "completely different" there will be a borderline case,
> although exactly where the border lies would require a court case
> to determine.

Well yes, technically if you remove so much code that what remains
of the original is so de minimis that it can't be considered
copyrightable then you're good. But that doesn't seem that useful
to know, because if you've removed that much then what remains,
pretty much by definition, isn't going to be useful. You'd be
better off simply starting from scratch and having an unimpeachable
claim to own the entire copyright yourself.

From bowman at montana.com  Fri Feb  3 00:46:10 2023
From: bowman at montana.com (rbowman)
Date: 3 Feb 2023 05:46:10 GMT
Subject: evaluation question
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me> <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me> <k423siF7s1jU1@mid.individual.net>
 <trgn74$v0sn$1@dont-email.me>
Message-ID: <k43ld2FeulaU3@mid.individual.net>

On Thu, 2 Feb 2023 16:09:09 -0000 (UTC), Muttley wrote:


> What if its not a few scripts? What if its 10s of thousands of lines of
> core production code? If the company it belongs to wants to add new
> Python 3 features it can't just plug them into the code because it won't
> run under Python 3, they have to do a full overhaul or even complete
> rewrite and that costs a lot of time and money.

Tell me about it...  Esri is the 500 pound gorilla in the GIS industry. 
They haven't been secretive about their roadmap but the tools people have 
been using for almost 20 years are going, going, GONE. 

Part of that is their scripting language ArcPy moved to 3.8. It's a minor 
inconvenience for me to update some old scripts and to develop with 3 but 
a lot of firms have those thousands of lines of code they've developed 
over the years in ArcPy for GIS data manipulation.

More painful for me is the C++ API is also gone. Legacy sites have a 
couple of more years before it's all over. That leaves me with a foot in 
both worlds. 

> Unfortunately a lot of people who've never worked in large institutions
> with huge code bases don't understand this.

I don't work in a large company but I deal with about 25 years of legacy 
code every day. The core functionality is good but the time and money is 
spent putting lipstick on a pig. 

Technical debt is an industry wide problem. 


From transreductionist at gmail.com  Fri Feb  3 16:18:46 2023
From: transreductionist at gmail.com (transreductionist)
Date: Fri, 3 Feb 2023 13:18:46 -0800 (PST)
Subject: Organizing modules and their code
Message-ID: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>

Here is the situation. There is a top-level module (see designs below) containing code, that as the name suggests, manages an ETL pipeline. A directory is created called etl_helpers that organizes several modules responsible for making up the pipeline. The discussion concerns the Python language, which supports OOP as well as Structural/Functional approaches to programming.

I am interested in opinions on which design adheres best to standard architectural practices and the SOLID principles. I understand that this is one of those topics where people may have strong opinions one way or the other. I am interested in those opinions. 
 
Allow me to give my thoughts. First, I don't think there would be much difference if I was using OOP for the functionality, or using a structural paradigm. A structural paradigm in my opinion, along the lines of Rich Hickey's comments on simple versus complex, would be a simpler implementation. In this case there is no reason to create a construct with state. So let's assume the code is structural and not OOP.

I would go with Design I. Succinctly stated, Design I supports readability and maintainability at least as well, if not better than the other designs. The goal of the SOLID principles are the creation of mid-level software structures that (Software Architecture: SA Martin). I think Design I best adheres to these principles of:
---- Tolerate change,
---- Are easy to understand, and
---- Are the basis of components that can be used in many software systems.

I could point to the Single Responsibility Principle which is defined as (SA Martin): a module should be responsible to one, and only one, actor. It should satisfy the Liskov Substitution Principle as well. Further, each module in the etl_helpers directory is at the same level of abstraction.

I could also mention that as Dijkstra stressed, at every level, from the smallest function to the largest component, software is like a science and, therefore, is driven by falsifiability. Software architects strive to define modules, components, and services that are easily falsifiable (testable). To do so, they employ restrictive disciplines similar to structured programming,
albeit at a much higher level (SA Martin).

One can point to multiple reasons why Design I might be preferred, but what are the compelling reasons, if there are any, that would suggest another design was superior.

Finally, let me reference an interesting research paper I read recently that seems to support the other designs as anti-patterns: Architecture_Anti-patterns_Automatically.pdf 

 ---- (https://www.cs.drexel.edu/~yfcai/papers/2019/tse2019.pdf)

SEVERAL DESIGNS FOR COMPARISON

DESIGN I:

---- manage_the_etl_pipeline.py
---- etl_helpers
      ---- extract.py
      ---- transform.py
      ---- load.py

Of course one could also

DESIGN II:

---- manage_the_etl_pipeline.py
---- etl_helpers
      ---- extract_transform_load.py

or probably even:

DESIGN III: 

---- manage_the_etl_pipeline.py
---- extract_transform_load.py

From 2QdxY4RzWzUUiLuE at potatochowder.com  Fri Feb  3 17:14:51 2023
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Fri, 3 Feb 2023 17:14:51 -0500
Subject: Organizing modules and their code
In-Reply-To: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
Message-ID: <Y92HW9iEZ9AudUDf@anomaly>

On 2023-02-03 at 13:18:46 -0800,
transreductionist <transreductionist at gmail.com> wrote:

> Here is the situation. There is a top-level module (see designs below)
> containing code, that as the name suggests, manages an ETL pipeline. A
> directory is created called etl_helpers that organizes several modules
> responsible for making up the pipeline. The discussion concerns the
> Python language, which supports OOP as well as Structural/Functional
> approaches to programming.

> I am interested in opinions on which design adheres best to standard
> architectural practices and the SOLID principles. I understand that
> this is one of those topics where people may have strong opinions one
> way or the other. I am interested in those opinions.

Okay, I'll start:  unless one of extract, transform, or load is already,
or will certainly at some point become, complex/complicated enough to be
its own architectural module with its own architectural substructure; or
you're constructing specific ETL pipelines for specific ETL jobs at the
times the jobs are defined; then I think you're overthinking it.

Note that I say that speaking as a notorious overthinker.  ;-)

Keep It Simple:  Put all four modules at the top level, and run with it
until you falsify it.  Yes, I would give you that same advice no matter
what language you're using.

FWIW, I'm not a big fan of OO, but based on what little I know about
your ETL pipelines, I agree with you that it probably doesn't make a big
difference at this level.  Define solid (in pretty much any/every sense
of the word, capitalized or not) interfaces between your modules, and
write your code against those interfaces, whether OO or any other
paradigm.

From list1 at tompassin.net  Fri Feb  3 17:31:26 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 3 Feb 2023 17:31:26 -0500
Subject: Organizing modules and their code
In-Reply-To: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
Message-ID: <bccfa720-6b02-53e1-d451-94b8f150e4b7@tompassin.net>

On 2/3/2023 4:18 PM, transreductionist wrote:
> Here is the situation. There is a top-level module (see designs below) containing code, that as the name suggests, manages an ETL pipeline. A directory is created called etl_helpers that organizes several modules responsible for making up the pipeline. The discussion concerns the Python language, which supports OOP as well as Structural/Functional approaches to programming.
> 
> I am interested in opinions on which design adheres best to standard architectural practices and the SOLID principles. I understand that this is one of those topics where people may have strong opinions one way or the other. I am interested in those opinions.

Well, you have pretty well stacked the deck to make DESIGN 1 the 
obviously preferred choice.  I don't think it has much to do with Python 
per se, or even with OO vs imperative style.

As a practical matter, once you got into working with 
extract_transform_load.py (for the other designs), I would expect that 
you would start wanting to refactor it and eventually end up more like 
DESIGN 1.  So you might as well start out that way.

The reasons are 1) what you said about separation of concerns, 2) a 
desire to keep each module or file relatively coherent and easy to read, 
and 3, as you also suggested, making each of them easier to test. 
Decoupling is important too (one of the SOLID prescriptions), but you 
can violate that with any architecture if you don't think carefully 
about what you are doing.

On the subject of OO, I think it is a very good approach to think about 
architecture and design in object terms - meaning conceptual objects 
from the users' point of view.  For example, here you have a pipeline (a 
metaphorical or userland object).  It will need functionality to load, 
transform, and output data so logically it can be composed of a loader, 
one or more transformers, and one or more output formatters (more 
objects).  You may also need a scheduler and a configuration manager 
(more objects).

(*Please* let's not have any quibbling about "class" vs "object".  We 
are at a conceptual level here!)

When it comes to implementation, you can choose to implement those 
userland objects with either imperative, OO, or functional techniques, 
or a mixture.


> Allow me to give my thoughts. First, I don't think there would be much difference if I was using OOP for the functionality, or using a structural paradigm. A structural paradigm in my opinion, along the lines of Rich Hickey's comments on simple versus complex, would be a simpler implementation. In this case there is no reason to create a construct with state. So let's assume the code is structural and not OOP.
> 
> I would go with Design I. Succinctly stated, Design I supports readability and maintainability at least as well, if not better than the other designs. The goal of the SOLID principles are the creation of mid-level software structures that (Software Architecture: SA Martin). I think Design I best adheres to these principles of:
> ---- Tolerate change,
> ---- Are easy to understand, and
> ---- Are the basis of components that can be used in many software systems.
> 
> I could point to the Single Responsibility Principle which is defined as (SA Martin): a module should be responsible to one, and only one, actor. It should satisfy the Liskov Substitution Principle as well. Further, each module in the etl_helpers directory is at the same level of abstraction.
> 
> I could also mention that as Dijkstra stressed, at every level, from the smallest function to the largest component, software is like a science and, therefore, is driven by falsifiability. Software architects strive to define modules, components, and services that are easily falsifiable (testable). To do so, they employ restrictive disciplines similar to structured programming,
> albeit at a much higher level (SA Martin).
> 
> One can point to multiple reasons why Design I might be preferred, but what are the compelling reasons, if there are any, that would suggest another design was superior.
> 
> Finally, let me reference an interesting research paper I read recently that seems to support the other designs as anti-patterns: Architecture_Anti-patterns_Automatically.pdf
> 
>   ---- (https://www.cs.drexel.edu/~yfcai/papers/2019/tse2019.pdf)
> 
> SEVERAL DESIGNS FOR COMPARISON
> 
> DESIGN I:
> 
> ---- manage_the_etl_pipeline.py
> ---- etl_helpers
>        ---- extract.py
>        ---- transform.py
>        ---- load.py
> 
> Of course one could also
> 
> DESIGN II:
> 
> ---- manage_the_etl_pipeline.py
> ---- etl_helpers
>        ---- extract_transform_load.py
> 
> or probably even:
> 
> DESIGN III:
> 
> ---- manage_the_etl_pipeline.py
> ---- extract_transform_load.py


From list1 at tompassin.net  Fri Feb  3 22:24:03 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 3 Feb 2023 22:24:03 -0500
Subject: Organizing modules and their code
In-Reply-To: <Y92HW9iEZ9AudUDf@anomaly>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
 <Y92HW9iEZ9AudUDf@anomaly>
Message-ID: <059a1cd6-0c1f-b740-91cd-e1fbeeb8d00b@tompassin.net>

On 2/3/2023 5:14 PM, 2QdxY4RzWzUUiLuE at potatochowder.com wrote:
> Keep It Simple:  Put all four modules at the top level, and run with it
> until you falsify it.  Yes, I would give you that same advice no matter
> what language you're using.

In my recent message I supported DESIGN 1.  But I really don't care much 
about the directory organization.  It's designing modules whose business 
is to handle various kinds of operations that counts, not so much the 
actual directory organization.


From PythonList at DancesWithMice.info  Sat Feb  4 00:24:15 2023
From: PythonList at DancesWithMice.info (dn)
Date: Sat, 4 Feb 2023 18:24:15 +1300
Subject: Organizing modules and their code
In-Reply-To: <059a1cd6-0c1f-b740-91cd-e1fbeeb8d00b@tompassin.net>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
 <Y92HW9iEZ9AudUDf@anomaly>
 <059a1cd6-0c1f-b740-91cd-e1fbeeb8d00b@tompassin.net>
Message-ID: <f0d234e7-6519-c0da-982b-f302c4c5437c@DancesWithMice.info>

On 04/02/2023 16.24, Thomas Passin wrote:
> On 2/3/2023 5:14 PM, 2QdxY4RzWzUUiLuE at potatochowder.com wrote:
>> Keep It Simple:? Put all four modules at the top level, and run with it
>> until you falsify it.? Yes, I would give you that same advice no matter
>> what language you're using.
> 
> In my recent message I supported DESIGN 1.? But I really don't care much 
> about the directory organization.? It's designing modules whose business 
> is to handle various kinds of operations that counts, not so much the 
> actual directory organization.

+1 (and to comments made in preceding post)

With ETL the 'reasons to change' (SRP) come from different 'actors'. For 
example, the data-source may be altered either in format or by changing 
the tool you'll utilise to access. Accordingly, the virtue of keeping it 
separate from other parts. If you have multiple data-sources, then each 
should be separate for the same reason.

The transform is likely dictated by your client's specification. So, 
another separation. Hence Design 1.

There is a strong argument for suggesting that we're going out of our 
way to imagine problems or future-changes (which may never happen). If 
this is (definitely?) a one-off, then why-bother? If permanence is 
likely, (so many 'temporary' solutions end-up lasting years!) then 
re-use can?should be considered.

Thus, when it comes to loading the data into your own DB; perhaps this 
should be separate, because it is highly likely that the mechanisms you 
build for loading will be matched by at least one 'someone else' wanting 
to access the same data for the desired end-purposes. Accordingly, a 
shareable module and/or class for that.


We can't see the code-structure, so some of the other parts of your 
question(s) are too broad. Here's hoping you and Liskov have a good time 
together...


My preference is for (what I term) the 'circles' diagram (see copy at 
https://mahu.rangi.cloud/CraftingSoftware/CleanArchitecture.jpg). This 
illustrates the 'rule' that code handling the inner functionality not 
know what happens at the more detailed/lower-level functional level of 
the outer rings.

With ETL, there's precious little to embody various circles, but the 
content of the outer ring is obvious. The "T" rules comprise the inner 
"Use Case", even if you eschew "Entities" insofar as OOP-avoidance is 
concerned. This 'inversion', where the inner controls don't need to care 
about the details of outer-ring implementation (is it an RDBMS, MySQL or 
Postgres; or is it some NoSQL system?) brings to life the "D" of SOLID, 
ie Dependency Inversion.


You may pick-up some ideas or reassurance from "Making a Simple Data 
Pipeline Part 1: The ETL Pattern" 
(https://www.codeproject.com/Articles/5324207/Making-a-Simple-Data-Pipeline-Part-1-The-ETL-Patte).

Let us know how it turns-out...
-- 
Regards,
=dn

From list1 at tompassin.net  Sat Feb  4 01:01:45 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 4 Feb 2023 01:01:45 -0500
Subject: Organizing modules and their code
In-Reply-To: <f0d234e7-6519-c0da-982b-f302c4c5437c@DancesWithMice.info>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
 <Y92HW9iEZ9AudUDf@anomaly>
 <059a1cd6-0c1f-b740-91cd-e1fbeeb8d00b@tompassin.net>
 <f0d234e7-6519-c0da-982b-f302c4c5437c@DancesWithMice.info>
Message-ID: <5514bdf5-b968-aa24-9ef1-13c86af0fbfa@tompassin.net>

On 2/4/2023 12:24 AM, dn via Python-list wrote:
> The transform is likely dictated by your client's specification. So, 
> another separation. Hence Design 1.
> 
> There is a strong argument for suggesting that we're going out of our 
> way to imagine problems or future-changes (which may never happen). If 
> this is (definitely?) a one-off, then why-bother? If permanence is 
> likely, (so many 'temporary' solutions end-up lasting years!) then 
> re-use can?should be considered.

With practice, it gets to be more automatic to set things up from the 
beginning to more-or-less honor separation of concerns, decoupled 
modules and APIs, and so forth.  Doing this does not require a full, 
future-proof suite of alternative database adapters, for example, right 
from the start. On top of everything else, you can't know the future 
perfectly.  And you can't know enough at the beginning to get every 
design and architectural path optimal. You learn as you go.

I have a Tomcat application where I separated the output formatting from 
the calculation of results.  At the time I wrote only an XML formatter. 
A decade later, here comes JSON, and customers are asking about it.  I 
was able to write a JSON formatter with the same API in about half an 
hour, and now we have optional JSON output. Separating out the 
formatting functionality with its own API was not an example of wasting 
time with YAGNI (You Aren't Going To Need It), it was just plain good 
practice that probably didn't even cost me any more development time - 
since it simplified the calculation code.

OTOH, you may be just trying to learn how to do the bits and pieces. You 
may be learning how to connect to the database. You may be learning how 
to make a pipeline multithreaded.  You have to experiment a thousand 
ways, and in a hurry.  Until you learn how to do the basic techniques, 
sure, quick and dirty is fine.  But it shouldn't be the way you design 
your actual product, unless it's just for you and needs to be done 
quickly, and will probably be ephemeral.

Why do I get the feeling that the OP was asking about a homework problem?


From research at johnohagan.com  Sat Feb  4 05:42:46 2023
From: research at johnohagan.com (John O'Hagan)
Date: Sat, 04 Feb 2023 21:42:46 +1100
Subject: tkinter ttk.Treeview:  changing background colour of single item
 when selected
Message-ID: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>

Hi list

I'm using a ttk Treeview to display a hierarchical data structure. When
an error condition arises in a node, I want the corresponding item in
the Treeview to flash its background color.?

Using a tag to flash the item background works, except when the item is
selected, when the tag has no effect. I want the item to keep flashing
even when selected.

The code below is a minimal example of the issue. It displays a
Treeview containing two items with the first one flashing. If you
select it, you don't see the flashing anymore.

from tkinter import *
from tkinter.ttk import *

root = Tk()
t = Treeview(root)

t.insert('', 0, iid='item1', text='item1')
t.insert('', 1, text='item2')
t.tag_configure('flashtag', background='red')
t.pack()

def flash():
 tags = t.item('item1', 'tags')
 t.item('item1', tags='' if tags else 'flashtag')
 t.after(500, flash)

flash()
mainloop()

Other than tags, the only other way I've found to dynamically change
Treeview backgrounds is using styles, in particular Style.map to
configure the background of a selected item, but they apply to all
items in the Treeview and would flash all selected items, not just the
one I want to keep flashing.

Is there another way to do what I want?

Thanks

--

John


From gweatherby at uchc.edu  Sat Feb  4 06:17:46 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Sat, 4 Feb 2023 11:17:46 +0000
Subject: Organizing modules and their code
In-Reply-To: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
Message-ID: <SJ0PR14MB586488F6E3BA6E3394CD5C28B9D49@SJ0PR14MB5864.namprd14.prod.outlook.com>

You?re overthinking it. It doesn?t really matter. Having small chunks of codes in separate files can be hassle when trying to find out what the program does. Having one file with 2,000 lines in it can be a hassle.  This is art / opinion, not science.

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of transreductionist <transreductionist at gmail.com>
Date: Friday, February 3, 2023 at 4:48 PM
To: python-list at python.org <python-list at python.org>
Subject: Organizing modules and their code
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Here is the situation. There is a top-level module (see designs below) containing code, that as the name suggests, manages an ETL pipeline. A directory is created called etl_helpers that organizes several modules responsible for making up the pipeline. The discussion concerns the Python language, which supports OOP as well as Structural/Functional approaches to programming.

I am interested in opinions on which design adheres best to standard architectural practices and the SOLID principles. I understand that this is one of those topics where people may have strong opinions one way or the other. I am interested in those opinions.

Allow me to give my thoughts. First, I don't think there would be much difference if I was using OOP for the functionality, or using a structural paradigm. A structural paradigm in my opinion, along the lines of Rich Hickey's comments on simple versus complex, would be a simpler implementation. In this case there is no reason to create a construct with state. So let's assume the code is structural and not OOP.

I would go with Design I. Succinctly stated, Design I supports readability and maintainability at least as well, if not better than the other designs. The goal of the SOLID principles are the creation of mid-level software structures that (Software Architecture: SA Martin). I think Design I best adheres to these principles of:
---- Tolerate change,
---- Are easy to understand, and
---- Are the basis of components that can be used in many software systems.

I could point to the Single Responsibility Principle which is defined as (SA Martin): a module should be responsible to one, and only one, actor. It should satisfy the Liskov Substitution Principle as well. Further, each module in the etl_helpers directory is at the same level of abstraction.

I could also mention that as Dijkstra stressed, at every level, from the smallest function to the largest component, software is like a science and, therefore, is driven by falsifiability. Software architects strive to define modules, components, and services that are easily falsifiable (testable). To do so, they employ restrictive disciplines similar to structured programming,
albeit at a much higher level (SA Martin).

One can point to multiple reasons why Design I might be preferred, but what are the compelling reasons, if there are any, that would suggest another design was superior.

Finally, let me reference an interesting research paper I read recently that seems to support the other designs as anti-patterns: Architecture_Anti-patterns_Automatically.pdf

 ---- (https://urldefense.com/v3/__https://www.cs.drexel.edu/*yfcai/papers/2019/tse2019.pdf__;fg!!Cn_UX_p3!jcpCdxiLoPobR0IdlyJHwyPiNP4_iVC6dAMtg_HsLr5hStszx-WnYyZQHJ-4pJTOGsw4-6pEGJyDpSytZQqfpvATg06FMA$ )

SEVERAL DESIGNS FOR COMPARISON

DESIGN I:

---- manage_the_etl_pipeline.py
---- etl_helpers
      ---- extract.py
      ---- transform.py
      ---- load.py

Of course one could also

DESIGN II:

---- manage_the_etl_pipeline.py
---- etl_helpers
      ---- extract_transform_load.py

or probably even:

DESIGN III:

---- manage_the_etl_pipeline.py
---- extract_transform_load.py
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jcpCdxiLoPobR0IdlyJHwyPiNP4_iVC6dAMtg_HsLr5hStszx-WnYyZQHJ-4pJTOGsw4-6pEGJyDpSytZQqfpvBaJ2e2VA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jcpCdxiLoPobR0IdlyJHwyPiNP4_iVC6dAMtg_HsLr5hStszx-WnYyZQHJ-4pJTOGsw4-6pEGJyDpSytZQqfpvBaJ2e2VA$>

From list1 at tompassin.net  Sat Feb  4 11:25:09 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 4 Feb 2023 11:25:09 -0500
Subject: tkinter ttk.Treeview: changing background colour of single item
 when selected
In-Reply-To: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
References: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
Message-ID: <c19f8551-0af3-9e7d-1653-570b12b1760e@tompassin.net>

I haven't worked specifically with a Treeview, but I think you need to 
detect the onClick event (or an onSelect event if there is one) and have 
that trigger the flashing.  Otherwise the selection probably overwrites 
styling that you added. That's what I do for classic Tk buttons to make 
them flash when selected.  (I only flash once, though; I don't keep them 
flashing).

On 2/4/2023 5:42 AM, John O'Hagan wrote:
> Hi list
> 
> I'm using a ttk Treeview to display a hierarchical data structure. When
> an error condition arises in a node, I want the corresponding item in
> the Treeview to flash its background color.
> 
> Using a tag to flash the item background works, except when the item is
> selected, when the tag has no effect. I want the item to keep flashing
> even when selected.
> 
> The code below is a minimal example of the issue. It displays a
> Treeview containing two items with the first one flashing. If you
> select it, you don't see the flashing anymore.
> 
> from tkinter import *
> from tkinter.ttk import *
> 
> root = Tk()
> t = Treeview(root)
> 
> t.insert('', 0, iid='item1', text='item1')
> t.insert('', 1, text='item2')
> t.tag_configure('flashtag', background='red')
> t.pack()
> 
> def flash():
>   tags = t.item('item1', 'tags')
>   t.item('item1', tags='' if tags else 'flashtag')
>   t.after(500, flash)
> 
> flash()
> mainloop()
> 
> Other than tags, the only other way I've found to dynamically change
> Treeview backgrounds is using styles, in particular Style.map to
> configure the background of a selected item, but they apply to all
> items in the Treeview and would flash all selected items, not just the
> one I want to keep flashing.
> 
> Is there another way to do what I want?
> 
> Thanks
> 
> --
> 
> John
> 


From transreductionist at gmail.com  Fri Feb  3 19:08:36 2023
From: transreductionist at gmail.com (transreductionist)
Date: Fri, 3 Feb 2023 16:08:36 -0800 (PST)
Subject: Organizing modules and their code
In-Reply-To: <mailman.1755.1675463492.20444.python-list@python.org>
References: <bccfa720-6b02-53e1-d451-94b8f150e4b7@tompassin.net>
 <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
 <mailman.1755.1675463492.20444.python-list@python.org>
Message-ID: <319a77e6-6a3a-4dbf-9a26-a511a717fa1fn@googlegroups.com>

On Friday, February 3, 2023 at 5:31:56 PM UTC-5, Thomas Passin wrote:
> On 2/3/2023 4:18 PM, transreductionist wrote: 
> > Here is the situation. There is a top-level module (see designs below) containing code, that as the name suggests, manages an ETL pipeline. A directory is created called etl_helpers that organizes several modules responsible for making up the pipeline. The discussion concerns the Python language, which supports OOP as well as Structural/Functional approaches to programming. 
> > 
> > I am interested in opinions on which design adheres best to standard architectural practices and the SOLID principles. I understand that this is one of those topics where people may have strong opinions one way or the other. I am interested in those opinions.
> Well, you have pretty well stacked the deck to make DESIGN 1 the 
> obviously preferred choice. I don't think it has much to do with Python 
> per se, or even with OO vs imperative style. 
> 
> As a practical matter, once you got into working with 
> extract_transform_load.py (for the other designs), I would expect that 
> you would start wanting to refactor it and eventually end up more like 
> DESIGN 1. So you might as well start out that way. 
> 
> The reasons are 1) what you said about separation of concerns, 2) a 
> desire to keep each module or file relatively coherent and easy to read, 
> and 3, as you also suggested, making each of them easier to test. 
> Decoupling is important too (one of the SOLID prescriptions), but you 
> can violate that with any architecture if you don't think carefully 
> about what you are doing. 
> 
> On the subject of OO, I think it is a very good approach to think about 
> architecture and design in object terms - meaning conceptual objects 
> from the users' point of view. For example, here you have a pipeline (a 
> metaphorical or userland object). It will need functionality to load, 
> transform, and output data so logically it can be composed of a loader, 
> one or more transformers, and one or more output formatters (more 
> objects). You may also need a scheduler and a configuration manager 
> (more objects). 
> 
> (*Please* let's not have any quibbling about "class" vs "object". We 
> are at a conceptual level here!) 
> 
> When it comes to implementation, you can choose to implement those 
> userland objects with either imperative, OO, or functional techniques, 
> or a mixture.
> > Allow me to give my thoughts. First, I don't think there would be much difference if I was using OOP for the functionality, or using a structural paradigm. A structural paradigm in my opinion, along the lines of Rich Hickey's comments on simple versus complex, would be a simpler implementation. In this case there is no reason to create a construct with state. So let's assume the code is structural and not OOP. 
> > 
> > I would go with Design I. Succinctly stated, Design I supports readability and maintainability at least as well, if not better than the other designs. The goal of the SOLID principles are the creation of mid-level software structures that (Software Architecture: SA Martin). I think Design I best adheres to these principles of: 
> > ---- Tolerate change, 
> > ---- Are easy to understand, and 
> > ---- Are the basis of components that can be used in many software systems. 
> > 
> > I could point to the Single Responsibility Principle which is defined as (SA Martin): a module should be responsible to one, and only one, actor. It should satisfy the Liskov Substitution Principle as well. Further, each module in the etl_helpers directory is at the same level of abstraction. 
> > 
> > I could also mention that as Dijkstra stressed, at every level, from the smallest function to the largest component, software is like a science and, therefore, is driven by falsifiability. Software architects strive to define modules, components, and services that are easily falsifiable (testable). To do so, they employ restrictive disciplines similar to structured programming, 
> > albeit at a much higher level (SA Martin). 
> > 
> > One can point to multiple reasons why Design I might be preferred, but what are the compelling reasons, if there are any, that would suggest another design was superior. 
> > 
> > Finally, let me reference an interesting research paper I read recently that seems to support the other designs as anti-patterns: Architecture_Anti-patterns_Automatically.pdf 
> > 
> > ---- (https://www.cs.drexel.edu/~yfcai/papers/2019/tse2019.pdf) 
> > 
> > SEVERAL DESIGNS FOR COMPARISON 
> > 
> > DESIGN I: 
> > 
> > ---- manage_the_etl_pipeline.py 
> > ---- etl_helpers 
> > ---- extract.py 
> > ---- transform.py 
> > ---- load.py 
> > 
> > Of course one could also 
> > 
> > DESIGN II: 
> > 
> > ---- manage_the_etl_pipeline.py 
> > ---- etl_helpers 
> > ---- extract_transform_load.py 
> > 
> > or probably even: 
> > 
> > DESIGN III: 
> > 
> > ---- manage_the_etl_pipeline.py 
> > ---- extract_transform_load.py


On point that I think is worth making ,and I forgot to make it, is that namespaces are ubiquitous in Python: Built-in, Global,  Function, and Enclosing namespaces, as well as user namespaces, e.g. dictionaries, the SimpleNamespace, and DataClasses to list just a few. Modules ARE namespaces. Namespaces organize programming constructs like classes, functions, variables, etc. into coherent groups of "things". To have a namespace that complects extract constructs with transform constructs, and load constructs in one module seems un-pythonistic.

From gabriele1NOSPAM at hotmail.com  Sat Feb  4 10:05:18 2023
From: gabriele1NOSPAM at hotmail.com (^Bart)
Date: Sat, 4 Feb 2023 16:05:18 +0100
Subject: How to read file content and send email on Debian Bullseye
Message-ID: <trls7e$21467$1@dont-email.me>

Hi guys,

On a Debian Bullseye server I have a lftp upload and after it I should 
send an email.

I thought to read the lftp log file where I have these lines:

2023-01-30 18:30:02 
/home/my_user/local_folder/upload/my_file_30-01-2023_18-30.txt -> 
sftp://ftp_user at ftpserver_ip:2201/remote_ftp_folder/my_file_30-01-2023_18-30.txt 
0-1660576 4.92 MiB/s

2023-02-02 18:30:02 
/home/my_user/local_folder/upload/my_file_02-02-2023_18-30.txt -> 
sftp://ftp_user at ftpserver_ip:2201/remote_ftp_folder/my_file_02-02-2023_18-30.txt 
0-603093 3.39 MiB/s

I'd like to use Python to check, from monday to friday (the lftp script 
runs in crontab from monday to friday) when the upload works is finished 
and I should send an email.

I could read by Python lftp.log and after it if there's a line with the 
same day of the machine I could send an email with ok otherwise the 
email will send a message with "no upload".

How could I do by Python?

Regards.
^Bart

From cl at isbd.net  Sat Feb  4 10:17:38 2023
From: cl at isbd.net (Chris Green)
Date: Sat, 4 Feb 2023 15:17:38 +0000
Subject: DeprecationWarning but replacement doesn't work
Message-ID: <ih92bj-knv02.ln1@esprimo.zbmc.eu>

I am using Image from PIL and I'm getting a deprecation warning as
follows:-

/home/chris/bin/picShrink.py:80: DeprecationWarning: ANTIALIAS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.


The code is very simple:-

    ...
    ...
    from PIL import Image
    ...
    ...
        im = Image.open(srcFile)
        im.thumbnail(size, Image.ANTIALIAS)
        im.save(dstFile, "JPEG")


But if I change line 80 to:-

        im.thumbnail(size, Resampling.LANCZOS)

I simply get an error as follows:-

    picShrink.py
    Traceback (most recent call last):
      File "/home/chris/bin/picShrink.py", line 80, in <module>
        im.thumbnail(size, Resampling.LANCZOS)
    NameError: name 'Resampling' is not defined


So, presumably there's more I need to change.  Where can I find out what
I need to do?

-- 
Chris Green
?

From rosuav at gmail.com  Sat Feb  4 15:59:27 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 5 Feb 2023 07:59:27 +1100
Subject: DeprecationWarning but replacement doesn't work
In-Reply-To: <ih92bj-knv02.ln1@esprimo.zbmc.eu>
References: <ih92bj-knv02.ln1@esprimo.zbmc.eu>
Message-ID: <CAPTjJmqDyka2P78xquhz_HMwyVqvOBy6+dyz60VqqSfSRxk-=w@mail.gmail.com>

On Sun, 5 Feb 2023 at 07:52, Chris Green <cl at isbd.net> wrote:
>
> I am using Image from PIL and I'm getting a deprecation warning as
> follows:-
>
> /home/chris/bin/picShrink.py:80: DeprecationWarning: ANTIALIAS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
>
>
>         im = Image.open(srcFile)
>         im.thumbnail(size, Image.ANTIALIAS)
>         im.save(dstFile, "JPEG")
>
>         im.thumbnail(size, Resampling.LANCZOS)
>
> I simply get an error as follows:-
>
>     picShrink.py
>     Traceback (most recent call last):
>       File "/home/chris/bin/picShrink.py", line 80, in <module>
>         im.thumbnail(size, Resampling.LANCZOS)
>     NameError: name 'Resampling' is not defined
>

It looks like Resampling is an enumeration. The warning said
"ANTIALIAS" as a bare name, and you were using Image.ANTIALIAS; it
looks like Image.Resampling.LANCZOS has the same effect.

In fact, it looks like they're actually the same value:

>>> Image.ANTIALIAS is Image.Resampling.LANCZOS
<stdin>:1: DeprecationWarning: ANTIALIAS is deprecated and will be
removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
True

So that should be a safe move forward.

ChrisA

From roel at roelschroeven.net  Sat Feb  4 16:06:59 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Sat, 4 Feb 2023 22:06:59 +0100
Subject: DeprecationWarning but replacement doesn't work
In-Reply-To: <ih92bj-knv02.ln1@esprimo.zbmc.eu>
References: <ih92bj-knv02.ln1@esprimo.zbmc.eu>
Message-ID: <feedbf55-ffe0-0c5c-6f10-87e211534c53@roelschroeven.net>

Chris Green schreef op 4/02/2023 om 16:17:
> I am using Image from PIL and I'm getting a deprecation warning as
> follows:-
>
> /home/chris/bin/picShrink.py:80: DeprecationWarning: ANTIALIAS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.
>
>
> But if I change line 80 to:-
>
>          im.thumbnail(size, Resampling.LANCZOS)
>
> I simply get an error as follows:-
>
>      picShrink.py
>      Traceback (most recent call last):
>        File "/home/chris/bin/picShrink.py", line 80, in <module>
>          im.thumbnail(size, Resampling.LANCZOS)
>      NameError: name 'Resampling' is not defined
>
>
> So, presumably there's more I need to change.  Where can I find out what
> I need to do?
There's some information about changes in constants in the release notes 
for version 9.1.0, see 
https://pillow.readthedocs.io/en/stable/releasenotes/9.1.0.html. At 
first sight it looks like that indicates you have to do either

 ??? from PIL import Image.Resampling as Resampling
 ??? ...
 ??? im.thumbnail(size, Resampling.LANCZOS)

or

 ??? from PIL import Image.Resampling
 ??? ...
 ??? im.thumbnail(size, Image.Resampling.LANCZOS)

BUT according to the release notes of version 9.4.0 (see 
https://pillow.readthedocs.io/en/stable/releasenotes/9.4.0.html#restored-image-constants) 
the constants in Image will remain (it's not clear to me if the ones in 
Image.Resample will remain too). As far as I can see ANTIALIAS is still 
deprecated in favor of LANCZOS though. All in all, it looks like the way 
to go is to use Image.LANCZOS. In versions 9.1.0 up to but not including 
9.4.0 that will give you a deprecation warning but that can safely be 
ignored.

-- 
"Life ain't no fairy tale
Just give me another ale
And I'll drink to Rock 'n Roll"
         -- Barkeep (The Scabs)


From avi.e.gross at gmail.com  Sat Feb  4 17:10:19 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sat, 4 Feb 2023 17:10:19 -0500
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <trls7e$21467$1@dont-email.me>
References: <trls7e$21467$1@dont-email.me>
Message-ID: <00ad01d938e5$7836e4f0$68a4aed0$@gmail.com>

Bart, you may want to narrow down your request to something quite specific.
For example, try to do whatever parts you know how to do and when some part
fails or is missing, ask.

I might have replied to you directly if your email email address did not
look like you want no SPAM, LOL!

The cron stuff is not really relevant and it seems your idea is to read a
part or all of a log file, parse the lines in some way and find a line that
either matches what you need or fail to find it. Either way you want to send
an email out with an appropriate content.

Which part of that do you not know how to do in python? Have you done some
reading or looking?



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of ^Bart
Sent: Saturday, February 4, 2023 10:05 AM
To: python-list at python.org
Subject: How to read file content and send email on Debian Bullseye

Hi guys,

On a Debian Bullseye server I have a lftp upload and after it I should send
an email.

I thought to read the lftp log file where I have these lines:

2023-01-30 18:30:02
/home/my_user/local_folder/upload/my_file_30-01-2023_18-30.txt ->
sftp://ftp_user at ftpserver_ip:2201/remote_ftp_folder/my_file_30-01-2023_18-30
.txt
0-1660576 4.92 MiB/s

2023-02-02 18:30:02
/home/my_user/local_folder/upload/my_file_02-02-2023_18-30.txt ->
sftp://ftp_user at ftpserver_ip:2201/remote_ftp_folder/my_file_02-02-2023_18-30
.txt
0-603093 3.39 MiB/s

I'd like to use Python to check, from monday to friday (the lftp script runs
in crontab from monday to friday) when the upload works is finished and I
should send an email.

I could read by Python lftp.log and after it if there's a line with the same
day of the machine I could send an email with ok otherwise the email will
send a message with "no upload".

How could I do by Python?

Regards.
^Bart
--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Sat Feb  4 17:59:11 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 4 Feb 2023 17:59:11 -0500
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <trls7e$21467$1@dont-email.me>
References: <trls7e$21467$1@dont-email.me>
Message-ID: <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>

On 2/4/2023 10:05 AM, ^Bart wrote:
> Hi guys,
> 
> On a Debian Bullseye server I have a lftp upload and after it I should 
> send an email.
> 
> I thought to read the lftp log file where I have these lines:
> 
> 2023-01-30 18:30:02 
> /home/my_user/local_folder/upload/my_file_30-01-2023_18-30.txt -> 
> sftp://ftp_user at ftpserver_ip:2201/remote_ftp_folder/my_file_30-01-2023_18-30.txt 0-1660576 4.92 MiB/s
> 
> 2023-02-02 18:30:02 
> /home/my_user/local_folder/upload/my_file_02-02-2023_18-30.txt -> 
> sftp://ftp_user at ftpserver_ip:2201/remote_ftp_folder/my_file_02-02-2023_18-30.txt 0-603093 3.39 MiB/s
> 
> I'd like to use Python to check, from monday to friday (the lftp script 
> runs in crontab from monday to friday) when the upload works is finished 
> and I should send an email.
> 
> I could read by Python lftp.log and after it if there's a line with the 
> same day of the machine I could send an email with ok otherwise the 
> email will send a message with "no upload".
> 
> How could I do by Python?

Not Python, but you could run a shell script that sends the file with 
lftp, and depending on the return code uses xdg-email to send one of the 
two messages. This would probably be simpler than using Python.

Otherwise you haven't said what part of the process you need help with. 
Reading the log file?  Checking the date?  Triggering the Python script? 
  Sending the email message with Python?

For the latter, it's covered in the Python docs -

https://docs.python.org/3.10/library/email.examples.html



From greg.ewing at canterbury.ac.nz  Sat Feb  4 19:26:53 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 5 Feb 2023 13:26:53 +1300
Subject: Organizing modules and their code
In-Reply-To: <0edf1674-4a11-4686-aee4-2774eaabfa43n@googlegroups.com>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
 <0edf1674-4a11-4686-aee4-2774eaabfa43n@googlegroups.com>
Message-ID: <k48begF6t8sU1@mid.individual.net>

On 5/02/23 11:18 am, transreductionist wrote:
> This analogy came to me the other day. For me, I would rather walk into a grocery store where the bananas, apples, and oranges are separated in to their own bins, instead of one common crate.

On the other hand, if the store has an entire aisle devoted to each
fruit, but only ever one crate of fruit in each aisle, one would think
they could make better use of their shelf space.

-- 
Greg

From transreductionist at gmail.com  Sat Feb  4 17:18:35 2023
From: transreductionist at gmail.com (transreductionist)
Date: Sat, 4 Feb 2023 14:18:35 -0800 (PST)
Subject: Organizing modules and their code
In-Reply-To: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
Message-ID: <0edf1674-4a11-4686-aee4-2774eaabfa43n@googlegroups.com>

Thank you for all the helpful replies and consideration. I do hope for other opinions

I would rather say it is more like engineering than art. Whether it is a matter of overthinking, or under thinking, is another matter. I enjoyed Dijkstra's letter to his colleagues on the role of scientific thought in computer programming. It is located at:

---- https://www.cs.utexas.edu/users/EWD/transcriptions/EWD04xx/EWD447.html 

It is my academic training in physics that makes me enjoy picking up an idea and examining  it from all sides, and sharing thoughts with friends. Just inquisitive curiosity, and not a homework problem,. Thanks for the great link to the ETL site. That was a good read. A few years back I built a prod ETL application in Golang using gRPC with a multiprocessing pipeline throughout. It handled GB of data and was fast.

This analogy came to me the other day. For me, I would rather walk into a grocery store where the bananas, apples, and oranges are separated in to their own bins, instead of one common crate.


On Friday, February 3, 2023 at 4:18:57 PM UTC-5, transreductionist wrote:
> Here is the situation. There is a top-level module (see designs below) containing code, that as the name suggests, manages an ETL pipeline. A directory is created called etl_helpers that organizes several modules responsible for making up the pipeline. The discussion concerns the Python language, which supports OOP as well as Structural/Functional approaches to programming. 
> 
> I am interested in opinions on which design adheres best to standard architectural practices and the SOLID principles. I understand that this is one of those topics where people may have strong opinions one way or the other. I am interested in those opinions. 
> 
> Allow me to give my thoughts. First, I don't think there would be much difference if I was using OOP for the functionality, or using a structural paradigm. A structural paradigm in my opinion, along the lines of Rich Hickey's comments on simple versus complex, would be a simpler implementation. In this case there is no reason to create a construct with state. So let's assume the code is structural and not OOP. 
> 
> I would go with Design I. Succinctly stated, Design I supports readability and maintainability at least as well, if not better than the other designs. The goal of the SOLID principles are the creation of mid-level software structures that (Software Architecture: SA Martin). I think Design I best adheres to these principles of: 
> ---- Tolerate change, 
> ---- Are easy to understand, and 
> ---- Are the basis of components that can be used in many software systems. 
> 
> I could point to the Single Responsibility Principle which is defined as (SA Martin): a module should be responsible to one, and only one, actor. It should satisfy the Liskov Substitution Principle as well. Further, each module in the etl_helpers directory is at the same level of abstraction. 
> 
> I could also mention that as Dijkstra stressed, at every level, from the smallest function to the largest component, software is like a science and, therefore, is driven by falsifiability. Software architects strive to define modules, components, and services that are easily falsifiable (testable). To do so, they employ restrictive disciplines similar to structured programming, 
> albeit at a much higher level (SA Martin). 
> 
> One can point to multiple reasons why Design I might be preferred, but what are the compelling reasons, if there are any, that would suggest another design was superior. 
> 
> Finally, let me reference an interesting research paper I read recently that seems to support the other designs as anti-patterns: Architecture_Anti-patterns_Automatically.pdf 
> 
> ---- (https://www.cs.drexel.edu/~yfcai/papers/2019/tse2019.pdf) 
> 
> SEVERAL DESIGNS FOR COMPARISON 
> 
> DESIGN I: 
> 
> ---- manage_the_etl_pipeline.py 
> ---- etl_helpers 
> ---- extract.py 
> ---- transform.py 
> ---- load.py 
> 
> Of course one could also 
> 
> DESIGN II: 
> 
> ---- manage_the_etl_pipeline.py 
> ---- etl_helpers 
> ---- extract_transform_load.py 
> 
> or probably even: 
> 
> DESIGN III: 
> 
> ---- manage_the_etl_pipeline.py 
> ---- extract_transform_load.py

From PythonList at DancesWithMice.info  Sat Feb  4 21:30:18 2023
From: PythonList at DancesWithMice.info (dn)
Date: Sun, 5 Feb 2023 15:30:18 +1300
Subject: Typing Number, PyCharm
Message-ID: <e818abc9-a531-67e1-1aca-d6770bca2c03@DancesWithMice.info>

Do we have a typing type-hint for numbers yet?


Often wanting to combine int and float, discovered that an application 
was doing a walk-through with/for uses three numeric types. Was 
intrigued to note variance, in that the code-set features two different 
methods for typing, in this situation:

def func( value ):
     ...using value...

where value may be an integer, a floating-point value, or a 
complex-number (but not decimal-type).
NB code snippets from memory (cf copy-paste)


Method 1 (possibly older code):-

from typing import Union
...
def fun( value:Union[ int, float, complex ] ):


Method 2:-

def fun( value:int|float|complex  ):


Pondering this, realised could use an alias to de-clutter the 
function-definition/signature:

Method 3:-

number_type = int|float|complex
...
def fun( value:number_type  ):


If it was important to have type consistency within the union, eg 
argument and return, could go for:

Method 4:-

from typing import TypeVar
number_type = TypeVar( 'number_type', int, float, complex )
...
def fun( value:number_type  ):


Then remembered the way we'd code an execution-time check for this using 
isinstance():

Method 5:-

from numbers import Number
...
def fun( value:Number  ):


Each of these will execute correctly.

All cause PyCharm to object if I try to call the fun(ction) with a 
string parameter - and execute an exception, as expected.


Accepting all the others, am curious as to why PyCharm objects to Method 
5 with "Expected type 'SupportsFloat | SupportsComplex | complex | 
SupportsIndex', got 'Number' instead? - yet still highlights the 
erroneous string parameter but none of the 'legal' data-types?

As soon as a list (in this case types) reaches three, my aged-eyes start 
to think de-cluttering is a good idea!

Do you know of another way to attack this/more properly?

-- 
Regards,
=dn

From 2QdxY4RzWzUUiLuE at potatochowder.com  Sat Feb  4 22:13:38 2023
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Sat, 4 Feb 2023 22:13:38 -0500
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
References: <trls7e$21467$1@dont-email.me>
 <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
Message-ID: <Y98e4oXIv1A6euCk@anomaly>

On 2023-02-04 at 17:59:11 -0500,
Thomas Passin <list1 at tompassin.net> wrote:

> On 2/4/2023 10:05 AM, ^Bart wrote:
> > Hi guys,
> > 
> > On a Debian Bullseye server I have a lftp upload and after it I should
> > send an email.

[...]

> [...] you could run a shell script that sends the file with lftp, and
> depending on the return code uses xdg-email to send one of the two
> messages. This would probably be simpler than using Python.

xdg-email appears to be for interactive use (it opens the user's
"preferred email composer"); I think sendmail would work much better
from a script.

Otherwise, I had the same initial thought, to add to and/or build a
wrapper around the existing lftp script.

From list1 at tompassin.net  Sat Feb  4 23:49:19 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 4 Feb 2023 23:49:19 -0500
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <Y98e4oXIv1A6euCk@anomaly>
References: <trls7e$21467$1@dont-email.me>
 <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
 <Y98e4oXIv1A6euCk@anomaly>
Message-ID: <b0fe549d-2ddf-aa2e-80b5-6844045d2909@tompassin.net>

On 2/4/2023 10:13 PM, 2QdxY4RzWzUUiLuE at potatochowder.com wrote:
> On 2023-02-04 at 17:59:11 -0500,
> Thomas Passin <list1 at tompassin.net> wrote:
> 
>> On 2/4/2023 10:05 AM, ^Bart wrote:
>>> Hi guys,
>>>
>>> On a Debian Bullseye server I have a lftp upload and after it I should
>>> send an email.
> 
> [...]
> 
>> [...] you could run a shell script that sends the file with lftp, and
>> depending on the return code uses xdg-email to send one of the two
>> messages. This would probably be simpler than using Python.
> 
> xdg-email appears to be for interactive use (it opens the user's
> "preferred email composer"); I think sendmail would work much better
> from a script.
> 
> Otherwise, I had the same initial thought, to add to and/or build a
> wrapper around the existing lftp script.

I have never used it, and I did see that wording in the man page, but it 
also showed putting all the info on the command line - from, to, 
subject, etc - so I thought it might be able to use a command line 
client without needed any GUI interaction.  But I don't know for sure.


From cl at isbd.net  Sun Feb  5 01:42:38 2023
From: cl at isbd.net (Chris Green)
Date: Sun, 5 Feb 2023 06:42:38 +0000
Subject: DeprecationWarning but replacement doesn't work
References: <ih92bj-knv02.ln1@esprimo.zbmc.eu>
 <feedbf55-ffe0-0c5c-6f10-87e211534c53@roelschroeven.net>
 <mailman.1763.1675544825.20444.python-list@python.org>
Message-ID: <unv3bj-2ac42.ln1@esprimo.zbmc.eu>

Roel Schroeven <roel at roelschroeven.net> wrote:
> Chris Green schreef op 4/02/2023 om 16:17:
> > I am using Image from PIL and I'm getting a deprecation warning as
> > follows:-
> >
> > /home/chris/bin/picShrink.py:80: DeprecationWarning: ANTIALIAS is deprecated 
> and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead. 
> 
> >
> >
> > But if I change line 80 to:-
> >
> >          im.thumbnail(size, Resampling.LANCZOS)
> >
> > I simply get an error as follows:-
> >
> >      picShrink.py
> >      Traceback (most recent call last):
> >        File "/home/chris/bin/picShrink.py", line 80, in <module>
> >          im.thumbnail(size, Resampling.LANCZOS)
> >      NameError: name 'Resampling' is not defined
> >
> >
> > So, presumably there's more I need to change.  Where can I find out what
> > I need to do?
> There's some information about changes in constants in the release notes 
> for version 9.1.0, see 
> https://pillow.readthedocs.io/en/stable/releasenotes/9.1.0.html. At 
> first sight it looks like that indicates you have to do either
> 
> ??? from PIL import Image.Resampling as Resampling
> ??? ...
> ??? im.thumbnail(size, Resampling.LANCZOS)
> 
> or
> 
> ??? from PIL import Image.Resampling
> ??? ...
> ??? im.thumbnail(size, Image.Resampling.LANCZOS)
> 
> BUT according to the release notes of version 9.4.0 (see 
> https://pillow.readthedocs.io/en/stable/releasenotes/9.4.0.html#restored-image-constants) 
> the constants in Image will remain (it's not clear to me if the ones in 
> Image.Resample will remain too). As far as I can see ANTIALIAS is still 
> deprecated in favor of LANCZOS though. All in all, it looks like the way 
> to go is to use Image.LANCZOS. In versions 9.1.0 up to but not including 
> 9.4.0 that will give you a deprecation warning but that can safely be 
> ignored.
> 
Thank you Roel (and Chris in the previous reply) for your helpful
explanations, it's all cleared up now.

-- 
Chris Green
?

From hjp-python at hjp.at  Sun Feb  5 09:13:49 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sun, 5 Feb 2023 15:13:49 +0100
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
References: <trls7e$21467$1@dont-email.me>
 <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
Message-ID: <20230205141349.5lf4ainamwm3utl5@hjp.at>

On 2023-02-04 17:59:11 -0500, Thomas Passin wrote:
> On 2/4/2023 10:05 AM, ^Bart wrote:
> > On a Debian Bullseye server I have a lftp upload and after it I should
> > send an email.
> > 
> > I thought to read the lftp log file where I have these lines:
[...]
> > I'd like to use Python to check, from monday to friday (the lftp script
> > runs in crontab from monday to friday) when the upload works is finished
> > and I should send an email.
> > 
> > I could read by Python lftp.log and after it if there's a line with the
> > same day of the machine I could send an email with ok otherwise the
> > email will send a message with "no upload".
> > 
> > How could I do by Python?
> 
> Not Python, but you could run a shell script

If you can run a shell script you can also run a python script.
Linux/Unix is almost completely agnostic about interpreters. You
generally just invoke "the program" and then the kernel examines the
first few bytes to determine what to do with it.

> that sends the file with lftp, and depending on the return code uses
> xdg-email to send one of the two messages. This would probably be
> simpler than using Python.

Probably. But Python gives you more control. Tradeoffs, tradeoffs ...

For something like this I'd say go with what you're most familiar with
and what takes the least effort to get the job done.
For a simple one line notification I'd write a shell script invoking
/usr/sbin/sendmail. If I wanted it to be a bit fancier (e.g. a table
with the uploads over the last few days) I'd probably write a Python
script to get the MIME/HTML structure right (and still use sendmail to
actually send the mail).

xdg-mail can be anything or not be configured at all - so it might not
be the best choice.

> Otherwise you haven't said what part of the process you need help with.

Yeah, that was the reason I didn't reply to the original mail. I simply
couldn't figure out where the problem was.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230205/10adc539/attachment.sig>

From gweatherby at uchc.edu  Sun Feb  5 10:23:20 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Sun, 5 Feb 2023 15:23:20 +0000
Subject: Organizing modules and their code
In-Reply-To: <0edf1674-4a11-4686-aee4-2774eaabfa43n@googlegroups.com>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
 <0edf1674-4a11-4686-aee4-2774eaabfa43n@googlegroups.com>
Message-ID: <SJ0PR14MB586461F3E3E32B2417877C96B9D59@SJ0PR14MB5864.namprd14.prod.outlook.com>

Well, first of all, while there is no doubt as to Dijkstra?s contribution to computer science, I don?t think his description of scientific thought is correct. The acceptance of Einstein?s theory of relativity has nothing to do with internal consistency or how easy or difficult to explain but rather repeatedly experimental results validating it. Or, more precisely, not disproving it. See Feynmann: https://www.youtube.com/watch?v=0KmimDq4cSU


Engineering is simply maximizing the ratio: benefit / cost. Highly recommend To Engineer is Human  by Henry Petroski.

Regarding the initial question: none of the suggested designs would work because they lack __init__.py file.

Once the __init__.py is added, the construct of the import statements within it will determine how the API looks. All three of Design I / Design II and Design III can be implemented with the same API. (I?m pretty sure that?s true. If it?s not, I?d be interested in a counterexample).





From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of transreductionist <transreductionist at gmail.com>
Date: Saturday, February 4, 2023 at 7:42 PM
To: python-list at python.org <python-list at python.org>
Subject: Re: Organizing modules and their code
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Thank you for all the helpful replies and consideration. I do hope for other opinions

I would rather say it is more like engineering than art. Whether it is a matter of overthinking, or under thinking, is another matter. I enjoyed Dijkstra's letter to his colleagues on the role of scientific thought in computer programming. It is located at:

---- https://urldefense.com/v3/__https://www.cs.utexas.edu/users/EWD/transcriptions/EWD04xx/EWD447.html__;!!Cn_UX_p3!nME8OhiOxAzmzM3jzg6uXZU851dhWWD9JGB8ZRZIzyUzGkmCN-C6SSXrL59eA2KVIh-y-W0VycJSNb8aYcNnc3jd5Pi2fw$<https://urldefense.com/v3/__https:/www.cs.utexas.edu/users/EWD/transcriptions/EWD04xx/EWD447.html__;!!Cn_UX_p3!nME8OhiOxAzmzM3jzg6uXZU851dhWWD9JGB8ZRZIzyUzGkmCN-C6SSXrL59eA2KVIh-y-W0VycJSNb8aYcNnc3jd5Pi2fw$>

It is my academic training in physics that makes me enjoy picking up an idea and examining  it from all sides, and sharing thoughts with friends. Just inquisitive curiosity, and not a homework problem,. Thanks for the great link to the ETL site. That was a good read. A few years back I built a prod ETL application in Golang using gRPC with a multiprocessing pipeline throughout. It handled GB of data and was fast.

This analogy came to me the other day. For me, I would rather walk into a grocery store where the bananas, apples, and oranges are separated in to their own bins, instead of one common crate.


On Friday, February 3, 2023 at 4:18:57 PM UTC-5, transreductionist wrote:
> Here is the situation. There is a top-level module (see designs below) containing code, that as the name suggests, manages an ETL pipeline. A directory is created called etl_helpers that organizes several modules responsible for making up the pipeline. The discussion concerns the Python language, which supports OOP as well as Structural/Functional approaches to programming.
>
> I am interested in opinions on which design adheres best to standard architectural practices and the SOLID principles. I understand that this is one of those topics where people may have strong opinions one way or the other. I am interested in those opinions.
>
> Allow me to give my thoughts. First, I don't think there would be much difference if I was using OOP for the functionality, or using a structural paradigm. A structural paradigm in my opinion, along the lines of Rich Hickey's comments on simple versus complex, would be a simpler implementation. In this case there is no reason to create a construct with state. So let's assume the code is structural and not OOP.
>
> I would go with Design I. Succinctly stated, Design I supports readability and maintainability at least as well, if not better than the other designs. The goal of the SOLID principles are the creation of mid-level software structures that (Software Architecture: SA Martin). I think Design I best adheres to these principles of:
> ---- Tolerate change,
> ---- Are easy to understand, and
> ---- Are the basis of components that can be used in many software systems.
>
> I could point to the Single Responsibility Principle which is defined as (SA Martin): a module should be responsible to one, and only one, actor. It should satisfy the Liskov Substitution Principle as well. Further, each module in the etl_helpers directory is at the same level of abstraction.
>
> I could also mention that as Dijkstra stressed, at every level, from the smallest function to the largest component, software is like a science and, therefore, is driven by falsifiability. Software architects strive to define modules, components, and services that are easily falsifiable (testable). To do so, they employ restrictive disciplines similar to structured programming,
> albeit at a much higher level (SA Martin).
>
> One can point to multiple reasons why Design I might be preferred, but what are the compelling reasons, if there are any, that would suggest another design was superior.
>
> Finally, let me reference an interesting research paper I read recently that seems to support the other designs as anti-patterns: Architecture_Anti-patterns_Automatically.pdf
>
> ---- (https://urldefense.com/v3/__https://www.cs.drexel.edu/*yfcai/papers/2019/tse2019.pdf__;fg!!Cn_UX_p3!nME8OhiOxAzmzM3jzg6uXZU851dhWWD9JGB8ZRZIzyUzGkmCN-C6SSXrL59eA2KVIh-y-W0VycJSNb8aYcNnc3jaresNFQ$ )
>
> SEVERAL DESIGNS FOR COMPARISON
>
> DESIGN I:
>
> ---- manage_the_etl_pipeline.py
> ---- etl_helpers
> ---- extract.py
> ---- transform.py
> ---- load.py
>
> Of course one could also
>
> DESIGN II:
>
> ---- manage_the_etl_pipeline.py
> ---- etl_helpers
> ---- extract_transform_load.py
>
> or probably even:
>
> DESIGN III:
>
> ---- manage_the_etl_pipeline.py
> ---- extract_transform_load.py
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nME8OhiOxAzmzM3jzg6uXZU851dhWWD9JGB8ZRZIzyUzGkmCN-C6SSXrL59eA2KVIh-y-W0VycJSNb8aYcNnc3hpaHTfyQ$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nME8OhiOxAzmzM3jzg6uXZU851dhWWD9JGB8ZRZIzyUzGkmCN-C6SSXrL59eA2KVIh-y-W0VycJSNb8aYcNnc3hpaHTfyQ$>

From gweatherby at uchc.edu  Sun Feb  5 10:03:23 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Sun, 5 Feb 2023 15:03:23 +0000
Subject: Typing Number, PyCharm
In-Reply-To: <e818abc9-a531-67e1-1aca-d6770bca2c03@DancesWithMice.info>
References: <e818abc9-a531-67e1-1aca-d6770bca2c03@DancesWithMice.info>
Message-ID: <SJ0PR14MB5864560E58F201EB62B0F200B9D59@SJ0PR14MB5864.namprd14.prod.outlook.com>

dn,

I?m missing something here. Method 5 seems to work fine in PyCharm. I?m interpreting your statement as:

from fractions import Fraction
from numbers import Number


def double(value: Number):
    if isinstance(value, Number):
        # noinspection PyTypeChecker
        return 2 * value
    raise ValueError(f"{value} of {type(value)} is not a Number")


print(double(7))
print(double(7.2))
print(double(complex(3.2, 4.5)))
print(double(Fraction(7, 8)))
# print(double("7")) PyCharm properly complains


From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of dn via Python-list <python-list at python.org>
Date: Saturday, February 4, 2023 at 9:32 PM
To: 'Python' <python-list at python.org>
Subject: Typing Number, PyCharm
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Do we have a typing type-hint for numbers yet?


Often wanting to combine int and float, discovered that an application
was doing a walk-through with/for uses three numeric types. Was
intrigued to note variance, in that the code-set features two different
methods for typing, in this situation:

def func( value ):
     ...using value...

where value may be an integer, a floating-point value, or a
complex-number (but not decimal-type).
NB code snippets from memory (cf copy-paste)


Method 1 (possibly older code):-

from typing import Union
...
def fun( value:Union[ int, float, complex ] ):


Method 2:-

def fun( value:int|float|complex  ):


Pondering this, realised could use an alias to de-clutter the
function-definition/signature:

Method 3:-

number_type = int|float|complex
...
def fun( value:number_type  ):


If it was important to have type consistency within the union, eg
argument and return, could go for:

Method 4:-

from typing import TypeVar
number_type = TypeVar( 'number_type', int, float, complex )
...
def fun( value:number_type  ):


Then remembered the way we'd code an execution-time check for this using
isinstance():

Method 5:-

from numbers import Number
...
def fun( value:Number  ):


Each of these will execute correctly.

All cause PyCharm to object if I try to call the fun(ction) with a
string parameter - and execute an exception, as expected.


Accepting all the others, am curious as to why PyCharm objects to Method
5 with "Expected type 'SupportsFloat | SupportsComplex | complex |
SupportsIndex', got 'Number' instead? - yet still highlights the
erroneous string parameter but none of the 'legal' data-types?

As soon as a list (in this case types) reaches three, my aged-eyes start
to think de-cluttering is a good idea!

Do you know of another way to attack this/more properly?

--
Regards,
=dn
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqIbcK1zaXmlo5y6741fRwcBUcDfxPNkiA4Jy_NHr9nEno2HaBGZYMuitXeivWrGwTJtds01pHdFfiY_Y5bnmsq_NQ$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqIbcK1zaXmlo5y6741fRwcBUcDfxPNkiA4Jy_NHr9nEno2HaBGZYMuitXeivWrGwTJtds01pHdFfiY_Y5bnmsq_NQ$>

From gabriele1NOSPAM at hotmail.com  Sun Feb  5 05:46:00 2023
From: gabriele1NOSPAM at hotmail.com (^Bart)
Date: Sun, 5 Feb 2023 11:46:00 +0100
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <mailman.1765.1675551557.20444.python-list@python.org>
References: <trls7e$21467$1@dont-email.me>
 <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
 <mailman.1765.1675551557.20444.python-list@python.org>
Message-ID: <tro1bs$2f94t$1@dont-email.me>

> Not Python, but you could run a shell script that sends the file with 
> lftp, and depending on the return code uses xdg-email to send one of the 
> two messages. This would probably be simpler than using Python.

This machine is a server without DE, just command line and I didn't 
configure mail client so maybe I should use something like ssmtp.

> Otherwise you haven't said what part of the process you need help with. 
> Reading the log file?? Checking the date?? Triggering the Python script? 
>  ?Sending the email message with Python?

I need to understand "just" if the upload job is finished and, I think, 
the best way is to read the log file so I should write a code like:

"if there's a line with the same date of the machine than send email 
with ok else send an email with failed"

I do just an upload every day, from monday to friday so I don't have in 
the same day other lines.

> For the latter, it's covered in the Python docs -
> 
> https://docs.python.org/3.10/library/email.examples.html

I'm reading it! :)

Regards.
^Bart


From gabriele1NOSPAM at hotmail.com  Sun Feb  5 06:09:33 2023
From: gabriele1NOSPAM at hotmail.com (^Bart)
Date: Sun, 5 Feb 2023 12:09:33 +0100
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <mailman.1767.1675566828.20444.python-list@python.org>
References: <trls7e$21467$1@dont-email.me>
 <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
 <Y98e4oXIv1A6euCk@anomaly>
 <mailman.1767.1675566828.20444.python-list@python.org>
Message-ID: <tro2o1$2fkp4$1@dont-email.me>

> xdg-email appears to be for interactive use (it opens the user's
> "preferred email composer"); I think sendmail would work much better
> from a script.

Like what I said in another post I think I could use ssmtp than 
xdg-email or sendmail...

> Otherwise, I had the same initial thought, to add to and/or build a
> wrapper around the existing lftp script.

I'd like to know if there's a code made from lftp to understand when an 
upload file is finished but certainly I can read it from the log file 
and I think it couldn't be hard to find a value in a *.txt file and if 
this value is inside of it to send an email like "ok" otherwise a 
message with "k.o.".

Regards.
^Bart

From gabriele1NOSPAM at hotmail.com  Sun Feb  5 06:13:15 2023
From: gabriele1NOSPAM at hotmail.com (^Bart)
Date: Sun, 5 Feb 2023 12:13:15 +0100
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <mailman.1768.1675572573.20444.python-list@python.org>
References: <trls7e$21467$1@dont-email.me>
 <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
 <Y98e4oXIv1A6euCk@anomaly>
 <b0fe549d-2ddf-aa2e-80b5-6844045d2909@tompassin.net>
 <mailman.1768.1675572573.20444.python-list@python.org>
Message-ID: <tro2uv$2fkp4$2@dont-email.me>

> I have never used it, and I did see that wording in the man page, but it 
> also showed putting all the info on the command line - from, to, 
> subject, etc - so I thought it might be able to use a command line 
> client without needed any GUI interaction.? But I don't know for sure.

This machine is a server without GUI so I must do everything by cli but 
it's not a problem! :)

So mainly I should write a code to find a match between the date found 
on the lftp's log file and the date of the pc, if there's a match I need 
to send an email with "ok" otherwise an email with "ko".

I think it shouldn't be so hard to compare a value inside of a file with 
the date of the machine but now I should understand which is the best 
way! :)

Regards.
Gabriele


From gabriele1NOSPAM at hotmail.com  Sun Feb  5 08:57:54 2023
From: gabriele1NOSPAM at hotmail.com (^Bart)
Date: Sun, 5 Feb 2023 14:57:54 +0100
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <mailman.1764.1675548624.20444.python-list@python.org>
References: <trls7e$21467$1@dont-email.me>
 <00ad01d938e5$7836e4f0$68a4aed0$@gmail.com>
 <mailman.1764.1675548624.20444.python-list@python.org>
Message-ID: <trocl2$2hbfm$1@dont-email.me>

> For example, try to do whatever parts you know how to do and when some part
> fails or is missing, ask.

You're right but first of all I wrote what I'd like to do and if Python 
could be the best choice about it! :)

> I might have replied to you directly if your email email address did not
> look like you want no SPAM, LOL!

Ahaha! I think you know what is spam and what is a reply\answer to a 
post request so you can feel free to use also my email! :)

> The cron stuff is not really relevant and it seems your idea is to read a
> part or all of a log file, parse the lines in some way and find a line that
> either matches what you need or fail to find it. Either way you want to send
> an email out with an appropriate content.

You got the point!

> Which part of that do you not know how to do in python? Have you done some
> reading or looking?

Like what I wrote above I didn't know if Python can does what I need and 
if to use Python is a good way I'll start to study how to do it! :)

In my past I used Python for Arduino programming or to do easy things, 
what I should do now is little more complex but I understood from years 
and years by the Python's powers you can do everything! LOL! :)

Regards.
^Bart

From jj at franjam.org.uk  Sun Feb  5 13:00:39 2023
From: jj at franjam.org.uk (Jim Jackson)
Date: Sun, 5 Feb 2023 18:00:39 -0000 (UTC)
Subject: How to read file content and send email on Debian Bullseye
References: <trls7e$21467$1@dont-email.me>
 <00ad01d938e5$7836e4f0$68a4aed0$@gmail.com>
 <mailman.1764.1675548624.20444.python-list@python.org>
 <trocl2$2hbfm$1@dont-email.me>
Message-ID: <slrnttvrm6.4ki.jj@iridium.wf32df>

On 2023-02-05, ^Bart <gabriele1NOSPAM at hotmail.com> wrote:
>> For example, try to do whatever parts you know how to do and when some part
>> fails or is missing, ask.
>
> You're right but first of all I wrote what I'd like to do and if Python 
> could be the best choice about it! :)

I'd say you want a simple shell script wrapped around your job, and a 
program to send email (bsdmail/sendmail or similar or mstmp on linux for 
instance).

>
>> I might have replied to you directly if your email email address did not
>> look like you want no SPAM, LOL!
>
> Ahaha! I think you know what is spam and what is a reply\answer to a 
> post request so you can feel free to use also my email! :)
>
>> The cron stuff is not really relevant and it seems your idea is to read a
>> part or all of a log file, parse the lines in some way and find a line that
>> either matches what you need or fail to find it. Either way you want to send
>> an email out with an appropriate content.
>
> You got the point!
>
>> Which part of that do you not know how to do in python? Have you done some
>> reading or looking?
>
> Like what I wrote above I didn't know if Python can does what I need and 
> if to use Python is a good way I'll start to study how to do it! :)
>
> In my past I used Python for Arduino programming or to do easy things, 
> what I should do now is little more complex but I understood from years 
> and years by the Python's powers you can do everything! LOL! :)
>
> Regards.
> ^Bart

From jj at franjam.org.uk  Sun Feb  5 13:06:43 2023
From: jj at franjam.org.uk (Jim Jackson)
Date: Sun, 5 Feb 2023 18:06:43 -0000 (UTC)
Subject: How to read file content and send email on Debian Bullseye
References: <trls7e$21467$1@dont-email.me>
 <1f5988f4-064a-af78-3a63-8c7b11d583ac@tompassin.net>
 <Y98e4oXIv1A6euCk@anomaly>
 <mailman.1767.1675566828.20444.python-list@python.org>
 <tro2o1$2fkp4$1@dont-email.me>
Message-ID: <slrnttvs1j.4ki.jj@iridium.wf32df>

On 2023-02-05, ^Bart <gabriele1NOSPAM at hotmail.com> wrote:
>> xdg-email appears to be for interactive use (it opens the user's
>> "preferred email composer"); I think sendmail would work much better
>> from a script.
>
> Like what I said in another post I think I could use ssmtp than 
> xdg-email or sendmail...
>
>> Otherwise, I had the same initial thought, to add to and/or build a
>> wrapper around the existing lftp script.
>
> I'd like to know if there's a code made from lftp to understand when an 
> upload file is finished

You make an lftp "script" to upload the file and write a log somewhere - 
write a script to run lftp with the lftp script, when the lftp has 
finished email the log. The log will show what happenned.

In cron run the script.

>  but certainly I can read it from the log file 
> and I think it couldn't be hard to find a value in a *.txt file and if 
> this value is inside of it to send an email like "ok" otherwise a 
> message with "k.o.".
>
> Regards.
> ^Bart

From PythonList at DancesWithMice.info  Sun Feb  5 14:53:37 2023
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 6 Feb 2023 08:53:37 +1300
Subject: Typing Number, PyCharm
In-Reply-To: <SJ0PR14MB5864560E58F201EB62B0F200B9D59@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <e818abc9-a531-67e1-1aca-d6770bca2c03@DancesWithMice.info>
 <SJ0PR14MB5864560E58F201EB62B0F200B9D59@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <a7d6d84b-066e-e556-9d3b-939514df2b62@DancesWithMice.info>

No @Gerard, YOU weren't missing anything: since posting, have upgraded 
PyCharm to 2022.3.2 and the complaints about 'Method 5' have 
disappeared. Evidently a PyCharm issue!

Which alters the top-line question to: is numbers.Number the preferred 
type-hint when multiple numeric types are to be accepted?

PS no Decimal(s) nor Fraction(s) in my situation, but may be worth 
adding to a wider discussion...


On 06/02/2023 04.03, Weatherby,Gerard wrote:
> dn,
> 
> I?m missing something here. Method 5 seems to work fine in PyCharm. I?m 
> interpreting your statement as:
> 
> from fractions import Fraction
> from numbers import Number
> 
> 
> def double(value: Number):
> if isinstance(value, Number):
> /# noinspection PyTypeChecker
> /return 2 * value
> raise ValueError(f"{value}of {type(value)}is not a Number")
> 
> 
> print(double(7))
> print(double(7.2))
> print(double(complex(3.2, 4.5)))
> print(double(Fraction(7, 8)))
> /# print(double("7")) PyCharm properly complains/
> 
> *From: *Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> 
> on behalf of dn via Python-list <python-list at python.org>
> *Date: *Saturday, February 4, 2023 at 9:32 PM
> *To: *'Python' <python-list at python.org>
> *Subject: *Typing Number, PyCharm
> 
> *** Attention: This is an external email. Use caution responding, 
> opening attachments or clicking on links. ***
> 
> Do we have a typing type-hint for numbers yet?
> 
> 
> Often wanting to combine int and float, discovered that an application
> was doing a walk-through with/for uses three numeric types. Was
> intrigued to note variance, in that the code-set features two different
> methods for typing, in this situation:
> 
> def func( value ):
>  ???? ...using value...
> 
> where value may be an integer, a floating-point value, or a
> complex-number (but not decimal-type).
> NB code snippets from memory (cf copy-paste)
> 
> 
> Method 1 (possibly older code):-
> 
> from typing import Union
> ...
> def fun( value:Union[ int, float, complex ] ):
> 
> 
> Method 2:-
> 
> def fun( value:int|float|complex? ):
> 
> 
> Pondering this, realised could use an alias to de-clutter the
> function-definition/signature:
> 
> Method 3:-
> 
> number_type = int|float|complex
> ...
> def fun( value:number_type? ):
> 
> 
> If it was important to have type consistency within the union, eg
> argument and return, could go for:
> 
> Method 4:-
> 
> from typing import TypeVar
> number_type = TypeVar( 'number_type', int, float, complex )
> ...
> def fun( value:number_type? ):
> 
> 
> Then remembered the way we'd code an execution-time check for this using
> isinstance():
> 
> Method 5:-
> 
> from numbers import Number
> ...
> def fun( value:Number? ):
> 
> 
> Each of these will execute correctly.
> 
> All cause PyCharm to object if I try to call the fun(ction) with a
> string parameter - and execute an exception, as expected.
> 
> 
> Accepting all the others, am curious as to why PyCharm objects to Method
> 5 with "Expected type 'SupportsFloat | SupportsComplex | complex |
> SupportsIndex', got 'Number' instead? - yet still highlights the
> erroneous string parameter but none of the 'legal' data-types?
> 
> As soon as a list (in this case types) reaches three, my aged-eyes start
> to think de-cluttering is a good idea!
> 
> Do you know of another way to attack this/more properly?
> 
> --
> Regards,
> =dn
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqIbcK1zaXmlo5y6741fRwcBUcDfxPNkiA4Jy_NHr9nEno2HaBGZYMuitXeivWrGwTJtds01pHdFfiY_Y5bnmsq_NQ$ <https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqIbcK1zaXmlo5y6741fRwcBUcDfxPNkiA4Jy_NHr9nEno2HaBGZYMuitXeivWrGwTJtds01pHdFfiY_Y5bnmsq_NQ$>
> 

-- 
Regards,
=dn


From greg.ewing at canterbury.ac.nz  Sun Feb  5 19:15:06 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 6 Feb 2023 13:15:06 +1300
Subject: Organizing modules and their code
In-Reply-To: <mailman.1770.1675610718.20444.python-list@python.org>
References: <e2c142ad-9100-49bb-8b07-e81a2bee638dn@googlegroups.com>
 <0edf1674-4a11-4686-aee4-2774eaabfa43n@googlegroups.com>
 <SJ0PR14MB586461F3E3E32B2417877C96B9D59@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <mailman.1770.1675610718.20444.python-list@python.org>
Message-ID: <k4av4eFj5taU1@mid.individual.net>

On 6/02/23 4:23 am, Weatherby,Gerard wrote:
> Well, first of all, while there is no doubt as to Dijkstra?s contribution to computer science, I don?t think his description of scientific thought is correct. The acceptance of Einstein?s theory of relativity has nothing to do with internal consistency or how easy or difficult to explain but rather repeatedly experimental results validating it.

I don't think Dijkstra was claiming that what he was talking about
was a *complete* description of scientific thought, only that the
ability to separate out independent concerns is an important part
of it, and that was something he saw his colleagues failing to do.

-- 
Greg

From avi.e.gross at gmail.com  Sun Feb  5 22:40:32 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sun, 5 Feb 2023 22:40:32 -0500
Subject: How to read file content and send email on Debian Bullseye
In-Reply-To: <trocl2$2hbfm$1@dont-email.me>
References: <trls7e$21467$1@dont-email.me>
 <00ad01d938e5$7836e4f0$68a4aed0$@gmail.com>
 <mailman.1764.1675548624.20444.python-list@python.org>
 <trocl2$2hbfm$1@dont-email.me>
Message-ID: <00c801d939dc$c475e7b0$4d61b710$@gmail.com>

Bart,

Some really decent cron jobs can be written without using anything complex.

I get it  now that perhaps your motivation is more about finding an excuse
to learn python better. The reality is there is not much that python cannot
do if other programming languages and environments can do them so asking if
it can do it feels a tad na?ve. Like many languages, python is very
extendable with all kinds of modules so often instead of doing something
totally on your own, you can find something that does much of the hard work
for you, too. 

Yes, python can nicely read in lines from a log and compare them to a fixed
string or pattern and either find or not find what you ask for.

But a shell script can be written in a few minutes that simply gets the
string for the current date in the format you want, interpolates it in a
regular expression, calls grep or a dozen other programs that handle a
command line argument, and if it returns a single line, you send one email
and if none you send another and if more than one, you may have done it
wrong. Some such programs, like AWK are designed mainly to do exactly
something like you ask and examine each input line against a series of
patterns. Sending an email though is not always something easy to do from
within a program like that but a shell script that checks how it ends may do
that part.

If you are on a specific machine and only need to solve the problem on that
machine or something similar, this seems straightforward. 

My impression is you want to match a line in the log file that may look like
it should  match "anything", then some keyword or so that specifies all
lines about this particular upload on every day, then matches another
"anything" up to something that exactly matches the date for today, and
maybe match another "anything" to the end of the line. It can be a fairly
straightforward regular expression if the data has a regular component in
the formatting. Grep, sed, awk, perl and others can do this and others. 

Could you do this faster in python? Maybe. Arguably if speed is a major
issue, write it in some compiled language like C++. 

But if your data is very regular such as the entry will have some key phrase
between the 12th and 19th characters and the date will be exactly in another
exact region, then you certainly can skip regular expressions and read each
line and examine the substrings for equality. You may also speed it a bit if
you exit any such loop as soon as you find what you are looking for. 

I note if your log file is big but not very busy, and you are pretty sure
the entry will be in the last few (maybe hundred) lines, some may use the
tail command and pipe the text it returns to whatever is processing the
data. There are many ways to do what you want.

But you improve your chances of getting an answer if you ask it more
clearly. There have been people (maybe just one) who have been posing
questions of a rather vague nature and then not responding as others debate
it in seemingly random directions. You are interacting nicely but some of us
have become hesitant to jump in until they see if the request is
well-intended. You do sound like you know quite a bit and your question
could have  been closer to saying that you know several ways to do it
(include examples or at least outlines) and wonder if some ways are better
or more pythonic or ...

So if people keep wondering what you want, it is because the people here are
not generally interested in doing homework or complete programs for people.
If you ask us how to generate a string with the current date, and cannot
just find it on your own, we might answer. If you want to know how to store
a date as an object including the current time, and also convert the text on
a line in the log file to make another such date object and then be able to
compare them and be able to include in your email how LONG AGO the upload
was done, that would be another more specific request. If you are not sure
how python does regular expressions, ...

Otherwise, what you are asking for may not be BASIC for some but seems
relatively straightforward to some of us and we sort of may be  wondering
if we are missing anything?

Good Luck,

^Avi

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of ^Bart
Sent: Sunday, February 5, 2023 8:58 AM
To: python-list at python.org
Subject: Re: RE: How to read file content and send email on Debian Bullseye

> For example, try to do whatever parts you know how to do and when some 
> part fails or is missing, ask.

You're right but first of all I wrote what I'd like to do and if Python
could be the best choice about it! :)

> I might have replied to you directly if your email email address did 
> not look like you want no SPAM, LOL!

Ahaha! I think you know what is spam and what is a reply\answer to a post
request so you can feel free to use also my email! :)

> The cron stuff is not really relevant and it seems your idea is to 
> read a part or all of a log file, parse the lines in some way and find 
> a line that either matches what you need or fail to find it. Either 
> way you want to send an email out with an appropriate content.

You got the point!

> Which part of that do you not know how to do in python? Have you done 
> some reading or looking?

Like what I wrote above I didn't know if Python can does what I need and if
to use Python is a good way I'll start to study how to do it! :)

In my past I used Python for Arduino programming or to do easy things, what
I should do now is little more complex but I understood from years and years
by the Python's powers you can do everything! LOL! :)

Regards.
^Bart
--
https://mail.python.org/mailman/listinfo/python-list


From gweatherby at uchc.edu  Mon Feb  6 07:11:31 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Mon, 6 Feb 2023 12:11:31 +0000
Subject: Typing Number, PyCharm
In-Reply-To: <a7d6d84b-066e-e556-9d3b-939514df2b62@DancesWithMice.info>
References: <e818abc9-a531-67e1-1aca-d6770bca2c03@DancesWithMice.info>
 <SJ0PR14MB5864560E58F201EB62B0F200B9D59@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <a7d6d84b-066e-e556-9d3b-939514df2b62@DancesWithMice.info>
Message-ID: <SJ0PR14MB586413905CB4C5AD76B4362FB9DA9@SJ0PR14MB5864.namprd14.prod.outlook.com>

?is numbers.Number the preferred type-hint when multiple numeric types are to be accepted??

I don?t know.

On the one hand, it is a well-known type, so it should be recognizable to users of an API. On the other hand, Number is entirely abstract, so it doesn?t provide useful type checking for the implementation; I had to add # noinspection PyTypeChecker to 2 * value to keep PyCharm from complaining. Additionally, it does not include the Decimal type.  Interestingly, It was added in 2007 in anticipation of ?if and when overloading based on types is added to the language.? This now seems unlikely to happen. ( https://peps.python.org/pep-3141/#rationale )

On the other hand, a specific type alias is clearer to type checkers and possibly to the user of an API?

Experimenting, it appears isinstance checks of Type Aliases don?t work before 3.10, and its TypeAlias construct. That is:

Numeric = Union[int, float, complex, Fraction]


def double(value: Numeric):
    if isinstance(value, Numeric):
        return 2 * value
    raise ValueError(f"{value} of {type(value)} is not Numeric")
Fails but

Numeric : TypeAlias = Union[int, float, complex, Fraction]


def double(value: Numeric):
    if isinstance(value, Numeric):
        return 2 * value
    raise ValueError(f"{value} of {type(value)} is not Numeric")

works (>= 3.10)


From: dn <PythonList at DancesWithMice.info>
Date: Sunday, February 5, 2023 at 2:54 PM
To: Weatherby,Gerard <gweatherby at uchc.edu>, 'Python' <python-list at python.org>
Subject: Re: Typing Number, PyCharm
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

No @Gerard, YOU weren't missing anything: since posting, have upgraded
PyCharm to 2022.3.2 and the complaints about 'Method 5' have
disappeared. Evidently a PyCharm issue!

Which alters the top-line question to: is numbers.Number the preferred
type-hint when multiple numeric types are to be accepted?

PS no Decimal(s) nor Fraction(s) in my situation, but may be worth
adding to a wider discussion...


On 06/02/2023 04.03, Weatherby,Gerard wrote:
> dn,
>
> I?m missing something here. Method 5 seems to work fine in PyCharm. I?m
> interpreting your statement as:
>
> from fractions import Fraction
> from numbers import Number
>
>
> def double(value: Number):
> if isinstance(value, Number):
> /# noinspection PyTypeChecker
> /return 2 * value
> raise ValueError(f"{value}of {type(value)}is not a Number")
>
>
> print(double(7))
> print(double(7.2))
> print(double(complex(3.2, 4.5)))
> print(double(Fraction(7, 8)))
> /# print(double("7")) PyCharm properly complains/
>
> *From: *Python-list <python-list-bounces+gweatherby=uchc.edu at python.org>
> on behalf of dn via Python-list <python-list at python.org>
> *Date: *Saturday, February 4, 2023 at 9:32 PM
> *To: *'Python' <python-list at python.org>
> *Subject: *Typing Number, PyCharm
>
> *** Attention: This is an external email. Use caution responding,
> opening attachments or clicking on links. ***
>
> Do we have a typing type-hint for numbers yet?
>
>
> Often wanting to combine int and float, discovered that an application
> was doing a walk-through with/for uses three numeric types. Was
> intrigued to note variance, in that the code-set features two different
> methods for typing, in this situation:
>
> def func( value ):
>       ...using value...
>
> where value may be an integer, a floating-point value, or a
> complex-number (but not decimal-type).
> NB code snippets from memory (cf copy-paste)
>
>
> Method 1 (possibly older code):-
>
> from typing import Union
> ...
> def fun( value:Union[ int, float, complex ] ):
>
>
> Method 2:-
>
> def fun( value:int|float|complex  ):
>
>
> Pondering this, realised could use an alias to de-clutter the
> function-definition/signature:
>
> Method 3:-
>
> number_type = int|float|complex
> ...
> def fun( value:number_type  ):
>
>
> If it was important to have type consistency within the union, eg
> argument and return, could go for:
>
> Method 4:-
>
> from typing import TypeVar
> number_type = TypeVar( 'number_type', int, float, complex )
> ...
> def fun( value:number_type  ):
>
>
> Then remembered the way we'd code an execution-time check for this using
> isinstance():
>
> Method 5:-
>
> from numbers import Number
> ...
> def fun( value:Number  ):
>
>
> Each of these will execute correctly.
>
> All cause PyCharm to object if I try to call the fun(ction) with a
> string parameter - and execute an exception, as expected.
>
>
> Accepting all the others, am curious as to why PyCharm objects to Method
> 5 with "Expected type 'SupportsFloat | SupportsComplex | complex |
> SupportsIndex', got 'Number' instead? - yet still highlights the
> erroneous string parameter but none of the 'legal' data-types?
>
> As soon as a list (in this case types) reaches three, my aged-eyes start
> to think de-cluttering is a good idea!
>
> Do you know of another way to attack this/more properly?
>
> --
> Regards,
> =dn
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqIbcK1zaXmlo5y6741fRwcBUcDfxPNkiA4Jy_NHr9nEno2HaBGZYMuitXeivWrGwTJtds01pHdFfiY_Y5bnmsq_NQ$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqIbcK1zaXmlo5y6741fRwcBUcDfxPNkiA4Jy_NHr9nEno2HaBGZYMuitXeivWrGwTJtds01pHdFfiY_Y5bnmsq_NQ$> <https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!gqIbcK1zaXmlo5y6741fRwcBUcDfxPNkiA4Jy_NHr9nEno2HaBGZYMuitXeivWrGwTJtds01pHdFfiY_Y5bnmsq_NQ$>
>

--
Regards,
=dn

From pbryan at anode.ca  Mon Feb  6 09:25:46 2023
From: pbryan at anode.ca (Paul Bryan)
Date: Mon, 06 Feb 2023 06:25:46 -0800
Subject: Typing Number, PyCharm
In-Reply-To: <SJ0PR14MB586413905CB4C5AD76B4362FB9DA9@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <e818abc9-a531-67e1-1aca-d6770bca2c03@DancesWithMice.info>
 <SJ0PR14MB5864560E58F201EB62B0F200B9D59@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <a7d6d84b-066e-e556-9d3b-939514df2b62@DancesWithMice.info>
 <SJ0PR14MB586413905CB4C5AD76B4362FB9DA9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <7d58afd0803058a0bbfd9a8a84c1091fbb2c2e2c.camel@anode.ca>

On Mon, 2023-02-06 at 12:11 +0000, Weatherby,Gerard wrote:

> On the one hand, it is a well-known type, so it should be
> recognizable to users of an API. On the other hand, Number is
> entirely abstract, so it doesn?t provide useful type checking for the
> implementation; I had to add # noinspection PyTypeChecker to 2 *
> value to keep PyCharm from complaining. Additionally, it does not
> include the Decimal type.

Hmm...

Python 3.10.9 (main, Dec 19 2022, 17:35:49) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from numbers import Number
>>> from decimal import Decimal as D
>>> isinstance(D("1.0"), Number)
True


From gweatherby at uchc.edu  Mon Feb  6 10:07:31 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Mon, 6 Feb 2023 15:07:31 +0000
Subject: Typing Number, PyCharm
In-Reply-To: <7d58afd0803058a0bbfd9a8a84c1091fbb2c2e2c.camel@anode.ca>
References: <e818abc9-a531-67e1-1aca-d6770bca2c03@DancesWithMice.info>
 <SJ0PR14MB5864560E58F201EB62B0F200B9D59@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <a7d6d84b-066e-e556-9d3b-939514df2b62@DancesWithMice.info>
 <SJ0PR14MB586413905CB4C5AD76B4362FB9DA9@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <7d58afd0803058a0bbfd9a8a84c1091fbb2c2e2c.camel@anode.ca>
Message-ID: <SJ0PR14MB5864E32C77D8D21DB2417C05B9DA9@SJ0PR14MB5864.namprd14.prod.outlook.com>

Yeah, I?m confused.
?The Decimal Type<https://peps.python.org/pep-3141/#the-decimal-type>

After consultation with its authors it has been decided that the Decimal type should not at this time be made part of the numeric tower.?  https://peps.python.org/pep-3141/#the-decimal-type

And the Decimal definition I?m finding says:
class Decimal(object):

yet

print(issubclass(Decimal,Number))

returns True.




From: Paul Bryan <pbryan at anode.ca>
Date: Monday, February 6, 2023 at 9:25 AM
To: Weatherby,Gerard <gweatherby at uchc.edu>, dn <PythonList at DancesWithMice.info>, 'Python' <python-list at python.org>
Subject: Re: Typing Number, PyCharm
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***
On Mon, 2023-02-06 at 12:11 +0000, Weatherby,Gerard wrote:

On the one hand, it is a well-known type, so it should be recognizable to users of an API. On the other hand, Number is entirely abstract, so it doesn?t provide useful type checking for the implementation; I had to add # noinspection PyTypeChecker to 2 * value to keep PyCharm from complaining. Additionally, it does not include the Decimal type.

Hmm...

Python 3.10.9 (main, Dec 19 2022, 17:35:49) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from numbers import Number
>>> from decimal import Decimal as D
>>> isinstance(D("1.0"), Number)
True


From stefalem79 at gmail.com  Mon Feb  6 13:19:50 2023
From: stefalem79 at gmail.com (stefalem)
Date: Mon, 6 Feb 2023 10:19:50 -0800 (PST)
Subject: tkinter ttk.Treeview: changing background colour of single item
 when selected
In-Reply-To: <mailman.1759.1675507388.20444.python-list@python.org>
References: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
 <mailman.1759.1675507388.20444.python-list@python.org>
Message-ID: <c47c9186-42b7-4931-bf75-d7d6812baf1cn@googlegroups.com>

Il giorno sabato 4 febbraio 2023 alle 11:43:29 UTC+1 John O'Hagan ha scritto:
...

> Is there another way to do what I want? 

from tkinter import *
from tkinter.ttk import *

root = Tk()
t = Treeview(root)

t.insert('', 0, iid='item1', text='item1')
t.insert('', 1, text='item2')
t.tag_configure('flashtag', background='red')
t.pack()
  
def flash():
 tags = t.item('item1', 'tags')
 t.item('item1', tags='' if tags else 'flashtag')
 t.after(500, flash)
 itemselected = t.selection()
 for x in itemselected:
  if (x == 'item1'):
   t.selection_remove(t.get_children())

flash()


> Thanks 
> 
> -- 
> 
> John

From stefalem79 at gmail.com  Mon Feb  6 13:26:35 2023
From: stefalem79 at gmail.com (stefalem)
Date: Mon, 6 Feb 2023 10:26:35 -0800 (PST)
Subject: tkinter ttk.Treeview: changing background colour of single item
 when selected
In-Reply-To: <mailman.1759.1675507388.20444.python-list@python.org>
References: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
 <mailman.1759.1675507388.20444.python-list@python.org>
Message-ID: <2865ec14-cc88-4c3b-a641-788c08f9f851n@googlegroups.com>

> Is there another way to do what I want? 

try this:

from tkinter import *
from tkinter.ttk import *

root = Tk()
t = Treeview(root)

t.insert('', 0, iid='item1', text='item1')
t.insert('', 1, text='item2')
t.tag_configure('flashtag', background='red')
t.pack()
  
def flash():
 tags = t.item('item1', 'tags')
 t.item('item1', tags='' if tags else 'flashtag')
 t.after(500, flash)
 itemselected = t.selection()
 for x in itemselected:
  if (x == 'item1'):
   t.selection_remove(t.get_children())

flash()
mainloop()


From stefalem79 at gmail.com  Mon Feb  6 16:14:36 2023
From: stefalem79 at gmail.com (stefalem)
Date: Mon, 6 Feb 2023 22:14:36 +0100
Subject: tkinter ttk.Treeview: changing background colour of single item
 when selected
In-Reply-To: <2865ec14-cc88-4c3b-a641-788c08f9f851n@googlegroups.com>
References: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
 <mailman.1759.1675507388.20444.python-list@python.org>
 <2865ec14-cc88-4c3b-a641-788c08f9f851n@googlegroups.com>
Message-ID: <trrqjs$374de$1@dont-email.me>

I apologize for the 3 messages sent, I could not access the usual news 
server and with Google Groups I messed up :)

From rob.cliffe at btinternet.com  Fri Feb  3 10:44:16 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Fri, 3 Feb 2023 15:44:16 +0000
Subject: evaluation question
In-Reply-To: <trfvu2$qhcd$1@dont-email.me>
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me> <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me>
Message-ID: <807535a2-dd1e-dd33-e6a6-cf17264ab955@btinternet.com>



On 02/02/2023 09:31, Muttley at dastardlyhq.com wrote:
> On Wed, 1 Feb 2023 18:28:04 +0100
> "Peter J. Holzer" <hjp-python at hjp.at> wrote:
>> --b2nljkb3mdefsdhx
>> Content-Type: text/plain; charset=us-ascii
>> Content-Disposition: inline
>> Content-Transfer-Encoding: quoted-printable
>>
>> On 2023-02-01 09:00:39 -0000, Muttley at dastardlyhq.com wrote:
>>> Its not evolution, its revolution. Evolution retains old functionality.
>> Tell a penguin that it can fly :-)
> Yeah ok :) But the ancestors of penguins didn't wake up one morning, flap
> their wings and fall out the tree, it happened gradually. Python2 syntax
> could have been retained for X versions of 3 just as C++ keeps old stuff
> until its eventually deprecated them removed.
Yeah?? So what would this do:
 ??? print ()
In Python 2 this prints an empty tuple.
In Python 3 this is a call to the print function with no arguments, 
which prints a blank line.
You can't have it both ways.
In any case, supporting two different syntaxes simultaneously would be 
messy and difficult to maintain.
Better a clean break, with Python 2 support continuing for a long time 
(as it was).
Best wishes
Rob Cliffe

From rosuav at gmail.com  Tue Feb  7 03:15:14 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 7 Feb 2023 19:15:14 +1100
Subject: evaluation question
In-Reply-To: <807535a2-dd1e-dd33-e6a6-cf17264ab955@btinternet.com>
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
 <k3r3rgF5l1hU1@mid.individual.net> <tramnp$3ohlk$1@dont-email.me>
 <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me>
 <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me>
 <807535a2-dd1e-dd33-e6a6-cf17264ab955@btinternet.com>
Message-ID: <CAPTjJmr56ZYp0g30u9Eyd8i=XRp6if4Unttw_ZQpg25yqzYF5A@mail.gmail.com>

On Tue, 7 Feb 2023 at 18:49, Rob Cliffe via Python-list
<python-list at python.org> wrote:
>
>
>
> On 02/02/2023 09:31, Muttley at dastardlyhq.com wrote:
> > On Wed, 1 Feb 2023 18:28:04 +0100
> > "Peter J. Holzer" <hjp-python at hjp.at> wrote:
> >> --b2nljkb3mdefsdhx
> >> Content-Type: text/plain; charset=us-ascii
> >> Content-Disposition: inline
> >> Content-Transfer-Encoding: quoted-printable
> >>
> >> On 2023-02-01 09:00:39 -0000, Muttley at dastardlyhq.com wrote:
> >>> Its not evolution, its revolution. Evolution retains old functionality.
> >> Tell a penguin that it can fly :-)
> > Yeah ok :) But the ancestors of penguins didn't wake up one morning, flap
> > their wings and fall out the tree, it happened gradually. Python2 syntax
> > could have been retained for X versions of 3 just as C++ keeps old stuff
> > until its eventually deprecated them removed.
> Yeah?  So what would this do:
>      print ()
> In Python 2 this prints an empty tuple.
> In Python 3 this is a call to the print function with no arguments,
> which prints a blank line.
> You can't have it both ways.
> In any case, supporting two different syntaxes simultaneously would be
> messy and difficult to maintain.

There are two solutions to this. The most obvious is "from __future__
import print_function", which gives you the full power and flexibility
of Py3 in anything back as far as 2.6; the other is to always pass a
single string argument to print:

print("")
print("spam %d ham %d" % (spam, ham))

This will work in pretty much ANY version of Python [1] and doesn't
require any sort of per-module configuration.

The idea that old syntax should be retained is only part of the story.
While it's definitely important to not break old code unnecessarily,
it is far more important to ensure that there's *some way* to write
code that works across multiple versions. That's what we have here:
even with the breaking changes, there was usually a way to make your
code run identically on multiple versions. Sometimes this means a
compatibility shim at the top, like "try: raw_input; except NameError:
raw_input = input", and sometimes it means following a discipline like
putting b"..." for all strings that need to be bytes. But there always
needs to be a way.

ChrisA

[1] This is the part where someone points out to me that it wouldn't
work in Python 1.3 or something

From rob.cliffe at btinternet.com  Tue Feb  7 10:12:08 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 7 Feb 2023 15:12:08 +0000
Subject: Evaluation of variable as f-string
In-Reply-To: <CAPTjJmpfdvb_yTMu91vNe5Wcb8_u+YO8L8c=w8AFbptj7X7Hiw@mail.gmail.com>
References: <tqmcbf$vdl$1@gioia.aioe.org>
 <CAPTjJmpKEBbYFxowgm4cDfDn0bmieo9sHLK_sNZh8WcqQ=Wqhw@mail.gmail.com>
 <ebb89997-603f-6283-742a-121128e55ae8@btinternet.com>
 <CAPTjJmoxero1FVGW8BNB619tS1XA_LOqn=Lnoq3ogw1JPh52Jg@mail.gmail.com>
 <8ebc73d2-e14a-b9ad-fa35-5078e5f9fd6c@btinternet.com>
 <CAPTjJmqbqNUfRvUt8uwVJLD--faMZVUqTD2LnhD6Cao9Fyx0Yg@mail.gmail.com>
 <33f48974-2ddc-67e4-7a79-ef9a758980a9@btinternet.com>
 <CAPTjJmpfdvb_yTMu91vNe5Wcb8_u+YO8L8c=w8AFbptj7X7Hiw@mail.gmail.com>
Message-ID: <d9e19a5a-6d9e-d0c0-22f1-731990790541@btinternet.com>

[re-sending this to both the list and to Chris, as a prior send to the 
list only was bounced back]
On 31/01/2023 22:33, Chris Angelico wrote:
>
>> Thanks for clarifying.
>> Hm.  So 'x' is neither in locals() nor in globals().  Which starts me
>> wondering (to go off on a tangent): Should there be a nonlocals()
>> dictionary?
> I don't think so, but there might be some value in a dictionary
> containing all available variables. It would have the same "don't
> depend on writing" caveats that locals() has (or would be specifically
> defined as a copy and thus disconnected), so its value would be
> limited. And it would probably STILL be imperfect, because perfection
> would require that it be a compiler construct, due to the way that
> nonlocals are implemented.
Does that mean that it is not possible to have a (built-in) function 
that would construct and return a dictionary of all available variables 
and their values?? If it were possible, it could be useful, and there 
would be no impact on Python run-time speed if it were only constructed 
on demand.

Best wishes
Rob

From rosuav at gmail.com  Tue Feb  7 12:46:52 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 8 Feb 2023 04:46:52 +1100
Subject: Evaluation of variable as f-string
In-Reply-To: <d9e19a5a-6d9e-d0c0-22f1-731990790541@btinternet.com>
References: <tqmcbf$vdl$1@gioia.aioe.org>
 <CAPTjJmpKEBbYFxowgm4cDfDn0bmieo9sHLK_sNZh8WcqQ=Wqhw@mail.gmail.com>
 <ebb89997-603f-6283-742a-121128e55ae8@btinternet.com>
 <CAPTjJmoxero1FVGW8BNB619tS1XA_LOqn=Lnoq3ogw1JPh52Jg@mail.gmail.com>
 <8ebc73d2-e14a-b9ad-fa35-5078e5f9fd6c@btinternet.com>
 <CAPTjJmqbqNUfRvUt8uwVJLD--faMZVUqTD2LnhD6Cao9Fyx0Yg@mail.gmail.com>
 <33f48974-2ddc-67e4-7a79-ef9a758980a9@btinternet.com>
 <CAPTjJmpfdvb_yTMu91vNe5Wcb8_u+YO8L8c=w8AFbptj7X7Hiw@mail.gmail.com>
 <d9e19a5a-6d9e-d0c0-22f1-731990790541@btinternet.com>
Message-ID: <CAPTjJmo_3tgunRswY7LE4mXuDykiPqFqN0W-d1NUbRv=td1G3w@mail.gmail.com>

On Wed, 8 Feb 2023 at 02:12, Rob Cliffe <rob.cliffe at btinternet.com> wrote:
>
> [re-sending this to both the list and to Chris, as a prior send to the
> list only was bounced back]
> On 31/01/2023 22:33, Chris Angelico wrote:
> >
> >> Thanks for clarifying.
> >> Hm.  So 'x' is neither in locals() nor in globals().  Which starts me
> >> wondering (to go off on a tangent): Should there be a nonlocals()
> >> dictionary?
> > I don't think so, but there might be some value in a dictionary
> > containing all available variables. It would have the same "don't
> > depend on writing" caveats that locals() has (or would be specifically
> > defined as a copy and thus disconnected), so its value would be
> > limited. And it would probably STILL be imperfect, because perfection
> > would require that it be a compiler construct, due to the way that
> > nonlocals are implemented.
> Does that mean that it is not possible to have a (built-in) function
> that would construct and return a dictionary of all available variables
> and their values?  If it were possible, it could be useful, and there
> would be no impact on Python run-time speed if it were only constructed
> on demand.
>

It can't be a built-in function and also be 100% reliable; and if it's
a special compiler construct, its presence in your code would have
semantic impact on all nested functions - even if you never call it:

def func():
    x = Obj()
    def inner():
        if False: get_vars
    return inner

x can no longer be disposed of, just in case you call inner and get the vars.

And quite frankly, I don't think this functionality justifies a magic
compiler construct and consequent keyword. But a builtin won't be
reliable.

ChrisA

From rosuav at gmail.com  Tue Feb  7 13:52:14 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 8 Feb 2023 05:52:14 +1100
Subject: Evaluation of variable as f-string
In-Reply-To: <nonlocal-20230207173920@ram.dialup.fu-berlin.de>
References: <tqmcbf$vdl$1@gioia.aioe.org>
 <CAPTjJmpKEBbYFxowgm4cDfDn0bmieo9sHLK_sNZh8WcqQ=Wqhw@mail.gmail.com>
 <ebb89997-603f-6283-742a-121128e55ae8@btinternet.com>
 <CAPTjJmoxero1FVGW8BNB619tS1XA_LOqn=Lnoq3ogw1JPh52Jg@mail.gmail.com>
 <8ebc73d2-e14a-b9ad-fa35-5078e5f9fd6c@btinternet.com>
 <CAPTjJmqbqNUfRvUt8uwVJLD--faMZVUqTD2LnhD6Cao9Fyx0Yg@mail.gmail.com>
 <33f48974-2ddc-67e4-7a79-ef9a758980a9@btinternet.com>
 <CAPTjJmpfdvb_yTMu91vNe5Wcb8_u+YO8L8c=w8AFbptj7X7Hiw@mail.gmail.com>
 <d9e19a5a-6d9e-d0c0-22f1-731990790541@btinternet.com>
 <mailman.1779.1675784381.20444.python-list@python.org>
 <nonlocal-20230207173920@ram.dialup.fu-berlin.de>
Message-ID: <CAPTjJmoXAsPGp9Q5QadwrEWS56RWSFTS9RnUGtOeiQ5=MACaow@mail.gmail.com>

On Wed, 8 Feb 2023 at 05:30, Stefan Ram <ram at zedat.fu-berlin.de> wrote:
>
> Rob Cliffe <rob.cliffe at btinternet.com> writes:
> >Does that mean that it is not possible to have a (built-in) function
> >that would construct and return a dictionary of all available variables
> >and their values?  If it were possible, it could be useful, and there
> >would be no impact on Python run-time speed if it were only constructed
> >on demand.
>
>   Here's a quick attempt to get local and non-local names as a set.
>   It might only work under some implementations of Python.
>
>   main.py
>
> set_class = set
>
> def f():
>     x = 0
>     def g():
>         set = set_class()
>         for n in g.__qualname__.split( '.' ):
>             if n != '<locals>':
>                 set = set.union( set_class( eval( n ).__code__.co_varnames ))
>         print( set )
>     g()
> f()
>
>   output
>
> {'n', 'set', 'x'}
>

Breaks on wrapped functions. Also, how are you going to get the values
of those variables?

ChrisA

From arequipeno at gmail.com  Tue Feb  7 15:32:39 2023
From: arequipeno at gmail.com (Ian Pilcher)
Date: Tue, 7 Feb 2023 14:32:39 -0600
Subject: Tool that can document private inner class?
Message-ID: <35e68b53-d64b-1b07-baf2-0c0db080c9e7@gmail.com>

I've been banging my head on Sphinx for a couple of days now, trying to
get it to include the docstrings of a private (name starts with two
underscores) inner class.  All I've managed to do is convince myself
that it really can't do it.

See https://github.com/sphinx-doc/sphinx/issues/11181.

Is there a tool out there that can do this?

-- 
========================================================================
Google                                      Where SkyNet meets Idiocracy
========================================================================

From nntp.mbourne at spamgourmet.com  Tue Feb  7 15:38:00 2023
From: nntp.mbourne at spamgourmet.com (Mark Bourne)
Date: Tue, 7 Feb 2023 20:38:00 +0000
Subject: A Function's name during its definition
In-Reply-To: <f-20230207175923@ram.dialup.fu-berlin.de>
References: <f-20230207175923@ram.dialup.fu-berlin.de>
Message-ID: <trucr8$3s6m6$1@dont-email.me>

Stefan Ram wrote:
>    When one defines a function, sometimes its name is only
>    half-existent.
> 
>    One can implicitly evaluate the name of the function:
> 
>    main.py
> 
> def g():
>      def f():
>         print( f )
>      f()
> g()
> 
>    output
> 
> <function g.<locals>.f at ...
> 
>    , but one gets an error when one tries to evaluate it explicitly:
> 
>    main.py
> 
> def g():
>      def f():
>         print( eval( 'f' ))
>      f()
> g()
> 
>    error output
> 
> NameError: name 'f' is not defined

I'm guessing that's because the name "f", referencing that function, is 
in the local scope of the "g" function.

In the first version, the interpreter searches the scope of the code 
which calls "print(f)".  It doesn't find anything named "f" there, so 
searches the next scope out, that of the function identified by "g". 
That contains "f", so it's found and can be printed.

In the second case, eval() only gets the globals and immediate locals, 
in this case the locals of "f".  The name "f" doesn't exist in the local 
scope of "f", and it doesn't exist in the global scope.  So the code 
executed by eval() can't see it.

The following does work:

def g():
     def f():
        print(eval('g'))
     f()
g()

...because in this case "g" is defined in the global scope, so the code 
in the eval call can see it.

The following also works:

def g():
     def f():
         pass
     print(eval('f'))
g()

...because in this case, eval() is called from within the scope of "g", 
so it can see the function "f" defined in that scope.


-- 
Mark.

From gweatherby at uchc.edu  Tue Feb  7 15:53:30 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Tue, 7 Feb 2023 20:53:30 +0000
Subject: Tool that can document private inner class?
In-Reply-To: <35e68b53-d64b-1b07-baf2-0c0db080c9e7@gmail.com>
References: <35e68b53-d64b-1b07-baf2-0c0db080c9e7@gmail.com>
Message-ID: <SJ0PR14MB5864F19E19149F60B23A5FF9B9DB9@SJ0PR14MB5864.namprd14.prod.outlook.com>

Yes.

Inspect module

import inspect


class Mine:

    def __init__(self):
        self.__value = 7

    def __getvalue(self):
        """Gets seven"""
        return self.__value


mine = Mine()
data = inspect.getdoc(mine)
for m in inspect.getmembers(mine):
    if '__getvalue' in m[0]:
        d = inspect.getdoc(m[1])
        print(d)


From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Ian Pilcher <arequipeno at gmail.com>
Date: Tuesday, February 7, 2023 at 3:34 PM
To: python-list at python.org <python-list at python.org>
Subject: Tool that can document private inner class?
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

I've been banging my head on Sphinx for a couple of days now, trying to
get it to include the docstrings of a private (name starts with two
underscores) inner class.  All I've managed to do is convince myself
that it really can't do it.

See https://urldefense.com/v3/__https://github.com/sphinx-doc/sphinx/issues/11181__;!!Cn_UX_p3!nTY5lSlEetgdaUdvelXo0VuYFt8-2nTnGIxK5whhkN8wXilJfeWOjyS9RJfSLbu9R6cwmmtjxYYDCfh9C4AH_g$<https://urldefense.com/v3/__https:/github.com/sphinx-doc/sphinx/issues/11181__;!!Cn_UX_p3!nTY5lSlEetgdaUdvelXo0VuYFt8-2nTnGIxK5whhkN8wXilJfeWOjyS9RJfSLbu9R6cwmmtjxYYDCfh9C4AH_g$> .

Is there a tool out there that can do this?

--
========================================================================
Google                                      Where SkyNet meets Idiocracy
========================================================================
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nTY5lSlEetgdaUdvelXo0VuYFt8-2nTnGIxK5whhkN8wXilJfeWOjyS9RJfSLbu9R6cwmmtjxYYDCfh_9Xz7jg$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nTY5lSlEetgdaUdvelXo0VuYFt8-2nTnGIxK5whhkN8wXilJfeWOjyS9RJfSLbu9R6cwmmtjxYYDCfh_9Xz7jg$>

From arequipeno at gmail.com  Tue Feb  7 16:01:23 2023
From: arequipeno at gmail.com (Ian Pilcher)
Date: Tue, 7 Feb 2023 15:01:23 -0600
Subject: Tool that can document private inner class?
In-Reply-To: <SJ0PR14MB5864F19E19149F60B23A5FF9B9DB9@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <35e68b53-d64b-1b07-baf2-0c0db080c9e7@gmail.com>
 <SJ0PR14MB5864F19E19149F60B23A5FF9B9DB9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <dd7ff4e9-1472-d7dc-bc2f-42f619b90cd3@gmail.com>

On 2/7/23 14:53, Weatherby,Gerard wrote:
> Yes.
> 
> Inspect module
> 
> import inspect
> 
> 
> class Mine:
> 
> def __init__(self):
> self.__value = 7
> 
> def __getvalue(self):
> /"""Gets seven"""
> /return self.__value
> 
> 
> mine = Mine()
> data = inspect.getdoc(mine)
> for m in inspect.getmembers(mine):
> if '__getvalue' in m[0]:
>  ??????? d = inspect.getdoc(m[1])
> print(d)
> 

Can inspect generate HTML documentation, ? la Sphinx and other tools?

-- 
========================================================================
Google                                      Where SkyNet meets Idiocracy
========================================================================


From rambiusparkisanius at gmail.com  Tue Feb  7 17:58:26 2023
From: rambiusparkisanius at gmail.com (Ivan "Rambius" Ivanov)
Date: Tue, 7 Feb 2023 17:58:26 -0500
Subject: Question about logging.config.dictConfig
Message-ID: <CAE9rwzMOXCEhsiqDnoGbzhLn1iM3P=PPsda9HrEgX2z_nqYTmA@mail.gmail.com>

Hello,

I am trying to configure my loggers using dictConfig, but they do not
print anything. Here are more details.

I have several python scripts that use a similar logging setup. I put
the common configuration in a separate module myloggingconf.py:

# myloggingconf.py
import logging

def configure_logging():
    default = logging.Formatter('%(asctime)s %(levelname)-7s %(name)s
%(funcName)s %(message)s')

    consoleHandler = logging.StreamHandler()
    consoleHandler.setFormatter(default)

    rootLogger = logging.getLogger()
    rootLogger.addHandler(consoleHandler)
    rootLogger.setLevel(logging.INFO)

Then I use it in the other scripts as:

# uselog.py
import logging
import os

from myloggingconf import configure_logging

logger = logging.getLogger(os.path.basename(__file__))

def dosmth():
    logger.debug("debug")
    logger.info("info")
    logger.warning("warning")
    logger.error("error")

def main():
    configure_logging()
    dosmth()

if __name__ == '__main__':
    main()

This works correctly and the log messages about INFO print out. Now I
want to try dictConfig. I changed myloggingconf.configure_logging to:

# myloggingconf.py
import logging.config

def configure_logging():
    config = {
        'version': 1,
        'disable_existing_loggers': True,
        'formatters': {
            'default': {
                'fmt': '%(asctime)s %(levelname)-7s %(name)s
%(funcName)s %(message)s'
            }
        },
        'handlers': {
            'stdout': {
                'level': 'DEBUG',
                'class': 'logging.StreamHandler',
                'formatter': 'default',
            }
        },
        'loggers': {
            '': {
                'handlers': ['stdout'],
                'level': 'DEBUG'
            }
        }
    }
    logging.config.dictConfig(config)

When I run uselog.py it prints nothing. I am wondering what is wrong
with the second configuration. I will appreciate any help.

Regards
rambius

-- 
Tangra Mega Rock: http://www.radiotangra.com

From nntp.mbourne at spamgourmet.com  Tue Feb  7 18:04:54 2023
From: nntp.mbourne at spamgourmet.com (Mark Bourne)
Date: Tue, 7 Feb 2023 23:04:54 +0000
Subject: A Function's name during its definition
In-Reply-To: <locals-20230207214421@ram.dialup.fu-berlin.de>
References: <f-20230207175923@ram.dialup.fu-berlin.de>
 <trucr8$3s6m6$1@dont-email.me>
 <locals-20230207214421@ram.dialup.fu-berlin.de>
Message-ID: <trulen$3tolq$1@dont-email.me>

Stefan Ram wrote:
> Mark Bourne <nntp.mbourne at spamgourmet.com> writes:
>> In the second case, eval() only gets the globals and immediate locals,
> 
>    Yes, I think you are right. Curiously, the following program would
>    mislead one to thing that eval /does/ see the intermediate names:
> 
>    main.py
> 
> def f():
>      x = 22
>      def g():
>          print( x )
>          print( eval( 'x' ))
>      g()
> f()
> 
>    output
> 
> 22
> 22
> 
>    . But "print( x )" had the effect of making that x local.
>    Without it, we see the error:
> 
>    main.py
> 
> def f():
>      x = 22
>      def g():
>          print( eval( 'x' ))
>      g()
> f()
> 
>    error output
> 
> NameError: name 'x' is not defined

That is interesting.  I know assigning to a value creates a local 
version (and the non-local then can't be accessed, even before the new 
value was assigned), but hadn't realised just referencing it brought it 
into local scope for reading.  I guess that's to do with closures, where 
any variables referenced within the function get bound to the function's 
scope.  e.g. if g() includes a reference to x [as it does in the first 
example above], and f() returned a reference to g(), calling g() later 
[from outside of f()] by using that reference would still be able to 
access x.

With just the eval() call and no other reference to x, the interpreter 
doesn't know at that time the closure is created that there is a 
reference to x.  At that point, the 'x' is just text in a string.  It's 
only when eval() gets called that it tries to execute that string as 
Python code - and finds that x isn't in scope.

I'm not all that familiar with the concept of closures, so may have got 
some of the terminology and details wrong - but it's something along 
those lines.  I'm sure others will correct me...

-- 
Mark.

From rosuav at gmail.com  Tue Feb  7 18:33:44 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 8 Feb 2023 10:33:44 +1100
Subject: A Function's name during its definition
In-Reply-To: <trulen$3tolq$1@dont-email.me>
References: <f-20230207175923@ram.dialup.fu-berlin.de>
 <trucr8$3s6m6$1@dont-email.me>
 <locals-20230207214421@ram.dialup.fu-berlin.de> <trulen$3tolq$1@dont-email.me>
Message-ID: <CAPTjJmrrBQ1fRTDe3wrDpw8rj3+xb20KM9rhReytMeijQVS-dQ@mail.gmail.com>

On Wed, 8 Feb 2023 at 10:18, Mark Bourne <nntp.mbourne at spamgourmet.com> wrote:
>
> Stefan Ram wrote:
> > Mark Bourne <nntp.mbourne at spamgourmet.com> writes:
> >> In the second case, eval() only gets the globals and immediate locals,
> >
> >    Yes, I think you are right. Curiously, the following program would
> >    mislead one to thing that eval /does/ see the intermediate names:
> >
> >    main.py
> >
> > def f():
> >      x = 22
> >      def g():
> >          print( x )
> >          print( eval( 'x' ))
> >      g()
> > f()
> >
> >    output
> >
> > 22
> > 22
> >
> >    . But "print( x )" had the effect of making that x local.
> >    Without it, we see the error:
> >
> >    main.py
> >
> > def f():
> >      x = 22
> >      def g():
> >          print( eval( 'x' ))
> >      g()
> > f()
> >
> >    error output
> >
> > NameError: name 'x' is not defined

Don't know if Stefan realises it, but newsgroup text that gets replies
ends up on the mailing list. So if this is a problem of copyright, you
should probably stop posting to the newsgroup - unless you want people
like Mark to unintentionally violate your copyright.

> That is interesting.  I know assigning to a value creates a local
> version (and the non-local then can't be accessed, even before the new
> value was assigned), but hadn't realised just referencing it brought it
> into local scope for reading.  I guess that's to do with closures, where
> any variables referenced within the function get bound to the function's
> scope.  e.g. if g() includes a reference to x [as it does in the first
> example above], and f() returned a reference to g(), calling g() later
> [from outside of f()] by using that reference would still be able to
> access x.

Yep, that's to do with closures. Your analysis is spot on. When a name
is local to an outer function and referenced in an inner function, it
becomes a nonlocal in both functions. You can also declare it with
"nonlocal x" in the inner function, which - like with a global
statement - allows you to assign to it.

> With just the eval() call and no other reference to x, the interpreter
> doesn't know at that time the closure is created that there is a
> reference to x.  At that point, the 'x' is just text in a string.  It's
> only when eval() gets called that it tries to execute that string as
> Python code - and finds that x isn't in scope.

Mostly true, yes. The eval function doesn't get any magic ability to
refer to nearby variables, so what it does is it depends on the
globals() and locals() mappings (which can be the same dictionary
under some circumstances).

I think you'll agree that globals() does not and should not contain a
reference to x here. It's definitely not a global.

But is x a local? Well, technically it's a nonlocal; but more
importantly, there's a difference between "theoretically could be a
nonlocal" and "is actually a nonlocal", as demonstrated by this:

def f():
    x = 42
    def g():
        print("without", locals())
    g()
    def h():
        x
        print("with", locals())
    h()

So, can eval() find x?

x = 1234
def f():
    x = 42
    def g():
        print("without", eval("x"))
    g()
    def h():
        x
        print("with", eval("x"))
    h()

>>> f()
without 1234
with 42

The compiler knows exactly which names are *theoretically available*
(which is a lexical concept), but at run time, the only names that are
actually available are those which have been captured.

ChrisA

From hjp-python at hjp.at  Tue Feb  7 19:31:56 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Wed, 8 Feb 2023 01:31:56 +0100
Subject: Question about logging.config.dictConfig
In-Reply-To: <CAE9rwzMOXCEhsiqDnoGbzhLn1iM3P=PPsda9HrEgX2z_nqYTmA@mail.gmail.com>
References: <CAE9rwzMOXCEhsiqDnoGbzhLn1iM3P=PPsda9HrEgX2z_nqYTmA@mail.gmail.com>
Message-ID: <20230208003156.mfhoozbkr2rv2dde@hjp.at>

On 2023-02-07 17:58:26 -0500, Ivan "Rambius" Ivanov wrote:
> I am trying to configure my loggers using dictConfig, but they do not
> print anything. Here are more details.
[...]
> from myloggingconf import configure_logging
> 
> logger = logging.getLogger(os.path.basename(__file__))
[...]
> 
> def main():
>     configure_logging()
>     dosmth()
[...]
> def configure_logging():
>     config = {
>         'version': 1,
>         'disable_existing_loggers': True,

          'disable_existing_loggers': False,

I think I just found out why I've been cargo-culting this line since my
early Django days ;-)-

If you call getLogger very early (as you do), you don't want it disabled
by a later call to dictConfig().


>         'formatters': {
>             'default': {
>                 'fmt': '%(asctime)s %(levelname)-7s %(name)s %(funcName)s %(message)s'
                  'format'
>             }
>         },
[...]
>     }
>     logging.config.dictConfig(config)
> 
> When I run uselog.py it prints nothing. I am wondering what is wrong
> with the second configuration.

See above.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230208/96e5ba01/attachment.sig>

From rob.cliffe at btinternet.com  Tue Feb  7 18:56:39 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 7 Feb 2023 23:56:39 +0000
Subject: evaluation question
In-Reply-To: <CAPTjJmr56ZYp0g30u9Eyd8i=XRp6if4Unttw_ZQpg25yqzYF5A@mail.gmail.com>
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me> <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me>
 <807535a2-dd1e-dd33-e6a6-cf17264ab955@btinternet.com>
 <CAPTjJmr56ZYp0g30u9Eyd8i=XRp6if4Unttw_ZQpg25yqzYF5A@mail.gmail.com>
Message-ID: <ada9f42e-3328-9ca7-ffd3-946c8de4d8fc@btinternet.com>



On 07/02/2023 08:15, Chris Angelico wrote:
> On Tue, 7 Feb 2023 at 18:49, Rob Cliffe via Python-list
> <python-list at python.org> wrote:
>>
>>
>> On 02/02/2023 09:31, Muttley at dastardlyhq.com wrote:
>>> On Wed, 1 Feb 2023 18:28:04 +0100
>>> "Peter J. Holzer" <hjp-python at hjp.at> wrote:
>>>> --b2nljkb3mdefsdhx
>>>> Content-Type: text/plain; charset=us-ascii
>>>> Content-Disposition: inline
>>>> Content-Transfer-Encoding: quoted-printable
>>>>
>>>> On 2023-02-01 09:00:39 -0000, Muttley at dastardlyhq.com wrote:
>>>>> Its not evolution, its revolution. Evolution retains old functionality.
>>>> Tell a penguin that it can fly :-)
>>> Yeah ok :) But the ancestors of penguins didn't wake up one morning, flap
>>> their wings and fall out the tree, it happened gradually. Python2 syntax
>>> could have been retained for X versions of 3 just as C++ keeps old stuff
>>> until its eventually deprecated them removed.
>> Yeah?  So what would this do:
>>       print ()
>> In Python 2 this prints an empty tuple.
>> In Python 3 this is a call to the print function with no arguments,
>> which prints a blank line.
>> You can't have it both ways.
>> In any case, supporting two different syntaxes simultaneously would be
>> messy and difficult to maintain.
> There are two solutions to this. The most obvious is "from __future__
> import print_function", which gives you the full power and flexibility
> of Py3 in anything back as far as 2.6; the other is to always pass a
> single string argument to print:
>
> print("")
> print("spam %d ham %d" % (spam, ham))
>
> This will work in pretty much ANY version of Python [1] and doesn't
> require any sort of per-module configuration.
>
> The idea that old syntax should be retained is only part of the story.
> While it's definitely important to not break old code unnecessarily,
> it is far more important to ensure that there's *some way* to write
> code that works across multiple versions. That's what we have here:
> even with the breaking changes, there was usually a way to make your
> code run identically on multiple versions. Sometimes this means a
> compatibility shim at the top, like "try: raw_input; except NameError:
> raw_input = input", and sometimes it means following a discipline like
> putting b"..." for all strings that need to be bytes. But there always
> needs to be a way.
>
> ChrisA
>
> [1] This is the part where someone points out to me that it wouldn't
> work in Python 1.3 or something
You are quite right Chris, and indeed I have used both solutions in my 
own code to keep 2-3 compatibility.
I was just pointing out that continuing to support Python 2 syntax in 
Python 3 was not an option.
Best wishes
Rob Cliffe

From Karsten.Hilbert at gmx.net  Wed Feb  8 06:20:48 2023
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Wed, 8 Feb 2023 12:20:48 +0100
Subject: pylint scoping question
Message-ID: <Y+OFkPB+JTmp0cWN@hermes.hilbert.loc>

Dear readers,

I have a pylint scoping (or how-to) question.

	pylint 2.7.2
	astroid 2.5.1
	Python 3.9.2 (default, Feb 28 2021, 17:03:44)

Objective:

to disable all pylint errors/warnings starting from a
particular source line until EOF (that part contains
to-be-run-manually scratch/test code for that file)

As an MWE consider this code:

#-----------------------------------------------
usr/bin/python3
"""MWE"""

print(does_not_exist_1)
# Xpylint: disable=undefined-variable
# Xpylint: disable=all
print(does_not_exist_2)
print(does_not_exist_3)
#-----------------------------------------------

Pylint docs say that disables are per-scope. Thusly a

	# pylint: disable=undefined-variable

inside the file-global scope should (?) disable
undefined-variable for the entire (?) file-global scope.

It does not, however, but rather seems to disable it inside
the file-global scope *from the line of occurrence onwards*.
To see this, remove the X from the relevant disable in the
MWE. This is the behaviour I desire to achieve.

However, when using the "disable=all" it *does* disable all
output, including the

	x.py:4:6: E0602: Undefined variable 'does_not_exist_1' (undefined-variable)

despite the

	# pylint: disable=all

sitting *after* the

	print(does_not_exist_1)

So:

Am I doing something wrong ?

Am I misinterpreting the docs ?

Do the docs explain this difference in disable-scoping ?

How should I properly go about my objective (apart from
fixing my code, of course :-D ) ?

Thanks for insights,
Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B

From Karsten.Hilbert at gmx.net  Wed Feb  8 06:25:41 2023
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Wed, 8 Feb 2023 12:25:41 +0100
Subject: pylint scoping question
In-Reply-To: <Y+OFkPB+JTmp0cWN@hermes.hilbert.loc>
References: <Y+OFkPB+JTmp0cWN@hermes.hilbert.loc>
Message-ID: <Y+OGtbW/5Wr6/nH5@hermes.hilbert.loc>

Am Wed, Feb 08, 2023 at 12:20:48PM +0100 schrieb Karsten Hilbert:

> I have a pylint scoping (or how-to) question.
...
> Objective:
>
> to disable all pylint errors/warnings starting from a
> particular source line until EOF (that part contains
> to-be-run-manually scratch/test code for that file)

This

	https://stackoverflow.com/questions/66914050/what-is-the-scope-of-pylint-comments

BTW, is the seemingly closest related information I could
find but it does not explain the difference in scoping
between disable=something and disable=all.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B

From shishaozhong at gmail.com  Wed Feb  8 06:39:40 2023
From: shishaozhong at gmail.com (Shaozhong SHI)
Date: Wed, 8 Feb 2023 11:39:40 +0000
Subject: tree representation of Python data
In-Reply-To: <016498db-aca3-9f53-4118-448ce35bb92d@tompassin.net>
References: <tqgus0$1vtt$1@gioia.aioe.org>
 <016498db-aca3-9f53-4118-448ce35bb92d@tompassin.net>
Message-ID: <CA+i5Jwb7+o0_-78iMB8TVqtjpTwQ0AEoDAT5Hu9APBV4k31xuQ@mail.gmail.com>

What is the robust way to use Python to read in an XML and turn it into a
JSON file?

JSON dictionary is actually a tree.  It is much easier to manage the
tree-structured data.

Regards,

David

From gweatherby at uchc.edu  Wed Feb  8 09:25:59 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Wed, 8 Feb 2023 14:25:59 +0000
Subject: Tool that can document private inner class?
In-Reply-To: <dd7ff4e9-1472-d7dc-bc2f-42f619b90cd3@gmail.com>
References: <35e68b53-d64b-1b07-baf2-0c0db080c9e7@gmail.com>
 <SJ0PR14MB5864F19E19149F60B23A5FF9B9DB9@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <dd7ff4e9-1472-d7dc-bc2f-42f619b90cd3@gmail.com>
Message-ID: <SJ0PR14MB5864C5B2C1075C0ACE28B183B9D89@SJ0PR14MB5864.namprd14.prod.outlook.com>

No.

I interpreted your query as ?is there something that can read docstrings of dunder methods??

Have you tried the Sphinx specific support forums? https://www.sphinx-doc.org/en/master/support.html

From: Ian Pilcher <arequipeno at gmail.com>
Date: Tuesday, February 7, 2023 at 4:01 PM
To: Weatherby,Gerard <gweatherby at uchc.edu>, python-list at python.org <python-list at python.org>
Subject: Re: Tool that can document private inner class?
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

On 2/7/23 14:53, Weatherby,Gerard wrote:
> Yes.
>
> Inspect module
>
> import inspect
>
>
> class Mine:
>
> def __init__(self):
> self.__value = 7
>
> def __getvalue(self):
> /"""Gets seven"""
> /return self.__value
>
>
> mine = Mine()
> data = inspect.getdoc(mine)
> for m in inspect.getmembers(mine):
> if '__getvalue' in m[0]:
>          d = inspect.getdoc(m[1])
> print(d)
>

Can inspect generate HTML documentation, ? la Sphinx and other tools?

--
========================================================================
Google                                      Where SkyNet meets Idiocracy
========================================================================

From robbiemezazem at hotmail.com  Wed Feb  8 09:29:38 2023
From: robbiemezazem at hotmail.com (Robbie mezazem)
Date: Wed, 8 Feb 2023 14:29:38 +0000
Subject: evaluation question
In-Reply-To: <ada9f42e-3328-9ca7-ffd3-946c8de4d8fc@btinternet.com>
References: <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <k3r3rgF5l1hU1@mid.individual.net>
 <tramnp$3ohlk$1@dont-email.me> <k3tkqhFhgf7U1@mid.individual.net>
 <062f691c-d4b7-09f9-c949-a14ef18119f7@DancesWithMice.info>
 <mailman.1733.1675210672.20444.python-list@python.org>
 <trd9nn$94gv$1@dont-email.me> <20230201172804.whx4lqlngs4qnkic@hjp.at>
 <mailman.1741.1675273066.20444.python-list@python.org>
 <trfvu2$qhcd$1@dont-email.me>
 <807535a2-dd1e-dd33-e6a6-cf17264ab955@btinternet.com>
 <CAPTjJmr56ZYp0g30u9Eyd8i=XRp6if4Unttw_ZQpg25yqzYF5A@mail.gmail.com>
 <ada9f42e-3328-9ca7-ffd3-946c8de4d8fc@btinternet.com>
Message-ID: <BYAPR04MB50629525DB0154DC5CCCB543BFD89@BYAPR04MB5062.namprd04.prod.outlook.com>

Ok I understand


Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows

From: Rob Cliffe via Python-list<mailto:python-list at python.org>
Sent: Tuesday, February 7, 2023 6:54 PM
To: Chris Angelico<mailto:rosuav at gmail.com>; python-list at python.org<mailto:python-list at python.org>
Subject: Re: evaluation question



On 07/02/2023 08:15, Chris Angelico wrote:
> On Tue, 7 Feb 2023 at 18:49, Rob Cliffe via Python-list
> <python-list at python.org> wrote:
>>
>>
>> On 02/02/2023 09:31, Muttley at dastardlyhq.com wrote:
>>> On Wed, 1 Feb 2023 18:28:04 +0100
>>> "Peter J. Holzer" <hjp-python at hjp.at> wrote:
>>>> --b2nljkb3mdefsdhx
>>>> Content-Type: text/plain; charset=us-ascii
>>>> Content-Disposition: inline
>>>> Content-Transfer-Encoding: quoted-printable
>>>>
>>>> On 2023-02-01 09:00:39 -0000, Muttley at dastardlyhq.com wrote:
>>>>> Its not evolution, its revolution. Evolution retains old functionality.
>>>> Tell a penguin that it can fly :-)
>>> Yeah ok :) But the ancestors of penguins didn't wake up one morning, flap
>>> their wings and fall out the tree, it happened gradually. Python2 syntax
>>> could have been retained for X versions of 3 just as C++ keeps old stuff
>>> until its eventually deprecated them removed.
>> Yeah?  So what would this do:
>>       print ()
>> In Python 2 this prints an empty tuple.
>> In Python 3 this is a call to the print function with no arguments,
>> which prints a blank line.
>> You can't have it both ways.
>> In any case, supporting two different syntaxes simultaneously would be
>> messy and difficult to maintain.
> There are two solutions to this. The most obvious is "from __future__
> import print_function", which gives you the full power and flexibility
> of Py3 in anything back as far as 2.6; the other is to always pass a
> single string argument to print:
>
> print("")
> print("spam %d ham %d" % (spam, ham))
>
> This will work in pretty much ANY version of Python [1] and doesn't
> require any sort of per-module configuration.
>
> The idea that old syntax should be retained is only part of the story.
> While it's definitely important to not break old code unnecessarily,
> it is far more important to ensure that there's *some way* to write
> code that works across multiple versions. That's what we have here:
> even with the breaking changes, there was usually a way to make your
> code run identically on multiple versions. Sometimes this means a
> compatibility shim at the top, like "try: raw_input; except NameError:
> raw_input = input", and sometimes it means following a discipline like
> putting b"..." for all strings that need to be bytes. But there always
> needs to be a way.
>
> ChrisA
>
> [1] This is the part where someone points out to me that it wouldn't
> work in Python 1.3 or something
You are quite right Chris, and indeed I have used both solutions in my
own code to keep 2-3 compatibility.
I was just pointing out that continuing to support Python 2 syntax in
Python 3 was not an option.
Best wishes
Rob Cliffe
--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Wed Feb  8 09:37:19 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 8 Feb 2023 09:37:19 -0500
Subject: tree representation of Python data
In-Reply-To: <CA+i5Jwb7+o0_-78iMB8TVqtjpTwQ0AEoDAT5Hu9APBV4k31xuQ@mail.gmail.com>
References: <tqgus0$1vtt$1@gioia.aioe.org>
 <016498db-aca3-9f53-4118-448ce35bb92d@tompassin.net>
 <CA+i5Jwb7+o0_-78iMB8TVqtjpTwQ0AEoDAT5Hu9APBV4k31xuQ@mail.gmail.com>
Message-ID: <604b5c9e-e61e-105f-1f47-ab5a3759745d@tompassin.net>

On 2/8/2023 6:39 AM, Shaozhong SHI wrote:
> What is the robust way to use Python to read in an XML and turn it into 
> a JSON file?
> 
> JSON dictionary is actually a tree.? It is much easier to manage the 
> tree-structured data.

XML and JSON are both for interchanging data.  What are you trying to 
accomplish? XML elements form a tree, XML attributes can be thought of 
as a dictionary.   JSON can contain both lists and associative arrays 
(dictionaries).  Personally, I would not say it's easier to "manage" 
tree-structured data than dictionaries, but either way it sounds like 
you want to send or receive data, rather than "managing" data in program 
data structures.

So why do you want to convert data in XML form to JSON?  Once you have 
XML ingested into some Python data structure, you can use the json 
library to output JSON- see

https://docs.python.org/3/library/json.html

There is a pip-installable program (which I have never used), xmltodict, 
that stores an XML file as a Python dictionary.  You then can write it 
to JSON as above.  I don't know how general this program is - XML is not 
isomorphic to JSON, but maybe your XML sources never use anything 
besides elements and attributes, and don't use namespaces.  Then it's 
pretty easy.


From rambiusparkisanius at gmail.com  Wed Feb  8 10:27:22 2023
From: rambiusparkisanius at gmail.com (Ivan "Rambius" Ivanov)
Date: Wed, 8 Feb 2023 10:27:22 -0500
Subject: Question about logging.config.dictConfig
In-Reply-To: <20230208003156.mfhoozbkr2rv2dde@hjp.at>
References: <CAE9rwzMOXCEhsiqDnoGbzhLn1iM3P=PPsda9HrEgX2z_nqYTmA@mail.gmail.com>
 <20230208003156.mfhoozbkr2rv2dde@hjp.at>
Message-ID: <CAE9rwzO9e0i9XqKHiew25Z7tqBUL0gZ1oUXsz_Xh+mtgzzw_gw@mail.gmail.com>

On Tue, Feb 7, 2023 at 7:35 PM Peter J. Holzer <hjp-python at hjp.at> wrote:
>
> On 2023-02-07 17:58:26 -0500, Ivan "Rambius" Ivanov wrote:
> > I am trying to configure my loggers using dictConfig, but they do not
> > print anything. Here are more details.
> [...]
> > from myloggingconf import configure_logging
> >
> > logger = logging.getLogger(os.path.basename(__file__))
> [...]
> >
> > def main():
> >     configure_logging()
> >     dosmth()
> [...]
> > def configure_logging():
> >     config = {
> >         'version': 1,
> >         'disable_existing_loggers': True,
>
>           'disable_existing_loggers': False,

Thank you! That helped a lot!

Regards
rambius

>
> I think I just found out why I've been cargo-culting this line since my
> early Django days ;-)-
>
> If you call getLogger very early (as you do), you don't want it disabled
> by a later call to dictConfig().
>
>
> >         'formatters': {
> >             'default': {
> >                 'fmt': '%(asctime)s %(levelname)-7s %(name)s %(funcName)s %(message)s'
>                   'format'
> >             }
> >         },
> [...]
> >     }
> >     logging.config.dictConfig(config)
> >
> > When I run uselog.py it prints nothing. I am wondering what is wrong
> > with the second configuration.
>
> See above.
>
>         hp
>
> --
>    _  | Peter J. Holzer    | Story must make more sense than reality.
> |_|_) |                    |
> | |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |       challenge!"
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
Tangra Mega Rock: http://www.radiotangra.com

From arequipeno at gmail.com  Wed Feb  8 14:22:16 2023
From: arequipeno at gmail.com (Ian Pilcher)
Date: Wed, 8 Feb 2023 13:22:16 -0600
Subject: Tool that can document private inner class?
In-Reply-To: <SJ0PR14MB5864C5B2C1075C0ACE28B183B9D89@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <35e68b53-d64b-1b07-baf2-0c0db080c9e7@gmail.com>
 <SJ0PR14MB5864F19E19149F60B23A5FF9B9DB9@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <dd7ff4e9-1472-d7dc-bc2f-42f619b90cd3@gmail.com>
 <SJ0PR14MB5864C5B2C1075C0ACE28B183B9D89@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <5f2f869c-e9c1-da8f-d05c-fccb61d1d291@gmail.com>

On 2/8/23 08:25, Weatherby,Gerard wrote:
> No.
> 
> I interpreted your query as ?is there something that can read docstrings 
> of dunder methods??
> 
> Have you tried the Sphinx specific support forums? 
> https://www.sphinx-doc.org/en/master/support.html 

Yes.  I've posted to both the -user and -dev groups, and I've also filed
an issue[1].  I haven't received a response.

Thus my conclusion that Sphinx, specifically autodoc, simply can't do
this.

[1] https://github.com/sphinx-doc/sphinx/issues/11181

-- 
========================================================================
Google                                      Where SkyNet meets Idiocracy
========================================================================


From python at bladeshadow.org  Fri Feb 10 14:59:07 2023
From: python at bladeshadow.org (Python)
Date: Fri, 10 Feb 2023 13:59:07 -0600
Subject: evaluation question
In-Reply-To: <tr83bf$381e1$1@dont-email.me>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
Message-ID: <20230210195907.GC4979@bladeshadow.org>

On Mon, Jan 30, 2023 at 09:41:03AM -0000, Muttley at dastardlyhq.com wrote:
> >Because print() returns nothing (i.e., the statement x is None is True). 
> 
> I don't understand this. What was the point of the upheaval of converting
> the print command in python 2 into a function in python 3 if as a function
> print() doesn't return anything useful? Surely even the length of the 
> formatted string as per C's sprintf() function would be helpful?

Python is far from the only language that allows functions to return
nothing explicit.  Statically typed languages typically use the void
keyword or something similar to indicate this, and dynamically typed
languages often just don't explicitly return anything.  [Some
languages, like Perl, just implicitly return the value of the last
expression, or something of the sort, but some do not.] In Pascal, in
fact, there are two different types of subroutines to distinguish
between those things: functions return a value, and procedures do not.
Modern languages commonly don't provide this distinction because by
and large it is unnecessary.

Typically the reason NOT to return a value is that the function is
designed to DO something, rather than to calculate some value.
Examples might be updating the appearance of a UI widget, printing
some information to the screen, or twiddling some bits in a register
(i.e. in-place update of a value).  They don't need to return anything
because the "output" is whatever was done.  In some languages it might
be typical to return a status indicating success or failure, but in
many languages this is unnecessary; status may be returned via a
parameter provided for the purpose, or in many languages, success can
be assumed, whereas failure will raise an exception.

If it's the case that you simply want to know the length of the string
that will be printed, you can, rather than expecting the I/O function
to tell that to you, figure it out for yourself ahead of time, e.g.
instead of:

    username = "John Smith"
    job = "Python programmer"

    # this doesn't work as desired
    len = print(f"{username} has occupation {job}.")
    print(len)
    ...

You would do this instead:

    message = f"{username} has the occupation {job}."
    message_length = len(message)
    print(message)
    print(message_length)
    ...


From Marco.Sulla.Python at gmail.com  Fri Feb 10 16:10:39 2023
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Fri, 10 Feb 2023 22:10:39 +0100
Subject: Am I banned from Discuss forum?
Message-ID: <CABbU2U9cY5o8UUYj8tJaum87_OrvtxSFUOFdZrxno6smx4aYhw@mail.gmail.com>

I was banned from the mailing list and Discuss forum for a very long time.
Too much IMHO, but I paid my dues.

Now this is my state in the forum:
- I never posted something unrespectful in the last months
- I have a limitation of three posts per threads, but only on some threads
- Some random posts of mine are obscured and must be restored manually by
moderators
- I opened a thread about the proposal of a new section called
Brainstorming. It was closed without a reason.
- I can't post links
- Two discussions I posted in section Idea were moved to Help, without a
single line of explanation.

If I'm not appreciated, I want to be publicly banned with a good reason, or
at least a reason.

From rosuav at gmail.com  Fri Feb 10 16:30:22 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 11 Feb 2023 08:30:22 +1100
Subject: evaluation question
In-Reply-To: <20230210195907.GC4979@bladeshadow.org>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
 <20230210195907.GC4979@bladeshadow.org>
Message-ID: <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>

On Sat, 11 Feb 2023 at 07:36, Python <python at bladeshadow.org> wrote:
> If it's the case that you simply want to know the length of the string
> that will be printed, you can, rather than expecting the I/O function
> to tell that to you, figure it out for yourself ahead of time, e.g.
> instead of:
>
>     username = "John Smith"
>     job = "Python programmer"
>
>     # this doesn't work as desired
>     len = print(f"{username} has occupation {job}.")
>     print(len)
>     ...
>
> You would do this instead:
>
>     message = f"{username} has the occupation {job}."
>     message_length = len(message)
>     print(message)
>     print(message_length)
>     ...
>

It's worth noting WHY output functions often return a byte count. It's
primarily for use with nonblocking I/O, with something like this:

buffer = b".............."
buffer = buffer[os.write(fd, buffer):]

It's extremely important to be able to do this sort of thing, but not
with the print function, which has a quite different job.

ChrisA

From python at bladeshadow.org  Fri Feb 10 16:55:37 2023
From: python at bladeshadow.org (Python)
Date: Fri, 10 Feb 2023 15:55:37 -0600
Subject: evaluation question
In-Reply-To: <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
 <20230210195907.GC4979@bladeshadow.org>
 <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>
Message-ID: <20230210215537.GD4979@bladeshadow.org>

On Sat, Feb 11, 2023 at 08:30:22AM +1100, Chris Angelico wrote:
> On Sat, 11 Feb 2023 at 07:36, Python <python at bladeshadow.org> wrote:
> > You would do this instead:
> >
> >     message = f"{username} has the occupation {job}."
> >     message_length = len(message)
> >     print(message)
> >     print(message_length)
> >     ...
> >
> 
> It's worth noting WHY output functions often return a byte count. It's
> primarily for use with nonblocking I/O, with something like this:
> 
> buffer = b".............."
> buffer = buffer[os.write(fd, buffer):]
> 
> It's extremely important to be able to do this sort of thing, but not
> with the print function, which has a quite different job.

I would agree with this only partially.  Your case applies to
os.write(), which is essentially just a wrapper around the write()
system call, which has that sort of property... though it applies also
to I/O in blocking mode, particularly on network sockets, where the
number of bytes you asked to write (or read) may not all have been
transferred, necessitating trying in a loop.

However, Python's print() function is more analogous to C's printf(),
which returns the number of characters converted for an entirely
different reason... It's precisely so that you'll know what the length
of the string that was converted is.  This is most useful with the
*snprintf() variants where you're actually concerned about overrunning
the buffer you've provided for the output string, so you can realloc()
the buffer if it was indeed too small, but it is also useful in the
context of, say, a routine to format text according to the size of
your terminal.  In that context it really has nothing to do with
blocking I/O or socket behavior.


From list1 at tompassin.net  Fri Feb 10 17:48:53 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 10 Feb 2023 17:48:53 -0500
Subject: evaluation question
In-Reply-To: <20230210215537.GD4979@bladeshadow.org>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <20230210195907.GC4979@bladeshadow.org>
 <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>
 <20230210215537.GD4979@bladeshadow.org>
Message-ID: <b3d18627-a622-4279-118b-9cbfb8e9b2ab@tompassin.net>

On 2/10/2023 4:55 PM, Python wrote:
> However, Python's print() function is more analogous to C's printf(),
> which returns the number of characters converted for an entirely
> different reason... It's precisely so that you'll know what the length
> of the string that was converted is.  This is most useful with the
> *snprintf() variants where you're actually concerned about overrunning
> the buffer you've provided for the output string, so you can realloc()
> the buffer if it was indeed too small, but it is also useful in the
> context of, say, a routine to format text according to the size of
> your terminal.  In that context it really has nothing to do with
> blocking I/O or socket behavior.

But none of that applies to the Python print() function. There are no 
buffers to overrun, no reason to know the length of the printed string, 
no re-allocating of a buffer.  It's certainly possible that one might 
want to know the actual physical length of a displayed string - perhaps 
to display it on a graphic - but now we're getting into font metrics and 
such things, and we'll be doing something more active than displaying on 
a terminal via stdout.

I don't know why the print() function doesn't return anything, but I'm 
fine with it. I've never felt that I needed to know.

From rosuav at gmail.com  Fri Feb 10 18:10:33 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 11 Feb 2023 10:10:33 +1100
Subject: evaluation question
In-Reply-To: <b3d18627-a622-4279-118b-9cbfb8e9b2ab@tompassin.net>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
 <20230210195907.GC4979@bladeshadow.org>
 <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>
 <20230210215537.GD4979@bladeshadow.org>
 <b3d18627-a622-4279-118b-9cbfb8e9b2ab@tompassin.net>
Message-ID: <CAPTjJmq_=cqUZz8fzxyhDTJfwNyUUttU1q2erXLy9QwLo7PEDw@mail.gmail.com>

On Sat, 11 Feb 2023 at 10:07, Thomas Passin <list1 at tompassin.net> wrote:
>
> On 2/10/2023 4:55 PM, Python wrote:
> > However, Python's print() function is more analogous to C's printf(),
> > which returns the number of characters converted for an entirely
> > different reason... It's precisely so that you'll know what the length
> > of the string that was converted is.  This is most useful with the
> > *snprintf() variants where you're actually concerned about overrunning
> > the buffer you've provided for the output string, so you can realloc()
> > the buffer if it was indeed too small, but it is also useful in the
> > context of, say, a routine to format text according to the size of
> > your terminal.  In that context it really has nothing to do with
> > blocking I/O or socket behavior.
>
> But none of that applies to the Python print() function. There are no
> buffers to overrun, no reason to know the length of the printed string,
> no re-allocating of a buffer.  It's certainly possible that one might
> want to know the actual physical length of a displayed string - perhaps
> to display it on a graphic - but now we're getting into font metrics and
> such things, and we'll be doing something more active than displaying on
> a terminal via stdout.

It's sometimes possible to have that go to stdout (maybe you want to
do columnar text or something), but yeah, you'll generally do that by
formatting first, then doing the measurement, then displaying.

ChrisA

From avi.e.gross at gmail.com  Fri Feb 10 18:17:42 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Fri, 10 Feb 2023 18:17:42 -0500
Subject: evaluation question
In-Reply-To: <20230210215537.GD4979@bladeshadow.org>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <20230210195907.GC4979@bladeshadow.org>
 <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>
 <20230210215537.GD4979@bladeshadow.org>
Message-ID: <003501d93da5$e0b00330$a2100990$@gmail.com>

There are no doubt many situations someone wants to know how long something
will be when printed but often at lower levels.

In variable-width fonts, for example, the number of characters does not
really line up precisely with how many characters. Some encodings use a
varying number of bytes and, again, the width of the output varies.

So for people who want to make 2-D formatted output like tables, or who want
to wrap lines longer than N characters, you more often let some deeper
software accept your data and decide on formatting it internally and either
print it at once, when done calculating, or in the case of some old-style
terminals, use something like the curses package that may use escape
sequences to update the screen more efficiently in various ways.

If someone wants more control over what they print, rather than asking the
print() function to return something that is mostly going to be ignored,
they can do the things others have already suggested here. You can make your
message parts in advance and measure their length or anything else before
you print. Or make a wrapper that does something for you before calling
print, perhaps only for common cases and then returns the length to you
after printing.

I wonder if the next request will be for  print() to know what your output
device is and other current settings so it return the width your text takes
up in pixels in the current font/size ...

I add a tidbit that many ways of printing allow you to specify the width you
want something printed in such as you want a floating point value with so
many digits after the decimal point in a zero or space padded field on the
left. So there are ways to calculate in advance for many common cases as to
how long each part will be if you specify it. Besides, I am not really sure
if "print" even knows easily how many characters it is putting out as it
chews away on the many things in your request and calls dunder methods in
objects so they display themselves and so on. I assume it can be made to
keep track, albeit I can imagine printing out an APL program with lots of
overwritten characters where the number of bytes sent is way more than the
number of spaces in the output.

Why are we even still talking about this? The answer to the question of why
print() does not return anything, let alone the number of characters
printed, is BECAUSE.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Python
Sent: Friday, February 10, 2023 4:56 PM
To: python-list at python.org
Subject: Re: evaluation question

On Sat, Feb 11, 2023 at 08:30:22AM +1100, Chris Angelico wrote:
> On Sat, 11 Feb 2023 at 07:36, Python <python at bladeshadow.org> wrote:
> > You would do this instead:
> >
> >     message = f"{username} has the occupation {job}."
> >     message_length = len(message)
> >     print(message)
> >     print(message_length)
> >     ...
> >
> 
> It's worth noting WHY output functions often return a byte count. It's 
> primarily for use with nonblocking I/O, with something like this:
> 
> buffer = b".............."
> buffer = buffer[os.write(fd, buffer):]
> 
> It's extremely important to be able to do this sort of thing, but not 
> with the print function, which has a quite different job.

I would agree with this only partially.  Your case applies to os.write(),
which is essentially just a wrapper around the write() system call, which
has that sort of property... though it applies also to I/O in blocking mode,
particularly on network sockets, where the number of bytes you asked to
write (or read) may not all have been transferred, necessitating trying in a
loop.

However, Python's print() function is more analogous to C's printf(), which
returns the number of characters converted for an entirely different
reason... It's precisely so that you'll know what the length of the string
that was converted is.  This is most useful with the
*snprintf() variants where you're actually concerned about overrunning the
buffer you've provided for the output string, so you can realloc() the
buffer if it was indeed too small, but it is also useful in the context of,
say, a routine to format text according to the size of your terminal.  In
that context it really has nothing to do with blocking I/O or socket
behavior.

--
https://mail.python.org/mailman/listinfo/python-list


From greg.ewing at canterbury.ac.nz  Fri Feb 10 20:31:36 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 11 Feb 2023 14:31:36 +1300
Subject: ChatGPT Generated news poster code
In-Reply-To: <ts1vv8$io6i$1@dont-email.me>
References: <ts1vv8$io6i$1@dont-email.me>
Message-ID: <k4o9foFm0asU1@mid.individual.net>

For a moment I thought this was going to be a script that
uses ChatGPT to generate a random news post and post it
to Usenet...

Which would also have been kind of cool, as long as it wasn't
overused.

-- 
Greg

From python at bladeshadow.org  Fri Feb 10 21:10:43 2023
From: python at bladeshadow.org (Python)
Date: Fri, 10 Feb 2023 20:10:43 -0600
Subject: evaluation question
In-Reply-To: <b3d18627-a622-4279-118b-9cbfb8e9b2ab@tompassin.net>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me>
 <20230210195907.GC4979@bladeshadow.org>
 <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>
 <20230210215537.GD4979@bladeshadow.org>
 <b3d18627-a622-4279-118b-9cbfb8e9b2ab@tompassin.net>
Message-ID: <20230211021043.GE4979@bladeshadow.org>

On Fri, Feb 10, 2023 at 05:48:53PM -0500, Thomas Passin wrote:
> On 2/10/2023 4:55 PM, Python wrote:
> > However, Python's print() function is more analogous to C's printf(),
> > which returns the number of characters converted for an entirely
> > different reason... It's precisely so that you'll know what the length
> > of the string that was converted is.  This is most useful with the
> > *snprintf() variants where you're actually concerned about overrunning
> > the buffer you've provided for the output string, so you can realloc()
> > the buffer if it was indeed too small, but it is also useful in the
> > context of, say, a routine to format text according to the size of
> > your terminal.  In that context it really has nothing to do with
> > blocking I/O or socket behavior.
> 
> But none of that applies to the Python print() function. There are no
> buffers to overrun, no reason to know the length of the printed string, no
> re-allocating of a buffer.

Indeed.  But the OP originally compared print to printf, and I was
specifically addressing Chris' point about why I/O functions return
the number of bytes written, which was relevant to, but maybe a bit of
tangent to the original post.  
 
> I don't know why the print() function doesn't return anything

You do though! :)  You actually just explained why yourself... it's
because it just doesn't need to.  But FWIW, I wasn't addressing this
point, because it had already been adequately covered in the thread.

There's good reason why Python's print and C's printf work
differently.  In languages like C it makes sense that printf returns
the length of the string. printf is the means of all three of:

  - formatting the string
  - counting its length
  - actually outputting the string to stdout (albeit indirectly).

This sort of breaks the rule of, "do one thing, and do it well," but
it does so in the name of efficiency.  You might or might not want to
actually know the length of the formatted string, depending on what
you're doing with it.  But the printf function basically needs to
calculate it anyway so that it can tell the underlying system calls
how many bytes to write (or tell stdio how many bytes it is adding to
its buffers, or whatever), and then stuffing that length in a register
to be returned to the caller is roughly free (it's probably using the
register that it's going to return already to do the counting), and
the caller can ignore it if it wants to.  C aims to be as efficient as
possible so this is a good strategy.

Unlike C[*], since Python can already separate the formatting and
length calculation from sending the data to stdout (I demonstrated how
in my first post in the thread), it has no need for print to return
the length.  As I mentioned in my earlier post, if Python encounters
an error condition it will raise an exception--which C can't do--so
there's no reason to return a status either.  What else would it
return?  Nothing else would really make sense.  

-=-=-=-
* In C you could sprintf the string into a buffer, which would return
its length, and then call printf on the buffer, but that would be much
less efficient and a bit silly... UNLESS you actually needed to know
the length of the string beforehand, e.g. to calculate where to put
line breaks in a text justification routine, or something.


From torriem at gmail.com  Fri Feb 10 22:59:39 2023
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 10 Feb 2023 20:59:39 -0700
Subject: Am I banned from Discuss forum?
In-Reply-To: <CABbU2U9cY5o8UUYj8tJaum87_OrvtxSFUOFdZrxno6smx4aYhw@mail.gmail.com>
References: <CABbU2U9cY5o8UUYj8tJaum87_OrvtxSFUOFdZrxno6smx4aYhw@mail.gmail.com>
Message-ID: <932700c6-dda5-213f-2197-b7885d605262@gmail.com>

On 2/10/23 14:10, Marco Sulla wrote:
> I was banned from the mailing list and Discuss forum for a very long time.
> Too much IMHO, but I paid my dues.
> 
> Now this is my state in the forum:
> - I never posted something unrespectful in the last months
> - I have a limitation of three posts per threads, but only on some threads
> - Some random posts of mine are obscured and must be restored manually by
> moderators
> - I opened a thread about the proposal of a new section called
> Brainstorming. It was closed without a reason.
> - I can't post links
> - Two discussions I posted in section Idea were moved to Help, without a
> single line of explanation.
> 
> If I'm not appreciated, I want to be publicly banned with a good reason, or
> at least a reason.

Your posts are showing up on the mailing list here just it seems.  I
didn't know there was a Discourse forum.  Is it supposed to be sync with
the mailing list and USENET?  Or is it intended to replace this mailing
list?  I rarely see Python devs on this list, so maybe they've chosen to
hang out exclusively in Discourse, which would be unfortunate.

The Discourse format has never appealed to me. It's way to unstructured
and gamified.  I much prefer the mailing list, but alas it seems like
most open source projects are moving to Discourse.

From roel at roelschroeven.net  Sat Feb 11 07:21:04 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Sat, 11 Feb 2023 13:21:04 +0100
Subject: Am I banned from Discuss forum?
In-Reply-To: <932700c6-dda5-213f-2197-b7885d605262@gmail.com>
References: <CABbU2U9cY5o8UUYj8tJaum87_OrvtxSFUOFdZrxno6smx4aYhw@mail.gmail.com>
 <932700c6-dda5-213f-2197-b7885d605262@gmail.com>
Message-ID: <41cef28a-775e-ae74-831e-aa4e93929978@roelschroeven.net>

Michael Torrie schreef op 11/02/2023 om 4:59:
> I didn't know there was a Discourse forum.  Is it supposed to be sync  > with the mailing list and USENET? Or is it intended to replace this 
 > mailing list? I rarely see Python devs on this list, so maybe > 
they've chosen to hang out exclusively in Discourse, which would be > 
unfortunate.
I don't think it's meant to replace this mailing list; it's still 
mentioned on the Python website. At least initially the forum was 
intended mostly for core dev discussions, as far as I understand it.

The initial announcement (which only talks about python-comitters being 
migrated): 
https://www.mail-archive.com/python-committers at python.org/msg06151.html
The site with the discussions: https://discuss.python.org/

-- 
"Honest criticism is hard to take, particularly from a relative, a friend,
an acquaintance, or a stranger."
         -- Franklin P. Jones

From cs at cskk.id.au  Sat Feb 11 16:09:00 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Sun, 12 Feb 2023 08:09:00 +1100
Subject: Am I banned from Discuss forum?
In-Reply-To: <41cef28a-775e-ae74-831e-aa4e93929978@roelschroeven.net>
References: <41cef28a-775e-ae74-831e-aa4e93929978@roelschroeven.net>
Message-ID: <Y+gD7PAQsCJhc0n9@cskk.homeip.net>

On 11Feb2023 13:21, Roel Schroeven <roel at roelschroeven.net> wrote:
>Michael Torrie schreef op 11/02/2023 om 4:59:
>>I didn't know there was a Discourse forum.  Is it supposed to be 
>>sync  > with the mailing list and USENET? Or is it intended to 
>>replace this
>> mailing list? I rarely see Python devs on this list, so maybe > 
>they've chosen to hang out exclusively in Discourse, which would be > 
>unfortunate.
>I don't think it's meant to replace this mailing list; it's still 
>mentioned on the Python website. At least initially the forum was 
>intended mostly for core dev discussions, as far as I understand it.
>
>The initial announcement (which only talks about python-comitters 
>being migrated): https://www.mail-archive.com/python-committers at python.org/msg06151.html
>The site with the discussions: https://discuss.python.org/

There are several sections on the Discourse forum; the "Help" section 
broadly has the same purpose as this mailing list. I use both, and use 
Discourse in email mode - the forum's there, but >90% of my interaction 
is via email - I file it and python-list to the same "python" folder 
here and lossely treat them the same.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From research at johnohagan.com  Sun Feb 12 06:10:20 2023
From: research at johnohagan.com (John O'Hagan)
Date: Sun, 12 Feb 2023 22:10:20 +1100
Subject: tkinter ttk.Treeview: changing background colour of single item
 when selected
In-Reply-To: <2865ec14-cc88-4c3b-a641-788c08f9f851n@googlegroups.com>
References: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
 <mailman.1759.1675507388.20444.python-list@python.org>
 <2865ec14-cc88-4c3b-a641-788c08f9f851n@googlegroups.com>
Message-ID: <fddb7bca4e3bd55d44abdef40fc7f3cbffb7ecc4.camel@johnohagan.com>

On Mon, 2023-02-06 at 10:19 -0800, stefalem wrote:
> Il giorno sabato 4 febbraio 2023 alle 11:43:29 UTC+1 John O'Hagan ha
> scritto:
> ...
> 
> > Is there another way to do what I want??
> 
> from tkinter import *
> from tkinter.ttk import *
> 
> root = Tk()
> t = Treeview(root)
> 
> t.insert('', 0, iid='item1', text='item1')
> t.insert('', 1, text='item2')
> t.tag_configure('flashtag', background='red')
> t.pack()
> ??
> def flash():
> ?tags = t.item('item1', 'tags')
> ?t.item('item1', tags='' if tags else 'flashtag')
> ?t.after(500, flash)
> ?itemselected = t.selection()
> ?for x in itemselected:
> ? if (x == 'item1'):
> ?? t.selection_remove(t.get_children())
> 
> flash()

Thank you for your reply. Unfortunately that's not quite what I'm
after, because it unselects the flashing item.

My goal was to be able to change the colour of an individual item
regardless of whether it is selected or not. To do that, it is
necessary to be able to change the colour of an individual selected
item, without changing the selection or changing the colour of other
selected items. It seems this isn't possible.

Thanks

John

From list1 at tompassin.net  Sun Feb 12 08:59:03 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 12 Feb 2023 08:59:03 -0500
Subject: tkinter ttk.Treeview: changing background colour of single item
 when selected
In-Reply-To: <fddb7bca4e3bd55d44abdef40fc7f3cbffb7ecc4.camel@johnohagan.com>
References: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
 <mailman.1759.1675507388.20444.python-list@python.org>
 <2865ec14-cc88-4c3b-a641-788c08f9f851n@googlegroups.com>
 <fddb7bca4e3bd55d44abdef40fc7f3cbffb7ecc4.camel@johnohagan.com>
Message-ID: <3a4a97b7-dd85-6d64-4a20-dc65270b60d3@tompassin.net>

On 2/12/2023 6:10 AM, John O'Hagan wrote:
> On Mon, 2023-02-06 at 10:19 -0800, stefalem wrote:
>> Il giorno sabato 4 febbraio 2023 alle 11:43:29 UTC+1 John O'Hagan ha
>> scritto:
>> ...
>>
>>> Is there another way to do what I want?
>>
>> from tkinter import *
>> from tkinter.ttk import *
>>
>> root = Tk()
>> t = Treeview(root)
>>
>> t.insert('', 0, iid='item1', text='item1')
>> t.insert('', 1, text='item2')
>> t.tag_configure('flashtag', background='red')
>> t.pack()
>>    
>> def flash():
>>  ?tags = t.item('item1', 'tags')
>>  ?t.item('item1', tags='' if tags else 'flashtag')
>>  ?t.after(500, flash)
>>  ?itemselected = t.selection()
>>  ?for x in itemselected:
>>  ? if (x == 'item1'):
>>  ?? t.selection_remove(t.get_children())
>>
>> flash()
> 
> Thank you for your reply. Unfortunately that's not quite what I'm
> after, because it unselects the flashing item.
> 
> My goal was to be able to change the colour of an individual item
> regardless of whether it is selected or not. To do that, it is
> necessary to be able to change the colour of an individual selected
> item, without changing the selection or changing the colour of other
> selected items. It seems this isn't possible.

I haven't worked with ttk objects or Treeviews, but judging from old 
style objects, I think you have to re-apply your color and flashing when 
the item becomes selected and possibly again when it becomes unselected.

Depending on exactly what effect you want, you may also need to apply 
color and flashing when the mouse moves over the item and again when it 
leaves.  When I make changes in e.g. color, I like to save the previous 
value(s) in the object itself.  That way I can easily restore say a 
background color without having to work out what it used to be, which 
may be some color that Tk applies based on the desktop theme and who 
know what else.

Here's an example (simplified) for changing color on mouse hover and 
leave events:

BG_KEY = 'bg' if platform.lower().startswith('win') \
          else 'activebackground'  # Different for Linux!

def on_enter(event):
     w = event.widget
     w.old_bg = w.cget('bg')
     w[BG_KEY] = BUTTON_HOVER  # Color you have chosen

def on_leave(event):
     w = event.widget
     _bg = w.old_bg
     w[BG_KEY] = _bg



From roland.em0001 at googlemail.com  Sun Feb 12 09:58:24 2023
From: roland.em0001 at googlemail.com (=?UTF-8?Q?Roland_M=c3=bcller?=)
Date: Sun, 12 Feb 2023 16:58:24 +0200
Subject: ChatGPT Generated news poster code
In-Reply-To: <k4o9foFm0asU1@mid.individual.net>
References: <ts1vv8$io6i$1@dont-email.me> <k4o9foFm0asU1@mid.individual.net>
Message-ID: <cce1c2ef-bfde-30fe-cbce-8d536084177c@googlemail.com>

Hello,

On 2/11/23 03:31, Greg Ewing via Python-list wrote:
> For a moment I thought this was going to be a script that
> uses ChatGPT to generate a random news post and post it
> to Usenet...
>
> Which would also have been kind of cool, as long as it wasn't
> overused.
>
Actually, I like cynical humor too ... But this is too much

-Roland


From mats at wichmann.us  Sun Feb 12 16:58:15 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Sun, 12 Feb 2023 14:58:15 -0700
Subject: ChatGPT Generated news poster code
In-Reply-To: <cce1c2ef-bfde-30fe-cbce-8d536084177c@googlemail.com>
References: <ts1vv8$io6i$1@dont-email.me> <k4o9foFm0asU1@mid.individual.net>
 <cce1c2ef-bfde-30fe-cbce-8d536084177c@googlemail.com>
Message-ID: <1f21ace3-bd29-9204-e412-a7a554e80521@wichmann.us>

On 2/12/23 07:58, Roland M?ller via Python-list wrote:
> Hello,
> 
> On 2/11/23 03:31, Greg Ewing via Python-list wrote:
>> For a moment I thought this was going to be a script that
>> uses ChatGPT to generate a random news post and post it
>> to Usenet...
>>
>> Which would also have been kind of cool, as long as it wasn't
>> overused.
>>
> Actually, I like cynical humor too ... But this is too much

Meanwhile, I'm still wondering why I need a program to "chat" to the 
GUID Partition Table


From research at johnohagan.com  Mon Feb 13 00:42:03 2023
From: research at johnohagan.com (John O'Hagan)
Date: Mon, 13 Feb 2023 16:42:03 +1100
Subject: tkinter ttk.Treeview: changing background colour of single item
 when selected
In-Reply-To: <3a4a97b7-dd85-6d64-4a20-dc65270b60d3@tompassin.net>
References: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
 <mailman.1759.1675507388.20444.python-list@python.org>
 <2865ec14-cc88-4c3b-a641-788c08f9f851n@googlegroups.com>
 <fddb7bca4e3bd55d44abdef40fc7f3cbffb7ecc4.camel@johnohagan.com>
 <3a4a97b7-dd85-6d64-4a20-dc65270b60d3@tompassin.net>
Message-ID: <be86357e7a5f0cfbdd159363eba61cbd1fecd401.camel@johnohagan.com>

On Sun, 2023-02-12 at 08:59 -0500, Thomas Passin wrote:

[...]

> On 2/12/2023 6:10 AM, John O'Hagan wrote:

[...]

> > 
> > My goal was to be able to change the colour of an individual item
> > regardless of whether it is selected or not. To do that, it is
> > necessary to be able to change the colour of an individual selected
> > item, without changing the selection or changing the colour of
> > other
> > selected items. It seems this isn't possible.
> 
> I haven't worked with ttk objects or Treeviews, but judging from old 
> style objects, I think you have to re-apply your color and flashing
> when 
> the item becomes selected and possibly again when it becomes
> unselected.
> 
> Depending on exactly what effect you want, you may also need to apply
> color and flashing when the mouse moves over the item and again when
> it 
> leaves.? When I make changes in e.g. color, I like to save the
> previous 
> value(s) in the object itself.? That way I can easily restore say a 
> background color without having to work out what it used to be, which
> may be some color that Tk applies based on the desktop theme and who 
> know what else.
> 
> Here's an example (simplified) for changing color on mouse hover and 
> leave events:
> 
> BG_KEY = 'bg' if platform.lower().startswith('win') \
> ????????? else 'activebackground'? # Different for Linux!
> 
> def on_enter(event):
> ???? w = event.widget
> ???? w.old_bg = w.cget('bg')
> ???? w[BG_KEY] = BUTTON_HOVER? # Color you have chosen
> 
> def on_leave(event):
> ???? w = event.widget
> ???? _bg = w.old_bg
> ???? w[BG_KEY] = _bg
> 
> 

Thank you again for your reply, Thomas. I didn't know tkinter
configuration keys have different names on different platforms, that
seems unneccesary! I also like the idea of storing the original
settings on the object itself.

However, because of the nature of ttk widgets, I think I'm stuck with
my original issue. Unlike classic tkinter widgets, a lot of the ttk
configuration options are not available on individual widgets, but must
be done through styles or tags. On Treeviews (and possibly other ttk
widgets), the colours for selected items can only be set using
style.map (AFAIK). ?The colours for individual items can (only?) be set
using tags, but this is evidently overridden?when the item is
selected?by the selection colour dictated by the style. Treeview tags
do not have a 'selectbackground' option like many tkinter widgets do.
(Fonts _can_ be set this way, so I could flash, say, the font size, but
that's ugly!)

As as aside, IMO ttk looks great (and Treeview is really useful) but
this approach to configuration is a double-edged sword. On one hand
it's super-easy to change the whole look, but on the other, it can be
overly restrictive if you want to control the look of individual
widgets.?

I might try to put forward a feature request to add a
'selectbackground' option to ttk tags to see if there's any interest.

Thanks

--

John

From gweatherby at uchc.edu  Mon Feb 13 10:15:43 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Mon, 13 Feb 2023 15:15:43 +0000
Subject: evaluation question
In-Reply-To: <003501d93da5$e0b00330$a2100990$@gmail.com>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <20230210195907.GC4979@bladeshadow.org>
 <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>
 <20230210215537.GD4979@bladeshadow.org>
 <003501d93da5$e0b00330$a2100990$@gmail.com>
Message-ID: <SJ0PR14MB5864008AC2BC8C0CF96BDC4CB9DD9@SJ0PR14MB5864.namprd14.prod.outlook.com>

?Why are we even still talking about this??

Because humans are social creatures and some contributors to the list like to discuss things in depth.


From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of avi.e.gross at gmail.com <avi.e.gross at gmail.com>
Date: Friday, February 10, 2023 at 6:19 PM
To: python-list at python.org <python-list at python.org>
Subject: RE: evaluation question
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

There are no doubt many situations someone wants to know how long something
will be when printed but often at lower levels.

In variable-width fonts, for example, the number of characters does not
really line up precisely with how many characters. Some encodings use a
varying number of bytes and, again, the width of the output varies.

So for people who want to make 2-D formatted output like tables, or who want
to wrap lines longer than N characters, you more often let some deeper
software accept your data and decide on formatting it internally and either
print it at once, when done calculating, or in the case of some old-style
terminals, use something like the curses package that may use escape
sequences to update the screen more efficiently in various ways.

If someone wants more control over what they print, rather than asking the
print() function to return something that is mostly going to be ignored,
they can do the things others have already suggested here. You can make your
message parts in advance and measure their length or anything else before
you print. Or make a wrapper that does something for you before calling
print, perhaps only for common cases and then returns the length to you
after printing.

I wonder if the next request will be for  print() to know what your output
device is and other current settings so it return the width your text takes
up in pixels in the current font/size ...

I add a tidbit that many ways of printing allow you to specify the width you
want something printed in such as you want a floating point value with so
many digits after the decimal point in a zero or space padded field on the
left. So there are ways to calculate in advance for many common cases as to
how long each part will be if you specify it. Besides, I am not really sure
if "print" even knows easily how many characters it is putting out as it
chews away on the many things in your request and calls dunder methods in
objects so they display themselves and so on. I assume it can be made to
keep track, albeit I can imagine printing out an APL program with lots of
overwritten characters where the number of bytes sent is way more than the
number of spaces in the output.

Why are we even still talking about this? The answer to the question of why
print() does not return anything, let alone the number of characters
printed, is BECAUSE.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Python
Sent: Friday, February 10, 2023 4:56 PM
To: python-list at python.org
Subject: Re: evaluation question

On Sat, Feb 11, 2023 at 08:30:22AM +1100, Chris Angelico wrote:
> On Sat, 11 Feb 2023 at 07:36, Python <python at bladeshadow.org> wrote:
> > You would do this instead:
> >
> >     message = f"{username} has the occupation {job}."
> >     message_length = len(message)
> >     print(message)
> >     print(message_length)
> >     ...
> >
>
> It's worth noting WHY output functions often return a byte count. It's
> primarily for use with nonblocking I/O, with something like this:
>
> buffer = b".............."
> buffer = buffer[os.write(fd, buffer):]
>
> It's extremely important to be able to do this sort of thing, but not
> with the print function, which has a quite different job.

I would agree with this only partially.  Your case applies to os.write(),
which is essentially just a wrapper around the write() system call, which
has that sort of property... though it applies also to I/O in blocking mode,
particularly on network sockets, where the number of bytes you asked to
write (or read) may not all have been transferred, necessitating trying in a
loop.

However, Python's print() function is more analogous to C's printf(), which
returns the number of characters converted for an entirely different
reason... It's precisely so that you'll know what the length of the string
that was converted is.  This is most useful with the
*snprintf() variants where you're actually concerned about overrunning the
buffer you've provided for the output string, so you can realloc() the
buffer if it was indeed too small, but it is also useful in the context of,
say, a routine to format text according to the size of your terminal.  In
that context it really has nothing to do with blocking I/O or socket
behavior.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$>

From avi.e.gross at gmail.com  Mon Feb 13 12:41:16 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 13 Feb 2023 12:41:16 -0500
Subject: evaluation question
In-Reply-To: <SJ0PR14MB5864008AC2BC8C0CF96BDC4CB9DD9@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <tr0uki$1p2c5$1@dont-email.me> <Ko8BL.57228$jiuc.57141@fx44.iad>
 <slrnttdoeq.s20.elvis-85792@notatla.org.uk>
 <bd4ffe68-8c09-b716-9f74-aa2d6abf1705@tompassin.net>
 <mailman.1717.1675054683.20444.python-list@python.org>
 <tr83bf$381e1$1@dont-email.me> <20230210195907.GC4979@bladeshadow.org>
 <CAPTjJmpBhNtpv5kBTu2wCJRV1HLx+ujsZz6QGsASmkW4XbW0yQ@mail.gmail.com>
 <20230210215537.GD4979@bladeshadow.org>
 <003501d93da5$e0b00330$a2100990$@gmail.com>
 <SJ0PR14MB5864008AC2BC8C0CF96BDC4CB9DD9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <004401d93fd2$602dda00$20898e00$@gmail.com>

Weatherby,

Of course you are right and people can, and do, discuss whatever they feel like.

My question is a bit more about asking if I am missing something here as my personal view is that we are not really exploring in more depth or breadth and are getting fairly repetitive as if in a typical SF time loop. How does one break out?

Languages often change and people do come and go so some topics can often be re-opened. This though is a somewhat focused forum and it is legitimate to ask if a conversation might best be taken elsewhere for now. The main focus is, at least in theory, aspects of python and mostly about the core as compared to very specific modules, let alone those nobody here has ever even used. Within that, it is fair at times to compare something in python to other languages or ask about perceived bugs or about possible future enhancements. We could discuss if "YIELD FROM" is just syntactic sugar as it often can be done with just the naked YIELD statement, or whether it really allows you to do innovative things, as an example.

But I think where this conversation has gone is fairly simple. The question was why print() does not return the number of characters printed. The answers boiled down to that this was not the design chosen and is mostly consistent with how python handles similar functions that return nothing when the change is largely "internal" in a sense. In addition, plenty of us have suggested alternate ways to get what the OP asked for, and also pointed out there are other things that someone may have wanted instead or in addition, including the actual number of bytes generated for encodings other than vanilla ASCII, or pixels if the text was rendered on a screen using variable width fonts and so on.

Some of the above already counted, in my mind, as adding depth or breadth to the original question. But if the conversation degrades to two or more sides largely talking past each other and not accepting what the other says, then perhaps a natural ending point has been reached. Call it a draw, albeit maybe a forfeit.

So, as part of my process, I am now stuck on hearing many questions as valid and others as not productive. I don't mean just here, but in many areas of my life. The answer is that historically, and in other ways, python took a specific path. Others took other paths. But once locked into this path, you run into goals of trying to remain consistent and not have new releases break older software or at least take time to deprecate it and give people time to adjust.

I have seen huge growing pains due to growth. An example is languages that have added features, such as promises and their variants and used them for many purposes such as allowing asynchronous execution using multiple methods or evaluating things in a more lazy way so they are done just in time. Some end up with some simple function call being augmented with quite a few additional functions with slightly different names and often different arguments and ways they are called that mostly should no longer be mixed with other variants of the function. You need compatibility with the old while allowing the new and then the newer and newest.

Had the language been built anew from scratch, it might be simpler and also more complex, as they would skip the older versions and pretty much use the shinier new version all the time, even as it often introduces many costs where they are not needed. 

So it can be very valid to ask questions as long as you also LISTEN to the answers and try to accept them as aspects of reality. Yes, python could have had a different design and perhaps someday may have a different design. But that is not happening TODAY so for today, accept what is and accept advice on how you might get something like what you want when, and if, you need it. The goal often is to get the job done, not to do it the way you insist it has to be done.

At some point, unless someone has more to say with some new twist, it does become a bit annoying.

So let me say something slightly new now. I have been reading about interesting uses of the python WITH statement and how it works. Some of the examples are creating an object with dunder methods that get invoked on entry and exit that can do all kinds of things. One is the ability to modify a list in a way that can be rolled back if there is an error and it is quite simple. You make a copy of the original list on entry. Things are then done to the copy. And if you exit properly, you copy the modified list back on top of the original list. Errors that result simply unwind the serious of transactions by leaving the original list untouched. 

Another example has your output stream redirected within the WITH and then put back in place after. What this allows, among many things, is for everything printed to be  desrever. Now clearly, such a technique could also be used to capture what is going to be printed, and count how many bytes or characters it produced and make the result available after you exit the region.  Heck, if fed a paragraph of text, it could not only print it but create one or more objects containing a detailed analysis including guessing what language it is in, translating it to English, pointing out spelling and grammar errors, and mailing you a copy! You can imagine quite a few side effects of calling print() but again, why would you put the functionality within print() versus in a function you wrote that does all that as well as calling print()?

But even assuming you code that properly and rewrite all your code as something like:

with capture:
    print(...)

# use chars_written_within_width as a variable created within that holds what you want.


Is that really any better than several other ways we have suggested would work such as creating an f-string independently and then printing it which would handle quite a few use cases?

If others wish to keep debating this topic or enhancing it, fine. I am not judging but simply expressing the personal opinion that even if I might have more to add, I am not motivated to do so any longer. Then again, I may soon lose the motivation to be part of this forum and take up other hobbies ?


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Weatherby,Gerard
Sent: Monday, February 13, 2023 10:16 AM
To: python-list at python.org
Subject: Re: evaluation question

?Why are we even still talking about this??

Because humans are social creatures and some contributors to the list like to discuss things in depth.


From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of avi.e.gross at gmail.com <avi.e.gross at gmail.com>
Date: Friday, February 10, 2023 at 6:19 PM
To: python-list at python.org <python-list at python.org>
Subject: RE: evaluation question
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

There are no doubt many situations someone wants to know how long something will be when printed but often at lower levels.

In variable-width fonts, for example, the number of characters does not really line up precisely with how many characters. Some encodings use a varying number of bytes and, again, the width of the output varies.

So for people who want to make 2-D formatted output like tables, or who want to wrap lines longer than N characters, you more often let some deeper software accept your data and decide on formatting it internally and either print it at once, when done calculating, or in the case of some old-style terminals, use something like the curses package that may use escape sequences to update the screen more efficiently in various ways.

If someone wants more control over what they print, rather than asking the
print() function to return something that is mostly going to be ignored, they can do the things others have already suggested here. You can make your message parts in advance and measure their length or anything else before you print. Or make a wrapper that does something for you before calling print, perhaps only for common cases and then returns the length to you after printing.

I wonder if the next request will be for  print() to know what your output device is and other current settings so it return the width your text takes up in pixels in the current font/size ...

I add a tidbit that many ways of printing allow you to specify the width you want something printed in such as you want a floating point value with so many digits after the decimal point in a zero or space padded field on the left. So there are ways to calculate in advance for many common cases as to how long each part will be if you specify it. Besides, I am not really sure if "print" even knows easily how many characters it is putting out as it chews away on the many things in your request and calls dunder methods in objects so they display themselves and so on. I assume it can be made to keep track, albeit I can imagine printing out an APL program with lots of overwritten characters where the number of bytes sent is way more than the number of spaces in the output.

Why are we even still talking about this? The answer to the question of why
print() does not return anything, let alone the number of characters printed, is BECAUSE.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Python
Sent: Friday, February 10, 2023 4:56 PM
To: python-list at python.org
Subject: Re: evaluation question

On Sat, Feb 11, 2023 at 08:30:22AM +1100, Chris Angelico wrote:
> On Sat, 11 Feb 2023 at 07:36, Python <python at bladeshadow.org> wrote:
> > You would do this instead:
> >
> >     message = f"{username} has the occupation {job}."
> >     message_length = len(message)
> >     print(message)
> >     print(message_length)
> >     ...
> >
>
> It's worth noting WHY output functions often return a byte count. It's 
> primarily for use with nonblocking I/O, with something like this:
>
> buffer = b".............."
> buffer = buffer[os.write(fd, buffer):]
>
> It's extremely important to be able to do this sort of thing, but not 
> with the print function, which has a quite different job.

I would agree with this only partially.  Your case applies to os.write(), which is essentially just a wrapper around the write() system call, which has that sort of property... though it applies also to I/O in blocking mode, particularly on network sockets, where the number of bytes you asked to write (or read) may not all have been transferred, necessitating trying in a loop.

However, Python's print() function is more analogous to C's printf(), which returns the number of characters converted for an entirely different reason... It's precisely so that you'll know what the length of the string that was converted is.  This is most useful with the
*snprintf() variants where you're actually concerned about overrunning the buffer you've provided for the output string, so you can realloc() the buffer if it was indeed too small, but it is also useful in the context of, say, a routine to format text according to the size of your terminal.  In that context it really has nothing to do with blocking I/O or socket behavior.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!nyKmKANNzMqWDt2IGH9-Vv63_bioBGOYeokJy5GupmZVZIelplk15rvc_5NNbt6afc9yukh8y5X5mZXDVgr_PhY$>
--
https://mail.python.org/mailman/listinfo/python-list


From dino at no.spam.ar  Fri Feb 10 19:39:48 2023
From: dino at no.spam.ar (Dino)
Date: Fri, 10 Feb 2023 19:39:48 -0500
Subject: Comparing caching strategies
Message-ID: <otBFL.3111707$WRz3.215206@fx03.ams4>

First off, a big shout out to Peter J. Holzer, who mentioned roaring 
bitmaps a few days ago and led me to quite a discovery.

Now I am stuck with an internal dispute with another software architect 
(well, with a software architect, I should say, as I probably shouldn't 
define myself a software architect when confronted with people with more 
experience than me in building more complex systems).
Anyway, now that I know what roaring bitmaps are (and what they can 
do!), my point is that we should abandon other attempts to build a 
caching layer for our project and just veer decidedly towards relying on 
those magic bitmaps and screw anything else. Sure, there is some 
overhead  marshaling our entries into integers and back, but the sheer 
speed and compactness of RBMs trump any other consideration (according 
to me, not according to the other guy, obviously).

Long story short: I want to prototype a couple of caching strategies in 
Python using bitmaps, and measure both performance and speed.

So, here are a few questions from an inexperienced programmer for you, 
friends. Apologies if they are a bit "open ended".

- How would you structure the caching so that different caching 
strategies are "pluggable"? change one line of code (or even a config 
file) and a different caching strategy is used in the next run. Is this 
the job for a design pattern such as factory or facade?

- what tool should I use to measure/log performance and memory 
occupation of my script? Google is coming up with quite a few options, 
but I value the opinion of people here a lot.

Thank you for any feedback you may be able to provide.

Dino

From s20201060 at htsps.edu.hk  Sun Feb 12 02:44:56 2023
From: s20201060 at htsps.edu.hk (outlook_F8D50C5B9E059743@outlook.com)
Date: Sun, 12 Feb 2023 15:44:56 +0800
Subject: l can't use pip in python 3.11.2
Message-ID: <E9441CC1-6FAA-41BD-BB18-B22358A7AA95@hxcore.ol>

   ?

   ?

   ? Windows ?[1]????

   ?

References

   Visible links
   1. https://go.microsoft.com/fwlink/?LinkId=550986

From stefalem79 at gmail.com.invalid  Sun Feb 12 06:31:44 2023
From: stefalem79 at gmail.com.invalid (stefalem)
Date: Sun, 12 Feb 2023 12:31:44 +0100
Subject: tkinter ttk.Treeview: changing background colour of single item
 when selected
In-Reply-To: <mailman.1808.1676200246.20444.python-list@python.org>
References: <4dd9c35651eb62c95808e42ce66f0ab1bfdab7c2.camel@johnohagan.com>
 <mailman.1759.1675507388.20444.python-list@python.org>
 <2865ec14-cc88-4c3b-a641-788c08f9f851n@googlegroups.com>
 <fddb7bca4e3bd55d44abdef40fc7f3cbffb7ecc4.camel@johnohagan.com>
 <mailman.1808.1676200246.20444.python-list@python.org>
Message-ID: <tsain0$1onrl$1@dont-email.me>

Il 12/02/23 12:10, John O'Hagan ha scritto:

 > My goal was to be able to change the colour of an individual item
 > regardless of whether it is selected or not. To do that, it is
 > necessary to be able to change the colour of an individual selected
 > item, without changing the selection or changing the colour of other
 > selected items. It seems this isn't possible.

ok sorry. As another alternative I had thought of this:

from tkinter import *
from tkinter.ttk import *

root = Tk()
t = Treeview(root)

t.insert('', 0, iid='item1', text='item1')
t.insert('', 1, text='item2')
t.tag_configure('flashtag', background='red')
t.pack()
style = Style()
styleDefault = (style.map("Treeview"))

def flash():
  tags = t.item('item1', 'tags')
  t.item('item1', tags='' if tags else 'flashtag')
  t.after(500, flash)
  itemselected = t.selection()
  for x in itemselected:
   if (x == 'item1'):
    style.configure('Treeview', selectbackground='red')
    style.map('Treeview', background=[('disabled', 'white')], 
foreground=[('disabled', 'red')])
   else:
    style.map('Treeview', background=[('selected', 
styleDefault['background'][1][1])],
                          foreground=[('selected', 
styleDefault['background'][0][1])])
flash()
mainloop()

Maybe it can be useful for other situations.

Bye bye

From nntp.mbourne at spamgourmet.com  Mon Feb 13 06:50:46 2023
From: nntp.mbourne at spamgourmet.com (Mark Bourne)
Date: Mon, 13 Feb 2023 11:50:46 +0000
Subject: ChatGPT Generated news poster code
In-Reply-To: <mailman.1811.1676239100.20444.python-list@python.org>
References: <ts1vv8$io6i$1@dont-email.me> <k4o9foFm0asU1@mid.individual.net>
 <cce1c2ef-bfde-30fe-cbce-8d536084177c@googlemail.com>
 <1f21ace3-bd29-9204-e412-a7a554e80521@wichmann.us>
 <mailman.1811.1676239100.20444.python-list@python.org>
Message-ID: <tsd86n$24tgk$1@dont-email.me>

Mats Wichmann wrote:
> Meanwhile, I'm still wondering why I need a program to "chat" to the 
> GUID Partition Table

Perhaps to keep on good terms with it so that it doesn't run away and 
hide?  I had to go looking for the GPT on one of my disks after it went 
AWOL a couple of years ago.  Eventually found it hiding in another sector.

(Turns out the USB-SATA adapter I previously used to format the disk 
misreports the logical block size for some disks.  Subsequently 
connecting the disk directly to a SATA port, the GPT was no longer found 
at logical block 1 where it should have been, because the block size was 
reported differently.)

-- 
Mark.

From stephen_tucker at sil.org  Tue Feb 14 02:09:37 2023
From: stephen_tucker at sil.org (Stephen Tucker)
Date: Tue, 14 Feb 2023 07:09:37 +0000
Subject: Precision Tail-off?
Message-ID: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>

Hi,

I have just produced the following log in IDLE (admittedly, in Python
2.7.10 and, yes I know that it has been superseded).

It appears to show a precision tail-off as the supplied float gets bigger.

I have two questions:
1. Is there a straightforward explanation for this or is it a bug?
2. Is the same behaviour exhibited in Python 3.x?

For your information, the first 20 significant figures of the cube root in
question are:
   49793385921817447440

Stephen Tucker.
----------------------------------------------
>>> 123.456789 ** (1.0 / 3.0)
4.979338592181744
>>> 123456.789 ** (1.0 / 3.0)
49.79338592181744
>>> 123456789. ** (1.0 / 3.0)
497.9338592181743
>>> 123456789000. ** (1.0 / 3.0)
4979.338592181743
>>> 123456789000000. ** (1.0 / 3.0)
49793.38592181742
>>> 123456789000000000. ** (1.0 / 3.0)
497933.8592181741
>>> 123456789000000000000. ** (1.0 / 3.0)
4979338.59218174
>>> 123456789000000000000000. ** (1.0 / 3.0)
49793385.9218174
>>> 123456789000000000000000000. ** (1.0 / 3.0)
497933859.2181739
>>> 123456789000000000000000000000. ** (1.0 / 3.0)
4979338592.181739
>>> 123456789000000000000000000000000. ** (1.0 / 3.0)
49793385921.81738
>>> 123456789000000000000000000000000000. ** (1.0 / 3.0)
497933859218.1737
>>> 123456789000000000000000000000000000000. ** (1.0 / 3.0)
4979338592181.736
>>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
49793385921817.36
>>> 123456789000000000000000000000000000000000000. ** (1.0 / 3.0)
497933859218173.56
>>> 123456789000000000000000000000000000000000000000. ** (1.0 / 3.0)
4979338592181735.0
>>> 123456789000000000000000000000000000000000000000000. ** (1.0 / 3.0)
4.979338592181734e+16
>>> 123456789000000000000000000000000000000000000000000000. ** (1.0 / 3.0)
4.979338592181734e+17
>>> 123456789000000000000000000000000000000000000000000000000. ** (1.0 /
3.0)
4.979338592181733e+18
>>> 123456789000000000000000000000000000000000000000000000000000. ** (1.0 /
3.0)
4.979338592181732e+19
>>> 123456789000000000000000000000000000000000000000000000000000000. **
(1.0 / 3.0)
4.9793385921817313e+20
----------------------------------------------

From oscar.j.benjamin at gmail.com  Tue Feb 14 06:17:20 2023
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 14 Feb 2023 11:17:20 +0000
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
Message-ID: <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>

On Tue, 14 Feb 2023 at 07:12, Stephen Tucker <stephen_tucker at sil.org> wrote:
>
> Hi,
>
> I have just produced the following log in IDLE (admittedly, in Python
> 2.7.10 and, yes I know that it has been superseded).
>
> It appears to show a precision tail-off as the supplied float gets bigger.
>
> I have two questions:
> 1. Is there a straightforward explanation for this or is it a bug?
> 2. Is the same behaviour exhibited in Python 3.x?
>
> For your information, the first 20 significant figures of the cube root in
> question are:
>    49793385921817447440
>
> Stephen Tucker.
> ----------------------------------------------
> >>> 123.456789 ** (1.0 / 3.0)
> 4.979338592181744
> >>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
> 49793385921817.36

You need to be aware that 1.0/3.0 is a float that is not exactly equal
to 1/3 and likewise the other float cannot have as many accurate
digits as is suggested by the number of zeros shown. Therefore you
should compare what exactly it means for the numbers you really have
rather than comparing with an exact cube root of the number that you
intended. Here I will do this with SymPy and calculate many more
digits than are needed. First here is the exact cube root:

In [29]: from sympy import *

In [30]: n = 123456789000000000000000000000000000000000

In [31]: cbrt(n).evalf(50)
Out[31]: 49793385921817.447440261250171604380899353243631762

So that's 50 digits of the exact cube root of the exact number and the
first 20 match what you showed. However in your calculation you use
floats so the exact expression that you evaluate is:

In [32]: e = Pow(Rational(float(n)), Rational(1.0/3.0), evaluate=False)

In [33]: print(e)
123456788999999998830049821836693930508288**(6004799503160661/18014398509481984)

Neither base or exponent is really the number that you intended it to
be. The first 50 decimal digits of this number are:

In [34]: e.evalf(50)
Out[34]: 49793385921817.360106660998131166304296436896582873

All of the digits in the calculation you showed match with the first
digits given here. The output from the float calculation is correct
given what the inputs actually are and also the available precision
for 64 bit floats (53 bits or ~16 decimal digits).

The reason that the results get further from your expectations as the
base gets larger is because the exponent is always less than 1/3 and
the relative effect of that difference is magnified for larger bases.
You can see this in a series expansion of a^x around x=1/3. Using
SymPy again:

In [37]: a, x = symbols('a, x')

In [38]: print(series(a**x, x, Rational(1, 3), 2))
a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))

You can see that the leading relative error term from x being not
quite equal to 1/3 is proportional to the log of the base. You should
expect this difference to grow approximately linearly as you keep
adding more zeros in the base.

--
Oscar

From gweatherby at uchc.edu  Tue Feb 14 07:58:42 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Tue, 14 Feb 2023 12:58:42 +0000
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
Message-ID: <SJ0PR14MB5864E83B9CAF430C7DF6EE08B9A29@SJ0PR14MB5864.namprd14.prod.outlook.com>

Use Python3

Use the decimal module:  https://docs.python.org/3/library/decimal.html


From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Stephen Tucker <stephen_tucker at sil.org>
Date: Tuesday, February 14, 2023 at 2:11 AM
To: Python <python-list at python.org>
Subject: Precision Tail-off?
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Hi,

I have just produced the following log in IDLE (admittedly, in Python
2.7.10 and, yes I know that it has been superseded).

It appears to show a precision tail-off as the supplied float gets bigger.

I have two questions:
1. Is there a straightforward explanation for this or is it a bug?
2. Is the same behaviour exhibited in Python 3.x?

For your information, the first 20 significant figures of the cube root in
question are:
   49793385921817447440

Stephen Tucker.
----------------------------------------------
>>> 123.456789 ** (1.0 / 3.0)
4.979338592181744
>>> 123456.789 ** (1.0 / 3.0)
49.79338592181744
>>> 123456789. ** (1.0 / 3.0)
497.9338592181743
>>> 123456789000. ** (1.0 / 3.0)
4979.338592181743
>>> 123456789000000. ** (1.0 / 3.0)
49793.38592181742
>>> 123456789000000000. ** (1.0 / 3.0)
497933.8592181741
>>> 123456789000000000000. ** (1.0 / 3.0)
4979338.59218174
>>> 123456789000000000000000. ** (1.0 / 3.0)
49793385.9218174
>>> 123456789000000000000000000. ** (1.0 / 3.0)
497933859.2181739
>>> 123456789000000000000000000000. ** (1.0 / 3.0)
4979338592.181739
>>> 123456789000000000000000000000000. ** (1.0 / 3.0)
49793385921.81738
>>> 123456789000000000000000000000000000. ** (1.0 / 3.0)
497933859218.1737
>>> 123456789000000000000000000000000000000. ** (1.0 / 3.0)
4979338592181.736
>>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
49793385921817.36
>>> 123456789000000000000000000000000000000000000. ** (1.0 / 3.0)
497933859218173.56
>>> 123456789000000000000000000000000000000000000000. ** (1.0 / 3.0)
4979338592181735.0
>>> 123456789000000000000000000000000000000000000000000. ** (1.0 / 3.0)
4.979338592181734e+16
>>> 123456789000000000000000000000000000000000000000000000. ** (1.0 / 3.0)
4.979338592181734e+17
>>> 123456789000000000000000000000000000000000000000000000000. ** (1.0 /
3.0)
4.979338592181733e+18
>>> 123456789000000000000000000000000000000000000000000000000000. ** (1.0 /
3.0)
4.979338592181732e+19
>>> 123456789000000000000000000000000000000000000000000000000000000. **
(1.0 / 3.0)
4.9793385921817313e+20
----------------------------------------------
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kSE4mNp5KxTEp6SKzpQeBukScLYsmEoDfLpSTuc2Zv8Z3pZQhTm0usq-k4eVquxM08u8VSUX1X6id9IICJHA2B4mzw$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kSE4mNp5KxTEp6SKzpQeBukScLYsmEoDfLpSTuc2Zv8Z3pZQhTm0usq-k4eVquxM08u8VSUX1X6id9IICJHA2B4mzw$>

From christoph.sobotta at arcor.de  Tue Feb 14 12:54:37 2023
From: christoph.sobotta at arcor.de (christoph sobotta)
Date: Tue, 14 Feb 2023 18:54:37 +0100
Subject: Python 3.8 pip installation on Windows 7
Message-ID: <a9c754ed-bb3b-baf9-a5ec-ed5355b87ca1@arcor.de>

Hello,
The installation file Python-3.8.10.exe (last version compatible with 
Windows 7 (32 bit) ? )
does not automatically install pip on Windows 7.
Are there compatibility problems with Windows 7 ?
See attachment log file
Greetings
C.Sobotta

...
MSI (s) (3C:FC) [13:32:28:993]: Hello, I'm your 32bit Impersonated 
custom action server.
CAQuietExec:? Traceback (most recent call last):
CAQuietExec:??? File "C:\Program Files\Python38-32\Lib\runpy.py", line 
194, in _run_module_as_main
CAQuietExec:????? return _run_code(code, main_globals, None,
CAQuietExec:??? File "C:\Program Files\Python38-32\Lib\runpy.py", line 
87, in _run_code
CAQuietExec:????? exec(code, run_globals)
CAQuietExec:??? File "C:\Program 
Files\Python38-32\Lib\ensurepip\__main__.py", line 5, in <module>
CAQuietExec:????? sys.exit(ensurepip._main())
CAQuietExec:??? File "C:\Program 
Files\Python38-32\Lib\ensurepip\__init__.py", line 206, in _main
CAQuietExec:????? return _bootstrap(
CAQuietExec:??? File "C:\Program 
Files\Python38-32\Lib\ensurepip\__init__.py", line 125, in _bootstrap
CAQuietExec:????? return _run_pip(args + , additional_paths)
CAQuietExec:??? File "C:\Program 
Files\Python38-32\Lib\ensurepip\__init__.py", line 34, in _run_pip
CAQuietExec:????? return subprocess.run(, check=True).returncode
CAQuietExec:??? File "C:\Program Files\Python38-32\Lib\subprocess.py", 
line 493, in run
CAQuietExec:????? with Popen(*popenargs, **kwargs) as process:
CAQuietExec:??? File "C:\Program Files\Python38-32\Lib\subprocess.py", 
line 858, in __init__
CAQuietExec:????? self._execute_child(args, executable, preexec_fn, 
close_fds,
CAQuietExec:??? File "C:\Program Files\Python38-32\Lib\subprocess.py", 
line 1311, in _execute_child
CAQuietExec:????? hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
CAQuietExec:? FileNotFoundError:? Das System kann die angegebene Datei 
nicht finden
CAQuietExec:? Error 0x80070001: Command line returned an error.
CAQuietExec:? Error 0x80070001: QuietExec Failed
CAQuietExec:? Error 0x80070001: Failed in ExecCommon method
MSI (s) (3C:F0) [13:32:29:617]: Executing op: 
End(Checksum=0,ProgressTotalHDWord=0,ProgressTotalLDWord=291152)
CustomAction UpdatePip returned actual error code 1603 but will be 
translated to success due to continue marking
MSI (s) (3C:F0) [13:32:29:617]: User policy value 'DisableRollback' is 0
MSI (s) (3C:F0) [13:32:29:617]: Machine policy value 'DisableRollback' is 0
MSI (s) (3C:F0) [13:32:29:617]: Note: 1: 2318 2:
MSI (s) (3C:F0) [13:32:29:617]: No System Restore sequence number for 
this installation.
MSI (s) (3C:F0) [13:32:29:617]: Unlocking Server
MSI (s) (3C:F0) [13:32:29:914]: PROPERTY CHANGE: Deleting UpdateStarted 
property. Its current value is '1'.
Action ended 13:32:29: InstallFinalize. Return value 1.
Action ended 13:32:29: INSTALL. Return value 1.

...


From jicman at gmail.com  Tue Feb 14 15:30:47 2023
From: jicman at gmail.com (jose isaias cabrera)
Date: Tue, 14 Feb 2023 15:30:47 -0500
Subject: Changing the original SQLite version to the latest
Message-ID: <CAO+295tdAifRpq6PSSix=feZFAfO-=ERxkn=aL5eB25BDU=ghw@mail.gmail.com>

Greetings.

I have tried both Cygwin and SQLite support, and I have received very
little ideas from them, so I am trying this to see if anyone has dealt
with such a problem before.

If I use Cygwin setup tool and install python39 and thus,

$ python
Python 3.9.10 (main, Jan 20 2022, 21:37:52) [GCC 11.2.0] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.34.0'
>>>

As you can see, the SQlite3 version installed is v3.34.0. Right now,
the version available is 3.40.1. What I would like to do is to work
with the latest SQLite version. I have 4 Cygwin instances, 2 of them
are working ok, 2 are not. This is what I have done to get python to
use 3.41.0:
- downloaded the latest SQlite check-in from the site
- $ tar -xvf SQLite-44200596.tar.gz
- cd to SQLite-44200596
- $ ./configure --prefix=/usr
- $ make install

These steps above have worked on two PCs, but I don't know what is the
difference that makes the other two work, and the other two not work.
I have started a few instances of Cygwin on the PC that is not
working, and I have been trying for a few days, and I am humbling
myself, and asking for help. So, the request is to get python3 to
change the SQLite3 library from 3.34.0 to 3.41.0. Any help would be
greatly appreciated. Since this is Windows 10, it's probably some
SQLite DLL somewhere that is being pulled instead of the one
installed. Perhaps some of you can provide a few suggestions to see
where I can  find a solution. I know the next step is to compile
python, but, I rather try to find how to fix this and get to the
bottom of it. Thanks, thanks and thanks.

jos?


-- 

What if eternity is real?  Hmmmm...

From dino at no.spam.ar  Tue Feb 14 17:07:00 2023
From: dino at no.spam.ar (Dino)
Date: Tue, 14 Feb 2023 17:07:00 -0500
Subject: LRU cache
Message-ID: <9CTGL.3255637$miq3.78888@fx02.ams4>


Here's my problem today. I am using a dict() to implement a quick and 
dirty in-memory cache.

I am stopping adding elements when I am reaching 1000 elements (totally 
arbitrary number), but I would like to have something slightly more 
sophisticated to free up space for newer and potentially more relevant 
entries.

I am thinking of the Least Recently Used principle, but how to implement 
that is not immediate. Before I embark on reinventing the wheel, is 
there a tool, library or smart trick that will allow me to remove 
elements with LRU logic?

thanks

Dino

From dino at no.spam.ar  Tue Feb 14 17:08:18 2023
From: dino at no.spam.ar (Dino)
Date: Tue, 14 Feb 2023 17:08:18 -0500
Subject: Comparing caching strategies
In-Reply-To: <otBFL.3111707$WRz3.215206@fx03.ams4>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
Message-ID: <nDTGL.3255638$miq3.1576571@fx02.ams4>

On 2/10/2023 7:39 PM, Dino wrote:
> 
> - How would you structure the caching so that different caching 
> strategies are "pluggable"? change one line of code (or even a config 
> file) and a different caching strategy is used in the next run. Is this 
> the job for a design pattern such as factory or facade?


turns out that the strategy pattern was the right one for me.




From rosuav at gmail.com  Tue Feb 14 17:45:50 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 15 Feb 2023 09:45:50 +1100
Subject: LRU cache
In-Reply-To: <9CTGL.3255637$miq3.78888@fx02.ams4>
References: <9CTGL.3255637$miq3.78888@fx02.ams4>
Message-ID: <CAPTjJmpZ+12xzGhZUsG4boKOArPpK7kRCz+EfYiBP5KBUCJahQ@mail.gmail.com>

On Wed, 15 Feb 2023 at 09:37, Dino <dino at no.spam.ar> wrote:
>
>
> Here's my problem today. I am using a dict() to implement a quick and
> dirty in-memory cache.
>
> I am stopping adding elements when I am reaching 1000 elements (totally
> arbitrary number), but I would like to have something slightly more
> sophisticated to free up space for newer and potentially more relevant
> entries.
>
> I am thinking of the Least Recently Used principle, but how to implement
> that is not immediate. Before I embark on reinventing the wheel, is
> there a tool, library or smart trick that will allow me to remove
> elements with LRU logic?
>

Check out functools.lru_cache :)

ChrisA

From torriem at gmail.com  Tue Feb 14 17:50:41 2023
From: torriem at gmail.com (Michael Torrie)
Date: Tue, 14 Feb 2023 15:50:41 -0700
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
Message-ID: <597ae645-2c4e-bbfa-5ed1-c826333f4db0@gmail.com>

On 2/14/23 00:09, Stephen Tucker wrote:
> I have two questions:
> 1. Is there a straightforward explanation for this or is it a bug?
To you 1/3 may be an exact fraction, and the definition of raising a
number to that power means a cube root which also has an exact answer,
but to the computer, 1/3 is 0.333333333333333 repeating in decimal,
which is some other fraction in binary.  And even rational numbers like
0.2, which are precise and exact, are not in binary
(0.01010101010101010101).  0.2 is .0011011011011011011 on and on forever.

IEEE floating point has very well known limitations.  All languages that
use IEEE floating point will be subject to these limitations.  So it's
not a bug in the sense that all languages will exhibit this behavior.

> 2. Is the same behaviour exhibited in Python 3.x?
Yes. And Java, C++, and any other language that uses IEEE floating point.


From avi.e.gross at gmail.com  Tue Feb 14 18:21:46 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 14 Feb 2023 18:21:46 -0500
Subject: LRU cache
In-Reply-To: <9CTGL.3255637$miq3.78888@fx02.ams4>
References: <9CTGL.3255637$miq3.78888@fx02.ams4>
Message-ID: <000d01d940cb$1c170630$54451290$@gmail.com>

Dino,

If your question is understood, you want to treat a dictionary as a sort of
queue with a maximum number of entries. And, you want to remove some kind of
least useful item to make room for any new one.

Most dictionaries now have entries in the order they were entered. There may
already be some variant out there that implements this but it should not be
hard to create.

So you could simply change the method that adds an item to the dictionary.
If the new next  item is not already in the dictionary, simply remove the
first item using whatever method you wish. 

Getting all the keys may be avoided by using an iterator once as in:
next(iter( my_dict.keys() )) or something like that. 

You can then remove that item using the key and insert your new item at the
end.

Of course,  that is not least recently used but least recently entered. 

To deal with keeping track of what was accessed last or never, is a problem
not trivially solved with just a dictionary. I mean you could store a tuple
for each item that included a date and a payload as the value, and you could
then search all the values and find the one with the oldest date. That seems
a bit much so another architecture could be to maintain another data
structure holding keys and dates that perhaps you keep sorted by date and
every time the cache accesses a value, it finds the item in the second
storage and updates the date and  moves the item to the end of the
structure. 

But note that some may want to keep an access count so you always know how
many times an item has been re-used and thus not remove them as easily.

The main goal of a dictionary is to speed up access and make it almost
linear. Your algorithm can add so much overhead, depending on how it is
done, that it can defeat the purpose. 

What some might do is skip the dictionary and use some kind of queue like a
dequeue that handles your thousand entries and new items are added at the
end, items accessed moved to the front, and a brute force search is used to
find an entry. But note some algorithms like that may end up removing the
newest item immediately as it is least recently used if placed at the end. 

It may be an Ordered Dict is one solution as shown here:

https://www.geeksforgeeks.org/lru-cache-in-python-using-ordereddict/



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Dino
Sent: Tuesday, February 14, 2023 5:07 PM
To: python-list at python.org
Subject: LRU cache


Here's my problem today. I am using a dict() to implement a quick and dirty
in-memory cache.

I am stopping adding elements when I am reaching 1000 elements (totally
arbitrary number), but I would like to have something slightly more
sophisticated to free up space for newer and potentially more relevant
entries.

I am thinking of the Least Recently Used principle, but how to implement
that is not immediate. Before I embark on reinventing the wheel, is there a
tool, library or smart trick that will allow me to remove elements with LRU
logic?

thanks

Dino
--
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Tue Feb 14 20:40:13 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 14 Feb 2023 20:40:13 -0500
Subject: LRU cache
In-Reply-To: <CAPTjJmpZ+12xzGhZUsG4boKOArPpK7kRCz+EfYiBP5KBUCJahQ@mail.gmail.com>
References: <9CTGL.3255637$miq3.78888@fx02.ams4>
 <CAPTjJmpZ+12xzGhZUsG4boKOArPpK7kRCz+EfYiBP5KBUCJahQ@mail.gmail.com>
Message-ID: <002601d940de$72f115a0$58d340e0$@gmail.com>

Chris,

That is a nice decorator solution with some extra features.

We don't know if the OP needed a cache that was more general purpose and
could be accessed from multiple points, and shared across multiple
functions.



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Chris Angelico
Sent: Tuesday, February 14, 2023 5:46 PM
To: python-list at python.org
Subject: Re: LRU cache

On Wed, 15 Feb 2023 at 09:37, Dino <dino at no.spam.ar> wrote:
>
>
> Here's my problem today. I am using a dict() to implement a quick and 
> dirty in-memory cache.
>
> I am stopping adding elements when I am reaching 1000 elements 
> (totally arbitrary number), but I would like to have something 
> slightly more sophisticated to free up space for newer and potentially 
> more relevant entries.
>
> I am thinking of the Least Recently Used principle, but how to 
> implement that is not immediate. Before I embark on reinventing the 
> wheel, is there a tool, library or smart trick that will allow me to 
> remove elements with LRU logic?
>

Check out functools.lru_cache :)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Tue Feb 14 17:27:15 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 14 Feb 2023 17:27:15 -0500
Subject: Changing the original SQLite version to the latest
In-Reply-To: <CAO+295tdAifRpq6PSSix=feZFAfO-=ERxkn=aL5eB25BDU=ghw@mail.gmail.com>
References: <CAO+295tdAifRpq6PSSix=feZFAfO-=ERxkn=aL5eB25BDU=ghw@mail.gmail.com>
Message-ID: <a17405db-784f-4cfe-766c-10dd8272130d@tompassin.net>

On 2/14/2023 3:30 PM, jose isaias cabrera wrote:
> Greetings.
> 
> I have tried both Cygwin and SQLite support, and I have received very
> little ideas from them, so I am trying this to see if anyone has dealt
> with such a problem before.
> 
> If I use Cygwin setup tool and install python39 and thus,
> 
> $ python
> Python 3.9.10 (main, Jan 20 2022, 21:37:52) [GCC 11.2.0] on cygwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import sqlite3
>>>> sqlite3.sqlite_version
> '3.34.0'
>>>>
> 
> As you can see, the SQlite3 version installed is v3.34.0. Right now,
> the version available is 3.40.1. What I would like to do is to work
> with the latest SQLite version. I have 4 Cygwin instances, 2 of them
> are working ok, 2 are not. This is what I have done to get python to
> use 3.41.0:
> - downloaded the latest SQlite check-in from the site
> - $ tar -xvf SQLite-44200596.tar.gz
> - cd to SQLite-44200596
> - $ ./configure --prefix=/usr
> - $ make install
> 
> These steps above have worked on two PCs, but I don't know what is the
> difference that makes the other two work, and the other two not work.
> I have started a few instances of Cygwin on the PC that is not
> working, and I have been trying for a few days, and I am humbling
> myself, and asking for help. So, the request is to get python3 to
> change the SQLite3 library from 3.34.0 to 3.41.0. Any help would be
> greatly appreciated. Since this is Windows 10, it's probably some
> SQLite DLL somewhere that is being pulled instead of the one
> installed. Perhaps some of you can provide a few suggestions to see
> where I can  find a solution. I know the next step is to compile
> python, but, I rather try to find how to fix this and get to the
> bottom of it. Thanks, thanks and thanks.

As a point of reference, the Python installation I've got on my Windows 
box (not a cygwin install) is

Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 
64 bit (AMD64)] on win32

and the sqlite_version is 3.39.4


From jicman at gmail.com  Tue Feb 14 21:29:11 2023
From: jicman at gmail.com (jose isaias cabrera)
Date: Tue, 14 Feb 2023 21:29:11 -0500
Subject: Changing the original SQLite version to the latest
In-Reply-To: <a17405db-784f-4cfe-766c-10dd8272130d@tompassin.net>
References: <CAO+295tdAifRpq6PSSix=feZFAfO-=ERxkn=aL5eB25BDU=ghw@mail.gmail.com>
 <a17405db-784f-4cfe-766c-10dd8272130d@tompassin.net>
Message-ID: <CAO+295sr_wBU1v3rrP5hv+tyidQmTnASWHFLPU1tjwv=3r-2nQ@mail.gmail.com>

On Tue, Feb 14, 2023 at 8:55 PM Thomas Passin <list1 at tompassin.net> wrote:
>
> As a point of reference, the Python installation I've got on my Windows
> box (not a cygwin install) is
>
> Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934
> 64 bit (AMD64)] on win32
>
> and the sqlite_version is 3.39.4

Thanks, previous to v3.40.1, the released version was v.3.39.4. I wish
it was that one that one. :-)

-- 

What if eternity is real?  Hmmmm...

From list1 at tompassin.net  Tue Feb 14 22:59:23 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 14 Feb 2023 22:59:23 -0500
Subject: Changing the original SQLite version to the latest
In-Reply-To: <CAO+295sr_wBU1v3rrP5hv+tyidQmTnASWHFLPU1tjwv=3r-2nQ@mail.gmail.com>
References: <CAO+295tdAifRpq6PSSix=feZFAfO-=ERxkn=aL5eB25BDU=ghw@mail.gmail.com>
 <a17405db-784f-4cfe-766c-10dd8272130d@tompassin.net>
 <CAO+295sr_wBU1v3rrP5hv+tyidQmTnASWHFLPU1tjwv=3r-2nQ@mail.gmail.com>
Message-ID: <d712110f-9a8f-bd08-2e6e-e80625d99d54@tompassin.net>

On 2/14/2023 9:29 PM, jose isaias cabrera wrote:
> On Tue, Feb 14, 2023 at 8:55 PM Thomas Passin <list1 at tompassin.net> wrote:
>>
>> As a point of reference, the Python installation I've got on my Windows
>> box (not a cygwin install) is
>>
>> Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934
>> 64 bit (AMD64)] on win32
>>
>> and the sqlite_version is 3.39.4
> 
> Thanks, previous to v3.40.1, the released version was v.3.39.4. I wish
> it was that one that one. :-)
> 

I haven't tried it but this may help:

https://stackoverflow.com/questions/61091438/how-to-upgrade-sqlite3-version-in-windows-10

"Download the latest release from http://www.sqlite.org/download.html 
and manually copy sqlite3.dll into Python's DLLs subfolder."

Not sure how you using cygwin might affect this.

From frank at chagford.com  Wed Feb 15 00:46:27 2023
From: frank at chagford.com (Frank Millman)
Date: Wed, 15 Feb 2023 07:46:27 +0200
Subject: Changing the original SQLite version to the latest
In-Reply-To: <d712110f-9a8f-bd08-2e6e-e80625d99d54@tompassin.net>
References: <CAO+295tdAifRpq6PSSix=feZFAfO-=ERxkn=aL5eB25BDU=ghw@mail.gmail.com>
 <a17405db-784f-4cfe-766c-10dd8272130d@tompassin.net>
 <CAO+295sr_wBU1v3rrP5hv+tyidQmTnASWHFLPU1tjwv=3r-2nQ@mail.gmail.com>
 <d712110f-9a8f-bd08-2e6e-e80625d99d54@tompassin.net>
Message-ID: <b2e9c901-b467-3de0-95e6-2815e3a234e8@chagford.com>

On 2023-02-15 5:59 AM, Thomas Passin wrote:
 >
 > "Download the latest release from http://www.sqlite.org/download.html
 > and manually copy sqlite3.dll into Python's DLLs subfolder."
 >

I have done exactly this a number of times and it has worked for me.

Frank Millman

		

From jicman at gmail.com  Wed Feb 15 07:49:21 2023
From: jicman at gmail.com (jose isaias cabrera)
Date: Wed, 15 Feb 2023 07:49:21 -0500
Subject: Changing the original SQLite version to the latest
In-Reply-To: <a17405db-784f-4cfe-766c-10dd8272130d@tompassin.net>
References: <CAO+295tdAifRpq6PSSix=feZFAfO-=ERxkn=aL5eB25BDU=ghw@mail.gmail.com>
 <a17405db-784f-4cfe-766c-10dd8272130d@tompassin.net>
Message-ID: <CAO+295ufnUfhJp4wQMjaUiJNW9ZP78aTzJN-T1vwuOUGFYorQQ@mail.gmail.com>

On Tue, Feb 14, 2023 at 8:55 PM Thomas Passin <list1 at tompassin.net> wrote:
>
> On 2/14/2023 3:30 PM, jose isaias cabrera wrote:
> > Greetings.
> >
> > I have tried both Cygwin and SQLite support, and I have received very
> > little ideas from them, so I am trying this to see if anyone has dealt
> > with such a problem before.
> >
> > If I use Cygwin setup tool and install python39 and thus,
> >
> > $ python
> > Python 3.9.10 (main, Jan 20 2022, 21:37:52) [GCC 11.2.0] on cygwin
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> import sqlite3
> >>>> sqlite3.sqlite_version
> > '3.34.0'
> >>>>
> >
> > As you can see, the SQlite3 version installed is v3.34.0. Right now,
> > the version available is 3.40.1. What I would like to do is to work
> > with the latest SQLite version. I have 4 Cygwin instances, 2 of them
> > are working ok, 2 are not. This is what I have done to get python to
> > use 3.41.0:
> > - downloaded the latest SQlite check-in from the site
> > - $ tar -xvf SQLite-44200596.tar.gz
> > - cd to SQLite-44200596
> > - $ ./configure --prefix=/usr
> > - $ make install
> >
> > These steps above have worked on two PCs, but I don't know what is the
> > difference that makes the other two work, and the other two not work.
> > I have started a few instances of Cygwin on the PC that is not
> > working, and I have been trying for a few days, and I am humbling
> > myself, and asking for help. So, the request is to get python3 to
> > change the SQLite3 library from 3.34.0 to 3.41.0. Any help would be
> > greatly appreciated. Since this is Windows 10, it's probably some
> > SQLite DLL somewhere that is being pulled instead of the one
> > installed. Perhaps some of you can provide a few suggestions to see
> > where I can  find a solution. I know the next step is to compile
> > python, but, I rather try to find how to fix this and get to the
> > bottom of it. Thanks, thanks and thanks.
>
> As a point of reference, the Python installation I've got on my Windows
> box (not a cygwin install) is
>
> Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934
> 64 bit (AMD64)] on win32
>
> and the sqlite_version is 3.39.4



In case anyone needs the answer, These steps worked for me:

In case anyone needs the answer, what worked for me was:
-- Downloaded the Pre-release Snapshots
$ wget https://sqlite.org/snapshot/sqlite-snapshot-202302131932.tar.gz
-- untared the snapshot
$ tar xvf sqlite-snapshot-202302131932.tar.gz
-- cd to the untared directory
$ cd sqlite-snapshot-202302131932
$ ./configure --prefix=/usr
$ make install

And this process has set the python SQLite version to the
sqlite-snapshot version. After that, you can download the trunk and
follow the same procedure, and the version of the trunk will be
changed also.

$ ./SQLiteVersion.py
3.41.0
['/usr/lib/python3.9/sqlite3']
3.41.0
2023-02-13 19:32:40
ecdeef43b27412b0b0b09e09a62ad3a03836a3fc80f2070268090e7ca8f02712

I hope this helps.

This script may be useful...
$ cat SQLiteVersion.py
#!/usr/bin/python3
import sqlite3

def ConnectToSharedDB(sdb):
    return sqlite3.connect(sdb)

print(sqlite3.sqlite_version)
print(sqlite3.__path__)
SharedDB = ":memory:"
con = ConnectToSharedDB(SharedDB)
cur = con.cursor()
cur.execute("SELECT sqlite_version(),sqlite_source_id();")
for row in cur:
    print(row[0] + '\r\n' + row[1])

con.close()

-- 

What if eternity is real?  Where will you spend it?  Hmmmm...

From gweatherby at uchc.edu  Wed Feb 15 09:36:24 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Wed, 15 Feb 2023 14:36:24 +0000
Subject: Precision Tail-off?
In-Reply-To: <597ae645-2c4e-bbfa-5ed1-c826333f4db0@gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <597ae645-2c4e-bbfa-5ed1-c826333f4db0@gmail.com>
Message-ID: <SA1SPRMB00405F34B2F8C611BBE12508B9A39@SA1SPRMB0040.namprd14.prod.outlook.com>

All languages that use IEEE floating point will indeed have the same limitations, but it is not true that Python3 only uses IEEE floating point. Using the Decimal class and cribbing a method from StackOverflow, https://stackoverflow.com/questions/47191533/how-to-efficiently-calculate-cube-roots-using-decimal-in-python


import decimal
from decimal import Decimal

decimal.getcontext().prec = 1_000_000


def cube_root(A: Decimal):
    guess = (A - Decimal(1)) / Decimal(3)
    x0 = (Decimal(2) * guess + A / Decimal(guess * guess)) / Decimal(3.0)
    while 1:
        xn = (Decimal(2) * x0 + A / Decimal(x0 * x0)) / Decimal(3.0)
        if xn == x0:
            break
        x0 = xn
    return xn


float_root = 5 ** (1.0 / 3)
float_r3 = float_root * float_root * float_root
print(5 - float_r3)
five = Decimal(5.0)
r = cube_root(five)
decimal_r3 = r * r * r
print(5 - decimal_r3)

----
8.881784197001252e-16
1E-999999

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Michael Torrie <torriem at gmail.com>
Date: Tuesday, February 14, 2023 at 5:52 PM
To: python-list at python.org <python-list at python.org>
Subject: Re: Precision Tail-off?
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

On 2/14/23 00:09, Stephen Tucker wrote:
> I have two questions:
> 1. Is there a straightforward explanation for this or is it a bug?
To you 1/3 may be an exact fraction, and the definition of raising a
number to that power means a cube root which also has an exact answer,
but to the computer, 1/3 is 0.333333333333333 repeating in decimal,
which is some other fraction in binary.  And even rational numbers like
0.2, which are precise and exact, are not in binary
(0.01010101010101010101).  0.2 is .0011011011011011011 on and on forever.

IEEE floating point has very well known limitations.  All languages that
use IEEE floating point will be subject to these limitations.  So it's
not a bug in the sense that all languages will exhibit this behavior.

> 2. Is the same behaviour exhibited in Python 3.x?
Yes. And Java, C++, and any other language that uses IEEE floating point.

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jjhLqksliV_IjxQAHxXvdnOLB00sJU_hfHNIfK2U1NK-yO2X2kOxJtk6nbqEzXZkyOPBOaMdIlz_sHGkpA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jjhLqksliV_IjxQAHxXvdnOLB00sJU_hfHNIfK2U1NK-yO2X2kOxJtk6nbqEzXZkyOPBOaMdIlz_sHGkpA$>

From mats at wichmann.us  Tue Feb 14 19:36:49 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Tue, 14 Feb 2023 17:36:49 -0700
Subject: LRU cache
In-Reply-To: <9CTGL.3255637$miq3.78888@fx02.ams4>
References: <9CTGL.3255637$miq3.78888@fx02.ams4>
Message-ID: <a00f333f-b3cf-1122-4efa-62fd07ce6748@wichmann.us>

On 2/14/23 15:07, Dino wrote:
> 
> Here's my problem today. I am using a dict() to implement a quick and 
> dirty in-memory cache.
> 
> I am stopping adding elements when I am reaching 1000 elements (totally 
> arbitrary number), but I would like to have something slightly more 
> sophisticated to free up space for newer and potentially more relevant 
> entries.
> 
> I am thinking of the Least Recently Used principle, but how to implement 
> that is not immediate. Before I embark on reinventing the wheel, is 
> there a tool, library or smart trick that will allow me to remove 
> elements with LRU logic?

It depends here how fancy you want to get.  If you really need the 
behavior of a size-limited cache and you expect there to be a lot of 
churn, then you'll probably want a combination of data structures... 
e.g. a dict-like thing for quick hashed lookups (means your "keys" need 
to be hashable) and a doubly-linked list for ordering so you can quickly 
move a hit to the most-recent position - and maybe you want to keep 
cache stats as well.

Do look at functools if that well-tested standard library module fits 
your needs. If you need, or are motivated (as a learning experience?) to 
roll your own, my memory tells me the RealPython crew did a tutorial on 
implementing an LRU cache, might be worth a look (not sure if this is 
one of the all-free ones, or one of the 
must-be-a-paid-subscriber-to-get-the-advanced-stuff ones.



From scruelt at hotmail.com  Wed Feb 15 02:33:37 2023
From: scruelt at hotmail.com (scruel tao)
Date: Wed, 15 Feb 2023 07:33:37 +0000
Subject: Add angle brackets for required args in argparse
In-Reply-To: <OSZP286MB0742B742ED736B07B5665A9DA5DD9@OSZP286MB0742.JPNP286.PROD.OUTLOOK.COM>
References: <OSZP286MB0742B742ED736B07B5665A9DA5DD9@OSZP286MB0742.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <OS3P286MB0740CCCB1D94B91E01BB4B79A5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>

If we have the following code:
```
parser = argparse.ArgumentParser(description="test")
parser.add_argument('path')
```

Run it without args, will get error message:
```
usage: test.py [-h] path
test.py: error: the following arguments are required: path
```

However, I hope the message can be as the following:
```
usage: test.py [-h] <path>
test.py: error: the following arguments are required: path
```

Or might can consider to provide a way to let user have there own style, like:
```
usage: test.py [-h] path
```

Thanks.


From scruelt at hotmail.com  Wed Feb 15 02:33:55 2023
From: scruelt at hotmail.com (scruel tao)
Date: Wed, 15 Feb 2023 07:33:55 +0000
Subject: Change the identation base value?
In-Reply-To: <OS3P286MB074001F1997110B9E32381D5A5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>
References: <OS3P286MB074001F1997110B9E32381D5A5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <OS3P286MB0740E315B090D29679DE630AA5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>

Currently, we have following PEP:
PEP 8: E114 indentation is not a multiple of 4 (comment)

However, I wonder how many people are using 2 spaces as their code indentation in projects? Should it be nesserary for us to change this PEP from ?multiple of 4? to ?multiple of 2??


From peter.slizik at gmail.com  Wed Feb 15 04:56:13 2023
From: peter.slizik at gmail.com (=?UTF-8?B?UGV0ZXIgU2zDrcW+aWs=?=)
Date: Wed, 15 Feb 2023 10:56:13 +0100
Subject: Python: How to use the 'trace' module programmatically?
Message-ID: <CAC-uhUDmoth-gAL+ac_ejrt=LMZ4d80WOiGXCn2LrYpvR=Ypxg@mail.gmail.com>

Hello,

I'm trying to analyze complex Python code. For some specific reasons, I
decided to use tracing instead of a debugger.

The first thing I tried was:

python -m trace -t /path/to/file.py

The output of this command turned out to be completely useless. The reason
is that there was a thread running in the background, doing some work
every *0.1
s* and this generated thousands of lines of tracing information. The useful
information (a reaction to my interaction with app GUI) scrolled away in a
blink.

For this reason, I decided to limit the scope of tracing. I did the
following.

The original code:

def caller():
    print("I'm the caller.")
    callee()
def callee():
    print("Callee here.")

Code modified for tracing:

import trace

tracer = trace.Tracer(
    count=0,
    trace=1,
)
def outer():
    print("I'm the caller.")
    tracer.runfunc(inner)
def inner():
    print("Callee here.")

Now I launched the program and the tracer did not generate any output. I
was hoping that this would provide complete tracing information, but only
for the limited scope of inner().

No success with tracer.run() either.

What I was able to do, when I set count=1, I was able to catch the coverage
data with tracer.results() and write them to a file. But the tracing
information was not generated even in this case.

Am I doing anything wrong?

Peter

From rosuav at gmail.com  Wed Feb 15 12:20:47 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 16 Feb 2023 04:20:47 +1100
Subject: Change the identation base value?
In-Reply-To: <OS3P286MB0740E315B090D29679DE630AA5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>
References: <OS3P286MB074001F1997110B9E32381D5A5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>
 <OS3P286MB0740E315B090D29679DE630AA5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>
Message-ID: <CAPTjJmoymDqzeQR6ZqCKjkCEWNdNY94BNLKS4Jw7Q2XgqZZR=g@mail.gmail.com>

On Thu, 16 Feb 2023 at 04:18, scruel tao <scruelt at hotmail.com> wrote:
>
> Currently, we have following PEP:
> PEP 8: E114 indentation is not a multiple of 4 (comment)
>
> However, I wonder how many people are using 2 spaces as their code indentation in projects? Should it be nesserary for us to change this PEP from ?multiple of 4? to ?multiple of 2??
>

Read PEP 8 itself before making a decision.

1) It is NOT rules for all Python code, only the standard library.
2) https://peps.python.org/pep-0008/#a-foolish-consistency-is-the-hobgoblin-of-little-minds

It is not necessary to change the PEP; it is necessary to fix tools
that claim more authority than they actually have.

ChrisA

From gweatherby at uchc.edu  Wed Feb 15 13:15:59 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Wed, 15 Feb 2023 18:15:59 +0000
Subject: Python: How to use the 'trace' module programmatically?
In-Reply-To: <CAC-uhUDmoth-gAL+ac_ejrt=LMZ4d80WOiGXCn2LrYpvR=Ypxg@mail.gmail.com>
References: <CAC-uhUDmoth-gAL+ac_ejrt=LMZ4d80WOiGXCn2LrYpvR=Ypxg@mail.gmail.com>
Message-ID: <SA1SPRMB004002012F1F0AD2C994E8DDB9A39@SA1SPRMB0040.namprd14.prod.outlook.com>

Have you tried the filter options?

?These options may be repeated multiple times.
--ignore-module=<mod>
Ignore each of the given module names and its submodules (if it is a package). The argument can be a list of names separated by a comma.
--ignore-dir=<dir>
Ignore all modules and packages in the named directory and subdirectories. The argument can be a list of directories separated by os.pathsep<https://docs.python.org/3/library/os.html#os.pathsep>.?



From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Peter Sl??ik <peter.slizik at gmail.com>
Date: Wednesday, February 15, 2023 at 12:22 PM
To: python-list at python.org <python-list at python.org>
Subject: Python: How to use the 'trace' module programmatically?
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Hello,

I'm trying to analyze complex Python code. For some specific reasons, I
decided to use tracing instead of a debugger.

The first thing I tried was:

python -m trace -t /path/to/file.py

The output of this command turned out to be completely useless. The reason
is that there was a thread running in the background, doing some work
every *0.1
s* and this generated thousands of lines of tracing information. The useful
information (a reaction to my interaction with app GUI) scrolled away in a
blink.

For this reason, I decided to limit the scope of tracing. I did the
following.

The original code:

def caller():
    print("I'm the caller.")
    callee()
def callee():
    print("Callee here.")

Code modified for tracing:

import trace

tracer = trace.Tracer(
    count=0,
    trace=1,
)
def outer():
    print("I'm the caller.")
    tracer.runfunc(inner)
def inner():
    print("Callee here.")

Now I launched the program and the tracer did not generate any output. I
was hoping that this would provide complete tracing information, but only
for the limited scope of inner().

No success with tracer.run() either.

What I was able to do, when I set count=1, I was able to catch the coverage
data with tracer.results() and write them to a file. But the tracing
information was not generated even in this case.

Am I doing anything wrong?

Peter
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!m_WsH0rFVMLw4RMkVvcu-ZoClOMWVTsdB8E99Qy5Sq7ZZF1iBw5_NpLvorEe3_hYvy2kdDwe2obDr1E2ZjFCM3Of$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!m_WsH0rFVMLw4RMkVvcu-ZoClOMWVTsdB8E99Qy5Sq7ZZF1iBw5_NpLvorEe3_hYvy2kdDwe2obDr1E2ZjFCM3Of$>

From dino at no.spam.ar  Wed Feb 15 14:58:37 2023
From: dino at no.spam.ar (Dino)
Date: Wed, 15 Feb 2023 14:58:37 -0500
Subject: LRU cache
In-Reply-To: <mailman.1831.1676481401.20444.python-list@python.org>
References: <9CTGL.3255637$miq3.78888@fx02.ams4>
 <a00f333f-b3cf-1122-4efa-62fd07ce6748@wichmann.us>
 <mailman.1831.1676481401.20444.python-list@python.org>
Message-ID: <NPaHL.2601707$14z3.887943@fx11.ams4>


Thank you Mats, Avi and Chris

btw, functools.lru_cache seems rather different from what I need, but 
maybe I am missing something. I'll look closer.

On 2/14/2023 7:36 PM, Mats Wichmann wrote:
> On 2/14/23 15:07, Dino wrote:
>>


From cs at cskk.id.au  Wed Feb 15 16:02:46 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Thu, 16 Feb 2023 08:02:46 +1100
Subject: Change the identation base value?
In-Reply-To: <CAPTjJmoymDqzeQR6ZqCKjkCEWNdNY94BNLKS4Jw7Q2XgqZZR=g@mail.gmail.com>
References: <CAPTjJmoymDqzeQR6ZqCKjkCEWNdNY94BNLKS4Jw7Q2XgqZZR=g@mail.gmail.com>
Message-ID: <Y+1Idg35u2ClBDG0@cskk.homeip.net>

On 16Feb2023 04:20, Chris Angelico <rosuav at gmail.com> wrote:
>On Thu, 16 Feb 2023 at 04:18, scruel tao <scruelt at hotmail.com> wrote:
>> Currently, we have following PEP:
>> PEP 8: E114 indentation is not a multiple of 4 (comment)
>>
>> However, I wonder how many people are using 2 spaces as their code indentation in projects? Should it be nesserary for us to change this PEP from ?multiple of 4? to ?multiple of 2??
>
>Read PEP 8 itself before making a decision.
>
>1) It is NOT rules for all Python code, only the standard library.
>2) https://peps.python.org/pep-0008/#a-foolish-consistency-is-the-hobgoblin-of-little-minds
>
>It is not necessary to change the PEP; it is necessary to fix tools
>that claim more authority than they actually have.

Speaking as a 2-space indenter myself (in my personal code):

The tool in question is likely `pycodestyle`, whose _default_ linting 
measures PEP8 criteria among other things.

I run a few different linters (via a script named "lint") and turn off 
quite a few of the `pycodestyle` checks, `E114` included.

`pycodestyle` doesn't clean any special "authority", and exlicitly 
offers ways to turn off whichever checks you find unsuitable to your 
code.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From barry at barrys-emacs.org  Thu Feb 16 03:55:59 2023
From: barry at barrys-emacs.org (Barry)
Date: Thu, 16 Feb 2023 08:55:59 +0000
Subject: Python: How to use the 'trace' module programmatically?
In-Reply-To: <CAC-uhUDmoth-gAL+ac_ejrt=LMZ4d80WOiGXCn2LrYpvR=Ypxg@mail.gmail.com>
References: <CAC-uhUDmoth-gAL+ac_ejrt=LMZ4d80WOiGXCn2LrYpvR=Ypxg@mail.gmail.com>
Message-ID: <F640DF52-A412-4CC8-BFAE-1238E00B1AFC@barrys-emacs.org>



> On 15 Feb 2023, at 17:23, Peter Sl??ik <peter.slizik at gmail.com> wrote:
> 
> ?Hello,
> 
> I'm trying to analyze complex Python code. For some specific reasons, I
> decided to use tracing instead of a debugger.
> 
> The first thing I tried was:
> 
> python -m trace -t /path/to/file.py
> 
> The output of this command turned out to be completely useless. The reason
> is that there was a thread running in the background, doing some work
> every *0.1
> s* and this generated thousands of lines of tracing information. The useful
> information (a reaction to my interaction with app GUI) scrolled away in a
> blink.
> 
> For this reason, I decided to limit the scope of tracing. I did the
> following.
> 
> The original code:
> 
> def caller():
>    print("I'm the caller.")
>    callee()
> def callee():
>    print("Callee here.")
> 
> Code modified for tracing:
> 
> import trace
> 
> tracer = trace.Tracer(
>    count=0,
>    trace=1,
> )
> def outer():
>    print("I'm the caller.")
>    tracer.runfunc(inner)


The docs show that you need to do either add the outfile to trace
or generate and write the report after runfunc returns.

I have not tested this, just read the docs out of curiosity
Here https://docs.python.org/3/library/trace.html

Barry

> def inner():
>    print("Callee here.")
> 
> Now I launched the program and the tracer did not generate any output. I
> was hoping that this would provide complete tracing information, but only
> for the limited scope of inner().
> 
> No success with tracer.run() either.
> 
> What I was able to do, when I set count=1, I was able to catch the coverage
> data with tracer.results() and write them to a file. But the tracing
> information was not generated even in this case.
> 
> Am I doing anything wrong?
> 
> Peter
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 

From peter.slizik at gmail.com  Thu Feb 16 05:01:19 2023
From: peter.slizik at gmail.com (=?UTF-8?B?UGV0ZXIgU2zDrcW+aWs=?=)
Date: Thu, 16 Feb 2023 11:01:19 +0100
Subject: Python: How to use the 'trace' module programmatically?
In-Reply-To: <F640DF52-A412-4CC8-BFAE-1238E00B1AFC@barrys-emacs.org>
References: <CAC-uhUDmoth-gAL+ac_ejrt=LMZ4d80WOiGXCn2LrYpvR=Ypxg@mail.gmail.com>
 <F640DF52-A412-4CC8-BFAE-1238E00B1AFC@barrys-emacs.org>
Message-ID: <CAC-uhUAFazu3kiD5oYSiETL-x87ZAJyxC-TkK43LXUWh5Yc0EQ@mail.gmail.com>

Gerard - I did not use the filtering options. Thank you for bringing them
to my attention.

Barry - thank you for the insight.

Now the tracing works as expected. I'm not sure why it didn't work
before... Maybe the program redirected stdout?

Thank you guys,
Peter

On Thu, Feb 16, 2023 at 9:56 AM Barry <barry at barrys-emacs.org> wrote:

>
>
> On 15 Feb 2023, at 17:23, Peter Sl??ik <peter.slizik at gmail.com> wrote:
>
> ?Hello,
>
> I'm trying to analyze complex Python code. For some specific reasons, I
> decided to use tracing instead of a debugger.
>
> The first thing I tried was:
>
> python -m trace -t /path/to/file.py
>
> The output of this command turned out to be completely useless. The reason
> is that there was a thread running in the background, doing some work
> every *0.1
> s* and this generated thousands of lines of tracing information. The useful
> information (a reaction to my interaction with app GUI) scrolled away in a
> blink.
>
> For this reason, I decided to limit the scope of tracing. I did the
> following.
>
> The original code:
>
> def caller():
>    print("I'm the caller.")
>    callee()
> def callee():
>    print("Callee here.")
>
> Code modified for tracing:
>
> import trace
>
> tracer = trace.Tracer(
>    count=0,
>    trace=1,
> )
> def outer():
>    print("I'm the caller.")
>    tracer.runfunc(inner)
>
>
>
> The docs show that you need to do either add the outfile to trace
> or generate and write the report after runfunc returns.
>
> I have not tested this, just read the docs out of curiosity
> Here https://docs.python.org/3/library/trace.html
>
> Barry
>
> def inner():
>    print("Callee here.")
>
> Now I launched the program and the tracer did not generate any output. I
> was hoping that this would provide complete tracing information, but only
> for the limited scope of inner().
>
> No success with tracer.run() either.
>
> What I was able to do, when I set count=1, I was able to catch the coverage
> data with tracer.results() and write them to a file. But the tracing
> information was not generated even in this case.
>
> Am I doing anything wrong?
>
> Peter
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>

From pkpearson at nowhere.invalid  Thu Feb 16 10:40:19 2023
From: pkpearson at nowhere.invalid (Peter Pearson)
Date: 16 Feb 2023 15:40:19 GMT
Subject: Precision Tail-off?
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
Message-ID: <k57132F3e1U1@mid.individual.net>

On Tue, 14 Feb 2023 11:17:20 +0000, Oscar Benjamin wrote:
> On Tue, 14 Feb 2023 at 07:12, Stephen Tucker <stephen_tucker at sil.org> wrote:
[snip]
>> I have just produced the following log in IDLE (admittedly, in Python
>> 2.7.10 and, yes I know that it has been superseded).
>>
>> It appears to show a precision tail-off as the supplied float gets bigger.
[snip]
>>
>> For your information, the first 20 significant figures of the cube root in
>> question are:
>>    49793385921817447440
>>
>> Stephen Tucker.
>> ----------------------------------------------
>> >>> 123.456789 ** (1.0 / 3.0)
>> 4.979338592181744
>> >>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
>> 49793385921817.36
>
> You need to be aware that 1.0/3.0 is a float that is not exactly equal
> to 1/3 ...
[snip]
> SymPy again:
>
> In [37]: a, x = symbols('a, x')
>
> In [38]: print(series(a**x, x, Rational(1, 3), 2))
> a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
>
> You can see that the leading relative error term from x being not
> quite equal to 1/3 is proportional to the log of the base. You should
> expect this difference to grow approximately linearly as you keep
> adding more zeros in the base.

Marvelous.  Thank you.


-- 
To email me, substitute nowhere->runbox, invalid->com.

From gweatherby at uchc.edu  Thu Feb 16 21:40:21 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Fri, 17 Feb 2023 02:40:21 +0000
Subject: LRU cache
In-Reply-To: <NPaHL.2601707$14z3.887943@fx11.ams4>
References: <9CTGL.3255637$miq3.78888@fx02.ams4>
 <a00f333f-b3cf-1122-4efa-62fd07ce6748@wichmann.us>
 <mailman.1831.1676481401.20444.python-list@python.org>
 <NPaHL.2601707$14z3.887943@fx11.ams4>
Message-ID: <SJ0PR14MB5864935B923047D8EFF0FD54B9A19@SJ0PR14MB5864.namprd14.prod.outlook.com>

I think this does the trick:

https://gist.github.com/Gerardwx/c60d200b4db8e7864cb3342dd19d41c9


#!/usr/bin/env python3
import collections
import random
from typing import Hashable, Any, Optional, Dict, Tuple


class LruCache:
    """Dictionary like storage of most recently inserted values"""

    def __init__(self, size: int = 1000):
        """:param size number of cached entries"""
        assert isinstance(size, int)
        self.size = size
        self.insert_counter = 0
       self.oldest = 0
        self._data : Dict[Hashable,Tuple[Any,int]]= {} # store values and age index
        self._lru: Dict[int, Hashable] = {} # age counter dictionary

    def insert(self, key: Hashable, value: Any) -> None:
        """Insert into dictionary"""
        existing = self._data.get(key, None)
        self._data[key] = (value, self.insert_counter)
        self._lru[self.insert_counter] = key
        if existing is not None:
            self._lru.pop(existing[1], None)  # remove old counter value, if it exists
        self.insert_counter += 1
        if (sz := len(self._data)) > self.size:  # is cache full?
            assert sz == self.size + 1
            while (
            key := self._lru.get(self.oldest, None)) is None:  # index may not be present, if value was reinserted
                self.oldest += 1
            del self._data[key]  # remove oldest key / value from dictionary
            del self._lru[self.oldest]
            self.oldest += 1  # next oldest index
        assert len(self._lru) == len(self._data)

    def get(self, key: Hashable) -> Optional[Any]:
        """Get value or return None if not in cache"""
        if (tpl := self._data.get(key, None)) is not None:
            return tpl[0]
        return None


if __name__ == "__main__":
    CACHE_SIZE = 1000
    TEST_SIZE = 1_000_000
    cache = LruCache(size=CACHE_SIZE)

    all = []
    for i in range(TEST_SIZE):
        all.append(random.randint(-5000, 5000))

    summary = collections.defaultdict(int)
    for value in all:
        cache.insert(value, value * value)
        summary[value] += 1
    smallest = TEST_SIZE
    largest = -TEST_SIZE
    total = 0
    for value, count in summary.items():
        smallest = min(smallest, count)
        largest = max(largest, count)
        total += count
    avg = total / len(summary)
    print(f"{len(summary)} values occurrences range from {smallest} to {largest}, average {avg:.1f}")

    recent = set()  # recent most recent entries
    for i in range(len(all) - 1, -1, -1):  # loop backwards to get the most recent entries
        value = all[i]
        if len(recent) < CACHE_SIZE:
            recent.add(value)
        if value in recent:
            if (r := cache.get(value)) != value * value:
                raise ValueError(f"Cache missing recent {value} {r}")
        else:
            if cache.get(value) != None:
                raise ValueError(f"Cache includes old {value}")

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Dino <dino at no.spam.ar>
Date: Wednesday, February 15, 2023 at 3:07 PM
To: python-list at python.org <python-list at python.org>
Subject: Re: LRU cache
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Thank you Mats, Avi and Chris

btw, functools.lru_cache seems rather different from what I need, but
maybe I am missing something. I'll look closer.

On 2/14/2023 7:36 PM, Mats Wichmann wrote:
> On 2/14/23 15:07, Dino wrote:
>>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jb3Gr2BFAPLJ2YuI5rFdJUtalqWcijhxHAfdmCI3afnLFDdcekALxDYAQwpE1L_JlJBBJ-BB3BuLdoSE$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jb3Gr2BFAPLJ2YuI5rFdJUtalqWcijhxHAfdmCI3afnLFDdcekALxDYAQwpE1L_JlJBBJ-BB3BuLdoSE$>

From rob.cliffe at btinternet.com  Tue Feb 14 17:20:12 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 14 Feb 2023 22:20:12 +0000
Subject: Comparing caching strategies
In-Reply-To: <otBFL.3111707$WRz3.215206@fx03.ams4>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
Message-ID: <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>

On 11/02/2023 00:39, Dino wrote:
> First off, a big shout out to Peter J. Holzer, who mentioned roaring 
> bitmaps a few days ago and led me to quite a discovery.
>
I was intrigued to hear about roaring bitmaps and discover they really 
were a thing (not a typo as I suspected at first).
What next, I wonder?
 ??? argumentative arrays
 ??? chattering classes (on second thoughts, we have those already)
 ??? dancing dictionaries
 ??? garrulous generators
 ??? laughing lists
 ??? piping pipelines
 ??? singing strings
 ??? speaking sets
 ??? stuttering sorts
 ??? talking tuples
 ??? whistling walruses?
The future awaits [pun not intended] ...

From python at mrabarnett.plus.com  Thu Feb 16 23:23:53 2023
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 17 Feb 2023 04:23:53 +0000
Subject: Comparing caching strategies
In-Reply-To: <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
Message-ID: <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>

On 2023-02-14 22:20, Rob Cliffe via Python-list wrote:
> On 11/02/2023 00:39, Dino wrote:
>> First off, a big shout out to Peter J. Holzer, who mentioned roaring 
>> bitmaps a few days ago and led me to quite a discovery.
>>
> I was intrigued to hear about roaring bitmaps and discover they really
> were a thing (not a typo as I suspected at first).
> What next, I wonder?
>   ??? argumentative arrays
>   ??? chattering classes (on second thoughts, we have those already)
>   ??? dancing dictionaries
>   ??? garrulous generators
>   ??? laughing lists
>   ??? piping pipelines
>   ??? singing strings
>   ??? speaking sets
>   ??? stuttering sorts
>   ??? talking tuples
>   ??? whistling walruses?

babbling bytestrings?

> The future awaits [pun not intended] ...


From rosuav at gmail.com  Fri Feb 17 00:04:46 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 17 Feb 2023 16:04:46 +1100
Subject: Comparing caching strategies
In-Reply-To: <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
Message-ID: <CAPTjJmpn1O2y4y1Vb0Lo3fN0axd8dx5bdk1ddvX3J=mJO=dsgA@mail.gmail.com>

On Fri, 17 Feb 2023 at 15:28, MRAB <python at mrabarnett.plus.com> wrote:
>
> On 2023-02-14 22:20, Rob Cliffe via Python-list wrote:
> > On 11/02/2023 00:39, Dino wrote:
> >> First off, a big shout out to Peter J. Holzer, who mentioned roaring
> >> bitmaps a few days ago and led me to quite a discovery.
> >>
> > I was intrigued to hear about roaring bitmaps and discover they really
> > were a thing (not a typo as I suspected at first).
> > What next, I wonder?
> >       argumentative arrays
> >       chattering classes (on second thoughts, we have those already)
> >       dancing dictionaries
> >       garrulous generators
> >       laughing lists
> >       piping pipelines
> >       singing strings
> >       speaking sets
> >       stuttering sorts
> >       talking tuples
> >       whistling walruses?
>
> babbling bytestrings?
>

Excited exceptions. I'm sure there's a particle physics crossover
waiting for this one.

ChrisA

From avi.e.gross at gmail.com  Fri Feb 17 00:07:12 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Fri, 17 Feb 2023 00:07:12 -0500
Subject: Comparing caching strategies
In-Reply-To: <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
Message-ID: <00bc01d9428d$b245c880$16d15980$@gmail.com>

I am less interested in the choice of names than the pro and con of when these Roaring bitmaps are worth using and when they are not.

It is a bit like discussing whether various compression techniques are worth using as the storage or memory costs can be weighed against the CPU or transient memory costs of compressing and uncompressing.  The value tends to depend on many factors and there may even be times you want to store the data in multiple data structures with each optimized to store that kind or amount of data.

Roaring bitmaps claim to be an improvement not only over uncompressed structures but some other compressed versions but my reading shows it may be limited to some uses. Bitsets in general seem to be useful only for a largely contiguous set of integers where each sequential bit represents whether the nth integer above the lowest is in the set or not. Of course, properly set up, this makes Unions and Intersections and some other operations fairly efficient. But sets are not the same as dictionaries and often you are storing other data types than smaller integers.

Many normal compression techniques can require lots of time to uncompress to find anything. My impression is that Roaring Bitmaps is a tad like the pkzip software that tries various compression techniques on each file and chooses whatever seems to work better on each one. That takes extra time when zipping, but when unzipping a file, it goes directly to the method used to compress it as the type is in a header and just expands it one way.

My impression is that Roaring bitmaps breaks up the range of integers into smaller chunks and depending on what is being stored in that chunk, may leave it as an uncompressed bitmap, or a list of the sparse contents, or other storage methods and can search each version fairly quickly. 

So, I have no doubt it works great for some applications such as treating social security numbers as integers. It likely would be overkill to store something like the components of an IP address between 0 and 255 inclusive.

But having said that, there may well be non-integer data that can be mapped into and out of integers. As an example, airports or radio stations have names like LAX or WPIX. If you limit yourself to ASCII letters then every one of them can be stored as a 32-bit integer, perhaps with some padding. Of course for such fairly simple data, some might choose to place the data in a balanced tree structure and get reasonable search speed.

I am curious about the size of some of these structures but obviously it depends. Are they stored on disk in this form too?


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of MRAB
Sent: Thursday, February 16, 2023 11:24 PM
To: python-list at python.org
Subject: Re: Comparing caching strategies

On 2023-02-14 22:20, Rob Cliffe via Python-list wrote:
> On 11/02/2023 00:39, Dino wrote:
>> First off, a big shout out to Peter J. Holzer, who mentioned roaring 
>> bitmaps a few days ago and led me to quite a discovery.
>>
> I was intrigued to hear about roaring bitmaps and discover they really 
> were a thing (not a typo as I suspected at first).
> What next, I wonder?
>       argumentative arrays
>       chattering classes (on second thoughts, we have those already)
>       dancing dictionaries
>       garrulous generators
>       laughing lists
>       piping pipelines
>       singing strings
>       speaking sets
>       stuttering sorts
>       talking tuples
>       whistling walruses?

babbling bytestrings?

> The future awaits [pun not intended] ...

--
https://mail.python.org/mailman/listinfo/python-list


From stephen_tucker at sil.org  Fri Feb 17 05:27:08 2023
From: stephen_tucker at sil.org (Stephen Tucker)
Date: Fri, 17 Feb 2023 10:27:08 +0000
Subject: Precision Tail-off?
In-Reply-To: <k57132F3e1U1@mid.individual.net>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
Message-ID: <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>

Thanks, one and all, for your reponses.

This is a hugely controversial claim, I know, but I would consider this
behaviour to be a serious deficiency in the IEEE standard.

Consider an integer N consisting of a finitely-long string of digits in
base 10.

Consider the infinitely-precise cube root of N (yes I know that it could
never be computed unless N is the cube of an integer, but this is a
mathematical argument, not a computational one), also in base 10. Let's
call it RootN.

Now consider appending three zeroes to the right-hand end of N (let's call
it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).

The *only *difference between RootN and RootNZZZ is that the decimal point
in RootNZZZ is one place further to the right than the decimal point in
RootN.

None of the digits in RootNZZZ's string should be different from the
corresponding digits in RootN.

I rest my case.

Perhaps this observation should be brought to the attention of the IEEE. I
would like to know their response to it.

Stephen Tucker.


On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson <pkpearson at nowhere.invalid>
wrote:

> On Tue, 14 Feb 2023 11:17:20 +0000, Oscar Benjamin wrote:
> > On Tue, 14 Feb 2023 at 07:12, Stephen Tucker <stephen_tucker at sil.org>
> wrote:
> [snip]
> >> I have just produced the following log in IDLE (admittedly, in Python
> >> 2.7.10 and, yes I know that it has been superseded).
> >>
> >> It appears to show a precision tail-off as the supplied float gets
> bigger.
> [snip]
> >>
> >> For your information, the first 20 significant figures of the cube root
> in
> >> question are:
> >>    49793385921817447440
> >>
> >> Stephen Tucker.
> >> ----------------------------------------------
> >> >>> 123.456789 ** (1.0 / 3.0)
> >> 4.979338592181744
> >> >>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
> >> 49793385921817.36
> >
> > You need to be aware that 1.0/3.0 is a float that is not exactly equal
> > to 1/3 ...
> [snip]
> > SymPy again:
> >
> > In [37]: a, x = symbols('a, x')
> >
> > In [38]: print(series(a**x, x, Rational(1, 3), 2))
> > a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
> >
> > You can see that the leading relative error term from x being not
> > quite equal to 1/3 is proportional to the log of the base. You should
> > expect this difference to grow approximately linearly as you keep
> > adding more zeros in the base.
>
> Marvelous.  Thank you.
>
>
> --
> To email me, substitute nowhere->runbox, invalid->com.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From stephen_tucker at sil.org  Fri Feb 17 06:01:06 2023
From: stephen_tucker at sil.org (Stephen Tucker)
Date: Fri, 17 Feb 2023 11:01:06 +0000
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
Message-ID: <CAP=-cKUhwGzwkOP5+n6=4fay-NXxCj9LtXrR4RCVs3tt1WXgdQ@mail.gmail.com>

As a follow-up to my previous message, I have just produced the following
log on IDLE, for your information:
------------------------------
>>> math.e ** (math.log
(123456789000000000000000000000000000000000000000000) / 3)
4.979338592181741e+16
>>> 10 ** (math.log10 (123456789000000000000000000000000000000000000000000)
/ 3)
4.979338592181736e+16
>>> 123456789000000000000000000000000000000000000000000 ** (1.0 / 3.0)
4.979338592181734e+16
>>> 123456789e42 ** (1.0 / 3.0)
4.979338592181734e+16
------------------------------

Stephen Tucker.


On Fri, Feb 17, 2023 at 10:27 AM Stephen Tucker <stephen_tucker at sil.org>
wrote:

> Thanks, one and all, for your reponses.
>
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
>
> Consider an integer N consisting of a finitely-long string of digits in
> base 10.
>
> Consider the infinitely-precise cube root of N (yes I know that it could
> never be computed unless N is the cube of an integer, but this is a
> mathematical argument, not a computational one), also in base 10. Let's
> call it RootN.
>
> Now consider appending three zeroes to the right-hand end of N (let's call
> it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).
>
> The *only *difference between RootN and RootNZZZ is that the decimal
> point in RootNZZZ is one place further to the right than the decimal point
> in RootN.
>
> None of the digits in RootNZZZ's string should be different from the
> corresponding digits in RootN.
>
> I rest my case.
>
> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.
>
> Stephen Tucker.
>
>
> On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson <pkpearson at nowhere.invalid>
> wrote:
>
>> On Tue, 14 Feb 2023 11:17:20 +0000, Oscar Benjamin wrote:
>> > On Tue, 14 Feb 2023 at 07:12, Stephen Tucker <stephen_tucker at sil.org>
>> wrote:
>> [snip]
>> >> I have just produced the following log in IDLE (admittedly, in Python
>> >> 2.7.10 and, yes I know that it has been superseded).
>> >>
>> >> It appears to show a precision tail-off as the supplied float gets
>> bigger.
>> [snip]
>> >>
>> >> For your information, the first 20 significant figures of the cube
>> root in
>> >> question are:
>> >>    49793385921817447440
>> >>
>> >> Stephen Tucker.
>> >> ----------------------------------------------
>> >> >>> 123.456789 ** (1.0 / 3.0)
>> >> 4.979338592181744
>> >> >>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
>> >> 49793385921817.36
>> >
>> > You need to be aware that 1.0/3.0 is a float that is not exactly equal
>> > to 1/3 ...
>> [snip]
>> > SymPy again:
>> >
>> > In [37]: a, x = symbols('a, x')
>> >
>> > In [38]: print(series(a**x, x, Rational(1, 3), 2))
>> > a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
>> >
>> > You can see that the leading relative error term from x being not
>> > quite equal to 1/3 is proportional to the log of the base. You should
>> > expect this difference to grow approximately linearly as you keep
>> > adding more zeros in the base.
>>
>> Marvelous.  Thank you.
>>
>>
>> --
>> To email me, substitute nowhere->runbox, invalid->com.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>

From list1 at tompassin.net  Fri Feb 17 08:51:55 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 17 Feb 2023 08:51:55 -0500
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
Message-ID: <25b8f4da-880f-3e76-2d96-1d424a5ec7cd@tompassin.net>

On 2/17/2023 5:27 AM, Stephen Tucker wrote:
> Thanks, one and all, for your reponses.
> 
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
> 
> Consider an integer N consisting of a finitely-long string of digits in
> base 10.

What you are not considering is that the IEEE standard is about trying 
to achieve a balance between resource use (memory and registers), 
precision, speed of computation, reliability (consistent and stable 
results), and compatibility.  So there have to be many tradeoffs.  One 
of them is the use of binary representation.  It has never been about 
achieving ideal mathematical perfection for some set of special cases.

Want a different set of tradeoffs?  Fine, go for it.  Python has Decimal 
and rational libraries among others.  They run more slowly than IEEE, 
but maybe that's a good tradeoff for you.  Use a symbolic math library. 
Trap special cases of interest to you and calculate them differently. 
Roll your own.  Trouble is, you have to know one heck of a lot to roll 
your own, and it may take decades of debugging to get it right.  Even 
then it won't have hardware assistance like IEEE floating point usually has.

> Consider the infinitely-precise cube root of N (yes I know that it could
> never be computed unless N is the cube of an integer, but this is a
> mathematical argument, not a computational one), also in base 10. Let's
> call it RootN.
> 
> Now consider appending three zeroes to the right-hand end of N (let's call
> it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).
> 
> The *only *difference between RootN and RootNZZZ is that the decimal point
> in RootNZZZ is one place further to the right than the decimal point in
> RootN.
> 
> None of the digits in RootNZZZ's string should be different from the
> corresponding digits in RootN.
> 
> I rest my case.
> 
> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.
> 
> Stephen Tucker.
> 
> 
> On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson <pkpearson at nowhere.invalid>
> wrote:
> 
>> On Tue, 14 Feb 2023 11:17:20 +0000, Oscar Benjamin wrote:
>>> On Tue, 14 Feb 2023 at 07:12, Stephen Tucker <stephen_tucker at sil.org>
>> wrote:
>> [snip]
>>>> I have just produced the following log in IDLE (admittedly, in Python
>>>> 2.7.10 and, yes I know that it has been superseded).
>>>>
>>>> It appears to show a precision tail-off as the supplied float gets
>> bigger.
>> [snip]
>>>>
>>>> For your information, the first 20 significant figures of the cube root
>> in
>>>> question are:
>>>>     49793385921817447440
>>>>
>>>> Stephen Tucker.
>>>> ----------------------------------------------
>>>>>>> 123.456789 ** (1.0 / 3.0)
>>>> 4.979338592181744
>>>>>>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
>>>> 49793385921817.36
>>>
>>> You need to be aware that 1.0/3.0 is a float that is not exactly equal
>>> to 1/3 ...
>> [snip]
>>> SymPy again:
>>>
>>> In [37]: a, x = symbols('a, x')
>>>
>>> In [38]: print(series(a**x, x, Rational(1, 3), 2))
>>> a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
>>>
>>> You can see that the leading relative error term from x being not
>>> quite equal to 1/3 is proportional to the log of the base. You should
>>> expect this difference to grow approximately linearly as you keep
>>> adding more zeros in the base.
>>
>> Marvelous.  Thank you.
>>
>>
>> --
>> To email me, substitute nowhere->runbox, invalid->com.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>


From gweatherby at uchc.edu  Fri Feb 17 09:39:42 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Fri, 17 Feb 2023 14:39:42 +0000
Subject: Precision Tail-off?
In-Reply-To: <25b8f4da-880f-3e76-2d96-1d424a5ec7cd@tompassin.net>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <25b8f4da-880f-3e76-2d96-1d424a5ec7cd@tompassin.net>
Message-ID: <SJ0PR14MB58647B930C505B3192C5CE5BB9A19@SJ0PR14MB5864.namprd14.prod.outlook.com>

IEEE did not define a standard for floating point arithmetics. They designed multiple standards, including a decimal float point one.  Although decimal floating point (DFP) hardware used to be manufactured, I couldn?t find any current manufacturers. There was a company that seemed to be active until a few years ago, but they seem to have gone dark: https://twitter.com/SilMinds



From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Thomas Passin <list1 at tompassin.net>
Date: Friday, February 17, 2023 at 9:02 AM
To: python-list at python.org <python-list at python.org>
Subject: Re: Precision Tail-off?
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

On 2/17/2023 5:27 AM, Stephen Tucker wrote:
> Thanks, one and all, for your reponses.
>
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
>
> Consider an integer N consisting of a finitely-long string of digits in
> base 10.

What you are not considering is that the IEEE standard is about trying
to achieve a balance between resource use (memory and registers),
precision, speed of computation, reliability (consistent and stable
results), and compatibility.  So there have to be many tradeoffs.  One
of them is the use of binary representation.  It has never been about
achieving ideal mathematical perfection for some set of special cases.

Want a different set of tradeoffs?  Fine, go for it.  Python has Decimal
and rational libraries among others.  They run more slowly than IEEE,
but maybe that's a good tradeoff for you.  Use a symbolic math library.
Trap special cases of interest to you and calculate them differently.
Roll your own.  Trouble is, you have to know one heck of a lot to roll
your own, and it may take decades of debugging to get it right.  Even
then it won't have hardware assistance like IEEE floating point usually has.

> Consider the infinitely-precise cube root of N (yes I know that it could
> never be computed unless N is the cube of an integer, but this is a
> mathematical argument, not a computational one), also in base 10. Let's
> call it RootN.
>
> Now consider appending three zeroes to the right-hand end of N (let's call
> it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).
>
> The *only *difference between RootN and RootNZZZ is that the decimal point
> in RootNZZZ is one place further to the right than the decimal point in
> RootN.
>
> None of the digits in RootNZZZ's string should be different from the
> corresponding digits in RootN.
>
> I rest my case.
>
> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.
>
> Stephen Tucker.
>
>
> On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson <pkpearson at nowhere.invalid>
> wrote:
>
>> On Tue, 14 Feb 2023 11:17:20 +0000, Oscar Benjamin wrote:
>>> On Tue, 14 Feb 2023 at 07:12, Stephen Tucker <stephen_tucker at sil.org>
>> wrote:
>> [snip]
>>>> I have just produced the following log in IDLE (admittedly, in Python
>>>> 2.7.10 and, yes I know that it has been superseded).
>>>>
>>>> It appears to show a precision tail-off as the supplied float gets
>> bigger.
>> [snip]
>>>>
>>>> For your information, the first 20 significant figures of the cube root
>> in
>>>> question are:
>>>>     49793385921817447440
>>>>
>>>> Stephen Tucker.
>>>> ----------------------------------------------
>>>>>>> 123.456789 ** (1.0 / 3.0)
>>>> 4.979338592181744
>>>>>>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
>>>> 49793385921817.36
>>>
>>> You need to be aware that 1.0/3.0 is a float that is not exactly equal
>>> to 1/3 ...
>> [snip]
>>> SymPy again:
>>>
>>> In [37]: a, x = symbols('a, x')
>>>
>>> In [38]: print(series(a**x, x, Rational(1, 3), 2))
>>> a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
>>>
>>> You can see that the leading relative error term from x being not
>>> quite equal to 1/3 is proportional to the log of the base. You should
>>> expect this difference to grow approximately linearly as you keep
>>> adding more zeros in the base.
>>
>> Marvelous.  Thank you.
>>
>>
>> --
>> To email me, substitute nowhere->runbox, invalid->com.
>> --
>> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jqgolDJWMiHsy0l-fRvM6Flcs478R5LIidNh2fAfa3kuPrtqTm0FC6uQmnUuyWLNypQZd3PkzzGyRzZlkbA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jqgolDJWMiHsy0l-fRvM6Flcs478R5LIidNh2fAfa3kuPrtqTm0FC6uQmnUuyWLNypQZd3PkzzGyRzZlkbA$>
>>

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jqgolDJWMiHsy0l-fRvM6Flcs478R5LIidNh2fAfa3kuPrtqTm0FC6uQmnUuyWLNypQZd3PkzzGyRzZlkbA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jqgolDJWMiHsy0l-fRvM6Flcs478R5LIidNh2fAfa3kuPrtqTm0FC6uQmnUuyWLNypQZd3PkzzGyRzZlkbA$>

From avi.e.gross at gmail.com  Fri Feb 17 12:46:52 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Fri, 17 Feb 2023 12:46:52 -0500
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
Message-ID: <001a01d942f7$d1ff38e0$75fdaaa0$@gmail.com>

Stephen,

What response do you expect from whatever people in the IEEE you want?

The specific IEEE standards were designed and agreed upon by groups working
in caveman times when the memory and CPU time were not so plentiful. The
design of many types, including floating point, had to work decently if not
perfectly so you stored data in ways the hardware of the time could process
it efficiently.

Note all kinds of related issues about what happens if you want an integer
larger than fits into 16 bits or 32 bits or even 64 bits. A Python integer
was designed to be effectively unlimited and uses as much storage as needed.
It can also get ever slower when doing things like looking for gigantic
primes. But it does not have overflow problems.

So could you design a floating point data type with similar features? It
would be some complex data structure that keeps track of the number of
bit/bytes/megabytes currently being used to store the mantissa or exponent
parts and then have some data structure that holds all the bits needed. When
doing any arithmetic like addition or division or more complex things, it
would need to compare the two objects being combined and calculate how to
perhaps expand/convert one to match the other and then do umpteen steps to
generate the result in as many pieces/steps as needed and create a data
structure that holds the result, optionally trimming off terminal parts not
needed or wanted. Then you would need all relevant functions that accept
regular floating point to handle these numbers and generate these numbers.

Can that be done well? Well, sure, but not necessarily WELL. Some would
point you to the Decimal type. It might take a somewhat different tack on
how to do this. But everything comes with a cost.

Perhaps the response from the IEEE would be that what they published was
meant for some purposes but not yours. It may be that a group needs to
formulate a new standard but leave the old ones in place for people willing
to use them as their needs are more modest. 

As an analogy, consider the lowly char that stored a single character in a
byte. II mean good old ASCII but also EBCDIC and the ISO family like ISO
8859-1 and so on. Those standards focused in on the needs of just a few
languages and if you wanted to write something in a mix of languages, it
could be a headache as I have had time I had to shift within one document to
say ISO 8859-8 to include some Hebrew, and ISO 8859-3 for Esperanto and so
on while ISO8859-1 was fine for English, French, German, Spanish and many
others. For some purposes, I had to use encodings like shift JIS to do
Japanese as many Asian languages were outside what ISO was doing.

The solutions since then vary but tend to allow or require multiple bytes
per character. But they retain limits and if we ever enter a Star Trek
Universe with infinite diversity and more languages and encodings, we might
need to again enlarge our viewpoint and perhaps be even more wasteful of our
computing resources to accommodate them all!

Standards are often not made to solve ALL possible problems but to make
clear what is supported and what is not required. Mathematical arguments can
be helpful but practical considerations and the limited time available (as
these darn things can take YEARS to be agreed on) are often dominant.
Frankly, by the tie many standards, such as for a programming language, are
finalized, the reality in the field has often changed. The language may
already have been supplanted largely by others for new work, or souped up
with not-yet-standard features.

I am not against striving for ever better standards and realities. But I do
think a better way to approach this is not to reproach what was done but ask
if we can focus on the near-future and make it better.

Arguably, there are now multiple features out there such as Decimal and they
may be quite different. That often happens without a standard.  But if you
now want everyone to get together and define a new standard that may break
some implementations, ...

As I see it, many computer courses teach the realities as well as the
mathematical fantasies that break down in the real world. One of those that
tend to be stressed is that floating point is not exact and that comparison
operators need to be used with caution. Often the suggestion is to subtract
one number from another and check if the result is fairly close to zero as
in the absolute value is less than an IEEE standard number where the last
few bits are ones. For more complex calculations where the errors can
accumulate, you may need to choose a small number with more such bits near
the end.

Extended precision arithmetic is perhaps cheaper now and can be done for a
reasonable number of digits. It probably is not realistic to do most such
calculations for billions of digits, albeit some of the calculations for the
first googolplex digits of pi might indeed need such methods, as soon as we
fin a way to keep that many digits in memory give the ten to the 80th or so
particles we think are in our observable universe. But knowing pi to that
precision may not be meaningful if an existing value already is so precise
that given an exact number for the diameter of something the size of the
universe (Yes, I know this is nonsense) you could calculate the
circumference (ditto) to less than the size (ditto) of a proton. Any errors
in such a measurement would be swamped by all kinds of things such as
uncertainties in what we can measure, or niggling details about how space
expands irregularly in the area as we speak and so on.

So if you want a new IEEE (or other such body) standard, would you be
satisfied with a new one for say a 16,384 byte monstrosity that holds
gigantic numbers with lots more precision, or hold out for a relatively
flexible and unlimited version that can be expanded until your computer or
planet runs out of storage room and provides answers after a few billion
years when used to just add two of them together?



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Stephen Tucker
Sent: Friday, February 17, 2023 5:27 AM
To: python-list at python.org
Subject: Re: Precision Tail-off?

Thanks, one and all, for your reponses.

This is a hugely controversial claim, I know, but I would consider this
behaviour to be a serious deficiency in the IEEE standard.

Consider an integer N consisting of a finitely-long string of digits in base
10.

Consider the infinitely-precise cube root of N (yes I know that it could
never be computed unless N is the cube of an integer, but this is a
mathematical argument, not a computational one), also in base 10. Let's call
it RootN.

Now consider appending three zeroes to the right-hand end of N (let's call
it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).

The *only *difference between RootN and RootNZZZ is that the decimal point
in RootNZZZ is one place further to the right than the decimal point in
RootN.

None of the digits in RootNZZZ's string should be different from the
corresponding digits in RootN.

I rest my case.

Perhaps this observation should be brought to the attention of the IEEE. I
would like to know their response to it.

Stephen Tucker.


On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson <pkpearson at nowhere.invalid>
wrote:

> On Tue, 14 Feb 2023 11:17:20 +0000, Oscar Benjamin wrote:
> > On Tue, 14 Feb 2023 at 07:12, Stephen Tucker 
> > <stephen_tucker at sil.org>
> wrote:
> [snip]
> >> I have just produced the following log in IDLE (admittedly, in 
> >> Python
> >> 2.7.10 and, yes I know that it has been superseded).
> >>
> >> It appears to show a precision tail-off as the supplied float gets
> bigger.
> [snip]
> >>
> >> For your information, the first 20 significant figures of the cube 
> >> root
> in
> >> question are:
> >>    49793385921817447440
> >>
> >> Stephen Tucker.
> >> ----------------------------------------------
> >> >>> 123.456789 ** (1.0 / 3.0)
> >> 4.979338592181744
> >> >>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
> >> 49793385921817.36
> >
> > You need to be aware that 1.0/3.0 is a float that is not exactly 
> > equal to 1/3 ...
> [snip]
> > SymPy again:
> >
> > In [37]: a, x = symbols('a, x')
> >
> > In [38]: print(series(a**x, x, Rational(1, 3), 2))
> > a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
> >
> > You can see that the leading relative error term from x being not 
> > quite equal to 1/3 is proportional to the log of the base. You 
> > should expect this difference to grow approximately linearly as you 
> > keep adding more zeros in the base.
>
> Marvelous.  Thank you.
>
>
> --
> To email me, substitute nowhere->runbox, invalid->com.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list


From pkpearson at nowhere.invalid  Fri Feb 17 10:57:29 2023
From: pkpearson at nowhere.invalid (Peter Pearson)
Date: 17 Feb 2023 15:57:29 GMT
Subject: Precision Tail-off?
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <mailman.1845.1676629651.20444.python-list@python.org>
Message-ID: <k59mf9Fcm8kU1@mid.individual.net>

On Fri, 17 Feb 2023 10:27:08, Stephen Tucker wrote:[Head-posting undone.]
> On Thu, Feb 16, 2023 at 6:49 PM Peter Pearson <pkpearson at nowhere.invalid>
> wrote:
>> On Tue, 14 Feb 2023 11:17:20 +0000, Oscar Benjamin wrote:
>> > On Tue, 14 Feb 2023 at 07:12, Stephen Tucker <stephen_tucker at sil.org>
>> wrote:
>> [snip]
>> >> I have just produced the following log in IDLE (admittedly, in Python
>> >> 2.7.10 and, yes I know that it has been superseded).
>> >>
>> >> It appears to show a precision tail-off as the supplied float gets
>> bigger.
>> [snip]
>> >>
>> >> For your information, the first 20 significant figures of the cube root
>> in
>> >> question are:
>> >>    49793385921817447440
>> >>
>> >> Stephen Tucker.
>> >> ----------------------------------------------
>> >> >>> 123.456789 ** (1.0 / 3.0)
>> >> 4.979338592181744
>> >> >>> 123456789000000000000000000000000000000000. ** (1.0 / 3.0)
>> >> 49793385921817.36
>> >
>> > You need to be aware that 1.0/3.0 is a float that is not exactly equal
>> > to 1/3 ...
>> [snip]
>> > SymPy again:
>> >
>> > In [37]: a, x = symbols('a, x')
>> >
>> > In [38]: print(series(a**x, x, Rational(1, 3), 2))
>> > a**(1/3) + a**(1/3)*(x - 1/3)*log(a) + O((x - 1/3)**2, (x, 1/3))
>> >
>> > You can see that the leading relative error term from x being not
>> > quite equal to 1/3 is proportional to the log of the base. You should
>> > expect this difference to grow approximately linearly as you keep
>> > adding more zeros in the base.
>>
>> Marvelous.  Thank you.
[snip]
> Now consider appending three zeroes to the right-hand end of N (let's call
> it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).
>
> The *only *difference between RootN and RootNZZZ is that the decimal point
> in RootNZZZ is one place further to the right than the decimal point in
> RootN.
>
> None of the digits in RootNZZZ's string should be different from the
> corresponding digits in RootN.
>
> I rest my case.
[snip]


I believe the pivotal point of Oscar Benjamin's explanation is
that within the constraints of limited-precision binary floating-point
numbers, the exponent of 1/3 cannot be represented precisely, and
is in practice represented by something slightly smaller than 1/3;
and accordingly, when you multiply your argument by 1000, its
not-quit-cube-root gets multiplied by something slightly smaller
than 10, which is why the number of figures matching the "right"
answer gets steadily smaller.

Put slightly differently, the crux of the problem lies not in the
complicated process of exponentiation, but simply in the failure
to represent 1/3 exactly.  The fact that the exponent is slightly
less than 1/3 means that you would observe the steady loss of
agreement that you report, even if the exponentiation process
were perfect.

-- 
To email me, substitute nowhere->runbox, invalid->com.

From torriem at gmail.com  Fri Feb 17 10:38:58 2023
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 17 Feb 2023 08:38:58 -0700
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
Message-ID: <96e74bab-c8ee-621c-660e-07651b0081a8@gmail.com>

On 2/17/23 03:27, Stephen Tucker wrote:
> Thanks, one and all, for your reponses.
> 
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.

No matter how you do it, there are always tradeoffs and inaccuracies
moving from real numbers in base 10 to base 2.  That's just the nature
of the math.  Any binary floating point representation is going to have
problems.  There are techniques for mitigating this:
https://en.wikipedia.org/wiki/Floating-point_error_mitigation
It's interesting to note that the article points out that floating point
error was first talked about in the 1930s.  So no matter what binary
scheme you choose there will be error. That's just the nature of
converting a real from one base to another.

Also we weren't clear on this, but the IEEE standard is not just
implemented in software. It's the way your CPU represents floating point
numbers in silicon.  And in your GPUs (where speed is preferred to
precision).  So it's not like Python could just arbitrarily do something
different unless you were willing to pay a huge penalty for speed.  For
example the decimal module which is arbitrary precision, but quite slow.

Have you tried the numpy cbrt() function?  It is probably going to be
more accurate than using power to 0.3333.

> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.
Rest assured the IEEE committee that formalized the format decades ago
knew all about the limitations and trade-offs.  Over the years CPUs have
increased in capacity and now we can use 128-bit floating point numbers
which mitigate some of the accuracy problems by simply having more
binary digits. But the fact remains that some rational numbers in
decimal are irrational in binary, so arbitrary decimal precision using
floating point is not possible.

From Richard at Damon-Family.org  Fri Feb 17 13:42:18 2023
From: Richard at Damon-Family.org (Richard Damon)
Date: Fri, 17 Feb 2023 13:42:18 -0500
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
Message-ID: <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>

On 2/17/23 5:27 AM, Stephen Tucker wrote:
> Thanks, one and all, for your reponses.
>
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
>
> Consider an integer N consisting of a finitely-long string of digits in
> base 10.
>
> Consider the infinitely-precise cube root of N (yes I know that it could
> never be computed unless N is the cube of an integer, but this is a
> mathematical argument, not a computational one), also in base 10. Let's
> call it RootN.
>
> Now consider appending three zeroes to the right-hand end of N (let's call
> it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).

The key factor here is IEEE floating point is storing numbers in BINARY, 
not DECIMAL, so a multiply by 1000 will change the representation of the 
number, and thus the possible resolution errors.

Store you numbers in IEEE DECIMAL floating point, and the variations by 
multiplying by powers of 10 go away.

>
> The *only *difference between RootN and RootNZZZ is that the decimal point
> in RootNZZZ is one place further to the right than the decimal point in
> RootN.

No, since the floating point number is stored as a fraction times a 
power of 2, the fraction has changed as well as the power of 2.

>
> None of the digits in RootNZZZ's string should be different from the
> corresponding digits in RootN.

Only if the storage format was DECIMAL.

>
> I rest my case.
>
> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.

That is why they have developed the Decimal Floating point format, to 
handle people with those sorts of problems.

They just aren't common enough for many things to have adopted the use 
of it.

>
> Stephen Tucker.

-- 
Richard Damon


From hjp-python at hjp.at  Fri Feb 17 13:47:02 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 17 Feb 2023 19:47:02 +0100
Subject: Comparing caching strategies
In-Reply-To: <00bc01d9428d$b245c880$16d15980$@gmail.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
Message-ID: <20230217184702.git7iwc46c7nusug@hjp.at>

On 2023-02-17 00:07:12 -0500, avi.e.gross at gmail.com wrote:
> Roaring bitmaps claim to be an improvement not only over uncompressed
> structures but some other compressed versions but my reading shows it
> may be limited to some uses. Bitsets in general seem to be useful only
> for a largely contiguous set of integers where each sequential bit
> represents whether the nth integer above the lowest is in the set or
> not.

They don't really have to be that contiguous. As long as your integers
fit into 32 bits you're fine.

> Of course, properly set up, this makes Unions and Intersections
> and some other operations fairly efficient. But sets are not the same
> as dictionaries and often you are storing other data types than
> smaller integers.

Of course. Different data types are useful for different applications.

> Many normal compression techniques can require lots of time to
> uncompress to find anything. My impression is that Roaring Bitmaps is
> a tad like the pkzip software that tries various compression
> techniques on each file and chooses whatever seems to work better on
> each one. That takes extra time when zipping, but when unzipping a
> file, it goes directly to the method used to compress it as the type
> is in a header and just expands it one way.

While not completely wrong, I don't think this comparison is very
helpful.


> My impression is that Roaring bitmaps breaks up the range of integers
> into smaller chunks and depending on what is being stored in that
> chunk, may leave it as an uncompressed bitmap, or a list of the sparse
> contents, or other storage methods and can search each version fairly
> quickly. 

It's been a few years since I looked at the implementation, but that's
the gist of it.


> So, I have no doubt it works great for some applications such as
> treating social security numbers as integers.

Not sure what you mean here. You mean storing sets of social security
numbers as roaring bitmaps? You might be able to do that (Last time I
looked RB's were limited to 32 bits, which may not be enough to
represent SSNs unmodified), but are set operations on SSNs something you
routinely do?

> It likely would be overkill to store something like the components of
> an IP address between 0 and 255 inclusive.
> 
> But having said that, there may well be non-integer data that can be
> mapped into and out of integers. As an example, airports or radio
> stations have names like LAX or WPIX. If you limit yourself to ASCII
> letters then every one of them can be stored as a 32-bit integer,
> perhaps with some padding.

Again: What would be the application?

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230217/c0b46d76/attachment.sig>

From oscar.j.benjamin at gmail.com  Fri Feb 17 13:53:38 2023
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 17 Feb 2023 18:53:38 +0000
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
Message-ID: <CAHVvXxQAjXFHoeukmWGZd+qUP4ETh5tbMpDi9==BJ_tUv5Hn-A@mail.gmail.com>

On Fri, 17 Feb 2023 at 10:29, Stephen Tucker <stephen_tucker at sil.org> wrote:
>
> Thanks, one and all, for your reponses.
>
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
[snip]
>
> Perhaps this observation should be brought to the attention of the IEEE. I
> would like to know their response to it.

Their response would be that they are well aware of what you are
saying and knew about this already since before writing any standards.
The basic limitation of the IEEE standard in this respect is that it
describes individual operations rather than composite operations. Your
calculation involves composing operations, specifically:

    result = x ** (n / d)

The problem is that there is more than one operation so we have to
evaluate this in two steps:

    e = n / d
    result = x ** e

Now the problem is that although n / d is correctly rounded e has a
small error because the exact value of n / d cannot be represented. In
the second operation taking this slightly off value of e as the
intended input means that the correctly rounded result for x ** e is
not the closest float to the true value of the *compound* operation.
The exponentiation operator in particular is very sensitive to changes
in the exponent when the base is large so the tiny error in e leads to
a more noticeable relative error in x ** e.

The only way to prevent this in full generality is to to have a system
in which no intermediate inexact operations are computed eagerly which
means representing expressions symbolically in some way. That is what
the SymPy code I showed does:

In [6]: from sympy import cbrt

In [7]: e = cbrt(123456789000000000000000000000000000000000)

In [8]: print(e)
100000000000*123456789**(1/3)

In [9]: e.evalf(50)
Out[9]: 49793385921817.447440261250171604380899353243631762

Because the *entire* expression is represented here *exactly* as e it
is then possible to evaluate different parts of the expression
repeatedly with different levels of precision and it is necessary to
do that for full accuracy in this case. Here evalf will use more than
50 digits of precision internally so that at the end you have a result
specified to 50 digits but where the error for the entire expression
is smaller than the final digit. If you give it a more complicated
expression then it will use even more digits internally for deeper
parts of the expression tree because that is what is needed to get a
correctly rounded result for the expression as a whole.

This kind of symbolic evaluation is completely outside the scope of
what the IEEE floating point standards are for. Any system based on
fixed precision and eager evaluation will show the same problem that
you have identified. It is very useful though to have a system with
fixed precision and eager evaluation despite these limitations. The
context for which the IEEE standards are mainly intended (e.g. FPU
instructions) is one in which fixed precision and eager evaluation are
the only option.

--
Oscar

From dino at no.spam.ar  Fri Feb 17 14:09:15 2023
From: dino at no.spam.ar (Dino)
Date: Fri, 17 Feb 2023 14:09:15 -0500
Subject: LRU cache
In-Reply-To: <mailman.1840.1676601634.20444.python-list@python.org>
References: <9CTGL.3255637$miq3.78888@fx02.ams4>
 <a00f333f-b3cf-1122-4efa-62fd07ce6748@wichmann.us>
 <mailman.1831.1676481401.20444.python-list@python.org>
 <NPaHL.2601707$14z3.887943@fx11.ams4>
 <SJ0PR14MB5864935B923047D8EFF0FD54B9A19@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <mailman.1840.1676601634.20444.python-list@python.org>
Message-ID: <uhQHL.3488380$MJk2.2293686@fx06.ams4>


Thank you, Gerard. I really appreciate your help

Dino

On 2/16/2023 9:40 PM, Weatherby,Gerard wrote:
> I think this does the trick:
> 
> https://gist.github.com/Gerardwx/c60d200b4db8e7864cb3342dd19d41c9
> 
> 
> #!/usr/bin/env python3
> import collections
> import random
> from typing import Hashable, Any, Optional, Dict, Tuple
> 
> 
> class LruCache:
>      """Dictionary like storage of most recently inserted values"""
> 
>      def __init__(self, size: int = 1000):
>          """:param size number of cached entries"""
>          assert isinstance(size, int)
>          self.size = size
>          self.insert_counter = 0
>         self.oldest = 0
>          self._data : Dict[Hashable,Tuple[Any,int]]= {} # store values and age index
>          self._lru: Dict[int, Hashable] = {} # age counter dictionary
> 
>      def insert(self, key: Hashable, value: Any) -> None:
>          """Insert into dictionary"""
>          existing = self._data.get(key, None)
>          self._data[key] = (value, self.insert_counter)
>          self._lru[self.insert_counter] = key
>          if existing is not None:
>              self._lru.pop(existing[1], None)  # remove old counter value, if it exists
>          self.insert_counter += 1
>          if (sz := len(self._data)) > self.size:  # is cache full?
>              assert sz == self.size + 1
>              while (
>              key := self._lru.get(self.oldest, None)) is None:  # index may not be present, if value was reinserted
>                  self.oldest += 1
>              del self._data[key]  # remove oldest key / value from dictionary
>              del self._lru[self.oldest]
>              self.oldest += 1  # next oldest index
>          assert len(self._lru) == len(self._data)
> 
>      def get(self, key: Hashable) -> Optional[Any]:
>          """Get value or return None if not in cache"""
>          if (tpl := self._data.get(key, None)) is not None:
>              return tpl[0]
>          return None
> 
> 
> if __name__ == "__main__":
>      CACHE_SIZE = 1000
>      TEST_SIZE = 1_000_000
>      cache = LruCache(size=CACHE_SIZE)
> 
>      all = []
>      for i in range(TEST_SIZE):
>          all.append(random.randint(-5000, 5000))
> 
>      summary = collections.defaultdict(int)
>      for value in all:
>          cache.insert(value, value * value)
>          summary[value] += 1
>      smallest = TEST_SIZE
>      largest = -TEST_SIZE
>      total = 0
>      for value, count in summary.items():
>          smallest = min(smallest, count)
>          largest = max(largest, count)
>          total += count
>      avg = total / len(summary)
>      print(f"{len(summary)} values occurrences range from {smallest} to {largest}, average {avg:.1f}")
> 
>      recent = set()  # recent most recent entries
>      for i in range(len(all) - 1, -1, -1):  # loop backwards to get the most recent entries
>          value = all[i]
>          if len(recent) < CACHE_SIZE:
>              recent.add(value)
>          if value in recent:
>              if (r := cache.get(value)) != value * value:
>                  raise ValueError(f"Cache missing recent {value} {r}")
>          else:
>              if cache.get(value) != None:
>                  raise ValueError(f"Cache includes old {value}")
> 
> From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Dino <dino at no.spam.ar>
> Date: Wednesday, February 15, 2023 at 3:07 PM
> To: python-list at python.org <python-list at python.org>
> Subject: Re: LRU cache
> *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***
> 
> Thank you Mats, Avi and Chris
> 
> btw, functools.lru_cache seems rather different from what I need, but
> maybe I am missing something. I'll look closer.
> 
> On 2/14/2023 7:36 PM, Mats Wichmann wrote:
>> On 2/14/23 15:07, Dino wrote:
>>>
> 
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jb3Gr2BFAPLJ2YuI5rFdJUtalqWcijhxHAfdmCI3afnLFDdcekALxDYAQwpE1L_JlJBBJ-BB3BuLdoSE$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jb3Gr2BFAPLJ2YuI5rFdJUtalqWcijhxHAfdmCI3afnLFDdcekALxDYAQwpE1L_JlJBBJ-BB3BuLdoSE$>


From hjp-python at hjp.at  Fri Feb 17 14:38:44 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 17 Feb 2023 20:38:44 +0100
Subject: Precision Tail-off?
In-Reply-To: <96e74bab-c8ee-621c-660e-07651b0081a8@gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <96e74bab-c8ee-621c-660e-07651b0081a8@gmail.com>
Message-ID: <20230217193844.6chboclcwcnpkwtn@hjp.at>

On 2023-02-17 08:38:58 -0700, Michael Torrie wrote:
> On 2/17/23 03:27, Stephen Tucker wrote:
> > Thanks, one and all, for your reponses.
> > 
> > This is a hugely controversial claim, I know, but I would consider this
> > behaviour to be a serious deficiency in the IEEE standard.
> 
> No matter how you do it, there are always tradeoffs and inaccuracies
> moving from real numbers in base 10 to base 2.

This is phrased ambiguosly. So just to clarify:

Real numbers are not in base 10. Or base 2 or base 37 or base e. A
positional system (which uses a base) is just a convenient way to write
a small subset of real numbers. By using any base you limit yourself to
rational numbers (no e or ? or ?2) and in fact only those rational
numbers where the denominator is a power of the base.

Converting numbers from one base to another with any finite precision
will generally involve rounding - so do that as little as possible.


> That's just the nature of the math.  Any binary floating point
> representation is going to have problems.

Any decimal floating point representation is also going to have
problems.

There is nothing magical about base 10. It's just what we are used to
(which also means that we are used to the rounding errors and aren't
surprised by them as much).

> Also we weren't clear on this, but the IEEE standard is not just
> implemented in software. It's the way your CPU represents floating point
> numbers in silicon.  And in your GPUs (where speed is preferred to
> precision).  So it's not like Python could just arbitrarily do something
> different unless you were willing to pay a huge penalty for speed.

I'm pretty sure that compared to the interpreter overhead of CPython the
overhead of a software FP implementation (whether binary or decimal)
would be rather small, maybe negligible.


> > Perhaps this observation should be brought to the attention of the IEEE. I
> > would like to know their response to it.
> Rest assured the IEEE committee that formalized the format decades ago
> knew all about the limitations and trade-offs.  Over the years CPUs have
> increased in capacity and now we can use 128-bit floating point numbers

The very first IEEE compliant processor (the Intel 8087) had an 80 bit
extended type (in fact it did all computations in 80 bit and only
rounded down to 64 or 32 bits when storing the result). By the 1990s, 96
and 128 bit was quite common.

> which mitigate some of the accuracy problems by simply having more
> binary digits. But the fact remains that some rational numbers in
> decimal are irrational in binary,

Be careful: "Rational" and "irrational" have a standard meaning in
mathematics and it's independent of base.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230217/077e8b26/attachment.sig>

From hjp-python at hjp.at  Fri Feb 17 14:50:03 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 17 Feb 2023 20:50:03 +0100
Subject: Precision Tail-off?
In-Reply-To: <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
Message-ID: <20230217195003.fooxpbedbfk2cqe7@hjp.at>

On 2023-02-17 10:27:08 +0000, Stephen Tucker wrote:
> This is a hugely controversial claim, I know, but I would consider this
> behaviour to be a serious deficiency in the IEEE standard.
> 
> Consider an integer N consisting of a finitely-long string of digits in
> base 10.
> 
> Consider the infinitely-precise cube root of N (yes I know that it could
> never be computed

However, computers exist to compute. Something which can never be
computed is outside of the realm of computing.

> unless N is the cube of an integer, but this is a mathematical
> argument, not a computational one), also in base 10. Let's call it
> RootN.
> 
> Now consider appending three zeroes to the right-hand end of N (let's call
> it NZZZ) and NZZZ's infinitely-precise cube root (RootNZZZ).
> 
> The *only *difference between RootN and RootNZZZ is that the decimal point
> in RootNZZZ is one place further to the right than the decimal point in
> RootN.

No. In mathematics there is no such thing as a decimal point. The only
difference is that RootNZZZ is RootN*10. But there is nothing special
about 10. You could multiply your original number by 512 and then the
new cube root would differ by a factor of 8 (which would show up as
shifted "binary point"[1] in binary but completely different digits in
decimal) or you could multiply by 1728 and then you would need base 12
to get the same digits with a shifted "duodecimal point".

        hp

[1] It's really unfortunate that the point which separates the integer
    and the fractional part of a number is called a "decimal point" in
    English. Makes it hard to talk about non-integer numbers in other
    bases.

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230217/7ccb17f8/attachment.sig>

From hjp-python at hjp.at  Fri Feb 17 14:58:31 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 17 Feb 2023 20:58:31 +0100
Subject: Precision Tail-off?
In-Reply-To: <SJ0PR14MB58647B930C505B3192C5CE5BB9A19@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <25b8f4da-880f-3e76-2d96-1d424a5ec7cd@tompassin.net>
 <SJ0PR14MB58647B930C505B3192C5CE5BB9A19@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <20230217195831.fwpq22444okfbps6@hjp.at>

On 2023-02-17 14:39:42 +0000, Weatherby,Gerard wrote:
> IEEE did not define a standard for floating point arithmetics. They
> designed multiple standards, including a decimal float point one.
> Although decimal floating point (DFP) hardware used to be
> manufactured, I couldn?t find any current manufacturers.

Doesn't IBM any more? Their POWER processors used to implement decimal
FP (starting with POWER8, if I remember correctly).

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230217/32e71092/attachment.sig>

From grant.b.edwards at gmail.com  Fri Feb 17 15:36:31 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 17 Feb 2023 12:36:31 -0800 (PST)
Subject: Precision Tail-off?
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
Message-ID: <63efe54f.050a0220.4d78.3abb@mx.google.com>

On 2023-02-17, Richard Damon <Richard at Damon-Family.org> wrote:
> [...]
>
>> Perhaps this observation should be brought to the attention of the IEEE. I
>> would like to know their response to it.
>
> That is why they have developed the Decimal Floating point format, to 
> handle people with those sorts of problems.
>
> They just aren't common enough for many things to have adopted the
> use of it.

Back before hardware floating point was common, support for deciaml
floating point was very common.  All of the popular C, Pascal, and
BASIC compilers (for microcomputers) I remember let you choose (at
compile time) whether you wanted to use binary floating point or
decimal (BCD) floating point. People doing scientific stuff usually
chose binary because it was a little faster and you got more
resolution for the same amount of stoage. If you were doing
accounting, you chose BCD (or used fixed-point).

Once hardware (binary) floating point became common, support for
software BCD floating point just sort of went away...

--
Grant





From mats at wichmann.us  Fri Feb 17 15:57:06 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Fri, 17 Feb 2023 13:57:06 -0700
Subject: Precision Tail-off?
In-Reply-To: <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
Message-ID: <155c30ab-3410-c1ba-2288-2021f135007e@wichmann.us>

On 2/17/23 11:42, Richard Damon wrote:
> On 2/17/23 5:27 AM, Stephen Tucker wrote:

> The key factor here is IEEE floating point is storing numbers in BINARY, 
> not DECIMAL, so a multiply by 1000 will change the representation of the 
> number, and thus the possible resolution errors.
> 
> Store you numbers in IEEE DECIMAL floating point, and the variations by 
> multiplying by powers of 10 go away.

The development of the original IEEE standard led eventually to 
consistent implementation in hardware (when they implement floating 
point at all, which embedded/IoT class chips in particular often don't) 
that aligned with how languages/compilers treated floating point, so 
that's been a really successful standard, whatever one might feel about 
the tradeoffs. Standards are all about finding a mutually acceptable way 
forward, once people admit there is no One Perfect Answer.

Newer editions of 754 (since 2008) have added this decimal floating 
point representation, which is supported by some software such as IBM 
and Intel floating-point libraries.  Hardware support has been slower to 
arrive.  The only ones I've heard of have been the IBM z series 
(mainframes) and somebody else mentioned Power though I'd never seen 
that. It's possible some of the GPU lines may be going this direction.

As far as Python goes... the decimal module has this comment:

 > It is a complete implementation of Mike Cowlishaw/IBM's General 
Decimal Arithmetic Specification.

Cowlishaw was the editor of the 2008 and 2019 editions of IEEE 754, fwiw.

And... this topic as a whole comes up over and over again, like 
everywhere.  See Stack Overflow for some amusement.

From grant.b.edwards at gmail.com  Fri Feb 17 17:03:09 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 17 Feb 2023 14:03:09 -0800 (PST)
Subject: Precision Tail-off?
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
 <155c30ab-3410-c1ba-2288-2021f135007e@wichmann.us>
Message-ID: <63eff99d.020a0220.bec00.1d43@mx.google.com>

On 2023-02-17, Mats Wichmann <mats at wichmann.us> wrote:

> And... this topic as a whole comes up over and over again, like
> everywhere.

That's an understatement.

I remember it getting rehashed over and over again in various USENET
groups 35 years ago when when the VAX 11/780 BSD machine on which I
read news exchanged postings with peers using a half-dozen dial-up
modems and UUCP.

One would have thought it would be a time-saver when David Goldberg
wrote "the paper" in 1991, and you could tell people to go away and
read this:

  https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html
  https://www.itu.dk/~sestoft/bachelor/IEEE754_article.pdf

It didn't help.

Every fall, the groups were again full of a new crop of people who had
just discovered all sorts of bugs in the way <software/hardware>
implemented floating point, and pointing them to a nicely written
document that explained it never did any good.

--
Grant



From avi.e.gross at gmail.com  Fri Feb 17 18:08:09 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Fri, 17 Feb 2023 18:08:09 -0500
Subject: Comparing caching strategies
In-Reply-To: <20230217184702.git7iwc46c7nusug@hjp.at>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
Message-ID: <001c01d94324$b411cf50$1c356df0$@gmail.com>

Peter,

Analogies I am sharing are mainly for me to wrap my head around an idea by
seeing if it matches any existing ideas or templates and is not meant to be
exact. Fair enough?

But in this case, from my reading, the analogy is rather reasonable. The
implementation of Roaring Bitmaps seems to logically break up the space of
all possible values it looks up into multiple "zones" that are somewhat
analogous to individual files, albeit probably in memory as the program
runs. In the pkzip analogy, each file is processed and stored independently
alongside a header that provides enough detail to uniquely open up the
contents when needed. The collection of such compressed files is then one
bigger file that is saved that uses up less space. Roaring bitmaps seems to
determine how best to store each zone and only processes that zone when
requested, hence some of the speedup as each zone is stored in a manner that
generally allows fast access.

I did not raise the issue and thus have no interest in promoting this
technology nor knocking it down. Just wondering what it was under the hood
and whether I might even have a need for it. I am not saying Social Security
numbers are a fit, simply that some form of ID number might fit. If a
company has a unique ID number for each client and uses it consistently,
then an implementation that holds a set stored this way of people using
product A, such as house insurance, and those using product B, such as car
insurance, and perhaps product C is an umbrella policy, might easily handle
some queries such as who uses two or all three (intersections of sets) or
who might merit a letter telling them how much they can save if they
subscribed to two or all three as a way to get more business. Again, just  a
made-up example I can think about. A company which has a million customers
over the years will have fairly large sets as described. 

What is helpful to me in thinking about something will naturally often not
be helpful to you or others but nothing you wrote makes me feel my first
take was in any serious way wrong. It still makes sense to me.

And FYI, the largest integer in signed 32 bits is 2_147_483_647 which is 10
digits. A Social Security number look like xxx-xx-xxxx at this time which is
only 9 digits.  Not that it matters, but it seems it still qualifies to be
used as I describe, as long as Roaring bitmaps allows it, minus any spaces
or hyphens and converted to an integer.

As for my other EXAMPLE, I fail to see why I need to provide a specific need
for an application. I don't care what they need it for. The thought was
about whether something that does not start as an integer can be uniquely
mapped into and out of integers of size 32 bits. So I considered a few
examples of short textual items such as three letter airport abbreviations.
But if you cannot imagine an application, consider one similar enough to the
above.  I think there are currently over 47,000 such airports  in the world
and apparently about 20,000 in the US. That seems to exceed the possible
combinations of 26 letters (cubed) so it seems there are 4-letter codes too
such as ZSPD. It should still fit into 4 bytes, for now.

So assume you have a variety of facts such as which airports have
handicapped accessible bathrooms, or have an extra long/strong runway that
can handle huge planes or anything else that is worth knowing. You might
have bitmaps (as is being discussed) that may be sparse for some such info
and fairly dense for other info like having jet fuel available. As above,
finding an airport that has various mixtures may be doable with these sets
and perhaps faster than say queries on a relational database storing the
same info.

I will end by saying this is a hypothetical discussion for me. I am not
doing any projects now where I expect to use Roaring bitmaps but am now
aware of them should any need or opportunity arise. My mind is very full
with such trivia and very little is needed albeit I never know what may come
in as useful. 

Respectfully,

Avi

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Peter J. Holzer
Sent: Friday, February 17, 2023 1:47 PM
To: python-list at python.org
Subject: Re: Comparing caching strategies

On 2023-02-17 00:07:12 -0500, avi.e.gross at gmail.com wrote:
> Roaring bitmaps claim to be an improvement not only over uncompressed 
> structures but some other compressed versions but my reading shows it 
> may be limited to some uses. Bitsets in general seem to be useful only 
> for a largely contiguous set of integers where each sequential bit 
> represents whether the nth integer above the lowest is in the set or 
> not.

They don't really have to be that contiguous. As long as your integers fit
into 32 bits you're fine.

> Of course, properly set up, this makes Unions and Intersections and 
> some other operations fairly efficient. But sets are not the same as 
> dictionaries and often you are storing other data types than smaller 
> integers.

Of course. Different data types are useful for different applications.

> Many normal compression techniques can require lots of time to 
> uncompress to find anything. My impression is that Roaring Bitmaps is 
> a tad like the pkzip software that tries various compression 
> techniques on each file and chooses whatever seems to work better on 
> each one. That takes extra time when zipping, but when unzipping a 
> file, it goes directly to the method used to compress it as the type 
> is in a header and just expands it one way.



> My impression is that Roaring bitmaps breaks up the range of integers 
> into smaller chunks and depending on what is being stored in that 
> chunk, may leave it as an uncompressed bitmap, or a list of the sparse 
> contents, or other storage methods and can search each version fairly 
> quickly.

It's been a few years since I looked at the implementation, but that's the
gist of it.


> So, I have no doubt it works great for some applications such as 
> treating social security numbers as integers.

Not sure what you mean here. You mean storing sets of social security
numbers as roaring bitmaps? You might be able to do that (Last time I looked
RB's were limited to 32 bits, which may not be enough to represent SSNs
unmodified), but are set operations on SSNs something you routinely do?

> It likely would be overkill to store something like the components of 
> an IP address between 0 and 255 inclusive.
> 
> But having said that, there may well be non-integer data that can be 
> mapped into and out of integers. As an example, airports or radio 
> stations have names like LAX or WPIX. If you limit yourself to ASCII 
> letters then every one of them can be stored as a 32-bit integer, 
> perhaps with some padding.

Again: What would be the application?

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"


From greg.ewing at canterbury.ac.nz  Fri Feb 17 20:37:44 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 18 Feb 2023 14:37:44 +1300
Subject: Precision Tail-off?
In-Reply-To: <mailman.1851.1676659340.20444.python-list@python.org>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
 <mailman.1851.1676659340.20444.python-list@python.org>
Message-ID: <k5aof8Fhs3qU1@mid.individual.net>

On 18/02/23 7:42 am, Richard Damon wrote:
> On 2/17/23 5:27 AM, Stephen Tucker wrote:
>> None of the digits in RootNZZZ's string should be different from the
>> corresponding digits in RootN.
> 
> Only if the storage format was DECIMAL.

Note that using decimal wouldn't eliminate this particular problem,
since 1/3 isn't exactly representable in decimal either.

To avoid it you would need to use an algorithm that computes nth
roots directly rather than raising to the power 1/n.

-- 
Greg

From rosuav at gmail.com  Fri Feb 17 20:44:34 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 18 Feb 2023 12:44:34 +1100
Subject: Precision Tail-off?
In-Reply-To: <k5aof8Fhs3qU1@mid.individual.net>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
 <mailman.1851.1676659340.20444.python-list@python.org>
 <k5aof8Fhs3qU1@mid.individual.net>
Message-ID: <CAPTjJmpEPzFx0kRo3t3ym-pMWvFwUnXks+5JYqr=vygQrsSKgQ@mail.gmail.com>

On Sat, 18 Feb 2023 at 12:41, Greg Ewing via Python-list
<python-list at python.org> wrote:
>
> On 18/02/23 7:42 am, Richard Damon wrote:
> > On 2/17/23 5:27 AM, Stephen Tucker wrote:
> >> None of the digits in RootNZZZ's string should be different from the
> >> corresponding digits in RootN.
> >
> > Only if the storage format was DECIMAL.
>
> Note that using decimal wouldn't eliminate this particular problem,
> since 1/3 isn't exactly representable in decimal either.
>
> To avoid it you would need to use an algorithm that computes nth
> roots directly rather than raising to the power 1/n.
>

It's somewhat curious that we don't really have that. We have many
other inverse operations - addition and subtraction (not just "negate
and add"), multiplication and division, log and exp - but we have
exponentiation without an arbitrary-root operation. For square roots,
that's not a problem, since we can precisely express the concept
"raise to the 0.5th power", but for anything else, we have to raise to
a fractional power that might be imprecise.

But maybe, in practice, this isn't even a problem?

ChrisA

From uri at speedy.net  Fri Feb 17 21:45:39 2023
From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=)
Date: Sat, 18 Feb 2023 04:45:39 +0200
Subject: Python 3.11.2, 3.10.10
In-Reply-To: <820CB8BD-49D6-469B-8DC0-0D7B2B5C984D@langa.pl>
References: <820CB8BD-49D6-469B-8DC0-0D7B2B5C984D@langa.pl>
Message-ID: <CABD5YeFFYBHau2Y2PcF0MCDS-78W7_bpqMNEqQWNRudbEh3k8Q@mail.gmail.com>

Hi,

I was surprised that Python 3.11.2 and 3.10.10 have been released without a
notice to this mailing list. What happened?

Thanks,
Uri.
????
uri at speedy.net


On Wed, Dec 7, 2022 at 1:03 AM ?ukasz Langa <lukasz at langa.pl> wrote:

> Greetings! We bring you a slew of releases this fine Saint Nicholas /
> Sinterklaas day. Six simultaneous releases has got to be some record.
> There?s one more record we broke this time, you?ll see below.
>
> In any case, updating is recommended due to security content:
>
> 3.7 - 3.12: gh-98739 <https://github.com/python/cpython/issues/98739>:
> Updated bundled libexpat to 2.5.0 to fix CVE-2022-43680 <
> https://nvd.nist.gov/vuln/detail/CVE-2022-43680> (heap use-after-free).
> 3.7 - 3.12: gh-98433 <https://github.com/python/cpython/issues/98433>:
> The IDNA codec decoder used on DNS hostnames by socket or asyncio related
> name resolution functions no longer involves a quadratic algorithm to fix
> CVE-2022-45061 <https://nvd.nist.gov/vuln/detail/CVE-2022-45061>. This
> prevents a potential CPU denial of service if an out-of-spec excessive
> length hostname involving bidirectional characters were decoded. Some
> protocols such as urllib http 3xx redirects potentially allow for an
> attacker to supply such a name.
> 3.7 - 3.12: gh-100001 <https://github.com/python/cpython/issues/100001>:
> python -m http.server no longer allows terminal control characters sent
> within a garbage request to be printed to the stderr server log.
> 3.8 - 3.12: gh-87604 <https://github.com/python/cpython/issues/87604>:
> Avoid publishing list of active per-interpreter audit hooks via the gc
> module.
> 3.9 - 3.10 (already released in 3.11+ before): gh-97514 <
> https://github.com/python/cpython/issues/97514>: On Linux the
> multiprocessing module returns to using filesystem backed unix domain
> sockets for communication with the forkserver process instead of the Linux
> abstract socket namespace. Only code that chooses to use the ?forkserver?
> start method is affected. This prevents Linux CVE-2022-42919 <
> https://nvd.nist.gov/vuln/detail/CVE-2022-42919> (potential privilege
> escalation) as abstract sockets have no permissions and could allow any
> user on the system in the same network namespace (often the whole system)
> to inject code into the multiprocessing forkserver process. This was a
> potential privilege escalation. Filesystem based socket permissions
> restrict this to the forkserver process user as was the default in Python
> 3.8 and earlier.
> 3.7 - 3.10: gh-98517 <https://github.com/python/cpython/issues/98517>:
> Port XKCP?s fix for the buffer overflows in SHA-3 to fix CVE-2022-37454 <
> https://nvd.nist.gov/vuln/detail/CVE-2022-37454>.
> 3.7 - 3.9 (already released in 3.10+ before): gh-68966 <
> https://github.com/python/cpython/issues/68966>: The deprecated mailcap
> module now refuses to inject unsafe text (filenames, MIME types,
> parameters) into shell commands to address CVE-2015-20107 <
> https://nvd.nist.gov/vuln/detail/CVE-2015-20107>. Instead of using such
> text, it will warn and act as if a match was not found (or for test
> commands, as if the test failed).
>  <
> https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3120-alpha-3-1>Python
> 3.12.0 alpha 3
>
> Get it here, read the change log, sing a GPT-3-generated Sinterklaas song:
>
> https://www.python.org/downloads/release/python-3120a3/ <
> https://www.python.org/downloads/release/python-3120a3/>
>
> 216 new commits since 3.12.0 alpha 2 last month.
>
>  <
> https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3111-2>Python
> 3.11.1
>
> Get it here, see the change log, read the recipe for quark soup:
>
> https://www.python.org/downloads/release/python-3111/ <
> https://www.python.org/downloads/release/python-3111/>
>
> A whopping 495 new commits since 3.11.0. This is a massive increase of
> changes comparing to 3.10 at the same stage in the release cycle: there
> were ?only? 339 commits between 3.10.0 and 3.10.1.
>
>  <
> https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3109-3>Python
> 3.10.9
>
> Get it here, read the change log, see circular patterns:
>
> https://www.python.org/downloads/release/python-3109/ <
> https://www.python.org/downloads/release/python-3109/>
>
> 165 new commits.
>
>  <
> https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3916-4>Python
> 3.9.16
>
> Get it here, read the change log, consider upgrading to a newer version:
>
> https://www.python.org/downloads/release/python-3916/ <
> https://www.python.org/downloads/release/python-3916/>
>
> Security-only release with no binaries. 10 commits.
>
>  <
> https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3816-5>Python
> 3.8.16
>
> Get it here, see the change log, definitely upgrade to a newer version:
>
> https://www.python.org/downloads/release/python-3816/ <
> https://www.python.org/downloads/release/python-3816/>
>
> Security-only release with no binaries. 9 commits.
>
>  <
> https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3716-6>Python
> 3.7.16
>
> Get it here, read the change log, check PEP 537  <
> https://peps.python.org/pep-0537/>to confirm EOL is coming to this
> version in June 2023:
>
> https://www.python.org/downloads/release/python-3716/ <
> https://www.python.org/downloads/release/python-3716/>
>
> Security-only release with no binaries. 8 commits.
>
>  <
> https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#we-hope-you-enjoy-the-new-releases-7>We
> hope you enjoy the new releases!
>
> Thanks to all of the many volunteers who help make Python Development and
> these releases possible! Please consider supporting our efforts by
> volunteering yourself or through organization contributions to the Python
> Software Foundation.
>
> https://www.python.org/psf/  <https://www.python.org/psf/>
> Your friendly release team,
>
> Ned Deily @nad <https://discuss.python.org/u/nad>
> Steve Dower @steve.dower <https://discuss.python.org/u/steve.dower>
> Pablo Galindo Salgado @pablogsal <https://discuss.python.org/u/pablogsal>
> ?ukasz Langa @ambv <https://discuss.python.org/u/ambv>
> Thomas Wouters @thomas <https://discuss.python.org/u/thomas>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From torriem at gmail.com  Fri Feb 17 21:54:29 2023
From: torriem at gmail.com (Michael Torrie)
Date: Fri, 17 Feb 2023 19:54:29 -0700
Subject: Precision Tail-off?
In-Reply-To: <63eff99d.020a0220.bec00.1d43@mx.google.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
 <155c30ab-3410-c1ba-2288-2021f135007e@wichmann.us>
 <63eff99d.020a0220.bec00.1d43@mx.google.com>
Message-ID: <74da1d78-5fbb-9c4d-50c9-a297870169d1@gmail.com>

On 2/17/23 15:03, Grant Edwards wrote:
> Every fall, the groups were again full of a new crop of people who had
> just discovered all sorts of bugs in the way <software/hardware>
> implemented floating point, and pointing them to a nicely written
> document that explained it never did any good.

But to be fair, Goldberg's article is pretty obtuse and formal for most
people, even programmers.  I don't need lots of formal proofs as he
shows.  Just a summary is sufficient I'd think.  Although I've been
programming for many years, I have no idea what he means with most of
the notation in that paper.

Although I have a vague notion of what's going on, as my last post
shows, I don't know any of the right terminology.


From oscar.j.benjamin at gmail.com  Fri Feb 17 22:52:51 2023
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 18 Feb 2023 03:52:51 +0000
Subject: Precision Tail-off?
In-Reply-To: <CAPTjJmpEPzFx0kRo3t3ym-pMWvFwUnXks+5JYqr=vygQrsSKgQ@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
 <mailman.1851.1676659340.20444.python-list@python.org>
 <k5aof8Fhs3qU1@mid.individual.net>
 <CAPTjJmpEPzFx0kRo3t3ym-pMWvFwUnXks+5JYqr=vygQrsSKgQ@mail.gmail.com>
Message-ID: <CAHVvXxSdr7Akn9Ofin=ia=ui67QdA=aAnVKKX=pW-fnGYYhpKQ@mail.gmail.com>

On Sat, 18 Feb 2023 at 01:47, Chris Angelico <rosuav at gmail.com> wrote:
>
> On Sat, 18 Feb 2023 at 12:41, Greg Ewing via Python-list
> <python-list at python.org> wrote:
> >
> > On 18/02/23 7:42 am, Richard Damon wrote:
> > > On 2/17/23 5:27 AM, Stephen Tucker wrote:
> > >> None of the digits in RootNZZZ's string should be different from the
> > >> corresponding digits in RootN.
> > >
> > > Only if the storage format was DECIMAL.
> >
> > Note that using decimal wouldn't eliminate this particular problem,
> > since 1/3 isn't exactly representable in decimal either.
> >
> > To avoid it you would need to use an algorithm that computes nth
> > roots directly rather than raising to the power 1/n.
> >
>
> It's somewhat curious that we don't really have that. We have many
> other inverse operations - addition and subtraction (not just "negate
> and add"), multiplication and division, log and exp - but we have
> exponentiation without an arbitrary-root operation. For square roots,
> that's not a problem, since we can precisely express the concept
> "raise to the 0.5th power", but for anything else, we have to raise to
> a fractional power that might be imprecise.

Various libraries can do this. Both SymPy and NumPy have cbrt for cube roots:

  >>> np.cbrt(123456789000000000000000000000000000000000000000000000000000000.)
  4.979338592181745e+20

SymPy can also evaluate any rational power either exactly or to any
desired accuracy. Under the hood SymPy uses mpmath for the approximate
numerical evaluation part of this and mpmath can also be used directly
with its cbrt and nthroot functions to do this working with any
desired precision.

> But maybe, in practice, this isn't even a problem?

I'd say it's a small problem. Few people would use such a feature but
it would be have a little usefulness for those people if it existed.
Libraries like mpmath and SymPy provide this and can offer a big step
up for those who are really concerned about exactness or accuracy
though so there are already options for those who care. These are a
lot slower though than working with plain old floats but on the other
hand offer vastly more than a math.cbrt function could offer to
someone who needs something more accurate than x**(1/3).

For those who are working with floats the compromise is clear: errors
can accumulate in calculations. Taking the OPs example to the extreme,
the largest result that does not overflow is:

  >>> (123456789. * 10**300) ** (1.0 / 3.0)
  4.979338592181679e+102

Only the last 3 digits are incorrect so the error is still small. It
is not hard to find other calculations where *all* the digits are
wrong though:

  >>> math.cos(3)**2 + math.sin(3)**2 - 1
  -1.1102230246251565e-16

So if you want to use floats then you need to learn to deal with this
as appropriate for your use case. IEEE standards do their best to make
results reproducible across machines as well as limiting avoidable
local errors so that global errors in larger operations are *less
likely* to dominate the result. Their guarantees are only local though
so as soon as you have more complicated calculations you need your own
error analysis somehow. IEEE guarantees are in that case also useful
for those who actually want to do a formal error analysis.

--
Oscar

From sjeik_appie at hotmail.com  Sat Feb 18 05:38:56 2023
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sat, 18 Feb 2023 11:38:56 +0100
Subject: LRU cache
In-Reply-To: <SJ0PR14MB5864935B923047D8EFF0FD54B9A19@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <DB6PR01MB38955805D4B3AFB4FCBB953783A69@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   I sometimes use this trick, which I learnt from a book by Martelli.
   Instead of try/except, membership testing with "in" (__contains__) might
   be faster. Probably "depends". Matter of measuring.
   def somefunc(arg, _cache={}):
   ? ? if len(_cache) > 10 ** 5:
   ? ? ? ? _cache.pop()
   ? ? try:
   ? ? ? ? return _cache[arg]
   ? ? except KeyError:
   ? ? ? ? result = expensivefunc(arg)
   ? ? ? ? _cache[arg] = result
   ? ? ? ? return result
   Albert-Jan

From hjp-python at hjp.at  Sat Feb 18 05:40:04 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 18 Feb 2023 11:40:04 +0100
Subject: Comparing caching strategies
In-Reply-To: <001c01d94324$b411cf50$1c356df0$@gmail.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
Message-ID: <20230218104004.gc7dcrsxxv235icw@hjp.at>

On 2023-02-17 18:08:09 -0500, avi.e.gross at gmail.com wrote:
> Analogies I am sharing are mainly for me to wrap my head around an idea by
> seeing if it matches any existing ideas or templates and is not meant to be
> exact. Fair enough?

Yeah. But if you are venting your musings into a public space you
shouldn't be surprised if people react to them. And we can only react to
what you write, not what you think.

> But in this case, from my reading, the analogy is rather reasonable.

Although that confused me a bit. You are clearly responding to something
I thought about but which you didn't quote below. Did I just think about
it and not write it, but you responded anyway because you're a mind
reader? Nope, it just turns out you (accidentally) deleted that sentence.


> The implementation of Roaring Bitmaps seems to logically break up the
> space of all possible values it looks up into multiple "zones" that
> are somewhat analogous to individual files,

That part is correct. But you presented that in the form of a
performance/space tradeoff, writing about "trying multiple methods" to
find the smallest, and that that makes compression slower. That may be
the case for pkzip, but it's not what RB does: Instead it uses a very
simple heuristic: If there are less than 25% of the bits set in a zone,
it uses a list of offsets, otherwise a plain bitmap. (according to
their 2016 paper which I just skimmed through again - maybe the
implementation is a bit more complex now). So I think your description
would lead the reader to anticipate problems which aren't there and
probably miss ones which are there. So I'll stay with my "not completely
wrong but not very helpful" assessment.


> I did not raise the issue and thus have no interest in promoting this
> technology nor knocking it down. Just wondering what it was under the hood
> and whether I might even have a need for it. I am not saying Social Security
> numbers are a fit, simply that some form of ID number might fit.

Yeah, that's the point: Any form of ID which is a small-ish integer
number fits.

And maybe it's just because I work with databases a lot, but
representing things with numeric ids (in relational databases we call
them "surrogate keys") is such a basic operation that it doesn't warrant
more than a sentence or two.


> If a company has a unique ID number for each client and uses it
> consistently, then an implementation that holds a set stored this way
> of people using product A, such as house insurance, and those using
> product B, such as car insurance, and perhaps product C is an umbrella
> policy, might easily handle some queries such as who uses two or all
> three (intersections of sets) or who might merit a letter telling them
> how much they can save if they subscribed to two or all three as a way
> to get more business. Again, just  a made-up example I can think
> about. A company which has a million customers over the years will
> have fairly large sets as described. 

A much better example. This is indeed how you would use roaring bitmaps.


> What is helpful to me in thinking about something will naturally often not
> be helpful to you or others but nothing you wrote makes me feel my first
> take was in any serious way wrong. It still makes sense to me.
> 
> And FYI, the largest integer in signed 32 bits is 2_147_483_647

I know. It's been some time since I could do hexadecimal arithmetic
in my head but the the limits of common data types are still burned into
my brain ;-).

> which is 10 digits. A Social Security number look like xxx-xx-xxxx at
> this time which is only 9 digits.

These are US social security numbers. Other countries have other
schemes. For example, Austrian SSNs have 10 digits, so you would need 34
bits to represent them exactly. However, they (obviously) contain some
redundancy (one of the digits is a checksum, and there aren't 999999
days in a century) so it could algorithmically be compressed down to 26
bits. But you probably wouldn't do that because in almost any real
application you wouldn't use the SSN as a primary key (some people don't
have one, and there have been mixups resulting in two people getting the
same SSN).


> As for my other EXAMPLE, I fail to see why I need to provide a specific need
> for an application. I don't care what they need it for. The thought was
> about whether something that does not start as an integer can be uniquely
> mapped into and out of integers of size 32 bits.

That's what confused me. You seemed to concentrate on the "map things to
integers" part which has been solved for decades and is absolutely
incidental to roaring bitmaps and completely ignored what you would be
using them for.

So I thought I was missing something, but it seems I wasn't.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230218/f69c50d3/attachment.sig>

From hjp-python at hjp.at  Sat Feb 18 06:16:01 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 18 Feb 2023 12:16:01 +0100
Subject: Precision Tail-off?
In-Reply-To: <CAHVvXxSdr7Akn9Ofin=ia=ui67QdA=aAnVKKX=pW-fnGYYhpKQ@mail.gmail.com>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
 <mailman.1851.1676659340.20444.python-list@python.org>
 <k5aof8Fhs3qU1@mid.individual.net>
 <CAPTjJmpEPzFx0kRo3t3ym-pMWvFwUnXks+5JYqr=vygQrsSKgQ@mail.gmail.com>
 <CAHVvXxSdr7Akn9Ofin=ia=ui67QdA=aAnVKKX=pW-fnGYYhpKQ@mail.gmail.com>
Message-ID: <20230218111601.vtlqaq5kpll3en2j@hjp.at>

On 2023-02-18 03:52:51 +0000, Oscar Benjamin wrote:
> On Sat, 18 Feb 2023 at 01:47, Chris Angelico <rosuav at gmail.com> wrote:
> > On Sat, 18 Feb 2023 at 12:41, Greg Ewing via Python-list
> > > To avoid it you would need to use an algorithm that computes nth
> > > roots directly rather than raising to the power 1/n.
> > >
> >
> > It's somewhat curious that we don't really have that. We have many
> > other inverse operations - addition and subtraction (not just "negate
> > and add"), multiplication and division, log and exp - but we have
> > exponentiation without an arbitrary-root operation. For square roots,
> > that's not a problem, since we can precisely express the concept
> > "raise to the 0.5th power", but for anything else, we have to raise to
> > a fractional power that might be imprecise.
> 
> Various libraries can do this. Both SymPy and NumPy have cbrt for cube roots:

Yes, but that's a special case. Chris was talking about arbitrary
(integer) roots. My calculator has a button labelled [x?y], but my
processor doesn't have an equivalent operation. Come to think of it, it
doesn't even have a a y**x operation - just some simpler operations
which can be used to implement it. GCC doesn't inline pow(y, x) on
x86/64 - it just calls the library function.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230218/419a6366/attachment.sig>

From oscar.j.benjamin at gmail.com  Sat Feb 18 07:46:46 2023
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 18 Feb 2023 12:46:46 +0000
Subject: Precision Tail-off?
In-Reply-To: <20230218111601.vtlqaq5kpll3en2j@hjp.at>
References: <CAP=-cKVbQ6yj_QuwykJ_WifJfnSggowPDwYtnfvQ6FyF2w5b_g@mail.gmail.com>
 <CAHVvXxTFqgurjCHHDpBRU_bGJ3fzXB=p7NY0tiGSJv5ng=fYkg@mail.gmail.com>
 <mailman.1817.1676373454.20444.python-list@python.org>
 <k57132F3e1U1@mid.individual.net>
 <CAP=-cKVgqutG=GufPYA4+4Frzcma9g_A9t+hDNf1S39C53PyGg@mail.gmail.com>
 <1373f7d0-a3b6-22c7-d409-8a4ee4208093@Damon-Family.org>
 <mailman.1851.1676659340.20444.python-list@python.org>
 <k5aof8Fhs3qU1@mid.individual.net>
 <CAPTjJmpEPzFx0kRo3t3ym-pMWvFwUnXks+5JYqr=vygQrsSKgQ@mail.gmail.com>
 <CAHVvXxSdr7Akn9Ofin=ia=ui67QdA=aAnVKKX=pW-fnGYYhpKQ@mail.gmail.com>
 <20230218111601.vtlqaq5kpll3en2j@hjp.at>
Message-ID: <CAHVvXxSe-2dfJ7XEwdiKPemhKhPLCfn5mh8ttizEoW_kvHKCiA@mail.gmail.com>

On Sat, 18 Feb 2023 at 11:19, Peter J. Holzer <hjp-python at hjp.at> wrote:
>
> On 2023-02-18 03:52:51 +0000, Oscar Benjamin wrote:
> > On Sat, 18 Feb 2023 at 01:47, Chris Angelico <rosuav at gmail.com> wrote:
> > > On Sat, 18 Feb 2023 at 12:41, Greg Ewing via Python-list
> > > > To avoid it you would need to use an algorithm that computes nth
> > > > roots directly rather than raising to the power 1/n.
> > > >
> > >
> > > It's somewhat curious that we don't really have that. We have many
> > > other inverse operations - addition and subtraction (not just "negate
> > > and add"), multiplication and division, log and exp - but we have
> > > exponentiation without an arbitrary-root operation. For square roots,
> > > that's not a problem, since we can precisely express the concept
> > > "raise to the 0.5th power", but for anything else, we have to raise to
> > > a fractional power that might be imprecise.
> >
> > Various libraries can do this. Both SymPy and NumPy have cbrt for cube roots:
>
> Yes, but that's a special case. Chris was talking about arbitrary
> (integer) roots. My calculator has a button labelled [x?y], but my
> processor doesn't have an equivalent operation.

All three of SymPy, mpmath and gmpy2 can do this as accurately as
desired for any integer root:

  >>> n = 123456789000000000000000000000000000000000000000000000000000000

  >>> sympy.root(n, 6)
  1000000000*13717421**(1/6)*3**(1/3)
  >>> sympy.root(n, 6).evalf(50)
  22314431635.562095902499928269233656421704825692573

  >>> mpmath.root(n, 6)
  mpf('22314431635.562096')
  >>> mpmath.mp.dps = 50
  >>> mpmath.root(n, 6)
  mpf('22314431635.562095902499928269233656421704825692572746')

  >>> gmpy2.root(n, 6)
  mpfr('22314431635.562096')
  >>> gmpy2.get_context().precision = 100
  >>> gmpy2.root(n, 6)
  mpfr('22314431635.56209590249992826924',100)

There are also specific integer only root routines like
sympy.integer_nthroot or gmpy2.iroot.

  >>> gmpy2.iroot(n, 6)
  (mpz(22314431635), False)
  >>> sympy.integer_nthroot(n, 6)
  (22314431635, False)

Other libraries like the stdlib math module and numpy define some
specific examples like cbrt or isqrt but not a full root or iroot.
What is lacking is a plain 64-bit floating point routine like:

  def root(x: float, n: int) -> float:
      return x ** (1/n) # except more accurate than this

It could be a good candidate for numpy and/or the math module. I just
noticed from the docs that the math module has a new in 3.11 cbrt
function that I didn't know about which suggests that a root function
might also be considered a reasonable addition in future. Similarly
isqrt was new in 3.8 and it is not a big leap from there to see
someone adding iroot.

--
Oscar

From list1 at tompassin.net  Sat Feb 18 10:29:32 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 18 Feb 2023 10:29:32 -0500
Subject: LRU cache
In-Reply-To: <DB6PR01MB38955805D4B3AFB4FCBB953783A69@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <DB6PR01MB38955805D4B3AFB4FCBB953783A69@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <28d0e37e-9487-847a-de97-0129ad2f86dc@tompassin.net>

On 2/18/2023 5:38 AM, Albert-Jan Roskam wrote:
>     I sometimes use this trick, which I learnt from a book by Martelli.
>     Instead of try/except, membership testing with "in" (__contains__) might
>     be faster. Probably "depends". Matter of measuring.
>     def somefunc(arg, _cache={}):
>     ? ? if len(_cache) > 10 ** 5:
>     ? ? ? ? _cache.pop()
>     ? ? try:
>     ? ? ? ? return _cache[arg]
>     ? ? except KeyError:
>     ? ? ? ? result = expensivefunc(arg)
>     ? ? ? ? _cache[arg] = result
>     ? ? ? ? return result
>     Albert-Jan

_cache.get(arg) should be a little faster and use slightly fewer 
resources than the try/except.


From rob.cliffe at btinternet.com  Sat Feb 18 11:28:28 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Sat, 18 Feb 2023 16:28:28 +0000
Subject: LRU cache
In-Reply-To: <28d0e37e-9487-847a-de97-0129ad2f86dc@tompassin.net>
References: <DB6PR01MB38955805D4B3AFB4FCBB953783A69@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
 <28d0e37e-9487-847a-de97-0129ad2f86dc@tompassin.net>
Message-ID: <41be91d5-660e-3f03-136d-08957ac7b58a@btinternet.com>



On 18/02/2023 15:29, Thomas Passin wrote:
> On 2/18/2023 5:38 AM, Albert-Jan Roskam wrote:
>> ??? I sometimes use this trick, which I learnt from a book by Martelli.
>> ??? Instead of try/except, membership testing with "in" 
>> (__contains__) might
>> ??? be faster. Probably "depends". Matter of measuring.
>> ??? def somefunc(arg, _cache={}):
>> ??? ? ? if len(_cache) > 10 ** 5:
>> ??? ? ? ? ? _cache.pop()
>> ??? ? ? try:
>> ??? ? ? ? ? return _cache[arg]
>> ??? ? ? except KeyError:
>> ??? ? ? ? ? result = expensivefunc(arg)
>> ??? ? ? ? ? _cache[arg] = result
>> ??? ? ? ? ? return result
>> ??? Albert-Jan
>
> _cache.get(arg) should be a little faster and use slightly fewer 
> resources than the try/except.
>
Provided that you can provide a default value to get() which will never 
be a genuine "result".


From sjeik_appie at hotmail.com  Sat Feb 18 12:19:22 2023
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sat, 18 Feb 2023 18:19:22 +0100
Subject: LRU cache
In-Reply-To: <41be91d5-660e-3f03-136d-08957ac7b58a@btinternet.com>
Message-ID: <DB6PR01MB3895ED139D2D63B217EC923983A69@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Feb 18, 2023 17:28, Rob Cliffe via Python-list <python-list at python.org>
   wrote:

     On 18/02/2023 15:29, Thomas Passin wrote:
     > On 2/18/2023 5:38 AM, Albert-Jan Roskam wrote:
     >> ??? I sometimes use this trick, which I learnt from a book by
     Martelli.
     >> ??? Instead of try/except, membership testing with "in"
     >> (__contains__) might
     >> ??? be faster. Probably "depends". Matter of measuring.
     >> ??? def somefunc(arg, _cache={}):
     >> ??? ? ? if len(_cache) > 10 ** 5:
     >> ??? ? ? ? ? _cache.pop()
     >> ??? ? ? try:
     >> ??? ? ? ? ? return _cache[arg]
     >> ??? ? ? except KeyError:
     >> ??? ? ? ? ? result = expensivefunc(arg)
     >> ??? ? ? ? ? _cache[arg] = result
     >> ??? ? ? ? ? return result
     >> ??? Albert-Jan
     >
     > _cache.get(arg) should be a little faster and use slightly fewer
     > resources than the try/except.
     >
     Provided that you can provide a default value to get() which will never
     be a genuine "result".

   =====
   This might be better than None:
   _cache.get(arg, Ellipsis)

From avi.e.gross at gmail.com  Sat Feb 18 14:59:37 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sat, 18 Feb 2023 14:59:37 -0500
Subject: Comparing caching strategies
In-Reply-To: <20230218104004.gc7dcrsxxv235icw@hjp.at>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
 <20230218104004.gc7dcrsxxv235icw@hjp.at>
Message-ID: <002901d943d3$87ed3310$97c79930$@gmail.com>

David,

This conversation strikes me as getting antagonistic and as such, I will not
continue it here after this message.

I can nitpick at least as well as you but have no interest. It is also
wandering away from the original point.

The analogy I gave remains VALID no matter if you do not accept it as being
precise. Analogies are not equalities. I did not say this does at all what
programs like pkzip do in entirety or even anything similarly. I simply said
that pkzip (as it no doubt evolves) has a series of compression methods to
choose from and each has a corresponding method to undo and get back the
original. As it happens, you can write any number of algorithms that
determine which method to use and get back the original. It depend on many
relative costs including not just the size of the compressed object but how
often it will be accessed and how much effort each takes. In some cases you
will compress everything once and extract it many times and in many places,
so it may be worth the effort to try various compression techniques and
measure size and even the effort to extract from that and decide which to
use.

I do not know the internals of any Roaring Bitmap implementation so all I
did gather was that once the problem is broken into accessing individual
things I chose to call zones for want of a more specific name, then each
zone is stored in one of an unknown number of ways depending on some logic.
You say an implementation chose two ways and that is fine. But in theory, it
may make sense to choose other ways as well and the basic outline of the
algorithm remains similar. I can imagine if a region/zone is all the even
numbers, then a function that checks if you are searching for an odd number
may be cheaper. That is an example, not something I expect to see or that is
useful enough. But the concept of storing a function as the determiner for a
region is general enough that it can support many more realistic ideas.

>From what you wrote, the algorithm chosen is fairly simple BUT I have to ask
if these bitmaps are static or can be changed at run time? I mean if you
have a region that is sparse and then you keep adding, does the software
pause and rewrite that region as a bitmap if it is a list of offsets? Or, if
a bitmap loses enough, ...

On to your other points. Again, understand I am talking way more abstractly
than you and thus it really does not matter what the length of a particular
ID in some country is for the main discussion. The assumption is that if you
are using something with limits, like a Roaring Bitmap, that you do things
within the limits. When I lived in Austria, I did not bother getting an
Austrian Sozialversicherungsnummer so I have no idea it is ten digits long.
In any case, many things grow over time such as the length of telephone
numbers. 

The same applies to things like airport codes. They can get longer for many
reasons and may well exceed 4 characters, and certainly UNICODE or other
such representations may exceed four bytes now if you allow non-ASCII
characters that map into multiple bytes. My point was to think about how
useful a Roaring bitmap is if it takes only 32 bit integers and one trivial
mapping was to use any four bytes to represent a unique integer. But clearly
you could map longer strings easily enough if you restrict yourself to 26
upper case letters and perhaps a few other symbols that can be encoded in 5
bits. I am not saying it is worth the effort, but that means 6 characters
can fit in 32 bits.

I do wonder if the basic idea has to be limited to 32 bits or if it can
expand to say 64 or use additional but fast methods of storing the data
beyond the two mentioned.

There are variants of other ideas I can think of like sparse arrays or
matrices such as you find in the scipy module in Python. If they hold a
Boolean value, they sound like they are a similar idea where you simply keep
track of the ones marked True, or if it makes sense, the ones considered
False.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Peter J. Holzer
Sent: Saturday, February 18, 2023 5:40 AM
To: python-list at python.org
Subject: Re: Comparing caching strategies

On 2023-02-17 18:08:09 -0500, avi.e.gross at gmail.com wrote:
> Analogies I am sharing are mainly for me to wrap my head around an 
> idea by seeing if it matches any existing ideas or templates and is 
> not meant to be exact. Fair enough?

Yeah. But if you are venting your musings into a public space you shouldn't
be surprised if people react to them. And we can only react to what you
write, not what you think.

> But in this case, from my reading, the analogy is rather reasonable.

Although that confused me a bit. You are clearly responding to something I
thought about but which you didn't quote below. Did I just think about it
and not write it, but you responded anyway because you're a mind reader?
Nope, it just turns out you (accidentally) deleted that sentence.


> The implementation of Roaring Bitmaps seems to logically break up the 
> space of all possible values it looks up into multiple "zones" that 
> are somewhat analogous to individual files,

That part is correct. But you presented that in the form of a
performance/space tradeoff, writing about "trying multiple methods" to find
the smallest, and that that makes compression slower. That may be the case
for pkzip, but it's not what RB does: Instead it uses a very simple
heuristic: If there are less than 25% of the bits set in a zone, it uses a
list of offsets, otherwise a plain bitmap. (according to their 2016 paper
which I just skimmed through again - maybe the implementation is a bit more
complex now). So I think your description would lead the reader to
anticipate problems which aren't there and probably miss ones which are
there. So I'll stay with my "not completely wrong but not very helpful"
assessment.


> I did not raise the issue and thus have no interest in promoting this 
> technology nor knocking it down. Just wondering what it was under the 
> hood and whether I might even have a need for it. I am not saying 
> Social Security numbers are a fit, simply that some form of ID number
might fit.

Yeah, that's the point: Any form of ID which is a small-ish integer number
fits.

And maybe it's just because I work with databases a lot, but representing
things with numeric ids (in relational databases we call them "surrogate
keys") is such a basic operation that it doesn't warrant more than a
sentence or two.


> If a company has a unique ID number for each client and uses it 
> consistently, then an implementation that holds a set stored this way 
> of people using product A, such as house insurance, and those using 
> product B, such as car insurance, and perhaps product C is an umbrella 
> policy, might easily handle some queries such as who uses two or all 
> three (intersections of sets) or who might merit a letter telling them 
> how much they can save if they subscribed to two or all three as a way 
> to get more business. Again, just  a made-up example I can think 
> about. A company which has a million customers over the years will 
> have fairly large sets as described.

A much better example. This is indeed how you would use roaring bitmaps.


> What is helpful to me in thinking about something will naturally often 
> not be helpful to you or others but nothing you wrote makes me feel my 
> first take was in any serious way wrong. It still makes sense to me.
> 
> And FYI, the largest integer in signed 32 bits is 2_147_483_647

I know. It's been some time since I could do hexadecimal arithmetic in my
head but the the limits of common data types are still burned into my brain
;-).

> which is 10 digits. A Social Security number look like xxx-xx-xxxx at 
> this time which is only 9 digits.

These are US social security numbers. Other countries have other schemes.
For example, Austrian SSNs have 10 digits, so you would need 34 bits to
represent them exactly. However, they (obviously) contain some redundancy
(one of the digits is a checksum, and there aren't 999999 days in a century)
so it could algorithmically be compressed down to 26 bits. But you probably
wouldn't do that because in almost any real application you wouldn't use the
SSN as a primary key (some people don't have one, and there have been mixups
resulting in two people getting the same SSN).


> As for my other EXAMPLE, I fail to see why I need to provide a 
> specific need for an application. I don't care what they need it for. 
> The thought was about whether something that does not start as an 
> integer can be uniquely mapped into and out of integers of size 32 bits.

That's what confused me. You seemed to concentrate on the "map things to
integers" part which has been solved for decades and is absolutely
incidental to roaring bitmaps and completely ignored what you would be using
them for.

So I thought I was missing something, but it seems I wasn't.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"


From python at mrabarnett.plus.com  Sat Feb 18 15:34:41 2023
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 18 Feb 2023 20:34:41 +0000
Subject: Comparing caching strategies
In-Reply-To: <002901d943d3$87ed3310$97c79930$@gmail.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
 <20230218104004.gc7dcrsxxv235icw@hjp.at>
 <002901d943d3$87ed3310$97c79930$@gmail.com>
Message-ID: <3fa61318-cd2c-7d4f-7ba7-3128d0b38316@mrabarnett.plus.com>

On 2023-02-18 19:59, avi.e.gross at gmail.com wrote:
[snip]
> I do not know the internals of any Roaring Bitmap implementation so all I
> did gather was that once the problem is broken into accessing individual
> things I chose to call zones for want of a more specific name, then each
> zone is stored in one of an unknown number of ways depending on some logic.
> You say an implementation chose two ways and that is fine. But in theory, it
> may make sense to choose other ways as well and the basic outline of the
> algorithm remains similar. I can imagine if a region/zone is all the even
> numbers, then a function that checks if you are searching for an odd number
> may be cheaper. That is an example, not something I expect to see or that is
> useful enough. But the concept of storing a function as the determiner for a
> region is general enough that it can support many more realistic ideas.
> 
>   From what you wrote, the algorithm chosen is fairly simple BUT I have to ask
> if these bitmaps are static or can be changed at run time? I mean if you
> have a region that is sparse and then you keep adding, does the software
> pause and rewrite that region as a bitmap if it is a list of offsets? Or, if
> a bitmap loses enough, ...
> 
A region is either "sparse" (<= 4096 entries) or "dense" (> 4096 
entries). It'll be converted to the other type as and when necessary.

> On to your other points. Again, understand I am talking way more abstractly
> than you and thus it really does not matter what the length of a particular
> ID in some country is for the main discussion. The assumption is that if you
> are using something with limits, like a Roaring Bitmap, that you do things
> within the limits. When I lived in Austria, I did not bother getting an
> Austrian Sozialversicherungsnummer so I have no idea it is ten digits long.
> In any case, many things grow over time such as the length of telephone
> numbers.
> 
> The same applies to things like airport codes. They can get longer for many
> reasons and may well exceed 4 characters, and certainly UNICODE or other
> such representations may exceed four bytes now if you allow non-ASCII
> characters that map into multiple bytes. My point was to think about how
> useful a Roaring bitmap is if it takes only 32 bit integers and one trivial
> mapping was to use any four bytes to represent a unique integer. But clearly
> you could map longer strings easily enough if you restrict yourself to 26
> upper case letters and perhaps a few other symbols that can be encoded in 5
> bits. I am not saying it is worth the effort, but that means 6 characters
> can fit in 32 bits.
> 
The Unicode Consortium have said that Unicode will be limited to 
U+0000..U+10FFFF (21 bits).

> I do wonder if the basic idea has to be limited to 32 bits or if it can
> expand to say 64 or use additional but fast methods of storing the data
> beyond the two mentioned.
> 
> There are variants of other ideas I can think of like sparse arrays or
> matrices such as you find in the scipy module in Python. If they hold a
> Boolean value, they sound like they are a similar idea where you simply keep
> track of the ones marked True, or if it makes sense, the ones considered
> False.
> 
[snip]


From list1 at tompassin.net  Sat Feb 18 15:59:32 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 18 Feb 2023 15:59:32 -0500
Subject: Comparing caching strategies
In-Reply-To: <002901d943d3$87ed3310$97c79930$@gmail.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
 <20230218104004.gc7dcrsxxv235icw@hjp.at>
 <002901d943d3$87ed3310$97c79930$@gmail.com>
Message-ID: <ec191d53-9757-ca61-d6e6-24e8c8652cbd@tompassin.net>

On 2/18/2023 2:59 PM, avi.e.gross at gmail.com wrote:
> I do not know the internals of any Roaring Bitmap implementation so all I
> did gather was that once the problem is broken into accessing individual
> things I chose to call zones for want of a more specific name, then each
> zone is stored in one of an unknown number of ways depending on some logic.

Somewhat tangential, but back quite a while ago there was a C library 
called IIRC "julia list".  It implemented lists in several different 
ways, some quite sophisticated, depending on the size and usage pattern. 
  I remembered it as soon as I took a look at Roaring Bitmap and saw 
that the latter can use different representations depending on size and 
access patterns.


From hjp-python at hjp.at  Sat Feb 18 17:55:44 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 18 Feb 2023 23:55:44 +0100
Subject: Comparing caching strategies
In-Reply-To: <ec191d53-9757-ca61-d6e6-24e8c8652cbd@tompassin.net>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
 <20230218104004.gc7dcrsxxv235icw@hjp.at>
 <002901d943d3$87ed3310$97c79930$@gmail.com>
 <ec191d53-9757-ca61-d6e6-24e8c8652cbd@tompassin.net>
Message-ID: <20230218225544.lbsymj654vzwyixu@hjp.at>

On 2023-02-18 15:59:32 -0500, Thomas Passin wrote:
> On 2/18/2023 2:59 PM, avi.e.gross at gmail.com wrote:
> > I do not know the internals of any Roaring Bitmap implementation so all I
> > did gather was that once the problem is broken into accessing individual
> > things I chose to call zones for want of a more specific name, then each
> > zone is stored in one of an unknown number of ways depending on some logic.
> 
> Somewhat tangential, but back quite a while ago there was a C library called
> IIRC "julia list".

ITYM Judy arrays. They were mentioned here already.

> It implemented lists in several different ways, some quite
> sophisticated, depending on the size and usage pattern.  I remembered
> it as soon as I took a look at Roaring Bitmap and saw that the latter
> can use different representations depending on size and access
> patterns.

Yes, Roaring Bitmaps are somewhat similar. Judy arrays are more
versatile (they support more data types) and in many ways more
sophisticated, despite being 10+ years older. OTOH Roaring Bitmaps are a
lot simpler which may have contributed to their popularity.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230218/d80e1e93/attachment.sig>

From avi.e.gross at gmail.com  Sat Feb 18 18:04:25 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sat, 18 Feb 2023 18:04:25 -0500
Subject: Comparing caching strategies
In-Reply-To: <ec191d53-9757-ca61-d6e6-24e8c8652cbd@tompassin.net>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
 <20230218104004.gc7dcrsxxv235icw@hjp.at>
 <002901d943d3$87ed3310$97c79930$@gmail.com>
 <ec191d53-9757-ca61-d6e6-24e8c8652cbd@tompassin.net>
Message-ID: <005401d943ed$58a7c380$09f74a80$@gmail.com>

It is not an unusual pattern, Thomas, to do something selective to some object rather than do all parts just one way.

The history of computing has often been one where you had to deal with scarcity of expensive resources.

Consider the Python "list" as a rather wasteful container that is best used for diverse contents. As soon as you make all the contents of the same basic type, you start wondering if you are better off with a restricted type that holds only one kind of item, often as something like a numpy array. Or, you start wondering if maybe a good way to store the contents is in a dictionary instead, as searching it is often way faster.

But fundamentally, there is nothing WRONG with uses of a Python list as it handles almost anything and for small sample sizes, it is often not work spending time redoing it. Still, if you implement a 3-D matrix as a list of lists of lists, and use slicing and other techniques to do things like matrix multiplication, it gets unwieldly enough so arguably it is the wrong tool.

If you look at the history of Python, they deliberately left out many of the containers other programming languages used and stressed lists and tuples. No vectors or arrays were the focus. Later stuff had to be added as people noted that generality has costs. If you look at a language like JavaScript, it is beyond weird how they decided to use attributes of an object to make an array. So an object might have attributes like "name" and "length" alongside some like "1" and "2" and "666" and when you wanted to treat the object like an array, it might look at the attributes and ignore the non-numerical ones and look like it has a sparsely populated array/vector of items indexed from 1 to 666. You can do all kinds of arithmetic operations and add missing indices or remove some and it still works like a typical array but with weird costs such as sometimes having to reindex lots of items if you want to insert a new item as the 3rd or 1st element so any above need renumbering the hard way! It works but strikes me as a kludge.

If you look within Python  at numpy and pandas and similar utilities, they are well aware of the idea of abstracting out concepts with different implementations and use lots of Python tools that access different storage methods as needed often by using objects that implement various "protocols" to the point where manipulating them similarly seems straightforward. In principle, you could create something like a pandas data.frame and then call a function that examines the columns, and perhaps rows, and returns a modified (or new) data.frame where the contents have been adjusted so the storage is smaller or can be accessed more efficiently, based on any other preferences and hints you supply, such as if it can be immutable. A column which is all 1's or Trues can obviously be done other ways, and one that has all values under 256, again, can use less storage. 

Of course this would need to be done very carefully so that changes that violate assumptions will result in a refactoring to a version that handles the changes, or results in an error. And a long-running system could be set to keep track of how an object is used and perhaps make adjustments. As one example, after some number of accesses, you might change a function "live" to begin caching, or have an object reconfigure to be faster to search even if occupying more room.

Back to Python lists, a typical change is simply to convert them to a tuple which makes them easier to store and often faster to search. And, if the keys are unique, now that dictionaries maintain order of insertion, sometimes you may want to convert the list to a dict. 

But I hasten to note some "improvements" may not really improve. In a language like R, many operations such as changing one vector in a data.frame are done lazily and the new copy is still sharing mostly the same storage as the old. Making some changes can result in negative effects. A common problem people have is that trying to store some objects in an existing vector can work except when done, the entire vector has been transformed into one that can carry any of the contents. A vector of integers may become a vector of doubles or even a vector of characters that now have entries like "777" as a character string. The flexibility comes with lots of programming errors!

Note how this can cause problems with the original idea here of caching strategies. Imagine a function that checks the environment as to what encoding or human language and so on to produce text in. If you cache it so it produces results that are stored in something like a dictionary with a key, and later the user changes the environment as it continues running, the cache may now contain invalid results. You might need to keep track of the environment and empty the cache if things change, or start a new cache and switch to it.  An example would be the way I use Google Translate. I sometimes am studying or using a language and want to look up a word or phrase or entire sentence. If Google Translate keeps track, it may notice repeated requests like "Valentines Day" and cache it for re-use. But I often click to switch languages and see if the other one uses a similar or different way to describe what it means or something similar but spelled another way. German does the latter as in Valentinstag which is a fairly literal translation as does Dutch (Valentijnsdag ) and  Hungarian (Valentin nap) . 

But Hebrew calls it the holiday of love, sort of (?? ?????). Portuguese is similar but includes day as well as love (Dia dos Namorados)

Esperanto tosses in more about sainthood (Sankta Valent?n) and in a sense Spanish does both ways with day and saint (D?a de San Valent?n).

You could continue playing with such phrases quite a bit as Translate translates N languages into N languages. So even caching from English to anything is problematic and the same word spelling can be found in many languages and the translation of that word into English also varies.

So if you had a function like ask_translate(from, to, phrase) then something like an @lru_cache() decorator would not suffice as caching might require either combining the three arguments into one longer string to cache, or multiple caches accessed by something like an NxN data structure to select which cache.

And it can also have other considerations such as not using naked queries as keys, but customizing them first into some canonical format such as removing leading and trailing whitespace as  well as multiple instances between words, shifting everything to lower case, removing some punctuation and perhaps replacing special characters with a single version. Otherwise, the cache may be too large and often less effective. 

As stated earlier, any analogies or comparisons I use are for the purpose of discussion only and opinions are mine.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Thomas Passin
Sent: Saturday, February 18, 2023 4:00 PM
To: python-list at python.org
Subject: Re: Comparing caching strategies

On 2/18/2023 2:59 PM, avi.e.gross at gmail.com wrote:
> I do not know the internals of any Roaring Bitmap implementation so 
> all I did gather was that once the problem is broken into accessing 
> individual things I chose to call zones for want of a more specific 
> name, then each zone is stored in one of an unknown number of ways depending on some logic.

Somewhat tangential, but back quite a while ago there was a C library called IIRC "julia list".  It implemented lists in several different ways, some quite sophisticated, depending on the size and usage pattern. 
  I remembered it as soon as I took a look at Roaring Bitmap and saw that the latter can use different representations depending on size and access patterns.

-- 
https://mail.python.org/mailman/listinfo/python-list


From python at mrabarnett.plus.com  Sat Feb 18 19:04:26 2023
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 19 Feb 2023 00:04:26 +0000
Subject: Comparing caching strategies
In-Reply-To: <005401d943ed$58a7c380$09f74a80$@gmail.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
 <20230218104004.gc7dcrsxxv235icw@hjp.at>
 <002901d943d3$87ed3310$97c79930$@gmail.com>
 <ec191d53-9757-ca61-d6e6-24e8c8652cbd@tompassin.net>
 <005401d943ed$58a7c380$09f74a80$@gmail.com>
Message-ID: <1e501602-bed9-317c-7d75-d7387b977e60@mrabarnett.plus.com>

On 2023-02-18 23:04, avi.e.gross at gmail.com wrote:
[snip]
> 
> Note how this can cause problems with the original idea here of caching strategies. Imagine a function that checks the environment as to what encoding or human language and so on to produce text in. If you cache it so it produces results that are stored in something like a dictionary with a key, and later the user changes the environment as it continues running, the cache may now contain invalid results. You might need to keep track of the environment and empty the cache if things change, or start a new cache and switch to it.  An example would be the way I use Google Translate. I sometimes am studying or using a language and want to look up a word or phrase or entire sentence. If Google Translate keeps track, it may notice repeated requests like "Valentines Day" and cache it for re-use. But I often click to switch languages and see if the other one uses a similar or different way to describe what it means or something similar but spelled another way. German does the latter as in Valentinstag which is a fairly literal translation as does Dutch (Valentijnsdag ) and  Hungarian (Valentin nap) .
> 
> But Hebrew calls it the holiday of love, sort of (?? ?????). Portuguese is similar but includes day as well as love (Dia dos Namorados)
> 
> Esperanto tosses in more about sainthood (Sankta Valent?n) and in a sense Spanish does both ways with day and saint (D?a de San Valent?n).
> 
The Esperanto didn't look right to me; it's "Valentena tago" or 
"Sankt-Valentena tago".

[snip]


From list1 at tompassin.net  Sat Feb 18 19:31:13 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 18 Feb 2023 19:31:13 -0500
Subject: Comparing caching strategies
In-Reply-To: <20230218225544.lbsymj654vzwyixu@hjp.at>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
 <20230218104004.gc7dcrsxxv235icw@hjp.at>
 <002901d943d3$87ed3310$97c79930$@gmail.com>
 <ec191d53-9757-ca61-d6e6-24e8c8652cbd@tompassin.net>
 <20230218225544.lbsymj654vzwyixu@hjp.at>
Message-ID: <05a273fe-0522-d376-c5b5-9aa5eeee9ae3@tompassin.net>

On 2/18/2023 5:55 PM, Peter J. Holzer wrote:
> On 2023-02-18 15:59:32 -0500, Thomas Passin wrote:
>> On 2/18/2023 2:59 PM, avi.e.gross at gmail.com wrote:
>>> I do not know the internals of any Roaring Bitmap implementation so all I
>>> did gather was that once the problem is broken into accessing individual
>>> things I chose to call zones for want of a more specific name, then each
>>> zone is stored in one of an unknown number of ways depending on some logic.
>>
>> Somewhat tangential, but back quite a while ago there was a C library called
>> IIRC "julia list".
> 
> ITYM Judy arrays. They were mentioned here already.

Ha! Fading memory, I guess.  Maybe that's why I couldn't find them with 
an internet search.

>> It implemented lists in several different ways, some quite
>> sophisticated, depending on the size and usage pattern.  I remembered
>> it as soon as I took a look at Roaring Bitmap and saw that the latter
>> can use different representations depending on size and access
>> patterns.
> 
> Yes, Roaring Bitmaps are somewhat similar. Judy arrays are more
> versatile (they support more data types) and in many ways more
> sophisticated, despite being 10+ years older. OTOH Roaring Bitmaps are a
> lot simpler which may have contributed to their popularity.
> 
>          hp
> 
> 


From avi.e.gross at gmail.com  Sat Feb 18 20:08:59 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sat, 18 Feb 2023 20:08:59 -0500
Subject: Comparing caching strategies
In-Reply-To: <1e501602-bed9-317c-7d75-d7387b977e60@mrabarnett.plus.com>
References: <otBFL.3111707$WRz3.215206@fx03.ams4>
 <ae81e937-0c48-4b99-a405-7362d4345944@btinternet.com>
 <3ddef37c-c1cc-7435-cd02-5f0c49b66bb0@mrabarnett.plus.com>
 <00bc01d9428d$b245c880$16d15980$@gmail.com>
 <20230217184702.git7iwc46c7nusug@hjp.at>
 <001c01d94324$b411cf50$1c356df0$@gmail.com>
 <20230218104004.gc7dcrsxxv235icw@hjp.at>
 <002901d943d3$87ed3310$97c79930$@gmail.com>
 <ec191d53-9757-ca61-d6e6-24e8c8652cbd@tompassin.net>
 <005401d943ed$58a7c380$09f74a80$@gmail.com>
 <1e501602-bed9-317c-7d75-d7387b977e60@mrabarnett.plus.com>
Message-ID: <007101d943fe$bfedfee0$3fc9fca0$@gmail.com>

MRAB,

I made it very clear I was using the translation provided by Google Translate. I copied exactly what it said and as I speak the languages involved, they seemed reasonable.  I often find it provides somewhat different translations than I expect and sometimes I need to supply a longer sentence to get it on track and sometimes it is just plain wrong. Getting it to provide female-specific sentences can be an exercise in frustration, for example.

What it produces is not important to the main point. 

It is hard to have conversations when people keep niggling on details and especially details that make absolutely no difference. 

My point was you can have functions that cannot be cached for results just any trivial way. The EXAMPLE I gave suggests if your had a Python program that did something long these lines between multiple languages, the RESULTS will depend on not just the phrase used. But they can be cached well in several ways if you want.

Let me point out another related use. When you type a message on your phone, you may have a sort of prediction algorithm running that keeps offering to auto-complete your words or fix spelling errors. It does this dynamically and learns new words and eventually may start remembering patterns. I have totally mucked up my ENGLISH keyboard because it now remembers snippets of my typing in languages like Spanish and German and lately Esperanto and it makes suggestions from a very confused composite state including lots of funny accented characters. I now switch keyboards when I remember but much of the damage is done! LOL!

That too is an example of some hidden data the program uses which is loaded with past data of what I have typed and of short phrases so it can make a guess that word A is often followed by word B and after seeing both A and B is extremely likely to be followed by C. Now obviously this would not be a trivial use of caching as part of guessing but could be part of a method that increments some keys like "A B" or "A B C" with a count of how often they have seen that combo. Yes, this is not direct caching, but also a side effect you might want to add to a function.

As for Esperanto, I am still learning it and as a created language, the snippet I used can likely be translated multiple ways and as someone who could care less about Valentines Day, I don't ever intend on using any of those ways in conversation. Most languages have alternate ways of saying things and it would not shock me if there are sentences like "The Holiday used in Capitalist Western Countries to commemorate a day of love based on a person some religions consider of merit and have given a religious title" or whatever.

So back on topic, an original question here was how to cache things, perhaps using a LRU algorithm with a data structure using some maximum.

My comment was that using a function decorator that caches any result may not be adequate in many cases. I presented several examples and my point is that in the above example, it may make more sense to have multiple caches that exist perhaps outside any one function, or a more complex cache that stores using a more complex key

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of MRAB
Sent: Saturday, February 18, 2023 7:04 PM
To: python-list at python.org
Subject: Re: Comparing caching strategies

On 2023-02-18 23:04, avi.e.gross at gmail.com wrote:
[snip]
> 
> Note how this can cause problems with the original idea here of caching strategies. Imagine a function that checks the environment as to what encoding or human language and so on to produce text in. If you cache it so it produces results that are stored in something like a dictionary with a key, and later the user changes the environment as it continues running, the cache may now contain invalid results. You might need to keep track of the environment and empty the cache if things change, or start a new cache and switch to it.  An example would be the way I use Google Translate. I sometimes am studying or using a language and want to look up a word or phrase or entire sentence. If Google Translate keeps track, it may notice repeated requests like "Valentines Day" and cache it for re-use. But I often click to switch languages and see if the other one uses a similar or different way to describe what it means or something similar but spelled another way. German does the latter as in Valentinstag which is a fairly literal translation as does Dutch (Valentijnsdag ) and  Hungarian (Valentin nap) .
> 
> But Hebrew calls it the holiday of love, sort of (?? ?????). 
> Portuguese is similar but includes day as well as love (Dia dos 
> Namorados)
> 
> Esperanto tosses in more about sainthood (Sankta Valent?n) and in a sense Spanish does both ways with day and saint (D?a de San Valent?n).
> 
The Esperanto didn't look right to me; it's "Valentena tago" or "Sankt-Valentena tago".

[snip]

--
https://mail.python.org/mailman/listinfo/python-list


From azizbek.khamdamov at gmail.com  Sun Feb 19 09:03:55 2023
From: azizbek.khamdamov at gmail.com (Azizbek Khamdamov)
Date: Sun, 19 Feb 2023 19:03:55 +0500
Subject: File write, weird behaviour
Message-ID: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code adds new record to the beginning of the file,
expected behaviour
file.write("new city\n")

file.close()


Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

file.readlines()
file.close()

I could not find anything in documentation to explain this strange
behaviour. Why is this happening?

From pablogsal at gmail.com  Sat Feb 18 06:33:04 2023
From: pablogsal at gmail.com (Pablo Galindo Salgado)
Date: Sat, 18 Feb 2023 11:33:04 +0000
Subject: [Python-Dev] Python 3.11.2, 3.10.10
In-Reply-To: <CABD5YeFFYBHau2Y2PcF0MCDS-78W7_bpqMNEqQWNRudbEh3k8Q@mail.gmail.com>
References: <CABD5YeFFYBHau2Y2PcF0MCDS-78W7_bpqMNEqQWNRudbEh3k8Q@mail.gmail.com>
Message-ID: <F37069A8-9A5C-46A2-A2B5-F593E06B3764@gmail.com>

   Apologies!
   It seems that I added python-comitters and python-announce but forgot to
   add python-dev. Here is the email to python-announce:

   [1]Mailman 3 [RELEASE]
   Python 3.11.2, Python
   3.10.10 and 3.12.0 alpha 5
   are available -              [2]favicon.ico
   Python-announce-list -
   python.org
   mail.python.org

   Apologies for the confusion!
   Regards from cloudy London,
   Pablo Galindo Salgado?
   Pablo Galindo Salgado

     On 18 Feb 2023, at 11:14, ???? <uri at speedy.net> wrote:

     ?
     Hi,
     I was surprised?that?Python 3.11.2 and 3.10.10 have been released
     without a notice to this mailing list. What happened?
     Thanks,
     Uri.
     ????
     [3]uri at speedy.net
     On Wed, Dec 7, 2022 at 1:03 AM ?ukasz Langa <[4]lukasz at langa.pl> wrote:

       Greetings! We bring you a slew of releases this fine Saint Nicholas /
       Sinterklaas day. Six simultaneous releases has got to be some record.
       There?s one more record we broke this time, you?ll see below.

       In any case, updating is recommended due to security content:

       3.7 - 3.12: gh-98739
       <[5]https://github.com/python/cpython/issues/98739>: Updated bundled
       libexpat to 2.5.0 to fix CVE-2022-43680
       <[6]https://nvd.nist.gov/vuln/detail/CVE-2022-43680> (heap
       use-after-free).
       3.7 - 3.12: gh-98433
       <[7]https://github.com/python/cpython/issues/98433>: The IDNA codec
       decoder used on DNS hostnames by socket or asyncio related name
       resolution functions no longer involves a quadratic algorithm to fix
       CVE-2022-45061 <[8]https://nvd.nist.gov/vuln/detail/CVE-2022-45061>.
       This prevents a potential CPU denial of service if an out-of-spec
       excessive length hostname involving bidirectional characters were
       decoded. Some protocols such as urllib http 3xx redirects potentially
       allow for an attacker to supply such a name.
       3.7 - 3.12: gh-100001
       <[9]https://github.com/python/cpython/issues/100001>: python -m
       http.server no longer allows terminal control characters sent within a
       garbage request to be printed to the stderr server log.
       3.8 - 3.12: gh-87604
       <[10]https://github.com/python/cpython/issues/87604>: Avoid publishing
       list of active per-interpreter audit hooks via the gc module.
       3.9 - 3.10 (already released in 3.11+ before): gh-97514
       <[11]https://github.com/python/cpython/issues/97514>: On Linux the
       multiprocessing module returns to using filesystem backed unix domain
       sockets for communication with the forkserver process instead of the
       Linux abstract socket namespace. Only code that chooses to use the
       ?forkserver? start method is affected. This prevents Linux
       CVE-2022-42919 <[12]https://nvd.nist.gov/vuln/detail/CVE-2022-42919>
       (potential privilege escalation) as abstract sockets have no
       permissions and could allow any user on the system in the same network
       namespace (often the whole system) to inject code into the
       multiprocessing forkserver process. This was a potential privilege
       escalation. Filesystem based socket permissions restrict this to the
       forkserver process user as was the default in Python 3.8 and earlier.
       3.7 - 3.10: gh-98517
       <[13]https://github.com/python/cpython/issues/98517>: Port XKCP?s fix
       for the buffer overflows in SHA-3 to fix CVE-2022-37454
       <[14]https://nvd.nist.gov/vuln/detail/CVE-2022-37454>.
       3.7 - 3.9 (already released in 3.10+ before): gh-68966
       <[15]https://github.com/python/cpython/issues/68966>: The deprecated
       mailcap module now refuses to inject unsafe text (filenames, MIME
       types, parameters) into shell commands to address CVE-2015-20107
       <[16]https://nvd.nist.gov/vuln/detail/CVE-2015-20107>. Instead of
       using such text, it will warn and act as if a match was not found (or
       for test commands, as if the test failed).
       ?<[17]https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3120-alpha-3-1>Python
       3.12.0 alpha 3

       Get it here, read the change log, sing a GPT-3-generated Sinterklaas
       song:

       [18]https://www.python.org/downloads/release/python-3120a3/
       <[19]https://www.python.org/downloads/release/python-3120a3/>

       216 new commits since 3.12.0 alpha 2 last month.

       ?<[20]https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3111-2>Python
       3.11.1

       Get it here, see the change log, read the recipe for quark soup:

       [21]https://www.python.org/downloads/release/python-3111/
       <[22]https://www.python.org/downloads/release/python-3111/>

       A whopping 495 new commits since 3.11.0. This is a massive increase of
       changes comparing to 3.10 at the same stage in the release cycle:
       there were ?only? 339 commits between 3.10.0 and 3.10.1.

       ?<[23]https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3109-3>Python
       3.10.9

       Get it here, read the change log, see circular patterns:

       [24]https://www.python.org/downloads/release/python-3109/
       <[25]https://www.python.org/downloads/release/python-3109/>

       165 new commits.

       ?<[26]https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3916-4>Python
       3.9.16

       Get it here, read the change log, consider upgrading to a newer
       version:

       [27]https://www.python.org/downloads/release/python-3916/
       <[28]https://www.python.org/downloads/release/python-3916/>

       Security-only release with no binaries. 10 commits.

       ?<[29]https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3816-5>Python
       3.8.16

       Get it here, see the change log, definitely upgrade to a newer
       version:

       [30]https://www.python.org/downloads/release/python-3816/
       <[31]https://www.python.org/downloads/release/python-3816/>

       Security-only release with no binaries. 9 commits.

       ?<[32]https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3716-6>Python
       3.7.16

       Get it here, read the change log, check PEP 537?
       <[33]https://peps.python.org/pep-0537/>to confirm EOL is coming to
       this version in June 2023:

       [34]https://www.python.org/downloads/release/python-3716/
       <[35]https://www.python.org/downloads/release/python-3716/>

       Security-only release with no binaries. 8 commits.

       ?<[36]https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#we-hope-you-enjoy-the-new-releases-7>We
       hope you enjoy the new releases!

       Thanks to all of the many volunteers who help make Python Development
       and these releases possible! Please consider supporting our efforts by
       volunteering yourself or through organization contributions to the
       Python Software Foundation.

       [37]https://www.python.org/psf/? <[38]https://www.python.org/psf/>
       Your friendly release team,

       Ned Deily @nad <[39]https://discuss.python.org/u/nad>
       Steve Dower @steve.dower
       <[40]https://discuss.python.org/u/steve.dower>
       Pablo Galindo Salgado @pablogsal
       <[41]https://discuss.python.org/u/pablogsal>
       ?ukasz Langa @ambv <[42]https://discuss.python.org/u/ambv>
       Thomas Wouters @thomas <[43]https://discuss.python.org/u/thomas>
       --
       [44]https://mail.python.org/mailman/listinfo/python-list

     _______________________________________________
     Python-Dev mailing list -- python-dev at python.org
     To unsubscribe send an email to python-dev-leave at python.org
     https://mail.python.org/mailman3/lists/python-dev.python.org/
     Message archived at
     https://mail.python.org/archives/list/python-dev at python.org/message/DK4YRRKBC4KLBNUY6VHLZ5Q6LWFWM3KY/
     Code of Conduct: http://python.org/psf/codeofconduct/

References

   Visible links
   1. https://mail.python.org/archives/list/python-announce-list at python.org/thread/P4JHHHAAO4L4KFZQ6PX5J3JRPAZUXJWJ/
   2. https://mail.python.org/archives/list/python-announce-list at python.org/thread/P4JHHHAAO4L4KFZQ6PX5J3JRPAZUXJWJ/
   3. mailto:uri at speedy.net
   4. mailto:lukasz at langa.pl
   5. https://github.com/python/cpython/issues/98739
   6. https://nvd.nist.gov/vuln/detail/CVE-2022-43680
   7. https://github.com/python/cpython/issues/98433
   8. https://nvd.nist.gov/vuln/detail/CVE-2022-45061
   9. https://github.com/python/cpython/issues/100001
  10. https://github.com/python/cpython/issues/87604
  11. https://github.com/python/cpython/issues/97514
  12. https://nvd.nist.gov/vuln/detail/CVE-2022-42919
  13. https://github.com/python/cpython/issues/98517
  14. https://nvd.nist.gov/vuln/detail/CVE-2022-37454
  15. https://github.com/python/cpython/issues/68966
  16. https://nvd.nist.gov/vuln/detail/CVE-2015-20107
  17. https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3120-alpha-3-1
  18. https://www.python.org/downloads/release/python-3120a3/
  19. https://www.python.org/downloads/release/python-3120a3/
  20. https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3111-2
  21. https://www.python.org/downloads/release/python-3111/
  22. https://www.python.org/downloads/release/python-3111/
  23. https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3109-3
  24. https://www.python.org/downloads/release/python-3109/
  25. https://www.python.org/downloads/release/python-3109/
  26. https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3916-4
  27. https://www.python.org/downloads/release/python-3916/
  28. https://www.python.org/downloads/release/python-3916/
  29. https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3816-5
  30. https://www.python.org/downloads/release/python-3816/
  31. https://www.python.org/downloads/release/python-3816/
  32. https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#python-3716-6
  33. https://peps.python.org/pep-0537/
  34. https://www.python.org/downloads/release/python-3716/
  35. https://www.python.org/downloads/release/python-3716/
  36. https://discuss.python.org/t/python-3-11-1-3-10-9-3-9-16-3-8-16-3-7-16-and-3-12-0-alpha-3-are-now-available/21724#we-hope-you-enjoy-the-new-releases-7
  37. https://www.python.org/psf/
  38. https://www.python.org/psf/
  39. https://discuss.python.org/u/nad
  40. https://discuss.python.org/u/steve.dower
  41. https://discuss.python.org/u/pablogsal
  42. https://discuss.python.org/u/ambv
  43. https://discuss.python.org/u/thomas
  44. https://mail.python.org/mailman/listinfo/python-list

From axy at declassed.art  Sun Feb 19 11:57:02 2023
From: axy at declassed.art (Axy)
Date: Sun, 19 Feb 2023 16:57:02 +0000
Subject: File write, weird behaviour
In-Reply-To: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
Message-ID: <e0d474c3-c792-0367-207c-20d5d1a9295d@declassed.art>

Looks like the data to be written is buffered, so actual write takes 
place after readlines(), when close() flushes buffers.

See io package documentation, BufferedIOBase.

The solution is file.flush() after file.write()

Can't deny, such a behaviour looks utterly weird. Try to avoid designing 
your software this way.

Axy.

On 19/02/2023 14:03, Azizbek Khamdamov wrote:
> Example 1 (works as expected)
>
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities
> # the following code adds new record to the beginning of the file,
> expected behaviour
> file.write("new city\n")
>
> file.close()
>
>
> Example 2 (weird behaviour)
>
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities
> # the following code DOES NOT add new record TO THE BEGINNING of the
> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
> new content should be added to the beginning of the file (as in
> Example 1)
> file.write("new city\n")
>
> file.readlines()
> file.close()
>
> I could not find anything in documentation to explain this strange
> behaviour. Why is this happening?

From hjp-python at hjp.at  Sun Feb 19 12:15:36 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sun, 19 Feb 2023 18:15:36 +0100
Subject: File write, weird behaviour
In-Reply-To: <e0d474c3-c792-0367-207c-20d5d1a9295d@declassed.art>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <e0d474c3-c792-0367-207c-20d5d1a9295d@declassed.art>
Message-ID: <20230219171536.gz5y5umc4v4uxc66@hjp.at>

On 2023-02-19 16:57:02 +0000, Axy via Python-list wrote:
> Looks like the data to be written is buffered, so actual write takes place
> after readlines(), when close() flushes buffers.
> 
> See io package documentation, BufferedIOBase.
> 
> The solution is file.flush() after file.write()

Or alternatively, file.seek() to the intended position when switching
between reading and writing. (The C standard says you have to do this. I
can't find it in the Python docs, but apparently Python behaves the
same.)

> On 19/02/2023 14:03, Azizbek Khamdamov wrote:
> > Example 2 (weird behaviour)
> > 
> > file = open("D:\Programming\Python\working_with_files\cities.txt",
> > 'r+') ## contains list cities
> > # the following code DOES NOT add new record TO THE BEGINNING of the
> > file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
> > new content should be added to the beginning of the file (as in
> > Example 1)
> > file.write("new city\n")

Also note that you can't ADD anything at the beginning (or in the
middle) of a file. You will overwrite existing content if you try this.
You can only add at the end of the file. If you want to insert
something, you have to rewrite everything from that position.

(So typically, for small files you wouldn't update a file in place, you
would just replace it completely. For large data sets which need to be
updated you would generally use some kind of database.)

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230219/1c259fed/attachment.sig>

From python at mrabarnett.plus.com  Sun Feb 19 12:18:54 2023
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 19 Feb 2023 17:18:54 +0000
Subject: File write, weird behaviour
In-Reply-To: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
Message-ID: <2cfc9aed-e4b2-7311-9687-c4dd2adf1270@mrabarnett.plus.com>

On 2023-02-19 14:03, Azizbek Khamdamov wrote:
> Example 1 (works as expected)
> 
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities
> # the following code adds new record to the beginning of the file,
> expected behaviour
> file.write("new city\n")
> 
> file.close()
> 
> 
> Example 2 (weird behaviour)
> 
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities
> # the following code DOES NOT add new record TO THE BEGINNING of the
> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
> new content should be added to the beginning of the file (as in
> Example 1)
> file.write("new city\n")
> 
> file.readlines()
> file.close()
> 
> I could not find anything in documentation to explain this strange
> behaviour. Why is this happening?

It works correctly if you use file.flush() or file.tell() before 
switching from writing to reading.

From list1 at tompassin.net  Sun Feb 19 12:59:43 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 19 Feb 2023 12:59:43 -0500
Subject: File write, weird behaviour
In-Reply-To: <e0d474c3-c792-0367-207c-20d5d1a9295d@declassed.art>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <e0d474c3-c792-0367-207c-20d5d1a9295d@declassed.art>
Message-ID: <6458f9a9-7b87-4e56-9db0-298a805d9f63@tompassin.net>

On 2/19/2023 11:57 AM, Axy via Python-list wrote:
> Looks like the data to be written is buffered, so actual write takes 
> place after readlines(), when close() flushes buffers.
> 
> See io package documentation, BufferedIOBase.
> 
> The solution is file.flush() after file.write()

Another possibility, from the Python docs:

"...TextIOWrapper (i.e., files opened with mode='r+') ... To disable 
buffering in TextIOWrapper, consider using the write_through flag for 
io.TextIOWrapper.reconfigure()"


Also from the docs:

"Warning: Calling f.write() without using the with keyword or calling 
f.close() might result in the arguments of f.write() not being 
completely written to the disk, even if the program exits successfully."

I realize that in the example, close() was called ... but not 
immediately after the write().

> Can't deny, such a behaviour looks utterly weird. Try to avoid designing 
> your software this way.
> 
> Axy.
> 
> On 19/02/2023 14:03, Azizbek Khamdamov wrote:
>> Example 1 (works as expected)
>>
>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> 'r+') ## contains list cities
>> # the following code adds new record to the beginning of the file,
>> expected behaviour
>> file.write("new city\n")
>>
>> file.close()
>>
>>
>> Example 2 (weird behaviour)
>>
>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> 'r+') ## contains list cities
>> # the following code DOES NOT add new record TO THE BEGINNING of the
>> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
>> new content should be added to the beginning of the file (as in
>> Example 1)
>> file.write("new city\n")
>>
>> file.readlines()
>> file.close()
>>
>> I could not find anything in documentation to explain this strange
>> behaviour. Why is this happening?


From rosuav at gmail.com  Sun Feb 19 13:53:25 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 20 Feb 2023 05:53:25 +1100
Subject: File write, weird behaviour
In-Reply-To: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
Message-ID: <CAPTjJmpsA7rr7N098SH2_-xKpUOqn4VKrMo-cwY2Ly0+y+0b0Q@mail.gmail.com>

On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
<azizbek.khamdamov at gmail.com> wrote:
>
> Example 1 (works as expected)
>
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities

Side note: You happened to get lucky with P, w, and c, but for the
future, I recommend using forward slashes for your paths:

open("D:/Programming/Python/working_with_files/cities.txt", "r+")

Otherwise, you may run into annoying and hard-to-solve problems. Or
alternatively, you'll upgrade to a newer Python and start getting
warnings, which would at least tell you that there's a problem.

ChrisA

From list1 at tompassin.net  Sun Feb 19 14:22:54 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 19 Feb 2023 14:22:54 -0500
Subject: File write, weird behaviour
In-Reply-To: <CAPTjJmpsA7rr7N098SH2_-xKpUOqn4VKrMo-cwY2Ly0+y+0b0Q@mail.gmail.com>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <CAPTjJmpsA7rr7N098SH2_-xKpUOqn4VKrMo-cwY2Ly0+y+0b0Q@mail.gmail.com>
Message-ID: <7e0ca67f-d47d-0a34-0663-5f887a0cd9b1@tompassin.net>

On 2/19/2023 1:53 PM, Chris Angelico wrote:
> On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
> <azizbek.khamdamov at gmail.com> wrote:
>>
>> Example 1 (works as expected)
>>
>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> 'r+') ## contains list cities
> 
> Side note: You happened to get lucky with P, w, and c, but for the
> future, I recommend using forward slashes for your paths:
> 
> open("D:/Programming/Python/working_with_files/cities.txt", "r+")
> 
> Otherwise, you may run into annoying and hard-to-solve problems. Or
> alternatively, you'll upgrade to a newer Python and start getting
> warnings, which would at least tell you that there's a problem.

Or use r'...' strings.  If you are copying a path to clipboard from 
Windows Explorer - a fairly common operation - it's much easier to 
prepend the "r" than to change all the backslashes to forward slashes.


From hjp-python at hjp.at  Sun Feb 19 14:29:31 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sun, 19 Feb 2023 20:29:31 +0100
Subject: File write, weird behaviour
In-Reply-To: <6458f9a9-7b87-4e56-9db0-298a805d9f63@tompassin.net>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <e0d474c3-c792-0367-207c-20d5d1a9295d@declassed.art>
 <6458f9a9-7b87-4e56-9db0-298a805d9f63@tompassin.net>
Message-ID: <20230219192931.qvlxvkgneuixliti@hjp.at>

On 2023-02-19 12:59:43 -0500, Thomas Passin wrote:
> On 2/19/2023 11:57 AM, Axy via Python-list wrote:
> > Looks like the data to be written is buffered, so actual write takes
> > place after readlines(), when close() flushes buffers.
> > 
> > See io package documentation, BufferedIOBase.
> > 
> > The solution is file.flush() after file.write()
> 
> Another possibility, from the Python docs:
> 
> "...TextIOWrapper (i.e., files opened with mode='r+') ... To disable
> buffering in TextIOWrapper, consider using the write_through flag for
> io.TextIOWrapper.reconfigure()"

That actually doesn't help (I tried it before writing my answer). The
binary layer below the text layer also buffers ...

> Also from the docs:
> 
> "Warning: Calling f.write() without using the with keyword or calling
> f.close() might result in the arguments of f.write() not being completely
> written to the disk, even if the program exits successfully."

He does call file.close():

> > > file.close()

so that doesn't seem relevant.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230219/179c35c6/attachment.sig>

From rosuav at gmail.com  Sun Feb 19 14:31:27 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 20 Feb 2023 06:31:27 +1100
Subject: File write, weird behaviour
In-Reply-To: <7e0ca67f-d47d-0a34-0663-5f887a0cd9b1@tompassin.net>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <CAPTjJmpsA7rr7N098SH2_-xKpUOqn4VKrMo-cwY2Ly0+y+0b0Q@mail.gmail.com>
 <7e0ca67f-d47d-0a34-0663-5f887a0cd9b1@tompassin.net>
Message-ID: <CAPTjJmrLCzQMHm94+ZEVqJxAGMpACfKhb5CmtFEaqQ_SpY9RKQ@mail.gmail.com>

On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1 at tompassin.net> wrote:
>
> On 2/19/2023 1:53 PM, Chris Angelico wrote:
> > On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
> > <azizbek.khamdamov at gmail.com> wrote:
> >>
> >> Example 1 (works as expected)
> >>
> >> file = open("D:\Programming\Python\working_with_files\cities.txt",
> >> 'r+') ## contains list cities
> >
> > Side note: You happened to get lucky with P, w, and c, but for the
> > future, I recommend using forward slashes for your paths:
> >
> > open("D:/Programming/Python/working_with_files/cities.txt", "r+")
> >
> > Otherwise, you may run into annoying and hard-to-solve problems. Or
> > alternatively, you'll upgrade to a newer Python and start getting
> > warnings, which would at least tell you that there's a problem.
>
> Or use r'...' strings.  If you are copying a path to clipboard from
> Windows Explorer - a fairly common operation - it's much easier to
> prepend the "r" than to change all the backslashes to forward slashes.
>

Yep, either way. I personally prefer using forward slashes since
there's no way to end an r-string with a single backslash, which is
often useful when building a path:

# won't work
path = r"c:\path\to\some\"
open(path + "file")

# will work
path = "c:/path/to/some/"
open(path + "file")

ChrisA

From python at mrabarnett.plus.com  Sun Feb 19 15:14:23 2023
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 19 Feb 2023 20:14:23 +0000
Subject: File write, weird behaviour
In-Reply-To: <CAPTjJmrLCzQMHm94+ZEVqJxAGMpACfKhb5CmtFEaqQ_SpY9RKQ@mail.gmail.com>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <CAPTjJmpsA7rr7N098SH2_-xKpUOqn4VKrMo-cwY2Ly0+y+0b0Q@mail.gmail.com>
 <7e0ca67f-d47d-0a34-0663-5f887a0cd9b1@tompassin.net>
 <CAPTjJmrLCzQMHm94+ZEVqJxAGMpACfKhb5CmtFEaqQ_SpY9RKQ@mail.gmail.com>
Message-ID: <98f6de54-8f5e-1f32-39f4-486a9c47f803@mrabarnett.plus.com>

On 2023-02-19 19:31, Chris Angelico wrote:
> On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1 at tompassin.net> wrote:
>>
>> On 2/19/2023 1:53 PM, Chris Angelico wrote:
>> > On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
>> > <azizbek.khamdamov at gmail.com> wrote:
>> >>
>> >> Example 1 (works as expected)
>> >>
>> >> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> >> 'r+') ## contains list cities
>> >
>> > Side note: You happened to get lucky with P, w, and c, but for the
>> > future, I recommend using forward slashes for your paths:
>> >
>> > open("D:/Programming/Python/working_with_files/cities.txt", "r+")
>> >
>> > Otherwise, you may run into annoying and hard-to-solve problems. Or
>> > alternatively, you'll upgrade to a newer Python and start getting
>> > warnings, which would at least tell you that there's a problem.
>>
>> Or use r'...' strings.  If you are copying a path to clipboard from
>> Windows Explorer - a fairly common operation - it's much easier to
>> prepend the "r" than to change all the backslashes to forward slashes.
>>
> 
> Yep, either way. I personally prefer using forward slashes since
> there's no way to end an r-string with a single backslash, which is
> often useful when building a path:
> 
> # won't work
> path = r"c:\path\to\some\"
> open(path + "file")
> 
> # will work
> path = "c:/path/to/some/"
> open(path + "file")
> 
When I build a path, I use os.path.join, so it's only a problem if one 
of the strings refers to the root of a drive, e.g. "C:\\".

From list1 at tompassin.net  Sun Feb 19 15:14:42 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 19 Feb 2023 15:14:42 -0500
Subject: File write, weird behaviour
In-Reply-To: <CAPTjJmrLCzQMHm94+ZEVqJxAGMpACfKhb5CmtFEaqQ_SpY9RKQ@mail.gmail.com>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <CAPTjJmpsA7rr7N098SH2_-xKpUOqn4VKrMo-cwY2Ly0+y+0b0Q@mail.gmail.com>
 <7e0ca67f-d47d-0a34-0663-5f887a0cd9b1@tompassin.net>
 <CAPTjJmrLCzQMHm94+ZEVqJxAGMpACfKhb5CmtFEaqQ_SpY9RKQ@mail.gmail.com>
Message-ID: <e33e6559-5f38-a4d4-0eb4-4ff2517f8cd9@tompassin.net>

On 2/19/2023 2:31 PM, Chris Angelico wrote:
> On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1 at tompassin.net> wrote:
>>
>> On 2/19/2023 1:53 PM, Chris Angelico wrote:
>>> On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
>>> <azizbek.khamdamov at gmail.com> wrote:
>>>>
>>>> Example 1 (works as expected)
>>>>
>>>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>>>> 'r+') ## contains list cities
>>>
>>> Side note: You happened to get lucky with P, w, and c, but for the
>>> future, I recommend using forward slashes for your paths:
>>>
>>> open("D:/Programming/Python/working_with_files/cities.txt", "r+")
>>>
>>> Otherwise, you may run into annoying and hard-to-solve problems. Or
>>> alternatively, you'll upgrade to a newer Python and start getting
>>> warnings, which would at least tell you that there's a problem.
>>
>> Or use r'...' strings.  If you are copying a path to clipboard from
>> Windows Explorer - a fairly common operation - it's much easier to
>> prepend the "r" than to change all the backslashes to forward slashes.
>>
> 
> Yep, either way. I personally prefer using forward slashes since
> there's no way to end an r-string with a single backslash, which is
> often useful when building a path:
> 
> # won't work
> path = r"c:\path\to\some\"
> open(path + "file")
> 
> # will work
> path = "c:/path/to/some/"
> open(path + "file")

Never hit that one before.  To be classified as "Learn something new 
every day"!

I've been using pathlib more lately.  It does clever things with "/" -

 >>> from pathlib import PurePath
 >>> p1 = PurePath('c:/this') / 'is' / 'a' /'test'
 >>> p1
PureWindowsPath('c:/this/is/a/test')
# on Linux, would be a PurePosixPath

and also

 >>> p2 = PurePath('c:/this/') /'is' /'a' /'test'
 >>> p2
PureWindowsPath('c:/this/is/a/test')

Despite the apparent forward slashes, the right separators get used for 
the OS. And the Paths can be used with open(), etc.



From dieter at handshake.de  Sun Feb 19 16:06:19 2023
From: dieter at handshake.de (Dieter Maurer)
Date: Sun, 19 Feb 2023 22:06:19 +0100
Subject: File write, weird behaviour
In-Reply-To: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
Message-ID: <25586.36683.279969.179776@ixdm.fritz.box>

Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:
> ...
>Example 2 (weird behaviour)
>
>file = open("D:\Programming\Python\working_with_files\cities.txt",
>'r+') ## contains list cities
># the following code DOES NOT add new record TO THE BEGINNING of the
>file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
>new content should be added to the beginning of the file (as in
>Example 1)
>file.write("new city\n")
>
>file.readlines()
>file.close()
>
>I could not find anything in documentation to explain this strange
>behaviour. Why is this happening?

The effect of "r+" (and friends) is specified by the C standard.

The Linux doc (of `fopen`) tells us that ANSI C requires that
a file positioning command (e.g. `seek`) must intervene
between input and output operations. Your example above violates
this condition. Therefore, weird behavior is to be expected.


From mats at wichmann.us  Sun Feb 19 18:10:06 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Sun, 19 Feb 2023 16:10:06 -0700
Subject: File write, weird behaviour
In-Reply-To: <25586.36683.279969.179776@ixdm.fritz.box>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <25586.36683.279969.179776@ixdm.fritz.box>
Message-ID: <65867cad-f45f-6cb5-1e14-6f1a96a4f423@wichmann.us>

On 2/19/23 14:06, Dieter Maurer wrote:
> Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:
>> ...
>> Example 2 (weird behaviour)
>>
>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> 'r+') ## contains list cities
>> # the following code DOES NOT add new record TO THE BEGINNING of the
>> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
>> new content should be added to the beginning of the file (as in
>> Example 1)
>> file.write("new city\n")
>>
>> file.readlines()
>> file.close()
>>
>> I could not find anything in documentation to explain this strange
>> behaviour. Why is this happening?
> 
> The effect of "r+" (and friends) is specified by the C standard.
> 
> The Linux doc (of `fopen`) tells us that ANSI C requires that
> a file positioning command (e.g. `seek`) must intervene
> between input and output operations. Your example above violates
> this condition. Therefore, weird behavior is to be expected.

If this isn't sufficiently described, someone should raise an issue 
against the Python docs.  I know that many concepts are "inherited from" 
environments generally in the POSIX space and the C language, because 
that's where Python was hatched (all of which makes perfect sense to me, 
who's been working in those spaces for...ever), but a Python programmer 
shouldn't have to read the ISO C standard (which is not free, although 
you can find copies on-line), or the POSIX standard (which also is not 
free, though manpages for systems like Linux cover the same material), 
in order to figure out how Python is going to work.

From list1 at tompassin.net  Sun Feb 19 21:13:45 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 19 Feb 2023 21:13:45 -0500
Subject: File write, weird behaviour
In-Reply-To: <65867cad-f45f-6cb5-1e14-6f1a96a4f423@wichmann.us>
References: <CAN14BN5=0sL-d8HQ4hyAYLicvGP4TzhbR=WEr6Ev-tV2TPJqWQ@mail.gmail.com>
 <25586.36683.279969.179776@ixdm.fritz.box>
 <65867cad-f45f-6cb5-1e14-6f1a96a4f423@wichmann.us>
Message-ID: <0d6a21f3-c88c-7256-b9cd-f1ef6209c0ff@tompassin.net>

On 2/19/2023 6:10 PM, Mats Wichmann wrote:
> On 2/19/23 14:06, Dieter Maurer wrote:
>> Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:
>>> ...
>>> Example 2 (weird behaviour)
>>>
>>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>>> 'r+') ## contains list cities
>>> # the following code DOES NOT add new record TO THE BEGINNING of the
>>> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
>>> new content should be added to the beginning of the file (as in
>>> Example 1)
>>> file.write("new city\n")
>>>
>>> file.readlines()
>>> file.close()
>>>
>>> I could not find anything in documentation to explain this strange
>>> behaviour. Why is this happening?
>>
>> The effect of "r+" (and friends) is specified by the C standard.
>>
>> The Linux doc (of `fopen`) tells us that ANSI C requires that
>> a file positioning command (e.g. `seek`) must intervene
>> between input and output operations. Your example above violates
>> this condition. Therefore, weird behavior is to be expected.
> 
> If this isn't sufficiently described, someone should raise an issue 
> against the Python docs.? I know that many concepts are "inherited from" 
> environments generally in the POSIX space and the C language, because 
> that's where Python was hatched (all of which makes perfect sense to me, 
> who's been working in those spaces for...ever), but a Python programmer 
> shouldn't have to read the ISO C standard (which is not free, although 
> you can find copies on-line), or the POSIX standard (which also is not 
> free, though manpages for systems like Linux cover the same material), 
> in order to figure out how Python is going to work.

The Python docs say that text file behavior is done by Python, not C, 
and so its behavior will be consistent across all platforms.  I didn't 
find any words about this case, though.

From nntp.mbourne at spamgourmet.com  Mon Feb 20 15:39:22 2023
From: nntp.mbourne at spamgourmet.com (Mark Bourne)
Date: Mon, 20 Feb 2023 20:39:22 +0000
Subject: Add angle brackets for required args in argparse
In-Reply-To: <mailman.1832.1676481401.20444.python-list@python.org>
References: <OSZP286MB0742B742ED736B07B5665A9DA5DD9@OSZP286MB0742.JPNP286.PROD.OUTLOOK.COM>
 <OS3P286MB0740CCCB1D94B91E01BB4B79A5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>
 <mailman.1832.1676481401.20444.python-list@python.org>
Message-ID: <tt0lpr$s0oc$1@dont-email.me>

scruel tao wrote:
> If we have the following code:
> ```
> parser = argparse.ArgumentParser(description="test")
> parser.add_argument('path')
> ```
> 
> Run it without args, will get error message:
> ```
> usage: test.py [-h] path
> test.py: error: the following arguments are required: path
> ```
> 
> However, I hope the message can be as the following:
> ```
> usage: test.py [-h] <path>
> test.py: error: the following arguments are required: path
> ```

The `metavar` argument to `add_argument` can be used to control how an 
argument is represented in the usage text:
```
import argparse
parser = argparse.ArgumentParser(description='test')
parser.add_argument('path', metavar='<path>')
parser.parse_args()
```

Which results in:
```
usage: test.py [-h] <path>
test.py: error: the following arguments are required: <path>
```

> Or might can consider to provide a way to let user have there own style, like:
> ```
> usage: test.py [-h] path
> ```

It's also possible to create a custom help formatter, overriding 
appropriate methods to control the formatting.  For example:
```
import argparse

class CustomHelpFormatter(argparse.HelpFormatter):
     def _get_default_metavar_for_positional(self, action):
         default = super()._get_default_metavar_for_positional(action)
         return f'<{default}>'

parser = argparse.ArgumentParser(
     description='test',
     formatter_class=CustomHelpFormatter)
parser.add_argument('path')
parser.parse_args()
```

Which results in:
```
usage: test.py [-h] <path>
test.py: error: the following arguments are required: path
```

That's a bit closer to what you asked for, since the required argument 
shown in the error message doesn't include the angle brackets.  It also 
avoids needing to specify a `metavar` for every positional argument. 
However, it is overriding a non-public method of the `HelpFormatter` 
class, so might not work across all Python versions if the name or 
signature of that method changes (even if it does work with all current 
versions, it might break in future).

-- 
Mark.

From henhanna at gmail.com  Mon Feb 20 20:01:33 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Mon, 20 Feb 2023 17:01:33 -0800 (PST)
Subject: why is a search thru a Tuple slower ? ---- (meaningless indentations)
Message-ID: <bde43164-bc89-455e-9c7b-22e4cd22f033n@googlegroups.com>

is Comp.Lang.Python very active????


why is a linear search  thru  a Tuple  slower 
                             (than  thru a  (corresponding)   List ) ???


sometimes,  i 'd  like to put  meaningless indentations
            like  i do (twice)  below
                                     ( how can it do this ?)

_______________________________

import time

X= [x for x in range(10000) ]
              TupX= tuple(X)
              SetX= set(X)

Items=[20, 30, 100, 200, 1000,   5000,    7000,   8000, 9000 ]
Count=1000

Time = time.time() 
for _ in range(Count):
    for i in X:    test= i*i; test= i*i *i
print( ' \t ({0:.4f})  X'.format(time.time()-Time) )

Time = time.time() 
for _ in range(Count):
    for i in TupX:    test= i*i; test= i*i *i
print( ' \t ({0:.4f})  TupX'.format(time.time()-Time) )



Time = time.time() 
for _ in range(Count):
    for i in Items:   test= i in X
print( ' \t ({0:.4f}) i in X'.format(time.time()-Time) )

Time = time.time() 
for _ in range(Count):
    for i in Items:   test= i in TupX
print( ' \t ({0:.4f}) i in TupX'.format(time.time()-Time) )


From henhanna at gmail.com  Mon Feb 20 20:06:26 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Mon, 20 Feb 2023 17:06:26 -0800 (PST)
Subject: is [comprehension]   the right word???
Message-ID: <fa01817b-3f0d-4653-a37a-ac405f65a400n@googlegroups.com>

is [comprehension]   the right word???

i swear  i never heard the word  before
          getting into Python  a few years ago.


What was it called in the Lisp (Scheme) world ????

ok...  in Common Lisp,  (and MacLisp)   it was  Loop and Collect

https://en.wikipedia.org/wiki/List_comprehension#History



From henhanna at gmail.com  Mon Feb 20 21:23:56 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Mon, 20 Feb 2023 18:23:56 -0800 (PST)
Subject: is [comprehension] the right word???
In-Reply-To: <87ttzfu6z4.fsf@nightsong.com>
References: <fa01817b-3f0d-4653-a37a-ac405f65a400n@googlegroups.com>
 <87ttzfu6z4.fsf@nightsong.com>
Message-ID: <07674cfa-ec04-4bc6-ab24-4c8620c0619cn@googlegroups.com>

On Monday, February 20, 2023 at 5:45:39 PM UTC-8, Paul Rubin wrote:
> Hen Hanna <henh... at gmail.com> writes: 
> > is [comprehension] the right word???
> Yes, it comes from math, particularly set theory. An expression like 
> 
> { n | n:integer, n mod 2 = 0 } 
> 
> is called a set comprehension, and then one there denotes the set of all 
> even integers. Axioms saying that the above denotes a legitimate set 
> are called comprehension axioms. In ZFC (an axiomitization of set 
> theory widely used in math), there is an infinite schema of such axioms. 
> 
> The Haskell language used a notation inspired by this for "list 
> comprehensions", and Python list (and later dictionary etc.) 
> comprehensions were inspired by Haskell's version.


thanks...   my curiosity was   re-aroused today
                             when i learned that....

In Italian .... they say:     [compresi]   as in...


                  Ho 5 libri di Eco, compresi quei 3 che vedete l?.

                  Ho 5 libri di Eco, inclusi 3 che non sono disponibili nella traduzione giapponese.




From henhanna at gmail.com  Mon Feb 20 22:36:15 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Mon, 20 Feb 2023 19:36:15 -0800 (PST)
Subject: Tuple   Comprehension ???
Message-ID: <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>

For a while,  i've been curious about a  [Tuple   Comprehension] 

So  finally   i tried it, and the result was a bit surprising...


X= [ x for x in range(10) ]
X= ( x for x in range(10) )
print(X)
a= list(X)
print(a)




From torriem at gmail.com  Mon Feb 20 22:56:46 2023
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 20 Feb 2023 20:56:46 -0700
Subject: Tuple Comprehension ???
In-Reply-To: <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
References: <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
Message-ID: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>

On 2/20/23 20:36, Hen Hanna wrote:
> For a while,  i've been curious about a  [Tuple   Comprehension] 

I've never heard of a "Tuple comprehension."  No such thing exists as
far as I know.

> So  finally   i tried it, and the result was a bit surprising...
> 
> 
> X= [ x for x in range(10) ]
> X= ( x for x in range(10) )
> print(X)
> a= list(X)
> print(a)

What was surprising? Don't keep us in suspense!

Using square brackets is a list comprehension. Using parenthesis creates
a generator expression. It is not a tuple. A generator expression can be
perhaps thought of as a lazy list.  Instead of computing each member
ahead of time, it returns a generator object which, when iterated over,
produces the members one at a time.  This can be a tremendous
optimization in terms of resource usage.  See
https://docs.python.org/3/reference/expressions.html#generator-expressions.
 Also you can search google for "generator expression" for other examples.


From torriem at gmail.com  Mon Feb 20 23:17:35 2023
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 20 Feb 2023 21:17:35 -0700
Subject: is [comprehension] the right word???
In-Reply-To: <fa01817b-3f0d-4653-a37a-ac405f65a400n@googlegroups.com>
References: <fa01817b-3f0d-4653-a37a-ac405f65a400n@googlegroups.com>
Message-ID: <23a7b155-968a-dd98-d93b-e3fe6b8fef46@gmail.com>

On 2/20/23 18:06, Hen Hanna wrote:
> is [comprehension]   the right word???
>
> i swear  i never heard the word  before
>           getting into Python  a few years ago.

Seems as though the term was borrowed from formal mathematics set theory.

A simple search reveals that the term "list comprehension" predates
Python.  Back to 1977 to be exact. The term was first coined by Phil
Wadler in the late 70s or early 80s.

https://en.wikipedia.org/wiki/List_comprehension

From torriem at gmail.com  Mon Feb 20 23:08:23 2023
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 20 Feb 2023 21:08:23 -0700
Subject: why is a search thru a Tuple slower ? ---- (meaningless
 indentations)
In-Reply-To: <bde43164-bc89-455e-9c7b-22e4cd22f033n@googlegroups.com>
References: <bde43164-bc89-455e-9c7b-22e4cd22f033n@googlegroups.com>
Message-ID: <59245355-5daf-5e4c-4a59-25f0401a3d60@gmail.com>

On 2/20/23 18:01, Hen Hanna wrote:
> is Comp.Lang.Python very active????
Fairly.  Apparently the cool kids are using the Python Discourse forum.

> why is a linear search  thru  a Tuple  slower 
>                              (than  thru a  (corresponding)   List ) ???

I cannot say, unfortunately.  Perhaps doing some analysis of the byte
code with the disasm module could tell you what the interpreter is doing
and why it is slower.  Since tuples are read only, I cannot think of any
reason to use them for large, generated structures. A list is far better
in my opinion.  I use tuples mainly for bundling small amounts of
information together, such as coordinates, or returning multiple values
from a function.

> sometimes,  i 'd  like to put  meaningless indentations
>             like  i do (twice)  below
>                                      ( how can it do this ?)

Fortunately you cannot.  Such indents are syntax errors.

And I have to say it makes your emails very hard to read and understand
when you indent your sentences as you do.  Looks poetic but hard to read.

Also your python example code was not run-able either thanks to those
two extra indents which are syntax errors.  It's always helpful to post
complete and working code examples when asking for help or wanting to
get discussion on a piece of code.

From avi.e.gross at gmail.com  Tue Feb 21 01:23:31 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 21 Feb 2023 01:23:31 -0500
Subject: Tuple Comprehension ???
In-Reply-To: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
References: <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
Message-ID: <004901d945bd$054eb500$0fec1f00$@gmail.com>

Tuples are immutable and sort of have to be created all at once. This does
not jive well wth being made incrementally in a comprehension. And, as
noted, the use of parentheses I too many contexts means that what looks like
a comprehension in parentheses is used instead as a generator.

If you really want a tuple made using a comprehension, you have some options
that are indirect.

One is to create a list using the comprehension and copy/convert that into a
tuple as in:

mytuple = tuple( [x for x in range(10) ] )

I think an alternative is to use a generator in a similar way that keeps
being iterated till done.

mytuple = tuple( (x for x in range(10) ) )

And similarly, you can use a set comprehension and convert that to a tuple
but only if nothing is repeated and perhaps order does not matter, albeit in
recent python versions, I think it remains ordered by insertion order!

mytuple = tuple( {x for x in range(10) } )

There are other more obscure and weird ways, of course but generally no
need.

Realistically, in many contexts, you do not have to store or use things in
tuples, albeit some sticklers think it is a good idea to use a tuple when
you want to make clear the data is to be immutable. There can be other
benefits such as storage space used. And in many ways, tuples are supposed
to be faster than lists.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Michael Torrie
Sent: Monday, February 20, 2023 10:57 PM
To: python-list at python.org
Subject: Re: Tuple Comprehension ???

On 2/20/23 20:36, Hen Hanna wrote:
> For a while,  i've been curious about a  [Tuple   Comprehension] 

I've never heard of a "Tuple comprehension."  No such thing exists as far as
I know.

> So  finally   i tried it, and the result was a bit surprising...
> 
> 
> X= [ x for x in range(10) ]
> X= ( x for x in range(10) )
> print(X)
> a= list(X)
> print(a)

What was surprising? Don't keep us in suspense!

Using square brackets is a list comprehension. Using parenthesis creates a
generator expression. It is not a tuple. A generator expression can be
perhaps thought of as a lazy list.  Instead of computing each member ahead
of time, it returns a generator object which, when iterated over, produces
the members one at a time.  This can be a tremendous optimization in terms
of resource usage.  See
https://docs.python.org/3/reference/expressions.html#generator-expressions.
 Also you can search google for "generator expression" for other examples.

--
https://mail.python.org/mailman/listinfo/python-list


From cs at cskk.id.au  Tue Feb 21 01:51:50 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 21 Feb 2023 17:51:50 +1100
Subject: Tuple   Comprehension ???
In-Reply-To: <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
References: <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
Message-ID: <Y/RqBgAsZfM6XaAP@cskk.homeip.net>

On 20Feb2023 19:36, Hen Hanna <henhanna at gmail.com> wrote:
>For a while,  i've been curious about a  [Tuple   Comprehension]
>
>So  finally   i tried it, and the result was a bit surprising...
>
>X= [ x for x in range(10) ]

This is a list comprehension, resulting in a list as its result.

>X= ( x for x in range(10) )

This is not a tuple comprehension - Python does not have one.

Instead, it is a generator expression:

     x for x in range(10)

inside some brackets, which are just group as you might find in an 
expression like:

     (3 + 4) * 7

If you want a tuple, you need to write:

     X = tuple( x for x in range(10) )

which makes a tuple from an iterable (such as a list, but anything 
iterable will do). Here the iterable is the generator expression:

     x for x in range(10)

Cheers,
Cameron Simpson <cs at cskk.id.au>

From henhanna at gmail.com  Mon Feb 20 23:13:59 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Mon, 20 Feb 2023 20:13:59 -0800 (PST)
Subject: Tuple Comprehension ???
In-Reply-To: <mailman.1895.1676951812.20444.python-list@python.org>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
Message-ID: <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>

On Monday, February 20, 2023 at 7:57:14 PM UTC-8, Michael Torrie wrote:
> On 2/20/23 20:36, Hen Hanna wrote: 
> > For a while, i've been curious about a [Tuple Comprehension]
> I've never heard of a "Tuple comprehension." No such thing exists as 
> far as I know.
> > So finally i tried it, and the result was a bit surprising... 
> > 
> > 
> > X= [ x for x in range(10) ] 
> > X= ( x for x in range(10) ) 
> > print(X) 
> > a= list(X) 
> > print(a)


> What was surprising? Don't keep us in suspense! 
> 
> Using square brackets is a list comprehension. Using parenthesis creates 
> a generator expression. It is not a tuple. 

ok!



    LisX= [x for x in range(10) ]

    print( sum( LisX ))
    print( max( LisX ))

    print( sum( x for x in range(10) ) )
    print( max( x for x in range(10) ) )

    print( * LisX )

    print( max( * LisX ))
    print( sum( LisX ))        # same as before
    # print( sum( * LisX )) <------- Bad syntax !!!

                       TypeError: sum() takes at most 2 arguments (10 given)


_____________________

                (A)   print( max( * LisX ))
                (B)   print( sum( * LisX ))        <------- Bad syntax !!!

What's most surprising is....     (A)  is ok, and  (B) is not.

           even tho'   max() and sum()  have   (basically)  the same syntax...  ( takes one arg ,  whch is a list )



i've been programming for many years...        ( just knew to Python )

From axy at declassed.art  Tue Feb 21 12:32:48 2023
From: axy at declassed.art (Axy)
Date: Tue, 21 Feb 2023 17:32:48 +0000
Subject: Tuple Comprehension ???
In-Reply-To: <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
Message-ID: <2301b1f7-02bd-3bae-b93e-90a28bcc0372@declassed.art>

On 21/02/2023 04:13, Hen Hanna wrote:
>
>                  (A)   print( max( * LisX ))
>                  (B)   print( sum( * LisX ))        <------- Bad syntax !!!
>
> What's most surprising is....     (A)  is ok, and  (B) is not.
>
>             even tho'   max() and sum()  have   (basically)  the same syntax...  ( takes one arg ,  whch is a list )
>
>
>
> i've been programming for many years...        ( just knew to Python )

LOL, python is full of surprises. I'd definitely step into the same 
piece of... Someday.

Of course 'Builtin functions' section explains that, but the 
inconsistency is weird.

My response is absolutely useless, just two cents on the issue. Maybe 
someone will fix that.

Axy.

From avi.e.gross at gmail.com  Tue Feb 21 12:41:07 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 21 Feb 2023 12:41:07 -0500
Subject: Tuple Comprehension ???
In-Reply-To: <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
Message-ID: <008a01d9461b$ae22fc50$0a68f4f0$@gmail.com>

There is a very common misunderstanding by people learning python that a
tuple has something to do with parentheses. It confused me too at first.

A tuple is made by the use of one or more commas and no parentheses are
needed except when, like everything else, they are used for grouping as in
the arithmetic for 

  (5 + 4) * 3

So (6) is not a tuple while a trailing comma makes (6,) to be a tuple with
one entry.

A tad confusingly is that () by itself is a tuple, containing nothing. While
(,) is a syntax error!

A serious design issue in most computer languages is that there are too few
unique symbols to go around and some get re-used in multiple ways that
usually are not ambiguous when viewed in context. As an example, sets and
dictionaries both use curly braces but {} by itself is considered ambiguous
and they chose to make it be an empty dictionary. To get an empty set, use
set() instead. Parentheses are way overused and thus it gets murky at times
as when they are used to sort of make it clear you are using a generator.

Consider how this fails without parentheses:

result = x*2 for x in [1,2,3]
SyntaxError: invalid syntax

But with parentheses works fine:

result = (x*2 for x in [1,2,3])
result
<generator object <genexpr> at 0x0000029A3CFCF030>

However if you want a generator that is expanded into a list, you do not
need the parentheses duplicated like this:

result = list( (x*2 for x in [1,2,3]) )

and can just use this without nested parentheses:

result = list(  x*2 for x in [1,2,3]  )

For completeness, you arguably should have a concept of a comprehension for
every possible case but the people at python chose not to for reasons like
the above and especially as it is fairly simple to use this version:

result = tuple(  x*2 for x in [1,2,3]  )

Yes, it is a tad indirect and requires making a generator first.






-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Hen Hanna
Sent: Monday, February 20, 2023 11:14 PM
To: python-list at python.org
Subject: Re: Tuple Comprehension ???

On Monday, February 20, 2023 at 7:57:14 PM UTC-8, Michael Torrie wrote:
> On 2/20/23 20:36, Hen Hanna wrote: 
> > For a while, i've been curious about a [Tuple Comprehension]
> I've never heard of a "Tuple comprehension." No such thing exists as 
> far as I know.
> > So finally i tried it, and the result was a bit surprising... 
> > 
> > 
> > X= [ x for x in range(10) ]
> > X= ( x for x in range(10) )
> > print(X)
> > a= list(X)
> > print(a)


> What was surprising? Don't keep us in suspense! 
> 
> Using square brackets is a list comprehension. Using parenthesis 
> creates a generator expression. It is not a tuple.

ok!



    LisX= [x for x in range(10) ]

    print( sum( LisX ))
    print( max( LisX ))

    print( sum( x for x in range(10) ) )
    print( max( x for x in range(10) ) )

    print( * LisX )

    print( max( * LisX ))
    print( sum( LisX ))        # same as before
    # print( sum( * LisX )) <------- Bad syntax !!!

                       TypeError: sum() takes at most 2 arguments (10 given)


_____________________

                (A)   print( max( * LisX ))
                (B)   print( sum( * LisX ))        <------- Bad syntax !!!

What's most surprising is....     (A)  is ok, and  (B) is not.

           even tho'   max() and sum()  have   (basically)  the same
syntax...  ( takes one arg ,  whch is a list )



i've been programming for many years...        ( just knew to Python )
--
https://mail.python.org/mailman/listinfo/python-list


From roel at roelschroeven.net  Tue Feb 21 13:11:07 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Tue, 21 Feb 2023 19:11:07 +0100
Subject: Tuple Comprehension ???
In-Reply-To: <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
Message-ID: <57e68c55-a025-7ae4-d1b4-301622f334f0@roelschroeven.net>

Hen Hanna schreef op 21/02/2023 om 5:13:
>                  (A)   print( max( * LisX ))
>                  (B)   print( sum( * LisX ))        <------- Bad syntax !!!
>
> What's most surprising is....     (A)  is ok, and  (B) is not.
>
>             even tho'   max() and sum()  have   (basically)  the same syntax...  ( takes one arg ,  whch is a list )
>
There's an important difference in syntax.

sum() takes an iterable:

sum(iterable, /, start=0)
 ??? Return the sum of a 'start' value (default: 0) plus an iterable of 
numbers

 ??? When the iterable is empty, return the start value.
 ??? This function is intended specifically for use with numeric values 
and may
 ??? reject non-numeric types.

max() on the other hand takes either an iterable or a number of 
individual elements:

max(...)
 ??? max(iterable, *[, default=obj, key=func]) -> value
 ??? max(arg1, arg2, *args, *[, key=func]) -> value

 ??? With a single iterable argument, return its biggest item. The
 ??? default keyword-only argument specifies an object to return if
 ??? the provided iterable is empty.
 ??? With two or more arguments, return the largest argument.

That second form of max is why max(*some_list) works while 
sum(*some_list) doesn't.

-- 
"You can fool some of the people all the time, and all of the people some
of the time, but you cannot fool all of the people all of the time."
         -- Abraham Lincoln
"You can fool too many of the people too much of the time."
         -- James Thurber


From list1 at tompassin.net  Tue Feb 21 13:22:37 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 21 Feb 2023 13:22:37 -0500
Subject: Tuple Comprehension ???
In-Reply-To: <2301b1f7-02bd-3bae-b93e-90a28bcc0372@declassed.art>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
 <2301b1f7-02bd-3bae-b93e-90a28bcc0372@declassed.art>
Message-ID: <ee16e4f9-3f40-d1d2-3d67-7ab0e27d3648@tompassin.net>

On 2/21/2023 12:32 PM, Axy via Python-list wrote:
> On 21/02/2023 04:13, Hen Hanna wrote:
>>
>> ???????????????? (A)?? print( max( * LisX ))
>> ???????????????? (B)?? print( sum( * LisX ))??????? <------- Bad 
>> syntax !!!
>>
>> What's most surprising is....???? (A)? is ok, and? (B) is not.
>>
>> ??????????? even tho'?? max() and sum()? have?? (basically)? the same 
>> syntax...? ( takes one arg ,? whch is a list )

They **don't** have basically the same signature, though.  max() takes 
either an iterable or two or more numbers.  Using max(*list_) presents 
it with a series of numbers, so that's OK.

sum() takes just one iterable (plus an optional start index).  Using 
sum(*list_) presents it with a series of numbers, and that does not 
match its signature.

Check what I said:

 >>> help(sum)
Help on built-in function sum in module builtins:

sum(iterable, /, start=0)
     Return the sum of a 'start' value (default: 0) plus an iterable of 
numbers

 >>> help(max)
Help on built-in function max in module builtins:

max(...)
     max(iterable, *[, default=obj, key=func]) -> value
     max(arg1, arg2, *args, *[, key=func]) -> value

Why they have different signatures may be lost to the whims of history 
and backwards compatibility...

>>
>>
>>
>> i've been programming for many years...??????? ( just knew to Python )
> 
> LOL, python is full of surprises. I'd definitely step into the same 
> piece of... Someday.
> 
> Of course 'Builtin functions' section explains that, but the 
> inconsistency is weird.
> 
> My response is absolutely useless, just two cents on the issue. Maybe 
> someone will fix that.
> 
> Axy.


From avi.e.gross at gmail.com  Tue Feb 21 14:11:38 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 21 Feb 2023 14:11:38 -0500
Subject: Tuple Comprehension ???
In-Reply-To: <57e68c55-a025-7ae4-d1b4-301622f334f0@roelschroeven.net>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
 <57e68c55-a025-7ae4-d1b4-301622f334f0@roelschroeven.net>
Message-ID: <009401d94628$52d5cdc0$f8816940$@gmail.com>

There are limits to anyone arguing for designs to be the way they want or expect and Roel has explained this one below.

When it comes to designing a function, lots of rules people expect are beyond irrelevant. Many functions can be implemented truly hundreds of ways with varying numbers of arguments and defaults and other effects. I can make a function that raises the first argument to a power specified in the second argument with no defaults and you get a syntax error for calling it with one argument or more than two arguments. Or I can make the second argument use a keyword with a default of 1, or 2 or whatever I wish and it can now be called with one argument to get the default or two but not more. Or, I can have the function absorb all additional arguments and ignore them or even use them as additional powers to be raised to so pow(2, 3, 4, 5) returns a tuple or list of 8, 16, 32.  Or maybe not and it would return ((2^3)^4)^5 or any other nonsense you design.

There IS NO CONSISTENCY possible in many cases unless you make a family of similarly named functions and add some thing to each name to make it clear.

Python arguably is harder than some languages in this regard as it allows way more flexibility. If a function accepts an iterator, and another does not, the call may superficially looks the same but is not. 

So, yes, max() could have been designed differently and you can even design your own mymax() and mysum() to check the arguments they receive and re-arrange them in a way that lets you call the original max/sum functions potentially in the same ways. 

But as a general rule, when using a function, don't GUESS what it does or infer what it does and then complain when someone says you should have read the manual. There are too many design choices, often done by different programmers and often motivated by ideas like efficiency. You likely can not guess many of them.

And lots of python functions you write can make use of all kinds of features such as caching results of previous computations or holding on to variables such as what you asked for last time so it can be used as a default. If I write a function like go(direction=something, distance=something) then perhaps my design will remember the last time it was invoked and if you call it again with no arguments, it may repeat the same action, or if only one is given, the other is repeated. But on a first call, it may fail as it has no memory yet of what you did. That may be intuitive to some and not others, but would it make as much sense for another function to be designed the same way so it tolerates being called with no arguments when this makes less sense? Do I often want to call for sin(x) and later for just sin() and expect it to mean that it be repeated?

But back to the original question about max/sum it gets weirder. Although max() takes any number of arguments, it really doesn't. There is no way to get the maximum of a single argument as in max(5) because it is designed to EITHER take one iterable OR more than one regular argument. 

So one case that normally fails is max([]) or any empty iterable and you can keep it from failing with something like max([], default=0) .

In your own code, you may want to either design your own functions, or use them as documented or perhaps create your own wrapper functions that carefully examine what you ask them to do and re-arrange as needed to call the function(s) you want as needed or return their own values or better error messages.  As a silly example, this fails:

max(1, "hello")

Max expects all arguments to be of compatible types. You could write your own function called charMax() that converts all arguments to be of type str before calling max() or maybe call max(... , key=mycompare) where compare as a function handles this case well.

The key point is that you need to adapt yourself to what some function you want to use offers, not expect the language to flip around at this point and start doing it your way and probably breaking many existing programs.

Yes, consistency is a good goal. Reality is a better goal.




-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Roel Schroeven
Sent: Tuesday, February 21, 2023 1:11 PM
To: python-list at python.org
Subject: Re: Tuple Comprehension ???

Hen Hanna schreef op 21/02/2023 om 5:13:
>                  (A)   print( max( * LisX ))
>                  (B)   print( sum( * LisX ))        <------- Bad syntax !!!
>
> What's most surprising is....     (A)  is ok, and  (B) is not.
>
>             even tho'   max() and sum()  have   (basically)  the same syntax...  ( takes one arg ,  whch is a list )
>
There's an important difference in syntax.

sum() takes an iterable:

sum(iterable, /, start=0)
     Return the sum of a 'start' value (default: 0) plus an iterable of numbers

     When the iterable is empty, return the start value.
     This function is intended specifically for use with numeric values and may
     reject non-numeric types.

max() on the other hand takes either an iterable or a number of individual elements:

max(...)
     max(iterable, *[, default=obj, key=func]) -> value
     max(arg1, arg2, *args, *[, key=func]) -> value

     With a single iterable argument, return its biggest item. The
     default keyword-only argument specifies an object to return if
     the provided iterable is empty.
     With two or more arguments, return the largest argument.

That second form of max is why max(*some_list) works while
sum(*some_list) doesn't.

--
"You can fool some of the people all the time, and all of the people some of the time, but you cannot fool all of the people all of the time."
         -- Abraham Lincoln
"You can fool too many of the people too much of the time."
         -- James Thurber

--
https://mail.python.org/mailman/listinfo/python-list


From henhanna at gmail.com  Tue Feb 21 13:18:48 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Tue, 21 Feb 2023 10:18:48 -0800 (PST)
Subject: Tuple Comprehension ???
In-Reply-To: <mailman.1900.1677000789.20444.python-list@python.org>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <2301b1f7-02bd-3bae-b93e-90a28bcc0372@declassed.art>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
 <mailman.1900.1677000789.20444.python-list@python.org>
Message-ID: <b6e6052a-e29d-4867-862c-05b8a709b722n@googlegroups.com>

On Tuesday, February 21, 2023 at 9:33:29 AM UTC-8, Axy wrote:
> On 21/02/2023 04:13, Hen Hanna wrote: 
> > 
> > (A) print( max( * LisX )) 
> > (B) print( sum( * LisX )) <------- Bad syntax !!! 
> > 
> > What's most surprising is.... (A) is ok, and (B) is not. 
> > 
> > even tho' max() and sum() have (basically) the same syntax... ( takes one arg , whch is a list ) 
> > 
> > 
> > 
> >           i've been programming for many years... ( just  new to Python )



> LOL, python is full of surprises. I'd definitely step into the same 
> piece of... Someday. 
> 
> Of course 'Builtin functions' section explains that, but the 
> inconsistency is weird. 
> 
> My response is absolutely useless, just two cents on the issue. Maybe 
> someone will fix that. 
> 
> Axy.


i'm glad you get it         ( that the inconsistency is weird. )

        (1)     print(1, sum( [1,2,3,4] ))
        (2)     print(2, max( [1,2,3,4] ))

        (3)   print(3, sum( * [1,2,3,4] ))
        (4)   print(4, max( * [1,2,3,4] ))

both 3,4 should be good    OR
both 3,4 should be bad. ------------ that's what i think!

ok....  i thnk i finally got it...   (just before  seeing  Roel Schroeven's  msg)


From axy at declassed.art  Tue Feb 21 14:37:22 2023
From: axy at declassed.art (Axy)
Date: Tue, 21 Feb 2023 19:37:22 +0000
Subject: Tuple Comprehension ???
In-Reply-To: <009401d94628$52d5cdc0$f8816940$@gmail.com>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
 <57e68c55-a025-7ae4-d1b4-301622f334f0@roelschroeven.net>
 <009401d94628$52d5cdc0$f8816940$@gmail.com>
Message-ID: <ea37ef24-13ba-b6de-0a1b-4d6d0526bc69@declassed.art>

On 21/02/2023 19:11, avi.e.gross at gmail.com wrote:
> In your own code, you may want to either design your own functions, or use them as documented or perhaps create your own wrapper functions that carefully examine what you ask them to do and re-arrange as needed to call the function(s) you want as needed or return their own values or better error messages.  As a silly example, this fails:
>
> max(1, "hello")
>
> Max expects all arguments to be of compatible types. You could write your own function called charMax() that converts all arguments to be of type str before calling max() or maybe call max(... , key=mycompare) where compare as a function handles this case well.
>
> The key point is that you need to adapt yourself to what some function you want to use offers, not expect the language to flip around at this point and start doing it your way and probably breaking many existing programs.
>
> Yes, consistency is a good goal. Reality is a better goal.

I don't think overengineering is a good thing. Good design utilizes 
associativity so a person don't get amazed by inconsistency in things 
that expected to be similar.

Axy.


From avi.e.gross at gmail.com  Tue Feb 21 16:09:04 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 21 Feb 2023 16:09:04 -0500
Subject: Tuple Comprehension ???
In-Reply-To: <ea37ef24-13ba-b6de-0a1b-4d6d0526bc69@declassed.art>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
 <57e68c55-a025-7ae4-d1b4-301622f334f0@roelschroeven.net>
 <009401d94628$52d5cdc0$f8816940$@gmail.com>
 <ea37ef24-13ba-b6de-0a1b-4d6d0526bc69@declassed.art>
Message-ID: <00ba01d94638$baeb5c80$30c21580$@gmail.com>

Axy,

Nobody denies some of the many ways you can make a good design. But people
have different priorities that include not just conflicts between elements
of a design but also equally important factors like efficiency and deadlines
and not breaking too badly with the past.

You can easily enough design your own sub-language of sorts within the
python universe. Simply write your own module(s) and import them. Choose
brand new names for many of your functions or replace existing functions
carefully by figuring out which namespace a function is defined in, then
creating a new function with the same name that may call the old function
within it by explicitly referring to it.

SO if you want a new max() then either create an axy_max() or perhaps link
the original max to original_max and make your own max() that after playing
around internally, might call original_max.

Here is an example. Suppose you want the maximum value in a nested structure
like:

nested = [ 1, [2, 3, [4, 5], 6], 7]

This contains parts at several levels including an inner list containing yet
another inner list. using max(nested) will not work even if some intuition
wants it to work.

If you want your own version of max to be flexible enough to deal with this
case too, then you might find a flattener function or  a function that
checks the depth of a structure such as a list or tuple, and apply it as
needed until you have arguments suitable to hand to original_max. Your max()
may end up being recursive as it keep peeling back one level and calling
itself on the results which may then peel back another level. 

But the people who built aspects of python chose not to solve every single
case, however elegant that might be, and chose to solve several common cases
fairly efficiently.

What we often end up doing here is pseudo-religious discussions that rarely
get anywhere. It really is of no importance to discuss what SHOULD HAVE BEEN
for many scenarios albeit with some exceptions. Some errors must be slated
to be fixed or at least have more warnings in the documentation. And,
proposals for future changes to Python can be submitted and mostly will be
ignored.

What people often do not do is to ask a question that is more easy to deal
with. Asking WHY a feature is like it is can be a decent question. Asking
how to get around a feature such as whether there is some module out there
that implements it another way using some other function call, is another
good question. COMPLAINING about what has been done and used for a long time
is sometimes viewed differently and especially if it suggests people doing
this were stupid or even inconsistent. 

Appealing to make-believe rules you choose to live your life by also tends
not to work. As you note, overengineering can cause even more problems than
a simple consistent design, albeit it can also create a rather boring and
useless product.

Too much of what happens under the table is hidden in python and if you
really study those details, you might see how a seemingly trivial task like
asking to create a new object of some class, can result in a cascade of code
being run that does things that themselves result in cascades of more code
as the object is assembled and modified using a weird number of searches and
executions for dunder methods in classes and metaclasses it is based on as
well as dealing with other objects/functions like descriptors and
decorators. Since our processors are faster, we might be able to afford a
design that does so much for you and we have garbage collection to deal with
the many bits and pieces created and abandoned in many processes. So our
higher level designs can often look fairly simple and even elegant but the
complexity now must be there somewhere.

I hate to bring up  an analogy as my experience suggests people will take it
as meaning way more (or less) than I intend. Many languages, especially
early on, hated to fail. Heck, machines crashed. So an elegant design was
required to be overlaid with endless testing to avoid the darn errors.
Compilers had to try to catch them even earlier so you did not provide any
argument to a function that was not the same type. You had to explicitly
test for other things at run time to avoid dividing by zero or take the
square root of a negative number or see if a list was empty ...

Python allows or even often encourages a different paradigm where you throw
errors when needed but mainly just TRY something and be prepared to deal
with failure. It too is an elegant design but a very different one. And, you
can do BOTH. Heck, you can do many styles of programming as the language
keeps being extended. There is no one right way most of the time even if
someone once said there is.

So if the standard library provides one way to do something, it may not be
the only way and may not match what you want. Sometimes the fix for a
request is made by adding options like the default=value for max, and
sometimes by allowing the user to specify the comparison function to use
with another keyword argument. This literally lets you use max() to actually
easily calculate what min() does or do something entirely different like
find the longest word in a sentence or the shortest:

>>> words = "A sentence with sesquipedalian words like
disestablishmentarianism to measure length".split()
...         
>>> words
...         
['A', 'sentence', 'with', 'sesquipedalian', 'words', 'like',
'disestablishmentarianism', 'to', 'measure', 'length']

So the length can be measured by len() and this asks for the maximum:

>>> max(words, key=len)
...         
'disestablishmentarianism'

The max() function here does not care and simply uses whatever key you
wanted. 

Simply making a function that returns a negative of the length suffices to
make this act like min():

>>> def neglen(arg): return( - len(arg))
... 
>>> neglen("hello")
-5
>>> max(words, key=neglen)
'A'

My point is that in one sense many standard library functions already have
had features added to them in ways that do not break existing programs and
they do allow fairly interesting problems to be solved. But sometimes such
features may mean either not accepting some formats as arguments that
another function supports, or perhaps needing to try to change lots of
functions at the same time to remain in sync. That is clearly often not
possible or may be highly expensive or it may simply be done more gradually
over many releases. I believe extending list comprehension or generators of
that sort to also do sets and dictionaries may have been done incrementally
even if today it seems like a unified set of things done in the same general
way.

So my PERSONAL view is that when looking at issues, we need to NOT find a
single principle and then insist it is the one and only principle to guide
us. We need to evaluate multiple ideas and not rule out any prematurely. We
then need to weigh the ideas in specific cases and see if anything dominates
or if perhaps a few in proper combination are a reasonable compromise. So I
submit as far as compilers and interpreters go, they would "love" really
short variable names and not see oodles of whitespace that just is there to
be ignored, as well as comments to ignore. But generally, we don't care
because what we focus on is the experience of human programmers and the ones
who read or write or maintain the code. For them, we want generally
variables long enough to hold some amount of meaningfulness. Similarly, many
other features are a tradeoff and although it is likely far cheaper to
write:

    deeply.nested.object.fieldname = 0 if deeply.nested.object.fieldname > 9

As more like:

    short = deeply.nested.object.fieldname
    short = 0 if short > 9

It makes sense to allow the programmer to choose either way and either have
a compiler or interpreter optimize code with regions like this that keep
referring to nested areas, or pay the costs to keep re-traversing it even if
it takes a millionth of a second longer.

Now if I am designing something under my own control, boy do I take short
cuts when prototyping and ignore lots of "rules" as I build the skeleton. If
I am somewhat satisfied, I might start applying many of the principles I use
including adding lots of comments, naming variables in more meaningful ways,
perhaps refactoring for more efficiency, changing the code to handle special
cases, setting up to detect some errors and catch them and do something
appropriate, write a manual page and so on. Lots of things are then
important but not so much in early stages.

Python is a work in progress built by lots of people and I am regularly
amazed at how well much of it hangs together despite some areas where it is
a tad inconsistent. But I am pretty sure many of the inconsistencies have
been handled in ways that are not seen unless you search. There must be
other implementations for functions that handle edge cases or that try many
ways and return the ones that do not generate errors. 

Do you prefer languages that pretty much either require you to create many
functions for each number and type of arguments like 
max(int, int) -> int
max(int, int, int) -> int
max(int, int, int, int) -> int
max(int, double) -> double
...
max(int, double, uint16) -> double

And so on? I know languages that allow you to write frameworks and then
internally create dozens of such variants to use as needed at compile time.
Each function can be optimized but so many functions may have their own
expenses.

But a language like python takes a different tack and one function alone can
do so many things with ease including handling any number of arguments,
defaults for some, many kinds of arguments and especially those that are
either compatible or implement some protocol, and so on. When you view
things that way, the design of max() and sum() may well make quite a bit
more sense and also why they are not identically designed.



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Axy via Python-list
Sent: Tuesday, February 21, 2023 2:37 PM
To: python-list at python.org
Subject: Re: Tuple Comprehension ???

On 21/02/2023 19:11, avi.e.gross at gmail.com wrote:
> In your own code, you may want to either design your own functions, or use
them as documented or perhaps create your own wrapper functions that
carefully examine what you ask them to do and re-arrange as needed to call
the function(s) you want as needed or return their own values or better
error messages.  As a silly example, this fails:
>
> max(1, "hello")
>
> Max expects all arguments to be of compatible types. You could write your
own function called charMax() that converts all arguments to be of type str
before calling max() or maybe call max(... , key=mycompare) where compare as
a function handles this case well.
>
> The key point is that you need to adapt yourself to what some function you
want to use offers, not expect the language to flip around at this point and
start doing it your way and probably breaking many existing programs.
>
> Yes, consistency is a good goal. Reality is a better goal.

I don't think overengineering is a good thing. Good design utilizes
associativity so a person don't get amazed by inconsistency in things that
expected to be similar.

Axy.

--
https://mail.python.org/mailman/listinfo/python-list


From henhanna at gmail.com  Tue Feb 21 20:52:21 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Tue, 21 Feb 2023 17:52:21 -0800 (PST)
Subject: Tuple Comprehension ???
In-Reply-To: <mailman.1903.1677004774.20444.python-list@python.org>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
 <ee16e4f9-3f40-d1d2-3d67-7ab0e27d3648@tompassin.net>
 <2301b1f7-02bd-3bae-b93e-90a28bcc0372@declassed.art>
 <mailman.1903.1677004774.20444.python-list@python.org>
Message-ID: <4d7a4eab-9d67-45a5-8c07-40f0aff1edcfn@googlegroups.com>

On Tuesday, February 21, 2023 at 10:39:54 AM UTC-8, Thomas Passin wrote:
> On 2/21/2023 12:32 PM, Axy via Python-list wrote: 
> > On 21/02/2023 04:13, Hen Hanna wrote: 
> >>
> >>                  (A)   print( max( * LisX )) 
> >>                  (B)   print( sum( * LisX ))        <------- Bad 
> >> syntax !!! 
> >> 
> >> What's most surprising is....     (A)  is ok, and  (B) is not. 
> >> 
> >>             even tho'   max() and sum()  have   (basically)  the same 
> >> syntax...  ( takes one arg ,  whch is a list )
> They **don't** have basically the same signature, though. max() takes 
> either an iterable or two or more numbers. Using max(*list_) presents 
> it with a series of numbers, so that's OK. 
> 
> sum() takes just one iterable (plus an optional start index). Using 
> sum(*list_) presents it with a series of numbers, and that does not 
> match its signature. 
> 
> Check what I said: 
> 
> >>> help(sum) 
> Help on built-in function sum in module builtins:
> sum(iterable, /, start=0) 

> >>> help(max) 

thakns...    i like the use of the word  [signature]


thanks for all the commetns...  i'll try to catch up later.


i think i understand it much better now.

regular  Python  (func-calling)  notation   is like  CL (Common Lisp) funcall.

and        fun( *  args )      notation is like a (compile-time)  macro


   ( max( * X ))               ----macroexpand--->        (apply  max    X)

   ( max( * [1,2,3,4] ))        ----macroexpand--->       (apply  max    '(1 2 3 4) )

and
      Max()   can take many arguments, but
      Sum()  can basically take only 1.



From henhanna at gmail.com  Tue Feb 21 21:00:20 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Tue, 21 Feb 2023 18:00:20 -0800 (PST)
Subject: Python + Vim editor
Message-ID: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>


what editor do you (all) use to write Python code?  (i use Vim)





yesterday (?)  i started noticing this with   Bing  search ....
e.g.   i type   [synonym    rich]   whch gives several synonyms,
and so  (next)  i try  to replace  [rich]  by  
                                 something like     Affluent

        so i try to    hit  [<-]    to erase  [rich] 
              but i can't... the [text entering]  box  is frozen.

so i give up and go to Google.

From gweatherby at uchc.edu  Tue Feb 21 22:03:12 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Wed, 22 Feb 2023 03:03:12 +0000
Subject: Python + Vim editor
In-Reply-To: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
Message-ID: <SJ0PR14MB58643E1F6CCDFD53D9CC5EDBB9AA9@SJ0PR14MB5864.namprd14.prod.outlook.com>

Vim 2% of the time, PyCharm (with VI plugin) 98% of the time.

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Hen Hanna <henhanna at gmail.com>
Date: Tuesday, February 21, 2023 at 9:38 PM
To: python-list at python.org <python-list at python.org>
Subject: Python + Vim editor
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

what editor do you (all) use to write Python code?  (i use Vim)





yesterday (?)  i started noticing this with   Bing  search ....
e.g.   i type   [synonym    rich]   whch gives several synonyms,
and so  (next)  i try  to replace  [rich]  by
                                 something like     Affluent

        so i try to    hit  [<-]    to erase  [rich]
              but i can't... the [text entering]  box  is frozen.

so i give up and go to Google.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jN9E2ZVh45QRBtJ1BgpTjJlghI54aPL2rcB7dlPx1IFw6h8P3vQXgVcNA0rjIEAt0WIg_9oMc7OLfSbcnoU$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!jN9E2ZVh45QRBtJ1BgpTjJlghI54aPL2rcB7dlPx1IFw6h8P3vQXgVcNA0rjIEAt0WIg_9oMc7OLfSbcnoU$>

From list1 at tompassin.net  Tue Feb 21 21:57:07 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 21 Feb 2023 21:57:07 -0500
Subject: Tuple Comprehension ???
In-Reply-To: <4d7a4eab-9d67-45a5-8c07-40f0aff1edcfn@googlegroups.com>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
 <ee16e4f9-3f40-d1d2-3d67-7ab0e27d3648@tompassin.net>
 <2301b1f7-02bd-3bae-b93e-90a28bcc0372@declassed.art>
 <mailman.1903.1677004774.20444.python-list@python.org>
 <4d7a4eab-9d67-45a5-8c07-40f0aff1edcfn@googlegroups.com>
Message-ID: <9814dd46-dd75-5e0e-353c-b10a3c2f01a1@tompassin.net>

On 2/21/2023 8:52 PM, Hen Hanna wrote:
> On Tuesday, February 21, 2023 at 10:39:54 AM UTC-8, Thomas Passin wrote:
>> On 2/21/2023 12:32 PM, Axy via Python-list wrote:
>>> On 21/02/2023 04:13, Hen Hanna wrote:
>>>>
>>>>                   (A)   print( max( * LisX ))
>>>>                   (B)   print( sum( * LisX ))        <------- Bad
>>>> syntax !!!
>>>>
>>>> What's most surprising is....     (A)  is ok, and  (B) is not.
>>>>
>>>>              even tho'   max() and sum()  have   (basically)  the same
>>>> syntax...  ( takes one arg ,  whch is a list )
>> They **don't** have basically the same signature, though. max() takes
>> either an iterable or two or more numbers. Using max(*list_) presents
>> it with a series of numbers, so that's OK.
>>
>> sum() takes just one iterable (plus an optional start index). Using
>> sum(*list_) presents it with a series of numbers, and that does not
>> match its signature.
>>
>> Check what I said:
>>
>>>>> help(sum)
>> Help on built-in function sum in module builtins:
>> sum(iterable, /, start=0)
> 
>>>>> help(max)
> 
> thakns...    i like the use of the word  [signature]
> 
> 
> thanks for all the commetns...  i'll try to catch up later.
> 
> 
> i think i understand it much better now.
> 
> regular  Python  (func-calling)  notation   is like  CL (Common Lisp) funcall.
> 
> and        fun( *  args )      notation is like a (compile-time)  macro
> 
> 
>     ( max( * X ))               ----macroexpand--->        (apply  max    X)
> 
>     ( max( * [1,2,3,4] ))        ----macroexpand--->       (apply  max    '(1 2 3 4) )
> 
> and
>        Max()   can take many arguments, but
>        Sum()  can basically take only 1.
... and that one has to be an iterable.


From bowman at montana.com  Tue Feb 21 22:20:21 2023
From: bowman at montana.com (rbowman)
Date: 22 Feb 2023 03:20:21 GMT
Subject: Python + Vim editor
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
Message-ID: <k5lfvlF71laU1@mid.individual.net>

On Tue, 21 Feb 2023 18:00:20 -0800 (PST), Hen Hanna wrote:

> what editor do you (all) use to write Python code?  (i use Vim)

I've used vim (gvim to be precise) for years and that has been my editor 
of choice for Python. Lately I've been using Spyder.

As background, most of my Python work is with Esri's ArcPy. Until recently 
it was 2.7 but they've moved to 3 and package it with conda. The package 
manager is integrated into ArcGIS Pro, the desktop tool for managing GIS 
data and Spyder is a relatively easy install into a virtual environment.

I won't discuss the conda system or Spyder but I can either go with the 
flow or spend a lot of time fighting it.  


From avi.e.gross at gmail.com  Tue Feb 21 23:08:53 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 21 Feb 2023 23:08:53 -0500
Subject: Tuple Comprehension ???
In-Reply-To: <4d7a4eab-9d67-45a5-8c07-40f0aff1edcfn@googlegroups.com>
References: <03e535c0-e4c8-0436-4b0f-bd4fea5c9d4c@gmail.com>
 <b3678abf-b18e-4626-a1b2-e596763ac066n@googlegroups.com>
 <mailman.1895.1676951812.20444.python-list@python.org>
 <eb71efdf-49fc-4fcb-9027-5a572ee38553n@googlegroups.com>
 <ee16e4f9-3f40-d1d2-3d67-7ab0e27d3648@tompassin.net>
 <2301b1f7-02bd-3bae-b93e-90a28bcc0372@declassed.art>
 <mailman.1903.1677004774.20444.python-list@python.org>
 <4d7a4eab-9d67-45a5-8c07-40f0aff1edcfn@googlegroups.com>
Message-ID: <00cd01d94673$60bd43a0$2237cae0$@gmail.com>

HH,

Just FYI, as a seeming newcomer to Python, there is a forum that may fit
some of your questions better as it is for sort of tutoring and related
purposes:

https://mail.python.org/mailman/listinfo/tutor

I am not discouraging you from posting here, just maybe not to overwhelm
this group with many questions and especially very basic ones.

Your choice.

But what I read below seems like an attempt by you to look for a cognate to
a python feature in a LISP dialect. There may be one but in many ways the
languages differ quite a bit.

As I see it, using the asterisk the way you tried is not all that common and
you are sometimes using it where it is not needed. Python is NOT a language
that is strongly typed so there is no need to take a list or other iterator
or container of numbers and fool the interpreter into making it look like
you placed them as multiple arguments. In many places, just handing over a
list is fine and it is expanded and used as needed.

When a function like sum() or max() is happy to take a single list argument,
then feed it the list, not *list. 

Where it can be helpful is a function like range() where range() takes up to
three arguments as in:

 >>> list(range(7))
[0, 1, 2, 3, 4, 5, 6]
>>> list(range(11, 20))
[11, 12, 13, 14, 15, 16, 17, 18, 19]
>>> list(range(31, 45, 3))
[31, 34, 37, 40, 43]

In the above, you are really asking for stop=num, or start/stop or
start/stop/step. 

But what if you have some data structure like a list that contains [31, 45,
3] or just a two number version or a single number and it is sitting in a
variable. You can ask Python to unpack all kinds of things in various ways
and the asterisk is one simple one. Unpacking is itself a huge topic I will
not discuss.

>>> mylist = [31, 45, 3]
>>> list(range(mylist))
Traceback (most recent call last):
  File "<pyshell#57>", line 1, in <module>
    list(range(mylist))
TypeError: 'list' object cannot be interpreted as an integer
>>> list(range(*mylist))
[31, 34, 37, 40, 43]

Range() takes only something like integers.  So you use the * in this
context to give it three integers individually.

Range does not take named arguments but many other functions do. So what if
I have an arbitrary function that accepts arguments like
myfunc(alpha=default, beta=default, gamma=default) where the defaults are
not the issue and may be anything such as "" or 0 or an empty set.

I could write code like this that creates a toy version of the function and
create a dictionary that supplies any combination of the required arguments
and use not the * operator that expands something like a list, but the
doubled ** operator that expands all entries of a dictionary into individual
items:

>>> def myfunc(alpha=1, beta="", gamma=""):
...     print(f"alpha={alpha}, beta={beta}, gamma={gamma}")
... 
...     
>>> myfunc()
alpha=1, beta=, gamma=
>>> myfunc(1, 2, 3)
alpha=1, beta=2, gamma=3

>>> mydict = { "alpha" : 101, "beta" : "hello", "gamma" : "buy bye" }
>>> mydict
{'alpha': 101, 'beta': 'hello', 'gamma': 'buy bye'}

>>> myfunc( **mydict )
alpha=101, beta=hello, gamma=buy bye

I have no idea if any of this is really like your macroexpand. It does not
need to be. It is what it is. If you went back to a language like C, their
macros might be used to make a "constant with "#define" but they would not
be a constant in the same way as a language that uses a keyword that makes a
variable name into a constant that cannot be changed without causing an
error. Similar but also not the same.

This is only the surface of some things commonly done in python when it
makes sense. But often there is no need and your examples are a good example
when the function happily take a list in the first place. So why fight it
especially when your expanded version is not happily received?

The takeaway is that you need to read a bit more of a textbook approach that
explains things and not use slightly more advanced features blindly. It is
NOT that sum() takes a single argument that matters. It fails on something
like sum(1) which is a single argument as well as sum("nonsense") and so on.
What sum takes is a wide variety of things in python which implement what it
takes to be considered an iterable. And it takes exactly one of them under
the current design.

sum((1,))
1
sum([1])
1
sum(n- 5 for n in range(10,15))
35

All kinds of things work. Tuples and lists are merely the easiest to see.
The latter exmple is a generator that returns 5 less than whatever range()
produces as another kind of iterable. The sum() function  will not take two
or more things, iterable or not. So the first below fails and the second
does not:

>>> sum([1, 2], [3,4])
Traceback (most recent call last):
  File "<pyshell#94>", line 1, in <module>
    sum([1, 2], [3,4])
TypeError: can only concatenate list (not "int") to list

>>> sum([1, 2] + [3,4])
10

Why? Because the plus sign asked the lists to combine into one larger list.
The sum function is only called after python has combined the lists into one
with no name.

Now you can reasonably ask how to add a bunch of numbers, no lists or other
gimmicks? Here is a short but working version:

>>> def multiSum(*args): return sum(list(args))
... 

>>> multiSum(1, 2, 3, 4 , 5)
15

Note I used an asterisk with a quite different meaning there in another sort
of mini language you use to explain what you want function arguments to be
seen as. *args there does not mean to do the expansion but the opposite and
collect them into a single argument I can then make into a list. But guess
what? args is already a list so this works just as well and since sum wants
something like a list, it gets it:

>>> def multiSum(*args): return sum(args)
... 
>>> multiSum(1, 2, 3, 4 , 5)
15

So although you can slow yourself down by trying to see how Python is like
some other language, consider almost forgetting everything else you know and
learn what python IS just for itself. I learned python after so many other
languages that I would never have had time to master it if I kept comparing
it to all the others.

Having said that, many programming languages tend to converge in some ways
and features in one are often adapted to or from another. There can be
cognates. But my advice remains to learn what is expected from a function
before using it and then you can adjust what you give it to match allowed
configurations. Many languages have such needs and some of them do things
differently. As an example, a language like R allows you to say
do.call(func, list) which goes and calls func() with the expanded arguments
taken from the list. Different idea for a related goal.

Good luck. Learning can be mental fun when you have the fundamentals.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Hen Hanna
Sent: Tuesday, February 21, 2023 8:52 PM
To: python-list at python.org
Subject: Re: Tuple Comprehension ???

On Tuesday, February 21, 2023 at 10:39:54 AM UTC-8, Thomas Passin wrote:
> On 2/21/2023 12:32 PM, Axy via Python-list wrote: 
> > On 21/02/2023 04:13, Hen Hanna wrote: 
> >>
> >>                  (A)   print( max( * LisX )) 
> >>                  (B)   print( sum( * LisX ))        <------- Bad 
> >> syntax !!! 
> >> 
> >> What's most surprising is....     (A)  is ok, and  (B) is not. 
> >> 
> >>             even tho'   max() and sum()  have   (basically)  the same 
> >> syntax...  ( takes one arg ,  whch is a list )
> They **don't** have basically the same signature, though. max() takes 
> either an iterable or two or more numbers. Using max(*list_) presents 
> it with a series of numbers, so that's OK.
> 
> sum() takes just one iterable (plus an optional start index). Using
> sum(*list_) presents it with a series of numbers, and that does not 
> match its signature.
> 
> Check what I said: 
> 
> >>> help(sum)
> Help on built-in function sum in module builtins:
> sum(iterable, /, start=0)

> >>> help(max)

thakns...    i like the use of the word  [signature]


thanks for all the commetns...  i'll try to catch up later.


i think i understand it much better now.

regular  Python  (func-calling)  notation   is like  CL (Common Lisp)
funcall.

and        fun( *  args )      notation is like a (compile-time)  macro


   ( max( * X ))               ----macroexpand--->        (apply  max    X)

   ( max( * [1,2,3,4] ))        ----macroexpand--->       (apply  max    '(1
2 3 4) )

and
      Max()   can take many arguments, but
      Sum()  can basically take only 1.


--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Tue Feb 21 21:59:25 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 21 Feb 2023 21:59:25 -0500
Subject: Python + Vim editor
In-Reply-To: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
Message-ID: <37aec7f4-2478-2a66-e94d-b09491cd1d48@tompassin.net>

On 2/21/2023 9:00 PM, Hen Hanna wrote:
> 
> what editor do you (all) use to write Python code?  (i use Vim)

I usually use the Leo-editor (https://github.com/leo-editor/leo-editor 
or PyPi).  It's wonderful once you get it figured out but it's got a 
real learning curve.


From orzodk at fastmail.com  Wed Feb 22 00:00:57 2023
From: orzodk at fastmail.com (orzodk)
Date: Tue, 21 Feb 2023 22:00:57 -0700
Subject: Python + Vim editor
In-Reply-To: <37aec7f4-2478-2a66-e94d-b09491cd1d48@tompassin.net> (Thomas
 Passin's message of "Tue, 21 Feb 2023 21:59:25 -0500")
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
 <37aec7f4-2478-2a66-e94d-b09491cd1d48@tompassin.net>
Message-ID: <m2ttzenvjq.fsf@ioa48nv.localdomain>

Thomas Passin <list1 at tompassin.net> writes:

> On 2/21/2023 9:00 PM, Hen Hanna wrote:
>> what editor do you (all) use to write Python code?  (i use Vim)
>
> I usually use the Leo-editor (https://github.com/leo-editor/leo-editor
> or PyPi).  It's wonderful once you get it figured out but it's got a
> real learning curve.

I had never heard of Leo before. This is a real rabbit-hole. Thanks for
sharing this interesting editor.

From list1 at tompassin.net  Wed Feb 22 00:36:34 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 22 Feb 2023 00:36:34 -0500
Subject: Python + Vim editor
In-Reply-To: <m2ttzenvjq.fsf@ioa48nv.localdomain>
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
 <37aec7f4-2478-2a66-e94d-b09491cd1d48@tompassin.net>
 <m2ttzenvjq.fsf@ioa48nv.localdomain>
Message-ID: <60a8c1e1-d081-8663-fba0-5bae67ccd659@tompassin.net>

On 2/22/2023 12:00 AM, orzodk wrote:
> Thomas Passin <list1 at tompassin.net> writes:
> 
>> On 2/21/2023 9:00 PM, Hen Hanna wrote:
>>> what editor do you (all) use to write Python code?  (i use Vim)
>>
>> I usually use the Leo-editor (https://github.com/leo-editor/leo-editor
>> or PyPi).  It's wonderful once you get it figured out but it's got a
>> real learning curve.
> 
> I had never heard of Leo before. This is a real rabbit-hole. Thanks for
> sharing this interesting editor.

There's also a Google Group: https://groups.google.com/g/leo-editor

Leo does dog-fooding: Leo's own code base (hundreds of Python files) is 
contained and developed in a Leo file (called an "outline").  I just 
this afternoon searched it to find where a particular core method is 
called in less than a second (two places, it turned out).  Leo lets you 
break down a program or other chunk of code into whatever pieces you 
like, not being limited to modules, classes, methods, and functions, and 
to rearrange them pretty much at will.

Much of Leo's documentation is in Sphinx documents that are authored and 
managed in Leo outlines - it's an excellent documentation tool.

More of how I use it is at 
https://leo-editor.github.io/leo-editor/testimonials.html#thomas-passin




From xiang_163_ok at 163.com  Wed Feb 22 05:14:33 2023
From: xiang_163_ok at 163.com (=?GBK?B?zfXP6A==?=)
Date: Wed, 22 Feb 2023 18:14:33 +0800 (CST)
Subject: __getattr__ is much slower in Python3.11
Message-ID: <7bf77399.5123.186789e1e9f.Coremail.xiang_163_ok@163.com>

hello everyone:
I upgrade my project from Py3.10.4 to Py3.11.1 recently and I noticed Py3.11.1 is faster than Py3.10.4 except for some testcase, like __getattr__.
This is my test code in python discussion:
https://discuss.python.org/t/getattr-is-much-slower-in-python3-11/24028


But no one knows why. I don't know whether this is a Python bug so I wrote this email. 
Thank you for your time.

From boblatest at yahoo.com  Wed Feb 22 03:49:53 2023
From: boblatest at yahoo.com (Robert Latest)
Date: 22 Feb 2023 08:49:53 GMT
Subject: Line continuation and comments
Message-ID: <k5m39hF9sj0U1@mid.individual.net>

I found myself building a complicated logical condition with many ands and ors
which I made more manageable by putting the various terms on individual lines
and breaking them with the "\" line continuation character. In this context it
would have been nice to be able to add comments to lines terms which of course
isn't possible because the backslash must be the last character on the line.

Question: If the Python syntax were changed to allow comments after line-ending
backslashes, would it break any existing code? I can't think of an example.

From edmondo.giovannozzi at gmail.com  Wed Feb 22 04:24:12 2023
From: edmondo.giovannozzi at gmail.com (Edmondo Giovannozzi)
Date: Wed, 22 Feb 2023 01:24:12 -0800 (PST)
Subject: Line continuation and comments
In-Reply-To: <k5m39hF9sj0U1@mid.individual.net>
References: <k5m39hF9sj0U1@mid.individual.net>
Message-ID: <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>

Il giorno mercoled? 22 febbraio 2023 alle 09:50:14 UTC+1 Robert Latest ha scritto:
> I found myself building a complicated logical condition with many ands and ors 
> which I made more manageable by putting the various terms on individual lines 
> and breaking them with the "\" line continuation character. In this context it 
> would have been nice to be able to add comments to lines terms which of course 
> isn't possible because the backslash must be the last character on the line. 
> 
> Question: If the Python syntax were changed to allow comments after line-ending 
> backslashes, would it break any existing code? I can't think of an example.

Well you can if you use parenthesis like in:
x = 5
a = (x > 3 and
#     x < 21 or
     x > 100
     )
You don't need the "\" to continue a line in this case


From anton.txt at g{oogle}mail.com  Wed Feb 22 05:32:39 2023
From: anton.txt at g{oogle}mail.com (Anton Shepelev)
Date: Wed, 22 Feb 2023 13:32:39 +0300
Subject: Introspecting the variable bound to a function argument
Message-ID: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>

Hello, all.

Does Python have an instrospection facility that can
determine to which outer variable a function argument is
bound, e.g.:

   v1 = 5;
   v2 = 5;

   def f(a):
       print(black_magic(a)) # or black_magic('a')

   f(v1) # prints: v1
   f(v2) # prints: v2

-- 
()  ascii ribbon campaign -- against html e-mail
/\  www.asciiribbon.org   -- against proprietary attachments

From dbineta3 at gmail.com  Wed Feb 22 06:18:58 2023
From: dbineta3 at gmail.com (Bibi)
Date: Wed, 22 Feb 2023 03:18:58 -0800 (PST)
Subject: Creating logs with Python
Message-ID: <5dcfa51b-834e-4979-9c22-42e06e2df9bdn@googlegroups.com>

Hello
I want to store and make available as part of my project, logs, for access to data. Do you have any proposals?
Kind regards

From gweatherby at uchc.edu  Wed Feb 22 10:03:29 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Wed, 22 Feb 2023 15:03:29 +0000
Subject: Creating logs with Python
In-Reply-To: <5dcfa51b-834e-4979-9c22-42e06e2df9bdn@googlegroups.com>
References: <5dcfa51b-834e-4979-9c22-42e06e2df9bdn@googlegroups.com>
Message-ID: <SJ0PR14MB58642496353EDBCE38BB75F9B9AA9@SJ0PR14MB5864.namprd14.prod.outlook.com>

https://docs.python.org/3/howto/logging.html

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Bibi <dbineta3 at gmail.com>
Date: Wednesday, February 22, 2023 at 9:44 AM
To: python-list at python.org <python-list at python.org>
Subject: Creating logs with Python
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Hello
I want to store and make available as part of my project, logs, for access to data. Do you have any proposals?
Kind regards
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ntBEz59Ey9Aq3lepQ3xSKwSYtD_mYgoMO3OPqxMUrzbHNliAV76yHKsVIbEDpznq3hXNHvGxG3RNUPVxj5k$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ntBEz59Ey9Aq3lepQ3xSKwSYtD_mYgoMO3OPqxMUrzbHNliAV76yHKsVIbEDpznq3hXNHvGxG3RNUPVxj5k$>

From pbryan at anode.ca  Wed Feb 22 10:23:00 2023
From: pbryan at anode.ca (Paul Bryan)
Date: Wed, 22 Feb 2023 07:23:00 -0800
Subject: Line continuation and comments
In-Reply-To: <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
Message-ID: <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>

Adding to this, there should be no reason now in recent versions of
Python to ever use line continuation. Black goes so far as to state
"backslashes are bad and should never be used":

https://black.readthedocs.io/en/stable/the_black_code_style/future_style.html#using-backslashes-for-with-statements

On Wed, 2023-02-22 at 01:24 -0800, Edmondo Giovannozzi wrote:
> Il giorno mercoled? 22 febbraio 2023 alle 09:50:14 UTC+1 Robert
> Latest ha scritto:
> > I found myself building a complicated logical condition with many
> > ands and ors 
> > which I made more manageable by putting the various terms on
> > individual lines 
> > and breaking them with the "\" line continuation character. In this
> > context it 
> > would have been nice to be able to add comments to lines terms
> > which of course 
> > isn't possible because the backslash must be the last character on
> > the line. 
> > 
> > Question: If the Python syntax were changed to allow comments after
> > line-ending 
> > backslashes, would it break any existing code? I can't think of an
> > example.
> 
> Well you can if you use parenthesis like in:
> x = 5
> a = (x > 3 and
> #???? x < 21 or
> ???? x > 100
> ???? )
> You don't need the "\" to continue a line in this case
> 


From gweatherby at uchc.edu  Wed Feb 22 10:02:05 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Wed, 22 Feb 2023 15:02:05 +0000
Subject: Line continuation and comments
In-Reply-To: <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
Message-ID: <SJ0PR14MB5864F3543AC693A864E07C2EB9AA9@SJ0PR14MB5864.namprd14.prod.outlook.com>

That?s a neat tip. End of line comments work, too

x = (3 > 4  #never
     and 7 == 7  # hopefully
     or datetime.datetime.now().day > 15 # sometimes
     )
print(x)

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Edmondo Giovannozzi <edmondo.giovannozzi at gmail.com>
Date: Wednesday, February 22, 2023 at 9:40 AM
To: python-list at python.org <python-list at python.org>
Subject: Re: Line continuation and comments
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Il giorno mercoled? 22 febbraio 2023 alle 09:50:14 UTC+1 Robert Latest ha scritto:
> I found myself building a complicated logical condition with many ands and ors
> which I made more manageable by putting the various terms on individual lines
> and breaking them with the "\" line continuation character. In this context it
> would have been nice to be able to add comments to lines terms which of course
> isn't possible because the backslash must be the last character on the line.
>
> Question: If the Python syntax were changed to allow comments after line-ending
> backslashes, would it break any existing code? I can't think of an example.

Well you can if you use parenthesis like in:
x = 5
a = (x > 3 and
#     x < 21 or
     x > 100
     )
You don't need the "\" to continue a line in this case

--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kck9yP0ubC7L_tbIUoMY-nkZJlkXFAiZdnjPtekuYQXN6F8K2wFMW5lO1xZ6gYv6vDdsSo5jxy1QYU_T-EgHsOJ6x7TvXQ$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kck9yP0ubC7L_tbIUoMY-nkZJlkXFAiZdnjPtekuYQXN6F8K2wFMW5lO1xZ6gYv6vDdsSo5jxy1QYU_T-EgHsOJ6x7TvXQ$>

From list1 at tompassin.net  Wed Feb 22 11:27:09 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 22 Feb 2023 11:27:09 -0500
Subject: Line continuation and comments
In-Reply-To: <SJ0PR14MB5864F3543AC693A864E07C2EB9AA9@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <SJ0PR14MB5864F3543AC693A864E07C2EB9AA9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <ec07a8b1-3761-d709-b1ba-29aba0c51ab6@tompassin.net>

On 2/22/2023 10:02 AM, Weatherby,Gerard wrote:
> That?s a neat tip. End of line comments work, too
> 
> x = (3 > 4  #never
>       and 7 == 7  # hopefully
>       or datetime.datetime.now().day > 15 # sometimes
>       )
> print(x)

I find myself doing this more and more often.  It can also help to make 
the code more readable and the intention more clear. Here's one example:

     return (getTerminalFromProcess()
             or getTerminalFromDirectory('/usr/bin')
             or getTerminalFromDirectory('/bin')
             or getCommonTerminal(('konsole', 'xterm'))
             )

It's easier to read than using a "\" at the end of lines, writing it all 
on one line would make for an unreadably long line, while building up 
the final result in steps would make the logic less clear.


> From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Edmondo Giovannozzi <edmondo.giovannozzi at gmail.com>
> Date: Wednesday, February 22, 2023 at 9:40 AM
> To: python-list at python.org <python-list at python.org>
> Subject: Re: Line continuation and comments
> *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***
> 
> Il giorno mercoled? 22 febbraio 2023 alle 09:50:14 UTC+1 Robert Latest ha scritto:
>> I found myself building a complicated logical condition with many ands and ors
>> which I made more manageable by putting the various terms on individual lines
>> and breaking them with the "\" line continuation character. In this context it
>> would have been nice to be able to add comments to lines terms which of course
>> isn't possible because the backslash must be the last character on the line.
>>
>> Question: If the Python syntax were changed to allow comments after line-ending
>> backslashes, would it break any existing code? I can't think of an example.
> 
> Well you can if you use parenthesis like in:
> x = 5
> a = (x > 3 and
> #     x < 21 or
>       x > 100
>       )
> You don't need the "\" to continue a line in this case
> 
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kck9yP0ubC7L_tbIUoMY-nkZJlkXFAiZdnjPtekuYQXN6F8K2wFMW5lO1xZ6gYv6vDdsSo5jxy1QYU_T-EgHsOJ6x7TvXQ$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kck9yP0ubC7L_tbIUoMY-nkZJlkXFAiZdnjPtekuYQXN6F8K2wFMW5lO1xZ6gYv6vDdsSo5jxy1QYU_T-EgHsOJ6x7TvXQ$>


From barry at barrys-emacs.org  Wed Feb 22 12:21:49 2023
From: barry at barrys-emacs.org (Barry)
Date: Wed, 22 Feb 2023 17:21:49 +0000
Subject: Introspecting the variable bound to a function argument
Message-ID: <E94BABCB-5C96-4ACF-B1B6-4E17DD2A07B8@barrys-emacs.org>

?please do not use an email address on a public list that cannot be replied to.


> On 22 Feb 2023, at 14:43, Anton Shepelev <anton.txt at g{oogle}mail.com> wrote:
> 
> ?Hello, all.
> 
> Does Python have an instrospection facility that can
> determine to which outer variable a function argument is
> bound, e.g.:

There is no requirement for a variable to be used in the call.
It could be an an int or string, 42 ?forty two?.

> 
>  v1 = 5;
>  v2 = 5;
> 
>  def f(a):
>      print(black_magic(a)) # or black_magic('a')
> 
>  f(v1) # prints: v1
>  f(v2) # prints: v2

There is the traceback module that lets you find where you are called from.
Also there is the inspect module that also lets tou get at the stack in more detail.

Barry

> 
> -- 
> ()  ascii ribbon campaign -- against html e-mail
> /\  www.asciiribbon.org   -- against proprietary attachments
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From Tramiv at doe.com  Wed Feb 22 13:16:48 2023
From: Tramiv at doe.com (Tramiv)
Date: Wed, 22 Feb 2023 18:16:48 -0000 (UTC)
Subject: Python + Vim editor
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
Message-ID: <slrntvcn0g.3i41.Tramiv@ketiv.vm>

On 2023-02-22, Hen Hanna <henhanna at gmail.com> wrote:
>
> what editor do you (all) use to write Python code?  (i use Vim)
>

For short editin I also use Vim and Pycharm IDE for bigger projects.


From orzodk at fastmail.com  Wed Feb 22 13:45:57 2023
From: orzodk at fastmail.com (orzodk)
Date: Wed, 22 Feb 2023 11:45:57 -0700
Subject: Python + Vim editor
In-Reply-To: <60a8c1e1-d081-8663-fba0-5bae67ccd659@tompassin.net> (Thomas
 Passin's message of "Wed, 22 Feb 2023 00:36:34 -0500")
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
 <37aec7f4-2478-2a66-e94d-b09491cd1d48@tompassin.net>
 <m2ttzenvjq.fsf@ioa48nv.localdomain>
 <60a8c1e1-d081-8663-fba0-5bae67ccd659@tompassin.net>
Message-ID: <m2pma1o7x6.fsf@ioa48nv.localdomain>

Thomas Passin <list1 at tompassin.net> writes:

> On 2/22/2023 12:00 AM, orzodk wrote:
>> Thomas Passin <list1 at tompassin.net> writes:
>> 
>>> On 2/21/2023 9:00 PM, Hen Hanna wrote:
>>>> what editor do you (all) use to write Python code?  (i use Vim)
>>>
>>> I usually use the Leo-editor (https://github.com/leo-editor/leo-editor
>>> or PyPi).  It's wonderful once you get it figured out but it's got a
>>> real learning curve.
>> I had never heard of Leo before. This is a real rabbit-hole. Thanks
>> for
>> sharing this interesting editor.
>
> There's also a Google Group: https://groups.google.com/g/leo-editor
>
> Leo does dog-fooding: Leo's own code base (hundreds of Python files)
> is contained and developed in a Leo file (called an "outline").  I
> just this afternoon searched it to find where a particular core method
> is called in less than a second (two places, it turned out).  Leo lets
> you break down a program or other chunk of code into whatever pieces
> you like, not being limited to modules, classes, methods, and
> functions, and to rearrange them pretty much at will.
>
> Much of Leo's documentation is in Sphinx documents that are authored
> and managed in Leo outlines - it's an excellent documentation tool.
>
> More of how I use it is at
> https://leo-editor.github.io/leo-editor/testimonials.html#thomas-passin

I watched a couple of the intro videos and saw the dog-fooding. Leo is a
total paradigm shift for me. Thanks for links, I'll be sure to check
them out.

From list1 at tompassin.net  Wed Feb 22 14:45:28 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 22 Feb 2023 14:45:28 -0500
Subject: Python + Vim editor
In-Reply-To: <m2pma1o7x6.fsf@ioa48nv.localdomain>
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
 <37aec7f4-2478-2a66-e94d-b09491cd1d48@tompassin.net>
 <m2ttzenvjq.fsf@ioa48nv.localdomain>
 <60a8c1e1-d081-8663-fba0-5bae67ccd659@tompassin.net>
 <m2pma1o7x6.fsf@ioa48nv.localdomain>
Message-ID: <173e1754-914a-574c-49b2-ad03b3b4bbd7@tompassin.net>

On 2/22/2023 1:45 PM, orzodk wrote:
> Thomas Passin <list1 at tompassin.net> writes:
> 
>> On 2/22/2023 12:00 AM, orzodk wrote:
>>> Thomas Passin <list1 at tompassin.net> writes:
>>>
>>>> On 2/21/2023 9:00 PM, Hen Hanna wrote:
>>>>> what editor do you (all) use to write Python code?  (i use Vim)
>>>>
>>>> I usually use the Leo-editor (https://github.com/leo-editor/leo-editor
>>>> or PyPi).  It's wonderful once you get it figured out but it's got a
>>>> real learning curve.
>>> I had never heard of Leo before. This is a real rabbit-hole. Thanks
>>> for
>>> sharing this interesting editor.
>>
>> There's also a Google Group: https://groups.google.com/g/leo-editor
>>
>> Leo does dog-fooding: Leo's own code base (hundreds of Python files)
>> is contained and developed in a Leo file (called an "outline").  I
>> just this afternoon searched it to find where a particular core method
>> is called in less than a second (two places, it turned out).  Leo lets
>> you break down a program or other chunk of code into whatever pieces
>> you like, not being limited to modules, classes, methods, and
>> functions, and to rearrange them pretty much at will.
>>
>> Much of Leo's documentation is in Sphinx documents that are authored
>> and managed in Leo outlines - it's an excellent documentation tool.
>>
>> More of how I use it is at
>> https://leo-editor.github.io/leo-editor/testimonials.html#thomas-passin
> 
> I watched a couple of the intro videos and saw the dog-fooding. Leo is a
> total paradigm shift for me. Thanks for links, I'll be sure to check
> them out.

Good luck!  Please contact me or the Google Group if you need help or 
guidance.


From dave at nk7z.net  Wed Feb 22 13:53:55 2023
From: dave at nk7z.net (Dave (NK7Z))
Date: Wed, 22 Feb 2023 10:53:55 -0800
Subject: Creating logs with Python
In-Reply-To: <SJ0PR14MB58642496353EDBCE38BB75F9B9AA9@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <5dcfa51b-834e-4979-9c22-42e06e2df9bdn@googlegroups.com>
 <SJ0PR14MB58642496353EDBCE38BB75F9B9AA9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <7eddd51c-5a08-99cf-43c6-fcdf662b56c3@nk7z.net>

Thank you Gerard!  I am working on a project and needed that...  :)

73, and thanks,
Dave (NK7Z)
https://www.nk7z.net
ARRL Volunteer Examiner
ARRL Technical Specialist, RFI
ARRL Asst. Director, NW Division, Technical Resources

On 2/22/23 07:03, Weatherby,Gerard wrote:
> https://docs.python.org/3/howto/logging.html
> 
> From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Bibi <dbineta3 at gmail.com>
> Date: Wednesday, February 22, 2023 at 9:44 AM
> To: python-list at python.org <python-list at python.org>
> Subject: Creating logs with Python
> *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***
> 
> Hello
> I want to store and make available as part of my project, logs, for access to data. Do you have any proposals?
> Kind regards
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ntBEz59Ey9Aq3lepQ3xSKwSYtD_mYgoMO3OPqxMUrzbHNliAV76yHKsVIbEDpznq3hXNHvGxG3RNUPVxj5k$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ntBEz59Ey9Aq3lepQ3xSKwSYtD_mYgoMO3OPqxMUrzbHNliAV76yHKsVIbEDpznq3hXNHvGxG3RNUPVxj5k$>

From henhanna at gmail.com  Wed Feb 22 15:05:21 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Wed, 22 Feb 2023 12:05:21 -0800 (PST)
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
Message-ID: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>


          >  py   bug.py
       Traceback (most recent call last):
         File "C:\Usenet\bug.py", line 5, in <module>
                     print( a + 12 )
          TypeError: can only concatenate str (not "int") to str


Why doesn't  Python (error msg) do the obvious thing and tell me
            WHAT   the actual   (offending,  arg)  values are ?

In many cases, it'd help to know what string the var  A  had  ,   when the error occurred.
             ------------  i wouldn't have to put      print(a)     just above,  to see.




( pypy    doesn't do that either,   but Python makes programming (debugging) so easy that i hardly feel any inconvenience.)

From henhanna at gmail.com  Wed Feb 22 15:12:10 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Wed, 22 Feb 2023 12:12:10 -0800 (PST)
Subject: Introspecting the variable bound to a function argument
In-Reply-To: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
Message-ID: <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>

On Wednesday, February 22, 2023 at 2:32:57 AM UTC-8, Anton Shepelev wrote:
> Hello, all. 
> 
> Does Python have an instrospection facility that can 
> determine to which outer variable a function argument is 
> bound, e.g.: 
> 
> v1 = 5; 
> v2 = 5; 


do some Python coders like to end lines with   ;   ?


> 
>         def f(a): 
>                    print(black_magic(a))            # or black_magic('a') 
> 
>         f(v1)            # prints: v1 
>         f(v2)            # prints: v2 
> 

the term  [call by name]  suggests  this should be possible.


30 years ago...  i used to think about this type of thing A LOT ---
         -------  CBR, CBV, CBN,   (call by value),    (call by name)....   etc.


From henhanna at gmail.com  Wed Feb 22 15:25:30 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Wed, 22 Feb 2023 12:25:30 -0800 (PST)
Subject: is [comprehension] the right word???
In-Reply-To: <87ttzfu6z4.fsf@nightsong.com>
References: <fa01817b-3f0d-4653-a37a-ac405f65a400n@googlegroups.com>
 <87ttzfu6z4.fsf@nightsong.com>
Message-ID: <47dc0f70-5c60-4937-91b9-72baee272ef8n@googlegroups.com>

On Monday, February 20, 2023 at 5:45:39 PM UTC-8, Paul Rubin wrote:
> Hen Hanna <henh... at gmail.com> writes: 
> > is [comprehension] the right word???

> Yes, it comes from math, particularly set theory. An expression like 
> 
> { n | n:integer, n mod 2 = 0 } 
> 
> is called a set comprehension, and then one there denotes the set of all 
> even integers. Axioms saying that the above denotes a legitimate set 
> are called comprehension axioms. In ZFC (an axiomitization of set 
> theory widely used in math), there is an infinite schema of such axioms. 
> 
> The Haskell language used a notation inspired by this for "list 
> comprehensions", and Python list (and later dictionary etc.) 
> comprehensions were inspired by Haskell's version.


thank you ....  i did a search thru   Google.Books  and found  just 1  hit  (before 1970).



The Cambridge History of Later Greek and Early Medieval ... - Page 133
books.google.com ? books
A. H. Armstrong ? 1967
                                              FOUND INSIDE ? PAGE 133
Too powerful , in fact : in 1902 , Russell showed that it is inconsistent , since it implies Russell's Antinomy ( see Russell's letter to Frege in van Heijenoort 1967 ) . Law V is close to what has become known as the Set Comprehension Principle (SCP)   ........................


From henhanna at gmail.com  Wed Feb 22 15:37:30 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Wed, 22 Feb 2023 12:37:30 -0800 (PST)
Subject: it seems like a few weeks ago... but actually it was more
 like 30 years ago that i was programming in C,      and
Message-ID: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>


it seems like a few weeks ago... but
           actually it was more like 30 years ago
that i was programming in C,      and

i'd get
                 [Segmentation Fault]     (core dumped)
                  [Bus Error]      (core dumped)
                  [access violation]     (core dumped)
                  [bad address]

and,   yes,  some of us   sometimes  analyzed the core file.



i  (vaguely) fantasized that someday   programming would be much more pleasant ---------- and  that  programming language would be  something like  Common Lisp  with a more   Algol, Pascal (C) like syntax.


i've been using Python for 3,4 years and .............. 

                for the first  several weeks... whenever i used Python...  all i could think of....was   --------  this is really Lisp (inside) with  a thin veil of   Java/Pascal syntax..........

               -----  that  everything is  first converted  (macro-expanded)  into (intermediated) Lisp code, and then.........

From avi.e.gross at gmail.com  Wed Feb 22 16:26:05 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Wed, 22 Feb 2023 16:26:05 -0500
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
Message-ID: <009701d94704$45dad1b0$d1907510$@gmail.com>

Hen or Hanna,

You keep asking WHY which may be reasonable but hard or irrelevant in many
cases.

I find the traceback perfectly informative.

It says you asked it to print NOT just "a" but "a + 12" and the error is
coming not from PRINT but from trying to invoke addition between two objects
that have not provided instructions on how to do so. Specifically, an object
of type str has not specified anything to do if asked to concatenate an
object of type int to it. And, an object of type int has not specified what
to do if asked to add itself to an object of type str to the left of it.
Deeper in python, the objects have dunder methods like __ADD__() and
___RADD__() to invoke for those situations that do some logic and decide
they cannot handle it and return an exception of sorts that ends up
generating your message.

If you want to know what "a" has at the moment, ask for just it, not adding
twelve to it. Perhaps you should add a line above your print asking to just
print(a).

Before you suggest what might be helpful, consider what it might mean in a
complex case with lots of variables and what work the interpreter might have
to do to dump the current values of anything relevant or just ANYTHING.

The way forward is less about asking why but asking what to do to get what
you want, or realize it is not attained the way you thought.

Avi

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Hen Hanna
Sent: Wednesday, February 22, 2023 3:05 PM
To: python-list at python.org
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg) values
are ?


          >  py   bug.py
       Traceback (most recent call last):
         File "C:\Usenet\bug.py", line 5, in <module>
                     print( a + 12 )
          TypeError: can only concatenate str (not "int") to str


Why doesn't  Python (error msg) do the obvious thing and tell me
            WHAT   the actual   (offending,  arg)  values are ?

In many cases, it'd help to know what string the var  A  had  ,   when the
error occurred.
             ------------  i wouldn't have to put      print(a)     just
above,  to see.




( pypy    doesn't do that either,   but Python makes programming (debugging)
so easy that i hardly feel any inconvenience.)
-- 
https://mail.python.org/mailman/listinfo/python-list


From mats at wichmann.us  Wed Feb 22 17:10:32 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Wed, 22 Feb 2023 15:10:32 -0700
Subject: Python + Vim editor
In-Reply-To: <slrntvcn0g.3i41.Tramiv@ketiv.vm>
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
 <slrntvcn0g.3i41.Tramiv@ketiv.vm>
Message-ID: <c0f68a08-64bb-35b3-6339-bac857b1adad@wichmann.us>

On 2/22/23 11:16, Tramiv wrote:
> On 2023-02-22, Hen Hanna <henhanna at gmail.com> wrote:
>>
>> what editor do you (all) use to write Python code?  (i use Vim)
>>
> 
> For short editin I also use Vim and Pycharm IDE for bigger projects.


The community has submitted some answers to that question here (two 
lists, some entrants don't fit neatly into one or the other):

https://wiki.python.org/moin/PythonEditors
https://wiki.python.org/moin/IntegratedDevelopmentEnvironments

There are a *lot* if you want to rathole... :)





From cs at cskk.id.au  Wed Feb 22 18:59:18 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Thu, 23 Feb 2023 10:59:18 +1100
Subject: Line continuation and comments
In-Reply-To: <ec07a8b1-3761-d709-b1ba-29aba0c51ab6@tompassin.net>
References: <ec07a8b1-3761-d709-b1ba-29aba0c51ab6@tompassin.net>
Message-ID: <Y/asViVLdPPlHSbH@cskk.homeip.net>

On 22Feb2023 11:27, Thomas Passin <list1 at tompassin.net> wrote:
>On 2/22/2023 10:02 AM, Weatherby,Gerard wrote:
>>That?s a neat tip. End of line comments work, too
>>
>>x = (3 > 4  #never
>>      and 7 == 7  # hopefully
>>      or datetime.datetime.now().day > 15 # sometimes
>>      )
>>print(x)
>
>I find myself doing this more and more often.  It can also help to 
>make the code more readable and the intention more clear. Here's one 
>example:
>
>    return (getTerminalFromProcess()
>            or getTerminalFromDirectory('/usr/bin')
>            or getTerminalFromDirectory('/bin')
>            or getCommonTerminal(('konsole', 'xterm'))
>            )

Aye, me too.

I autoformat my code using `yapf` (I _hate_ `black`) and append my 
personal code style below. In particular, note the 
`split_before_logical_operator` which does the above automatically when 
it reflows a logical expression (you need the brackets, but you need 
them anyway for multiline stuff if you're eschewing the backslash).

     [style]
     based_on_style = pep8
     align_closing_bracket_with_visual_indent = True
     arithmetic_precedence_indication = False
     blank_line_before_module_docstring = True
     blank_line_before_nested_class_or_def = True
     blank_lines_around_top_level_definition = 1
     dedent_closing_brackets = True
     indent_dictionary_value = False
     indent_width = 2
     space_between_ending_comma_and_closing_bracket = False
     spaces_before_comment = 2
     split_before_dot = True
     split_before_expression_after_opening_paren = True
     split_before_first_argument = True
     split_before_logical_operator = True
     split_complex_comprehension = True
     use_tabs = False

So basicly PEP8 with some tweaks.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From henhanna at gmail.com  Wed Feb 22 18:46:09 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Wed, 22 Feb 2023 15:46:09 -0800 (PST)
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
Message-ID: <f4aa2a4b-0cf9-415f-88dc-6d8cc6de8404n@googlegroups.com>

On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote:
> > py bug.py 
> Traceback (most recent call last): 
> File "C:\Usenet\bug.py", line 5, in <module> 
> print( a + 12 ) 
> TypeError: can only concatenate str (not "int") to str 
> 
> 
> Why doesn't Python (error msg) do the obvious thing and tell me 
> WHAT the actual (offending, arg) values are ? 
> 
> In many cases, it'd help to know what string the var A had , when the error occurred. 
> ------------ i wouldn't have to put print(a) just above, to see. 
> 
> 
> 
> 
> ( pypy doesn't do that either, but Python makes programming (debugging) so easy that i hardly feel any inconvenience.)



i  see that my example   would be clearER  with this one-line  change:


                  >  py   bug.py

                   Traceback (most recent call last):

                  File "C:\Usenet\bug.py", line 5, in <module>
                                         map( Func,    fooBar(  X,  Y,  X +  Y  ))
 
                   TypeError: can only concatenate str (not "int") to str


i hope that   NOW   a few of you  can  see this as a genuine,  (reasonable)  question.


From list1 at tompassin.net  Wed Feb 22 19:24:02 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 22 Feb 2023 19:24:02 -0500
Subject: Introspecting the variable bound to a function argument
In-Reply-To: <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
Message-ID: <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>

On 2/22/2023 3:12 PM, Hen Hanna wrote:
> On Wednesday, February 22, 2023 at 2:32:57 AM UTC-8, Anton Shepelev wrote:
>> Hello, all.
>>
>> Does Python have an instrospection facility that can
>> determine to which outer variable a function argument is
>> bound, e.g.:
>>
>> v1 = 5;
>> v2 = 5;
> 
> 
> do some Python coders like to end lines with   ;   ?

Very few, probably.  It's not harmful but adds unnecessary visual clutter.

>>
>>          def f(a):
>>                     print(black_magic(a))            # or black_magic('a')
>>
>>          f(v1)            # prints: v1
>>          f(v2)            # prints: v2
>>
> 
> the term  [call by name]  suggests  this should be possible.
> 
> 
> 30 years ago...  i used to think about this type of thing A LOT ---
>           -------  CBR, CBV, CBN,   (call by value),    (call by name)....   etc.
> 


From list1 at tompassin.net  Wed Feb 22 19:28:29 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 22 Feb 2023 19:28:29 -0500
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <f4aa2a4b-0cf9-415f-88dc-6d8cc6de8404n@googlegroups.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <f4aa2a4b-0cf9-415f-88dc-6d8cc6de8404n@googlegroups.com>
Message-ID: <7872371a-1c42-2308-df1b-7ac0786a2a2b@tompassin.net>

On 2/22/2023 6:46 PM, Hen Hanna wrote:
> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote:
>>> py bug.py
>> Traceback (most recent call last):
>> File "C:\Usenet\bug.py", line 5, in <module>
>> print( a + 12 )
>> TypeError: can only concatenate str (not "int") to str
>>
>>
>> Why doesn't Python (error msg) do the obvious thing and tell me
>> WHAT the actual (offending, arg) values are ?
>>
>> In many cases, it'd help to know what string the var A had , when the error occurred.
>> ------------ i wouldn't have to put print(a) just above, to see.
>>
>>
>>
>>
>> ( pypy doesn't do that either, but Python makes programming (debugging) so easy that i hardly feel any inconvenience.)
> 
> 
> 
> i  see that my example   would be clearER  with this one-line  change:
> 
> 
>                    >  py   bug.py
> 
>                     Traceback (most recent call last):
> 
>                    File "C:\Usenet\bug.py", line 5, in <module>
>                                           map( Func,    fooBar(  X,  Y,  X +  Y  ))
>   
>                     TypeError: can only concatenate str (not "int") to str
> 
> 
> i hope that   NOW   a few of you  can  see this as a genuine,  (reasonable)  question.

It tells me to go look at the function definition and how it's being 
invoked.  Even if I knew which of (X, Y) was an int and which a str, I'd 
still need to do that.

Or you could add type annotations to your code and run mypy on it...



From avi.e.gross at gmail.com  Wed Feb 22 19:58:42 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Wed, 22 Feb 2023 19:58:42 -0500
Subject: semi colonic
In-Reply-To: <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
Message-ID: <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>

Thomas,

This is one of many little twists I see between languages where one feature
impacts use or even the need for another feature.

So can anyone point to places in Python where a semicolon is part of a best
or even good way to do anything?

Some older languages had simple parsers/compilers that needed some way to
know when a conceptual line of code was DONE and the semi-colon was a choice
for making that clear. But some languages seem to only continue looking past
an end-of-line if they detect some serious reason to assume you are in
middle of something. An unmatched open parenthesis or square bracket might
be enough, and in some languages a curly brace.

Python mainly has a concept of indentation and blank lines as one part of
the guidance. Continuing lines is possible, if done carefully.

But consider the lowly comma. Some languages may assume more is to come if
it is dangled at the end of a line. But in a language that supports a
dangling comma such as in making a tuple, how is the interpreter to know
more is to come?

>>> a = 5,
>>> a
(5,)

>>> a = 5, \
...     6
>>> a
(5, 6)

Well, one possible use of a semi-colon is to make short one-liner functions
like this:

    def twoByFour(a): sq = a*a; forth = sq*sq; return((sq, forth))

There is no reason, of course, that could not be done in multiple indented
lines or other ways. 

So if it was allowed in something like a lambda creation, it could be useful
but it isn't!

About the only thing that I can think of is if someone wishes to compress a
file of python code a bit. The indentation can add up but a semi-colon does
not solve all such problems.

Would anything serious break if it was deprecated for use as a statement
terminator? Then again, is it hurting anything? If it stopped being used
this way, could it later be introduced as some new language feature or
operator such as we now have a := b as a reuse of the colon, maybe a
semicolon could be useful at least until someone decides to allow additional
Unicode characters!

Now if there are serious reasons to use semi-colon in python, great. If not,
it is a historical artifact.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Thomas Passin
Sent: Wednesday, February 22, 2023 7:24 PM
To: python-list at python.org
Subject: Re: Introspecting the variable bound to a function argument

On 2/22/2023 3:12 PM, Hen Hanna wrote:
> On Wednesday, February 22, 2023 at 2:32:57 AM UTC-8, Anton Shepelev wrote:
>> Hello, all.
>>
>> Does Python have an instrospection facility that can determine to 
>> which outer variable a function argument is bound, e.g.:
>>
>> v1 = 5;
>> v2 = 5;
> 
> 
> do some Python coders like to end lines with   ;   ?

Very few, probably.  It's not harmful but adds unnecessary visual clutter.

>>
>>          def f(a):
>>                     print(black_magic(a))            # or
black_magic('a')
>>
>>          f(v1)            # prints: v1
>>          f(v2)            # prints: v2
>>
> 
> the term  [call by name]  suggests  this should be possible.
> 
> 
> 30 years ago...  i used to think about this type of thing A LOT ---
>           -------  CBR, CBV, CBN,   (call by value),    (call by name)....
etc.
> 

--
https://mail.python.org/mailman/listinfo/python-list


From henhanna at gmail.com  Wed Feb 22 19:52:34 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Wed, 22 Feb 2023 16:52:34 -0800 (PST)
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <f4aa2a4b-0cf9-415f-88dc-6d8cc6de8404n@googlegroups.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <f4aa2a4b-0cf9-415f-88dc-6d8cc6de8404n@googlegroups.com>
Message-ID: <84b38a83-5113-481f-8eb1-720401908d24n@googlegroups.com>

On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote:
> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: 
> > > py bug.py 
> > Traceback (most recent call last): 
> > File "C:\Usenet\bug.py", line 5, in <module> 
> > print( a + 12 ) 
> > TypeError: can only concatenate str (not "int") to str 
> > 
> > 
> > Why doesn't Python (error msg) do the obvious thing and tell me 
> > WHAT the actual (offending, arg) values are ? 
> > 
> > In many cases, it'd help to know what string the var A had , when the error occurred. 
> > ------------ i wouldn't have to put print(a) just above, to see. 
> > 
> > 
> > 
> > 
> > ( pypy doesn't do that either, but Python makes programming (debugging) so easy that i hardly feel any inconvenience.)


 i see that my example would be (even)  clearER with this one-line change:

           py bug.py 
           
         Traceback (most recent call last): 
          
           File "C:\Usenet\bug.py", line 5, in <module>
                         map( Func, fooBar( X, Y, X + Y ))

         TypeError: can only concatenate str (not "int") to str
                            attempt to call +  with      'abc'  ,   123.45  <--------------

> i hope that NOW a few of you can see this as a genuine, (reasonable) question.

Python  seems so perfectly  User-friendly that
                   i 'm  so curious (puzzled)  that it doesn't do the very  obvious and easy thing 
                 of giving me this info:

                            attempt to call      +     with      'abc'  ,   123.45  <--------------

From cs at cskk.id.au  Wed Feb 22 17:35:28 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Thu, 23 Feb 2023 09:35:28 +1100
Subject: Python + Vim editor
In-Reply-To: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
References: <eac182f0-32c0-4eb4-aa06-7cec10c6c8a8n@googlegroups.com>
Message-ID: <Y/aYsNcel0tG0Ezw@cskk.homeip.net>

On 21Feb2023 18:00, Hen Hanna <henhanna at gmail.com> wrote:
>what editor do you (all) use to write Python code?  (i use Vim)

vim

From list1 at tompassin.net  Wed Feb 22 21:04:45 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 22 Feb 2023 21:04:45 -0500
Subject: semi colonic
In-Reply-To: <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
Message-ID: <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>

On 2/22/2023 7:58 PM, avi.e.gross at gmail.com wrote:
> Thomas,
> 
> This is one of many little twists I see between languages where one feature
> impacts use or even the need for another feature.
> 
> So can anyone point to places in Python where a semicolon is part of a best
> or even good way to do anything?

Mostly I use it to run small commands on the command line with python 
-c.  e.g.

python -c "import sys;print('\n'.join(sys.path))"

This is handy enough that I wouldn't like to do without.

Another place I use the semicolon (once in a while) is for quick 
debugging.  I might add as line like, perhaps,

import os; print(os.path.exists(filename))

This way I can get rid of the debugging statement by deleting that 
single line.  This is non only quicker but I'm less likely to delete too 
much by mistake.

> Some older languages had simple parsers/compilers that needed some way to
> know when a conceptual line of code was DONE and the semi-colon was a choice
> for making that clear. But some languages seem to only continue looking past
> an end-of-line if they detect some serious reason to assume you are in
> middle of something. An unmatched open parenthesis or square bracket might
> be enough, and in some languages a curly brace.
> 
> Python mainly has a concept of indentation and blank lines as one part of
> the guidance. Continuing lines is possible, if done carefully.
> 
> But consider the lowly comma. Some languages may assume more is to come if
> it is dangled at the end of a line. But in a language that supports a
> dangling comma such as in making a tuple, how is the interpreter to know
> more is to come?
> 
>>>> a = 5,
>>>> a
> (5,)
> 
>>>> a = 5, \
> ...     6
>>>> a
> (5, 6)
> 
> Well, one possible use of a semi-colon is to make short one-liner functions
> like this:
> 
>      def twoByFour(a): sq = a*a; forth = sq*sq; return((sq, forth))
> 
> There is no reason, of course, that could not be done in multiple indented
> lines or other ways.
> 
> So if it was allowed in something like a lambda creation, it could be useful
> but it isn't!
> 
> About the only thing that I can think of is if someone wishes to compress a
> file of python code a bit. The indentation can add up but a semi-colon does
> not solve all such problems.
> 
> Would anything serious break if it was deprecated for use as a statement
> terminator? Then again, is it hurting anything? If it stopped being used
> this way, could it later be introduced as some new language feature or
> operator such as we now have a := b as a reuse of the colon, maybe a
> semicolon could be useful at least until someone decides to allow additional
> Unicode characters!
> 
> Now if there are serious reasons to use semi-colon in python, great. If not,
> it is a historical artifact.
> 
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
> Behalf Of Thomas Passin
> Sent: Wednesday, February 22, 2023 7:24 PM
> To: python-list at python.org
> Subject: Re: Introspecting the variable bound to a function argument
> 
> On 2/22/2023 3:12 PM, Hen Hanna wrote:
>> On Wednesday, February 22, 2023 at 2:32:57 AM UTC-8, Anton Shepelev wrote:
>>> Hello, all.
>>>
>>> Does Python have an instrospection facility that can determine to
>>> which outer variable a function argument is bound, e.g.:
>>>
>>> v1 = 5;
>>> v2 = 5;
>>
>>
>> do some Python coders like to end lines with   ;   ?
> 
> Very few, probably.  It's not harmful but adds unnecessary visual clutter.
> 
>>>
>>>           def f(a):
>>>                      print(black_magic(a))            # or
> black_magic('a')
>>>
>>>           f(v1)            # prints: v1
>>>           f(v2)            # prints: v2
>>>
>>
>> the term  [call by name]  suggests  this should be possible.
>>
>>
>> 30 years ago...  i used to think about this type of thing A LOT ---
>>            -------  CBR, CBV, CBN,   (call by value),    (call by name)....
> etc.
>>
> 
> --
> https://mail.python.org/mailman/listinfo/python-list
> 


From rob.cliffe at btinternet.com  Wed Feb 22 21:20:28 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 23 Feb 2023 02:20:28 +0000
Subject: semi colonic
In-Reply-To: <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
Message-ID: <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>



On 23/02/2023 02:04, Thomas Passin wrote:
> On 2/22/2023 7:58 PM, avi.e.gross at gmail.com wrote:
>>
>>
>> So can anyone point to places in Python where a semicolon is part of 
>> a best
>> or even good way to do anything?
>
> ?I use the semicolon (once in a while) is for quick debugging.? I 
> might add as line like, perhaps,
>
> import os; print(os.path.exists(filename))
>
> This way I can get rid of the debugging statement by deleting that 
> single line.? This is non only quicker but I'm less likely to delete 
> too much by mistake.
>
I do exactly the same.
Rob Cliffe

From scruelt at hotmail.com  Wed Feb 22 21:43:23 2023
From: scruelt at hotmail.com (scruel tao)
Date: Thu, 23 Feb 2023 02:43:23 +0000
Subject: Add angle brackets for required args in argparse
In-Reply-To: <tt0lpr$s0oc$1@dont-email.me>
References: <OSZP286MB0742B742ED736B07B5665A9DA5DD9@OSZP286MB0742.JPNP286.PROD.OUTLOOK.COM>
 <OS3P286MB0740CCCB1D94B91E01BB4B79A5A39@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>
 <mailman.1832.1676481401.20444.python-list@python.org>
 <tt0lpr$s0oc$1@dont-email.me>
Message-ID: <OS3P286MB0740568C7894506D10154669A5AB9@OS3P286MB0740.JPNP286.PROD.OUTLOOK.COM>

Thank you for your workarounds, Mark Bourne.
`metavar` argument should be sufficient  for infrequent use scenarios, and I will consider to use the custom help formatter if necessary.

>>> That's a bit closer to what you asked for, since the required argument
>>> shown in the error message doesn't include the angle brackets.  It also
>>> avoids needing to specify a `metavar` for every positional argument.
>>> However, it is overriding a non-public method of the `HelpFormatter`
>>> class, so might not work across all Python versions if the name or
>>> signature of that method changes (even if it does work with all current
>>> versions, it might break in future).

Your are right to be concerned, that?s why I still think, might the `argparse` can provide a more stable way which can set such format strategy globally, your workarounds are fine to work now, just I have to write the same code to parse either `metaver` or `formatter` every times I use argparse.
Might can have different argparse subclasses, or make some HelpFormatter builtin, so that users won?t need to write them by themselves.

From henhanna at gmail.com  Wed Feb 22 21:25:00 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Wed, 22 Feb 2023 18:25:00 -0800 (PST)
Subject: semi colonic
In-Reply-To: <mailman.1930.1677118854.20444.python-list@python.org>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
Message-ID: <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>

On Wednesday, February 22, 2023 at 6:21:13 PM UTC-8, Rob Cliffe wrote:
> On 23/02/2023 02:04, Thomas Passin wrote: 
> > On 2/22/2023 7:58 PM, avi.e... at gmail.com wrote: 
> >> 
> >> 
> >> So can anyone point to places in Python where a semicolon is part of 
> >> a best 
> >> or even good way to do anything? 
> >
> >  I use the semicolon (once in a while) is for quick debugging.  I 
> >            might add as line like, perhaps, 
> > 
> > import os; print(os.path.exists(filename)) 
> > 
> > This way I can get rid of the debugging statement by deleting that 
> > single line.  This is non only quicker but I'm less likely to delete 
> > too much by mistake. 
> >

> I do exactly the same. 
> Rob Cliffe


i sometimes  put  extra  commas...  as:

                               [  1, 2,  3,  4,     ]

so it is (or may be)  easier  to add things   later.

           -----------  i can think of putting extra final  ;   for the same reason.

From avi.e.gross at gmail.com  Wed Feb 22 22:42:10 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Wed, 22 Feb 2023 22:42:10 -0500
Subject: semi colonic
In-Reply-To: <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
Message-ID: <000b01d94738$cfbe82d0$6f3b8870$@gmail.com>

That seems like a reasonable if limited use of a semi-colon, Thomas.

Of course, most shells will allow a multi-line argument too like some AWK
scripts I have written with a quote on the first line followed by multiple
lines of properly formatted code  and a closing quote.

Python though can get touchy about getting just the right amount of
indentation and simple attempts to break your program up into two lines 

python -c "import sys 
print('\n'.join(sys.path))"


DO not work so well on some shells.

So, yes, I agree. But I tried this on bash under Cygwin on windows using a
"here" document and it worked fine with multiple lines so something to
consider with no semicolons:

$ python <<!
> import sys
> print('\n'.join(sys.path))
> !

/usr/lib/python2.7/site-packages/pylint-1.3.1-py2.7.egg
/usr/lib/python2.7/site-packages/astroid-1.3.4-py2.7.egg
/usr/lib/python2.7/site-packages/six-1.9.0-py2.7.egg
/usr/lib/python27.zip
/usr/lib/python2.7
/usr/lib/python2.7/plat-cygwin
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/lib/python2.7/site-packages
/usr/lib/python2.7/site-packages/gtk-2.0

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Thomas Passin
Sent: Wednesday, February 22, 2023 9:05 PM
To: python-list at python.org
Subject: Re: semi colonic

On 2/22/2023 7:58 PM, avi.e.gross at gmail.com wrote:
> Thomas,
> 
> This is one of many little twists I see between languages where one 
> feature impacts use or even the need for another feature.
> 
> So can anyone point to places in Python where a semicolon is part of a 
> best or even good way to do anything?

Mostly I use it to run small commands on the command line with python -c.
e.g.

python -c "import sys;print('\n'.join(sys.path))"

This is handy enough that I wouldn't like to do without.

Another place I use the semicolon (once in a while) is for quick debugging.
I might add as line like, perhaps,

import os; print(os.path.exists(filename))

This way I can get rid of the debugging statement by deleting that single
line.  This is non only quicker but I'm less likely to delete too much by
mistake.

> Some older languages had simple parsers/compilers that needed some way 
> to know when a conceptual line of code was DONE and the semi-colon was 
> a choice for making that clear. But some languages seem to only 
> continue looking past an end-of-line if they detect some serious 
> reason to assume you are in middle of something. An unmatched open 
> parenthesis or square bracket might be enough, and in some languages a
curly brace.
> 
> Python mainly has a concept of indentation and blank lines as one part 
> of the guidance. Continuing lines is possible, if done carefully.
> 
> But consider the lowly comma. Some languages may assume more is to 
> come if it is dangled at the end of a line. But in a language that 
> supports a dangling comma such as in making a tuple, how is the 
> interpreter to know more is to come?
> 
>>>> a = 5,
>>>> a
> (5,)
> 
>>>> a = 5, \
> ...     6
>>>> a
> (5, 6)
> 
> Well, one possible use of a semi-colon is to make short one-liner 
> functions like this:
> 
>      def twoByFour(a): sq = a*a; forth = sq*sq; return((sq, forth))
> 
> There is no reason, of course, that could not be done in multiple 
> indented lines or other ways.
> 
> So if it was allowed in something like a lambda creation, it could be 
> useful but it isn't!
> 
> About the only thing that I can think of is if someone wishes to 
> compress a file of python code a bit. The indentation can add up but a 
> semi-colon does not solve all such problems.
> 
> Would anything serious break if it was deprecated for use as a 
> statement terminator? Then again, is it hurting anything? If it 
> stopped being used this way, could it later be introduced as some new 
> language feature or operator such as we now have a := b as a reuse of 
> the colon, maybe a semicolon could be useful at least until someone 
> decides to allow additional Unicode characters!
> 
> Now if there are serious reasons to use semi-colon in python, great. 
> If not, it is a historical artifact.
> 
> -----Original Message-----
> From: Python-list 
> <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of 
> Thomas Passin
> Sent: Wednesday, February 22, 2023 7:24 PM
> To: python-list at python.org
> Subject: Re: Introspecting the variable bound to a function argument
> 
> On 2/22/2023 3:12 PM, Hen Hanna wrote:
>> On Wednesday, February 22, 2023 at 2:32:57 AM UTC-8, Anton Shepelev
wrote:
>>> Hello, all.
>>>
>>> Does Python have an instrospection facility that can determine to 
>>> which outer variable a function argument is bound, e.g.:
>>>
>>> v1 = 5;
>>> v2 = 5;
>>
>>
>> do some Python coders like to end lines with   ;   ?
> 
> Very few, probably.  It's not harmful but adds unnecessary visual clutter.
> 
>>>
>>>           def f(a):
>>>                      print(black_magic(a))            # or
> black_magic('a')
>>>
>>>           f(v1)            # prints: v1
>>>           f(v2)            # prints: v2
>>>
>>
>> the term  [call by name]  suggests  this should be possible.
>>
>>
>> 30 years ago...  i used to think about this type of thing A LOT ---
>>            -------  CBR, CBV, CBN,   (call by value),    (call by
name)....
> etc.
>>
> 
> --
> https://mail.python.org/mailman/listinfo/python-list
> 

--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Wed Feb 22 23:26:19 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Wed, 22 Feb 2023 23:26:19 -0500
Subject: semi colonic
In-Reply-To: <000b01d94738$cfbe82d0$6f3b8870$@gmail.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <000b01d94738$cfbe82d0$6f3b8870$@gmail.com>
Message-ID: <08935fe1-3616-8c2b-3dc0-2ec066e5d37b@tompassin.net>

On 2/22/2023 10:42 PM, avi.e.gross at gmail.com wrote:
> That seems like a reasonable if limited use of a semi-colon, Thomas.
> 
> Of course, most shells will allow a multi-line argument too like some AWK
> scripts I have written with a quote on the first line followed by multiple
> lines of properly formatted code  and a closing quote.

"Most shells"... got to include cmd.exe, don't forget.

> Python though can get touchy about getting just the right amount of
> indentation and simple attempts to break your program up into two lines
> 
> python -c "import sys
> print('\n'.join(sys.path))"
> 
> 
> DO not work so well on some shells.
> 
> So, yes, I agree. But I tried this on bash under Cygwin on windows using a
> "here" document and it worked fine with multiple lines so something to
> consider with no semicolons:
> 
> $ python <<!
>> import sys
>> print('\n'.join(sys.path))
>> !
> 
> /usr/lib/python2.7/site-packages/pylint-1.3.1-py2.7.egg
> /usr/lib/python2.7/site-packages/astroid-1.3.4-py2.7.egg
> /usr/lib/python2.7/site-packages/six-1.9.0-py2.7.egg
> /usr/lib/python27.zip
> /usr/lib/python2.7
> /usr/lib/python2.7/plat-cygwin
> /usr/lib/python2.7/lib-tk
> /usr/lib/python2.7/lib-old
> /usr/lib/python2.7/lib-dynload
> /usr/lib/python2.7/site-packages
> /usr/lib/python2.7/site-packages/gtk-2.0
> 
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
> Behalf Of Thomas Passin
> Sent: Wednesday, February 22, 2023 9:05 PM
> To: python-list at python.org
> Subject: Re: semi colonic
> 
> On 2/22/2023 7:58 PM, avi.e.gross at gmail.com wrote:
>> Thomas,
>>
>> This is one of many little twists I see between languages where one
>> feature impacts use or even the need for another feature.
>>
>> So can anyone point to places in Python where a semicolon is part of a
>> best or even good way to do anything?
> 
> Mostly I use it to run small commands on the command line with python -c.
> e.g.
> 
> python -c "import sys;print('\n'.join(sys.path))"
> 
> This is handy enough that I wouldn't like to do without.
> 
> Another place I use the semicolon (once in a while) is for quick debugging.
> I might add as line like, perhaps,
> 
> import os; print(os.path.exists(filename))
> 
> This way I can get rid of the debugging statement by deleting that single
> line.  This is non only quicker but I'm less likely to delete too much by
> mistake.
> 
>> Some older languages had simple parsers/compilers that needed some way
>> to know when a conceptual line of code was DONE and the semi-colon was
>> a choice for making that clear. But some languages seem to only
>> continue looking past an end-of-line if they detect some serious
>> reason to assume you are in middle of something. An unmatched open
>> parenthesis or square bracket might be enough, and in some languages a
> curly brace.
>>
>> Python mainly has a concept of indentation and blank lines as one part
>> of the guidance. Continuing lines is possible, if done carefully.
>>
>> But consider the lowly comma. Some languages may assume more is to
>> come if it is dangled at the end of a line. But in a language that
>> supports a dangling comma such as in making a tuple, how is the
>> interpreter to know more is to come?
>>
>>>>> a = 5,
>>>>> a
>> (5,)
>>
>>>>> a = 5, \
>> ...     6
>>>>> a
>> (5, 6)
>>
>> Well, one possible use of a semi-colon is to make short one-liner
>> functions like this:
>>
>>       def twoByFour(a): sq = a*a; forth = sq*sq; return((sq, forth))
>>
>> There is no reason, of course, that could not be done in multiple
>> indented lines or other ways.
>>
>> So if it was allowed in something like a lambda creation, it could be
>> useful but it isn't!
>>
>> About the only thing that I can think of is if someone wishes to
>> compress a file of python code a bit. The indentation can add up but a
>> semi-colon does not solve all such problems.
>>
>> Would anything serious break if it was deprecated for use as a
>> statement terminator? Then again, is it hurting anything? If it
>> stopped being used this way, could it later be introduced as some new
>> language feature or operator such as we now have a := b as a reuse of
>> the colon, maybe a semicolon could be useful at least until someone
>> decides to allow additional Unicode characters!
>>
>> Now if there are serious reasons to use semi-colon in python, great.
>> If not, it is a historical artifact.
>>
>> -----Original Message-----
>> From: Python-list
>> <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of
>> Thomas Passin
>> Sent: Wednesday, February 22, 2023 7:24 PM
>> To: python-list at python.org
>> Subject: Re: Introspecting the variable bound to a function argument
>>
>> On 2/22/2023 3:12 PM, Hen Hanna wrote:
>>> On Wednesday, February 22, 2023 at 2:32:57 AM UTC-8, Anton Shepelev
> wrote:
>>>> Hello, all.
>>>>
>>>> Does Python have an instrospection facility that can determine to
>>>> which outer variable a function argument is bound, e.g.:
>>>>
>>>> v1 = 5;
>>>> v2 = 5;
>>>
>>>
>>> do some Python coders like to end lines with   ;   ?
>>
>> Very few, probably.  It's not harmful but adds unnecessary visual clutter.
>>
>>>>
>>>>            def f(a):
>>>>                       print(black_magic(a))            # or
>> black_magic('a')
>>>>
>>>>            f(v1)            # prints: v1
>>>>            f(v2)            # prints: v2
>>>>
>>>
>>> the term  [call by name]  suggests  this should be possible.
>>>
>>>
>>> 30 years ago...  i used to think about this type of thing A LOT ---
>>>             -------  CBR, CBV, CBN,   (call by value),    (call by
> name)....
>> etc.
>>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
> 
> --
> https://mail.python.org/mailman/listinfo/python-list
> 


From greg.ewing at canterbury.ac.nz  Thu Feb 23 01:15:59 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Feb 2023 19:15:59 +1300
Subject: Introspecting the variable bound to a function argument
In-Reply-To: <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
Message-ID: <k5oel2Fl6h0U1@mid.individual.net>

On 23/02/23 9:12 am, Hen Hanna wrote:
> On Wednesday, February 22, 2023 at 2:32:57 AM UTC-8, Anton Shepelev wrote:
>>          def f(a):
>>                     print(black_magic(a))            # or black_magic('a')
>>
>>          f(v1)            # prints: v1
>>          f(v2)            # prints: v2
>>
> 
> the term  [call by name]  suggests  this should be possible.

But Python doesn't use call-by-name or anything remotely like it.

(Even if it did, the word "name" in that context doesn't mean
what it sounds like it means. The Algol docs used some words in
weird ways.)

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Feb 23 01:27:30 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Feb 2023 19:27:30 +1300
Subject: semi colonic
In-Reply-To: <mailman.1927.1677113927.20444.python-list@python.org>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <mailman.1927.1677113927.20444.python-list@python.org>
Message-ID: <k5ofamFlaeeU1@mid.individual.net>

On 23/02/23 1:58 pm, avi.e.gross at gmail.com wrote:

> Would anything serious break if it was deprecated for use as a statement
> terminator?

Well, it would break all the code of people who like to
write code that way. They might get a bit miffed if we
decide that their code is not serious. :-)

On the other hand, if they really want to, they will still
be able to abuse semicolons by doing this sort of thing:

a = 5; pass
b = 7; pass
c = a * b; pass

Then everyone will know it's some really serious code!

-- 
Greg

From greg.ewing at canterbury.ac.nz  Thu Feb 23 01:37:37 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 23 Feb 2023 19:37:37 +1300
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
Message-ID: <k5oftlFlct6U1@mid.individual.net>

On 23/02/23 9:37 am, Hen Hanna wrote:
>                  for the first  several weeks... whenever i used Python...  all i could think of....was   --------  this is really Lisp (inside) with  a thin veil of   Java/Pascal syntax..........
> 
>                 -----  that  everything is  first converted  (macro-expanded)  into (intermediated) Lisp code, and then.........

I once toyed with the idea of implementing a Python compiler
by translating it into Scheme and then feeding it to a Scheme
compiler.

But I quickly realised that, although Scheme and Python are
both dynamically-typed languages, Python is way *more* dynamic
than Scheme.

So without doing some very serious static analysis, the best
I could do would be just putting together calls to runtime
support routines that implement all the dynamic dispatching
that Python does for its operators, etc., and the result
wouldn't be much better than an interpreter.

There are some similarities between Python and Lisp-family
languages, but really Python is its own thing.

-- 
Greg

From rob.cliffe at btinternet.com  Sat Feb 18 19:17:02 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Sun, 19 Feb 2023 00:17:02 +0000
Subject: LRU cache
In-Reply-To: <DB6PR01MB3895ED139D2D63B217EC923983A69@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <DB6PR01MB3895ED139D2D63B217EC923983A69@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <0b82d771-3e04-937f-9ef8-7c03bb11a646@btinternet.com>



On 18/02/2023 17:19, Albert-Jan Roskam wrote:
>
>
> On Feb 18, 2023 17:28, Rob Cliffe via Python-list 
> <python-list at python.org> wrote:
>
>     On 18/02/2023 15:29, Thomas Passin wrote:
>     > On 2/18/2023 5:38 AM, Albert-Jan Roskam wrote:
>     >> ??? I sometimes use this trick, which I learnt from a book by
>     Martelli.
>     >> ??? Instead of try/except, membership testing with "in"
>     >> (__contains__) might
>     >> ??? be faster. Probably "depends". Matter of measuring.
>     >> ??? def somefunc(arg, _cache={}):
>     >> ??? ? ? if len(_cache) > 10 ** 5:
>     >> ??? ? ? ? ? _cache.pop()
>     >> ??? ? ? try:
>     >> ??? ? ? ? ? return _cache[arg]
>     >> ??? ? ? except KeyError:
>     >> ??? ? ? ? ? result = expensivefunc(arg)
>     >> ??? ? ? ? ? _cache[arg] = result
>     >> ??? ? ? ? ? return result
>     >> ??? Albert-Jan
>     >
>     > _cache.get(arg) should be a little faster and use slightly fewer
>     > resources than the try/except.
>     >
>     Provided that you can provide a default value to get() which will
>     never
>     be a genuine "result".
>
>
> =====
>
> This might be better than None:
> _cache.get(arg, Ellipsis)
>
>
A common strategy is to have a dedicated sentinel object.? E.g. (untested):
IMPOSSIBLE_RESULT = object()
...
 ??? ??? if _cache.get(arg, IMPOSSIBLE_RESULT) == IMPOSSIBLE_RESULT:
 ??? ??? ??? # arg was not in the cache
 ??? ??? ??? ...

From barry at barrys-emacs.org  Thu Feb 23 02:57:09 2023
From: barry at barrys-emacs.org (Barry)
Date: Thu, 23 Feb 2023 07:57:09 +0000
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <84b38a83-5113-481f-8eb1-720401908d24n@googlegroups.com>
References: <84b38a83-5113-481f-8eb1-720401908d24n@googlegroups.com>
Message-ID: <EAA85A55-F12A-4DE4-803D-D08643E782F4@barrys-emacs.org>



> On 23 Feb 2023, at 01:39, Hen Hanna <henhanna at gmail.com> wrote:
> 
> ?On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote:
>> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: 
>>>> py bug.py 
>>> Traceback (most recent call last): 
>>> File "C:\Usenet\bug.py", line 5, in <module> 
>>> print( a + 12 ) 
>>> TypeError: can only concatenate str (not "int") to str 
>>> 
>>> 
>>> Why doesn't Python (error msg) do the obvious thing and tell me 
>>> WHAT the actual (offending, arg) values are ? 
>>> 
>>> In many cases, it'd help to know what string the var A had , when the error occurred. 
>>> ------------ i wouldn't have to put print(a) just above, to see. 
>>> 
>>> 
>>> 
>>> 
>>> ( pypy doesn't do that either, but Python makes programming (debugging) so easy that i hardly feel any inconvenience.)
> 
> 
> i see that my example would be (even)  clearER with this one-line change:
> 
>           py bug.py 
> 
>         Traceback (most recent call last): 
> 
>           File "C:\Usenet\bug.py", line 5, in <module>
>                         map( Func, fooBar( X, Y, X + Y ))
> 
>         TypeError: can only concatenate str (not "int") to str
>                            attempt to call +  with      'abc'  ,   123.45  <--------------
> 
>> i hope that NOW a few of you can see this as a genuine, (reasonable) question.
> 
> Python  seems so perfectly  User-friendly that
>                   i 'm  so curious (puzzled)  that it doesn't do the very  obvious and easy thing 
>                 of giving me this info:
> 
>                            attempt to call      +     with      'abc'  ,   123.45  <--------------

It is not easy to do that in a robust and reliable way for any object.
You can end up in the code to generate the error message itself breaking.
For example using unbounded CPU time when attempting to get the string repr of the variable.

Barry

> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From robin at reportlab.com  Thu Feb 23 08:03:26 2023
From: robin at reportlab.com (Robin Becker)
Date: Thu, 23 Feb 2023 13:03:26 +0000
Subject: lxml with python-3.12.0a5
Message-ID: <a7091c28-61ae-9e5f-9737-0ff21106fe72@everest.reportlab.co.uk>

I'm trying to test python-3.12.0a5 and need to install lxml.

My wheel build for lxml fails with errors like this


src/lxml/etree.c: In function ?__Pyx_PyIndex_AsSsize_t?:
src/lxml/etree.c:270404:45: error: ?PyLongObject? {aka ?struct _longobject?} has no member named ?ob_digit?
270404 |     const digit* digits = ((PyLongObject*)b)->ob_digit;
        |                                             ^~

I'm using archlinux which has gcc 12.2.1. I tested and lxml will build with 3.11.2.

I imagine this is part of ongoing changes to the python core, but perhaps there's some simple workaround.

Anyone know how to get around this problem. I did try rebuilding the cpython stuff using make, but that also failed.

--
Robin Becker

From mats at wichmann.us  Thu Feb 23 09:47:04 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Thu, 23 Feb 2023 07:47:04 -0700
Subject: lxml with python-3.12.0a5
In-Reply-To: <a7091c28-61ae-9e5f-9737-0ff21106fe72@everest.reportlab.co.uk>
References: <a7091c28-61ae-9e5f-9737-0ff21106fe72@everest.reportlab.co.uk>
Message-ID: <42284c49-430c-67d0-6dcf-e86f4d2041ea@wichmann.us>

On 2/23/23 06:03, Robin Becker wrote:
> I'm trying to test python-3.12.0a5 and need to install lxml.
> 
> My wheel build for lxml fails with errors like this
> 
> 
> src/lxml/etree.c: In function ?__Pyx_PyIndex_AsSsize_t?:
> src/lxml/etree.c:270404:45: error: ?PyLongObject? {aka ?struct 
> _longobject?} has no member named ?ob_digit?
> 270404 |???? const digit* digits = ((PyLongObject*)b)->ob_digit;
>  ?????? |???????????????????????????????????????????? ^~
> 
> I'm using archlinux which has gcc 12.2.1. I tested and lxml will build 
> with 3.11.2.
> 
> I imagine this is part of ongoing changes to the python core, but 
> perhaps there's some simple workaround.
> 
> Anyone know how to get around this problem. I did try rebuilding the 
> cpython stuff using make, but that also failed.

I seem to always have trouble with lxml (which I know doesn't help).

The cause would seem to be this:

     GH-101291: Refactor the `PyLongObject` struct into object header 
and PyLongValue struct. (GH-101292)






From avi.e.gross at gmail.com  Thu Feb 23 10:15:49 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Thu, 23 Feb 2023 10:15:49 -0500
Subject: semi colonic
In-Reply-To: <k5ofamFlaeeU1@mid.individual.net>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <mailman.1927.1677113927.20444.python-list@python.org>
 <k5ofamFlaeeU1@mid.individual.net>
Message-ID: <002101d94799$b67883a0$23698ae0$@gmail.com>

Greg,

How did you know that was the method I used to indicate I had properly
debugged and tested a line of code?

a = 5; pass
b = 7; pass
c = a * b; pass

Then I switched to using comments:

a = 5 #  pass
b = 7 # pass
c = a * b # fail

And would you believe it still worked!

OK, I am just kidding if anyone is taking this seriously.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Greg Ewing via Python-list
Sent: Thursday, February 23, 2023 1:28 AM
To: python-list at python.org
Subject: Re: semi colonic

On 23/02/23 1:58 pm, avi.e.gross at gmail.com wrote:

> Would anything serious break if it was deprecated for use as a 
> statement terminator?

Well, it would break all the code of people who like to write code that way.
They might get a bit miffed if we decide that their code is not serious. :-)

On the other hand, if they really want to, they will still be able to abuse
semicolons by doing this sort of thing:

a = 5; pass
b = 7; pass
c = a * b; pass

Then everyone will know it's some really serious code!

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list


From mats at wichmann.us  Thu Feb 23 13:09:29 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Thu, 23 Feb 2023 11:09:29 -0700
Subject: lxml with python-3.12.0a5
In-Reply-To: <42284c49-430c-67d0-6dcf-e86f4d2041ea@wichmann.us>
References: <a7091c28-61ae-9e5f-9737-0ff21106fe72@everest.reportlab.co.uk>
 <42284c49-430c-67d0-6dcf-e86f4d2041ea@wichmann.us>
Message-ID: <630af59b-fe9e-3bbc-7715-89ba3e27a26a@wichmann.us>

On 2/23/23 07:47, Mats Wichmann wrote:
> On 2/23/23 06:03, Robin Becker wrote:
>> I'm trying to test python-3.12.0a5 and need to install lxml.
>>
>> My wheel build for lxml fails with errors like this
>>
>>
>> src/lxml/etree.c: In function ?__Pyx_PyIndex_AsSsize_t?:
>> src/lxml/etree.c:270404:45: error: ?PyLongObject? {aka ?struct 
>> _longobject?} has no member named ?ob_digit?
>> 270404 |???? const digit* digits = ((PyLongObject*)b)->ob_digit;
>> ??????? |???????????????????????????????????????????? ^~
>>
>> I'm using archlinux which has gcc 12.2.1. I tested and lxml will build 
>> with 3.11.2.
>>
>> I imagine this is part of ongoing changes to the python core, but 
>> perhaps there's some simple workaround.
>>
>> Anyone know how to get around this problem. I did try rebuilding the 
>> cpython stuff using make, but that also failed.
> 
> I seem to always have trouble with lxml (which I know doesn't help).
> 
> The cause would seem to be this:
> 
>  ??? GH-101291: Refactor the `PyLongObject` struct into object header 
> and PyLongValue struct. (GH-101292)

So it looks to me like cython was affected, and has patched themselves. 
It's possible regenerating with a patched cython will clear up the build 
problem (something which the lxml project takes pains to tell you that 
you don't really want to do :)

From bowman at montana.com  Thu Feb 23 01:13:28 2023
From: bowman at montana.com (rbowman)
Date: 23 Feb 2023 06:13:28 GMT
Subject: semi colonic
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
Message-ID: <k5oeg7FkujpU3@mid.individual.net>

On Wed, 22 Feb 2023 18:25:00 -0800 (PST), Hen Hanna wrote:


> i sometimes  put  extra  commas...  as:
> 
>                                [  1, 2,  3,  4,     ]
> 
> so it is (or may be)  easier  to add things   later.


That can bite you with things like JSON that aren't very forgiving. The 
same can be said for single quotes that may or may not work as intended. 

From henhanna at gmail.com  Thu Feb 23 03:08:31 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Thu, 23 Feb 2023 00:08:31 -0800 (PST)
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <mailman.1935.1677139044.20444.python-list@python.org>
References: <EAA85A55-F12A-4DE4-803D-D08643E782F4@barrys-emacs.org>
 <84b38a83-5113-481f-8eb1-720401908d24n@googlegroups.com>
 <mailman.1935.1677139044.20444.python-list@python.org>
Message-ID: <59af3ca8-bad9-4f91-a631-f84e38bce02en@googlegroups.com>

On Wednesday, February 22, 2023 at 11:57:45 PM UTC-8, Barry wrote:
> > On 23 Feb 2023, at 01:39, Hen Hanna <henh... at gmail.com> wrote:
> > 
> > ?On Wednesday, February 22, 2023 at 3:46:21 PM UTC-8, Hen Hanna wrote: 
> >> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote: 
> >>>> py bug.py 
> >>> Traceback (most recent call last): 
> >>> File "C:\Usenet\bug.py", line 5, in <module> 
> >>> print( a + 12 ) 
> >>> TypeError: can only concatenate str (not "int") to str 
> >>> 
> >>> 
> >>> Why doesn't Python (error msg) do the obvious thing and tell me 
> >>> WHAT the actual (offending, arg) values are ? 
> >>> 
> >>> In many cases, it'd help to know what string the var A had , when the error occurred. 
> >>> ------------ i wouldn't have to put print(a) just above, to see. 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> ( pypy doesn't do that either, but Python makes programming (debugging) so easy that i hardly feel any inconvenience.) 
> > 
> > 
> > i see that my example would be (even) clearER with this one-line change: 
> > 
> > py bug.py 
> > 
> > Traceback (most recent call last): 
> > 
> > File "C:\Usenet\bug.py", line 5, in <module> 
> > map( Func, fooBar( X, Y, X + Y )) 
> > 
> > TypeError: can only concatenate str (not "int") to str 
> >                       attempt to call     +      with       'abc',      123       <-------------- 
> > 
> >> i hope that NOW a few of you can see this as a genuine, (reasonable) question. 
> > 
> > Python seems so perfectly User-friendly that 
> > i 'm so curious (puzzled) that it doesn't do the very obvious and easy thing 
> > of giving me this info: 
> > 
> >                         attempt to call     +    with       'abc',       123     <--------------

> It is not easy to do that in a robust and reliable way for any object. 
> You can end up in the code to generate the error message itself breaking. 
> For example using unbounded CPU time when attempting to get the string repr of the variable. 
> 
> Barry 
> 


 Python VM  is seeing an "int" object (123)   (and telling me that)   ...   so it should be easy to print that "int" object 


                What does  Python VM    know ?   and when does it know it ?


it seems like  it is being playful, teasing (or mean),    and   hiding  the ball from me

From dieter at handshake.de  Thu Feb 23 13:17:03 2023
From: dieter at handshake.de (Dieter Maurer)
Date: Thu, 23 Feb 2023 19:17:03 +0100
Subject: semi colonic
In-Reply-To: <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
Message-ID: <25591.44447.791951.857945@ixdm.fritz.box>

Thomas Passin wrote at 2023-2-22 21:04 -0500:
>On 2/22/2023 7:58 PM, avi.e.gross at gmail.com wrote:
> ...
>> So can anyone point to places in Python where a semicolon is part of a best
>> or even good way to do anything?
>
>Mostly I use it to run small commands on the command line with python
>-c.  e.g.
>
>python -c "import sys;print('\n'.join(sys.path))"
>
>This is handy enough that I wouldn't like to do without.
>
>Another place I use the semicolon (once in a while) is for quick
>debugging.  I might add as line like, perhaps,
>
>import os; print(os.path.exists(filename))

I also see, `;` occasionally in `*.pth` files.

From rob.cliffe at btinternet.com  Wed Feb 22 16:31:14 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Wed, 22 Feb 2023 21:31:14 +0000
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
Message-ID: <002f6ca7-e547-f566-51e2-fb1f030d6f0d@btinternet.com>



On 22/02/2023 20:05, Hen Hanna wrote:
> Python makes programming (debugging) so easy
I agree with that!
Rob Cliffe

From rob.cliffe at btinternet.com  Wed Feb 22 21:10:52 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 23 Feb 2023 02:10:52 +0000
Subject: semi colonic
In-Reply-To: <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
Message-ID: <8bed7807-d107-acd6-1456-15731d4e2773@btinternet.com>



On 23/02/2023 00:58, avi.e.gross at gmail.com wrote:
> So can anyone point to places in Python where a semicolon is part of a best
> or even good way to do anything?
>
>
Yes.? Take this bit of toy code which I just dreamed up.? (Of course it 
is toy code; don't bother telling me how it could be written better.)? 
If it looks a bit ragged, pretend it is in a fixed font.

if dow==0: day="Mon"; calcPay()
if dow==1: day="Tue"; calcPay()
if dow==2: day="Wed"; calcPay()
if dow==3: day="Thu"; calcPay()
if dow==4: day="Fri"; calcpay()
if dow==5: day="Sat"; calcPay(rate=1.5)
if dow==6: day="Sun"; calcPay(rate=2)

The point is: when you have several short bits of code with an identical 
or similar pattern, *vertically aligning* the corresponding parts can 
IMO make it much easier to read the code and easier to spot errors.
Compare this:

if dow==0:
 ??? day="Mon"
 ??? calcPay()
if dow==1:
 ??? day="Tue"
 ??? calcPay()
if dow==2:
 ??? day="Wed"
 ??? calcPay()
if dow==3:
 ??? day="Thu"
 ??? calcPay()
if dow==4:
 ??? day="Fri"
 ??? calcpay()
if dow==5:
 ??? day="Sat"
 ??? calcPay(rate=1.5)
if dow==6:
 ??? day="Sun"
 ??? calcPay(rate=2)

Not so easy to spot the mistake now, is it?
Not to mention the saving of vertical space.

Best wishes
Rob Cliffe
PS If you really care, I can send you a more complicated example of real 
code from one of my programs which is HUGELY more readable when laid out 
in this way.


From avi.e.gross at gmail.com  Thu Feb 23 13:57:09 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Thu, 23 Feb 2023 13:57:09 -0500
Subject: semi colonic
In-Reply-To: <8bed7807-d107-acd6-1456-15731d4e2773@btinternet.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <8bed7807-d107-acd6-1456-15731d4e2773@btinternet.com>
Message-ID: <000c01d947b8$a20659a0$e6130ce0$@gmail.com>


That is a reasonable use, Rob, albeit I would refactor that example in quite a few ways so the need for a semicolon disappears even for lining things up.

So to extrapolate, perhaps a related example might be as simple as wanting to initialialize multiple variables together might suffice as in:

if dow == 0: hours_worked = 8; overtime = False

Of course some monstrosities are now possible for such a scenario such as


if dow == 0:  hours_worked, overtime  =  8,  False

Not that readable. 

I repeat, there is nothing wrong with a language having a feature like a semi-colon even if it is mainly syntactic sugar. Just wondering if it was widely used or even essential. My thought was that python evolved when some languages really needed a terminator but as it went another way, using indentation and sometimes blank lines, ...

I am not sure what Dieter meant about seeing semicolons in .pth files. I expect to see them in all kinds of files containing python code or anything created with a structure that chooses to include it.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Rob Cliffe via Python-list
Sent: Wednesday, February 22, 2023 9:11 PM
To: python-list at python.org
Subject: Re: semi colonic



On 23/02/2023 00:58, avi.e.gross at gmail.com wrote:
> So can anyone point to places in Python where a semicolon is part of a 
> best or even good way to do anything?
>
>
Yes.  Take this bit of toy code which I just dreamed up.  (Of course it is toy code; don't bother telling me how it could be written better.) If it looks a bit ragged, pretend it is in a fixed font.

if dow==0: day="Mon"; calcPay()
if dow==1: day="Tue"; calcPay()
if dow==2: day="Wed"; calcPay()
if dow==3: day="Thu"; calcPay()
if dow==4: day="Fri"; calcpay()
if dow==5: day="Sat"; calcPay(rate=1.5)
if dow==6: day="Sun"; calcPay(rate=2)

The point is: when you have several short bits of code with an identical or similar pattern, *vertically aligning* the corresponding parts can IMO make it much easier to read the code and easier to spot errors.
Compare this:

if dow==0:
     day="Mon"
     calcPay()
if dow==1:
     day="Tue"
     calcPay()
if dow==2:
     day="Wed"
     calcPay()
if dow==3:
     day="Thu"
     calcPay()
if dow==4:
     day="Fri"
     calcpay()
if dow==5:
     day="Sat"
     calcPay(rate=1.5)
if dow==6:
     day="Sun"
     calcPay(rate=2)

Not so easy to spot the mistake now, is it?
Not to mention the saving of vertical space.

Best wishes
Rob Cliffe
PS If you really care, I can send you a more complicated example of real code from one of my programs which is HUGELY more readable when laid out in this way.

--
https://mail.python.org/mailman/listinfo/python-list


From rob.cliffe at btinternet.com  Thu Feb 23 06:07:42 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 23 Feb 2023 11:07:42 +0000
Subject: semi colonic
In-Reply-To: <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
Message-ID: <9ef85a5e-05d8-8887-a714-91dd47634e0d@btinternet.com>



On 23/02/2023 02:25, Hen Hanna wrote:
>
> i sometimes  put  extra  commas...  as:
>
>                                 [  1, 2,  3,  4,     ]
That is a good idea.
Even more so when the items are on separate lines:
 ??? [
 ??? ??? "spam",
 ??? ??? "eggs",
 ??? ??? "cheese",
 ??? ]
and you may want to change the order.
>
> so it is (or may be)  easier  to add things   later.
>
>             -----------  i can think of putting extra final  ;   for the same reason.
That may not be such a good idea.? Writing multiple statements on one 
line is generally discouraged (notwithstanding that IMO it is 
occasionally appropriate).

Rob Cliffe

From grant.b.edwards at gmail.com  Thu Feb 23 14:27:44 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Thu, 23 Feb 2023 11:27:44 -0800 (PST)
Subject: semi colonic
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
 <k5oeg7FkujpU3@mid.individual.net>
Message-ID: <63f7be30.020a0220.e3cd1.3e77@mx.google.com>

On 2023-02-23, rbowman <bowman at montana.com> wrote:
> On Wed, 22 Feb 2023 18:25:00 -0800 (PST), Hen Hanna wrote:
>
>> i sometimes  put  extra  commas...  as:
>> 
>>                                [  1, 2,  3,  4,     ]
>> 
>> so it is (or may be)  easier  to add things   later.
>
> That can bite you with things like JSON that aren't very forgiving.

Oh, how I hate that about JSON...



From rob.cliffe at btinternet.com  Wed Feb 22 14:08:08 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Wed, 22 Feb 2023 19:08:08 +0000
Subject: Line continuation and comments
In-Reply-To: <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
Message-ID: <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>



On 22/02/2023 15:23, Paul Bryan wrote:
> Adding to this, there should be no reason now in recent versions of
> Python to ever use line continuation. Black goes so far as to state
> "backslashes are bad and should never be used":
>
> https://black.readthedocs.io/en/stable/the_black_code_style/future_style.html#using-backslashes-for-with-statements

def someFunc():
 ??? HelpText = """\
Left click:???????????? Open spam
Shift + Left click:???? Cook spam
Right click:??????????? Crack egg
Shift + Right click:??? Fry egg
"""

The initial backslash aligns the first line with the others (in a fixed 
font of course).
Best wishes
Rob Cliffe

From jf_byrnes at comcast.net  Thu Feb 23 15:29:28 2023
From: jf_byrnes at comcast.net (Jim Byrnes)
Date: Thu, 23 Feb 2023 14:29:28 -0600
Subject: Not receiving posts
Message-ID: <f2a406d7-f12c-a8e3-c133-6279e0cc923f@comcast.net>

I have been reading the python-list for some time now. At first via 
gemane and since it's demise via a subscription. Recently I noticed that 
I have not received any emails for quite sometime.

I tried resubscribing but still have received no emails from the list. 
To my knowledge I have done nothing to warrant removal.

Can you please check and see what I need to do to start receiving emails 
once more.

regards,  Jim Byrnes

From grant.b.edwards at gmail.com  Thu Feb 23 15:56:24 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Thu, 23 Feb 2023 12:56:24 -0800 (PST)
Subject: Not receiving posts
References: <f2a406d7-f12c-a8e3-c133-6279e0cc923f@comcast.net>
Message-ID: <63f7d2f8.020a0220.6ce17.4bdc@mx.google.com>

On 2023-02-23, Jim Byrnes <jf_byrnes at comcast.net> wrote:

> I have been reading the python-list for some time now. At first via 
> gemane and since it's demise via a subscription.

FWIW, gmane is still there, and still working fine. I read this list
by pointing slrn at news.gmane.io

For whatever reason, python-list decided to stop accepting postings
via gmane, so postings do have to be e-mailed. I use a custom "inews"
app to handle that:

    https://github.com/GrantEdwards/hybrid-inews

[The old gmane webUI apparently died and couldn't be resurrected.]



From avi.e.gross at gmail.com  Thu Feb 23 15:56:54 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Thu, 23 Feb 2023 15:56:54 -0500
Subject: semi colonic
In-Reply-To: <63f7be30.020a0220.e3cd1.3e77@mx.google.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
 <k5oeg7FkujpU3@mid.individual.net>
 <63f7be30.020a0220.e3cd1.3e77@mx.google.com>
Message-ID: <002801d947c9$5cacb000$16061000$@gmail.com>

Grant,

I am not sure it is fair to blame JSON for a design choice.

Use of commas can be done many ways in many contexts. 

One context is a sort of placeholder. Can you have a language where a
function has multiple arguments and you can skip some as in:

Func(a,b,c)
Func(a, b,)
Func(a,,)

Or even
Func(a,,c)

The missing arguments in such a language may be viewed as some sort of NULL
or take a default value as a possibility.

So what if you have a language where a list or tuple or other data structure
incorporates a similar idea when created or used. If I have a matrix and I
want every entry in row 4, meaning all columns, can I ask for mat[4,] and it
means something else than mat[4] which may return a vector instead of a
matrix?

There are tons of such ideas that are choices. Python allows a SINGLE comma
here but multiple are an error:

>>> a=1
>>> a
1
>>> a=1,
>>> a
(1,)
>>> a=1,,
SyntaxError: incomplete input

So why not allow MULTIPLE commas and ignore them? It is a choice!

Here is a scenario where a trailing comma is an error:

>>> a,b,,, = range(5)
SyntaxError: invalid syntax
>>> a,b,_,_,_ = range(5)
>>> a,b,*_ = range(5)

The way to deal here with more items is to use * in front of the last one to
gather any strays.

But as _ is simply reused in the middle example and meant to be ignored, why
do you need it if you would simply allow multiple commas? Short answer is
they did not choose to design it that way. The places in python that do
allow a trailing "," will allow only one. Beyond that, they assume you are
making an error. So if someone wants to make a list of 5 things in
alphabetical order but forgets a few, they cannot write:

mylist = [first, , third,  , , ]

and then let the code run to be enhanced later with their reminder. What
they can do is write this:

mylist = [first,
          #,
          third,
          #,
          #,
          ]

The reminders are now simply well-placed comments.

Now we could debate the design of JSON and some enhancements people have
made for other more portable data structures. I think it reasonable that
they decided to stick to working with fully-formatted data structures and
guess what? If I make a list or tuple or other data structures in python
with a trailing comma, it is NOT stored that way and if you display it,
there is no trailing comma shown. It is fully JSON compatible in some sense:

>>> import json
>>> mynest = [1,2, [3, 4,], 5,]
>>> mynest
[1, 2, [3, 4], 5]
>>> json.dumps(mynest)
'[1, 2, [3, 4], 5]'
>>> json.dumps([1,2, [3, 4,], 5,])
'[1, 2, [3, 4], 5]'
>>> json.loads(json.dumps(mynest))
[1, 2, [3, 4], 5]

So when are you running into problems? Is it when reading something from a
file using a function expecting properly formatted JSON?





-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Grant Edwards
Sent: Thursday, February 23, 2023 2:28 PM
To: python-list at python.org
Subject: Re: semi colonic

On 2023-02-23, rbowman <bowman at montana.com> wrote:
> On Wed, 22 Feb 2023 18:25:00 -0800 (PST), Hen Hanna wrote:
>
>> i sometimes  put  extra  commas...  as:
>> 
>>                                [  1, 2,  3,  4,     ]
>> 
>> so it is (or may be)  easier  to add things   later.
>
> That can bite you with things like JSON that aren't very forgiving.

Oh, how I hate that about JSON...


-- 
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Thu Feb 23 15:26:53 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Thu, 23 Feb 2023 15:26:53 -0500
Subject: semi colonic
In-Reply-To: <9ef85a5e-05d8-8887-a714-91dd47634e0d@btinternet.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
 <9ef85a5e-05d8-8887-a714-91dd47634e0d@btinternet.com>
Message-ID: <002001d947c5$2ad08ba0$8071a2e0$@gmail.com>

Rob,

It depends. Some purists say python abhors one liners. Well, I politely disagree and I enjoyed this book which shows how to write some quite compressed one-liners or nearly so.

Python One-Liners: Write Concise, Eloquent Python Like a Professional Illustrated Edition
by Christian Mayer (Author)

https://www.amazon.com/Python-One-Liners-Concise-Eloquent-Professional/dp/1718500505/ref=sr_1_1?crid=2MMIRHGLR3GHN&keywords=python+one+liners&qid=1677183160&sprefix=python+one+liner%2Caps%2C93&sr=8-1

The reality is that python is chock full of constructs that make one-liners easy and perhaps make a need for semi-colons less crucial.

An example is a comprehension like:

[x*y for x in range(10) for y in range(10) if x != y ]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 3, 4, 5, 6, 7, 8, 9, 0, 2, 6, 8, 10, 12, 14, 16, 18, 0, 3, 6, 12, 15, 18, 21, 24, 27, 0, 4, 8, 12, 20, 24, 28, 32, 36, 0, 5, 10, 15, 20, 30, 35, 40, 45, 0, 6, 12, 18, 24, 30, 42, 48, 54, 0, 7, 14, 21, 28, 35, 42, 56, 63, 0, 8, 16, 24, 32, 40, 48, 56, 72, 0, 9, 18, 27, 36, 45, 54, 63, 72]

How many lines of code would it take to make than nonsense using an initializer for an empty list and nested loops and an "if" statement?

A barely longer one-liner add more functionality with no added lines or semicolons:

[(x*y, x+y, x>=y) for x in range(10) for y in range(10) if x != y  ]
[(0, 1, False), (0, 2, False), ..., (72, 17, True)]

Examples of all kinds of such things about including seemingly trivial things like how a "with" statement lets you hide lots of code to do when entering and exiting use of an object. I have earlier mentioned the way packing and unpacking can effectively replace many lines of code with one.

So the pythonic way often is not so much to do things one many lines but often to do things in a way that a unit of logic often can fit on one screen by using what the language offers judiciously even if you do not put multiple statement with semicolons on one line.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Rob Cliffe via Python-list
Sent: Thursday, February 23, 2023 6:08 AM
To: python-list at python.org
Subject: Re: semi colonic



On 23/02/2023 02:25, Hen Hanna wrote:
>
> i sometimes  put  extra  commas...  as:
>
>                                 [  1, 2,  3,  4,     ]
That is a good idea.
Even more so when the items are on separate lines:
     [
         "spam",
         "eggs",
         "cheese",
     ]
and you may want to change the order.
>
> so it is (or may be)  easier  to add things   later.
>
>             -----------  i can think of putting extra final  ;   for the same reason.
That may not be such a good idea.  Writing multiple statements on one line is generally discouraged (notwithstanding that IMO it is occasionally appropriate).

Rob Cliffe
--
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Thu Feb 23 15:08:49 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Thu, 23 Feb 2023 15:08:49 -0500
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <002f6ca7-e547-f566-51e2-fb1f030d6f0d@btinternet.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <002f6ca7-e547-f566-51e2-fb1f030d6f0d@btinternet.com>
Message-ID: <001b01d947c2$a4e2ba10$eea82e30$@gmail.com>

Rob,

There are lots of nifty features each of us might like and insist make much
more sense than what others say they want.

Sometimes the answer is to not satisfy most of those demands but provide
TOOLS they can use to do things for themselves.

As you agree, many of us have found all kinds of tools that help with
debugging and frankly, some of them use it to the point of annoying others
who would rather avoid them. An example is type hints that can get quite
detailed and obscure the outline of your program and are ignored by the
parser and only relevant for some linter or other such program.

I am thinking of what would happen if I created several fairly long or
complex data structures and tried to add them. Say I have a dictionary
containing millions of entries including every conceivable UNICODE character
as well as very complex values such as other dictionaries or strings
containing entire books. My other data structure might be a forest
containing many smaller trees, such as the output of some machine learning
models or perhaps showing every possible game of chess up to 50 moves deep
along with an evaluation of the relative strength of the board to one
player.

I then accidentally write code that contains:

   big_dic + forest_trees

Would I like my error message consume all the paper in my city (or scroll my
screen for a week) as it tells me a dict cannot be added to a forest and by
the way, here is a repr of each of them showing the current (highly
recursive) contents.

Now people have written functions that take something long and truncate it
so a list containing [1, 2, 3, ... 1_000_000] is shown in this condensed
form with the rest missing, but then someone will complain they are not
seeing all of it!

So the deal is to use your TOOLS. You can run a debugger or add print
statements or enclose it in a try/catch to keep it from stopping the program
and other techniques. You can examine the objects carefully just before, or
even after and do cautious things like ask for the length and then maybe ask
for the first few and last few items, or whatever makes sense.

In the original example, we were first asked about print(a + b) and later
given a somewhat weirder func(x, y, x +y) as examples. Now ask what order
things are evaluated and where the error happens. What is known by the party
handling the error?

If you put the offending statement in a try/catch scenario, then when the
error is triggered, YOU wrote the code that catches the exception and you
can often examine the payload of the exception, or know that the arguments
of a or b or x or y were involved and you can craft your own output to be
more clear. Or, you can even sometimes fix the problem and redo the code
using something like float(x) or str(y).

My impression here is that the error is not on the surface but caught
deeper. The symbols used may be x and y but what if we work with "12" + 13
and follow what happens?

Since the interpreter in python evaluates x+y before calling the function
using the result as an argument, the function never sees anything. What
should happen is that the interpreter sees a "12" which is normally an
object of a class of str and then it sees a "+" and then it sees anything
that follows as a second object it ignores for now. Python does not have a
fully defined operator that it invokes when it sees a "+" as the meaning
depends on what object is being asked to do whatever plus means to it. For a
string argument, it means concatenate to your current content and return a
new str object. The way that happens is that the class (or a relative) has
defined a method called __add__() or it hasn't. If it has, it takes an
argument of the second object and in this case it gets the integer object
containing 13. 

So it runs the function and it has not been programmed on how to append an
integer to a string of characters and it returns without an answer but an
exception. The interpreter evaluator does not admit defeat yet and
reasonable tries to see if the integer 13 has a __iadd__() which is similar
but different and again, an integer has not been programmed to append itself
to an object of type str. Could it have been? Sure. If you make your own
(sub)class you can create a kind of integer that will make a str version of
itself and append it o the "12" to make "1213" BUT in this case, that is not
an option. So the integer method fails and returns an exception too.

Now the parser functionality knows it has failed. "12" and 13 have both
refused to implement the plus sign and either it catches the exception OR is
does not and lets it flow upstream till any other functions in a chain catch
it. Any one can then generate some error message, or it can reach the top
level of the interpreter and it has to decide what to do.

But some errors are not fatal. If str had no __add__() that is not an error.
If it returns that it cannot do it, that is not a fatal error but a
temporary drawback. Only when int fails too is there a likely error but int
is not programmed to do anything lie reporting in __iadd__() and nor should
it except if you are debugging new functionality.

Can we get better error messages in many cases? Sure. Clearly the
interpreter level sees a call like func(x, y, x+y) and if evaluating it
causes an error, it could type out what all the variables were. 

But if you are running code at the console and it aborts like this, you
usually have the variables at your fingertips and can say print(x) or
print(y) but you cannot properly say print(x+y) yet. The error message did
specify you were trying to work with incompatible objects as one was of type
this and the other of type that. That suggests the values of both could have
been accessed and shown in this case ad perhaps the code could be modified
so it returns some semblance of the values at least for built-in objects.
All that though adds code and complexity and often slows things down. 



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Rob Cliffe via Python-list
Sent: Wednesday, February 22, 2023 4:31 PM
To: python-list at python.org
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg)
values are ?



On 22/02/2023 20:05, Hen Hanna wrote:
> Python makes programming (debugging) so easy
I agree with that!
Rob Cliffe
-- 
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Thu Feb 23 16:12:06 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Thu, 23 Feb 2023 16:12:06 -0500
Subject: Line continuation and comments
In-Reply-To: <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
Message-ID: <002d01d947cb$7c262c70$74728550$@gmail.com>

Good example, Rob, of how some people make what I consider RELIGIOUS edicts that one can easily violate if one wishes and it makes lots of sense in your example.

Let me extend that. The goal was to store a character string consisting of multiple lines when printed that are all left-aligned. Had you written:

     HelpText = """
Left click:             Open spam
...
Shift + Right click:    Fry egg
"""
Then it would begin with an extra carriage return you did not want. Your example also ends with a carriage return because you closed the quotes on another line, so a \ on the last line of text (or moving the quotes to the end of the line) would be a way of avoiding that.

Consider some alternatives I have seen that are in a sense ugly and may involve extra work for the interpreter unless it is byte compiled once.

def someFunc():
     HelpText =
     "Left click:             Open spam" + "\n" +
     "Shift + Left click:     Cook spam" + "\n" +
     ...

Or the variant of:
HelpText =  "Left click:             Open spam\n"
HelpText +=  " Shift + Left click:     Cook spam\n"
...

Or perhaps just dumping the multi-line text into a file beforehand and reading that into a string!

def someFunc():

The backslash is not looking like such a bad idea! LOL!

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Rob Cliffe via Python-list
Sent: Wednesday, February 22, 2023 2:08 PM
To: python-list at python.org
Subject: Re: Line continuation and comments



On 22/02/2023 15:23, Paul Bryan wrote:
> Adding to this, there should be no reason now in recent versions of 
> Python to ever use line continuation. Black goes so far as to state 
> "backslashes are bad and should never be used":
>
> https://black.readthedocs.io/en/stable/the_black_code_style/future_sty
> le.html#using-backslashes-for-with-statements

def someFunc():
     HelpText = """\
Left click:             Open spam
Shift + Left click:     Cook spam
Right click:            Crack egg
Shift + Right click:    Fry egg
"""

The initial backslash aligns the first line with the others (in a fixed font of course).
Best wishes
Rob Cliffe
--
https://mail.python.org/mailman/listinfo/python-list


From ethan at stoneleaf.us  Thu Feb 23 16:00:50 2023
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 23 Feb 2023 13:00:50 -0800
Subject: Not receiving posts
In-Reply-To: <f2a406d7-f12c-a8e3-c133-6279e0cc923f@comcast.net>
References: <f2a406d7-f12c-a8e3-c133-6279e0cc923f@comcast.net>
Message-ID: <2dfb83ad-a2c6-68a8-6e77-bdb334a5210d@stoneleaf.us>

On 2/23/23 12:29, Jim Byrnes wrote:

 > I have been reading the python-list for some time now. At first via gemane and since it's demise via a subscription.
 > Recently I noticed that I have not received any emails for quite sometime.
 >
 > I tried resubscribing but still have received no emails from the list. To my knowledge I have done nothing to warrant
 > removal.
 >
 > Can you please check and see what I need to do to start receiving emails once more.

It doesn't look like that address is subscribed.  I tried inviting you, let's see if that works.

--
~Ethan~

From rosuav at gmail.com  Thu Feb 23 17:40:03 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 24 Feb 2023 09:40:03 +1100
Subject: semi colonic
In-Reply-To: <63f7be30.020a0220.e3cd1.3e77@mx.google.com>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
 <k5oeg7FkujpU3@mid.individual.net>
 <63f7be30.020a0220.e3cd1.3e77@mx.google.com>
Message-ID: <CAPTjJmoEEG+jnQ3MUG-8kZajt1tFMoWrJLeyCxdJkGo_DpBWMw@mail.gmail.com>

On Fri, 24 Feb 2023 at 06:29, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>
> On 2023-02-23, rbowman <bowman at montana.com> wrote:
> > On Wed, 22 Feb 2023 18:25:00 -0800 (PST), Hen Hanna wrote:
> >
> >> i sometimes  put  extra  commas...  as:
> >>
> >>                                [  1, 2,  3,  4,     ]
> >>
> >> so it is (or may be)  easier  to add things   later.
> >
> > That can bite you with things like JSON that aren't very forgiving.
>
> Oh, how I hate that about JSON...
>

Also C#, it's incredibly frustrating that I'm not allowed to have a
trailing comma in a function's argument list. Ugh. Completely
unnecessary restriction.

ChrisA

From PythonList at DancesWithMice.info  Thu Feb 23 17:38:30 2023
From: PythonList at DancesWithMice.info (dn)
Date: Fri, 24 Feb 2023 11:38:30 +1300
Subject: Line continuation and comments
In-Reply-To: <k5m39hF9sj0U1@mid.individual.net>
References: <k5m39hF9sj0U1@mid.individual.net>
Message-ID: <d1ddbd81-89d3-726a-bf5b-e082e0f1ace2@DancesWithMice.info>

On 22/02/2023 21.49, Robert Latest via Python-list wrote:
> I found myself building a complicated logical condition with many ands and ors
> which I made more manageable by putting the various terms on individual lines
> and breaking them with the "\" line continuation character. In this context it
> would have been nice to be able to add comments to lines terms which of course
> isn't possible because the backslash must be the last character on the line.
> 
> Question: If the Python syntax were changed to allow comments after line-ending
> backslashes, would it break any existing code? I can't think of an example.

Alternative to suggestions thus far: break the complex* logical 
condition into 'labelled' components, and then combine those (likely 
shorter) into the condition:

if person.is_adult and person.is_qualified and person.has_funds ...

which presumes that at some previous time we have, for example:

def is_adult( self )->bool:
     return 21 <= self.age <= 65

(not that I'd like to see those two 'magic-constants' in anyone's code, 
but (hopefully) the idea has been conveyed...)


* "simple is better than..."

NB my PyCharm-settings grumble whenever I create an identifier which is 
only used once (and perhaps, soon after it was established). I 
understand the (space) optimisation, but prefer to trade that for 
'readability'.
-- 
Regards,
=dn

From henhanna at gmail.com  Thu Feb 23 17:58:47 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Thu, 23 Feb 2023 14:58:47 -0800 (PST)
Subject: Error-Msg  Jeannie's  charming,  teasing ways
Message-ID: <3e837f1f-cf35-472b-8782-b90e6f8b91edn@googlegroups.com>


Python's Error-Msg  genie  (Jeannie)  is cute and fickle...   She sometimes teases me by not telling me what the VALUE of the   "int"  is
                         ( "That's for me to know, and for you to find out!" )
as in:

                           TypeError: can only concatenate str (not "int") to str




Other times, she (Jeannie)  gives me a helpful comment:
ABCD (above and beyond... her job desc.)


                              if (x=0):
                                    ^^^
SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?



Maybe you'd share  other EXAMPLES   of her   charming,  teasing ways???



is there a Tool that can scan my code and tell me   such   (wink,wink)  type suggestions???? -----------  [Maybe you'd like to rewrite this line as ....... ?]
[Might i interest you in the coding-style wherein ...... ?]
  etc?




https://stackoverflow.com/questions/26000198/what-does-colon-equal-in-python-mean

                     There is new syntax       :=         that assigns values to variables as part of a larger expression. It is affectionately known as ?the walrus operator? due to its resemblance to the eyes and tusks of a walrus.


From gweatherby at uchc.edu  Thu Feb 23 18:45:44 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Thu, 23 Feb 2023 23:45:44 +0000
Subject: Line continuation and comments
In-Reply-To: <d1ddbd81-89d3-726a-bf5b-e082e0f1ace2@DancesWithMice.info>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d1ddbd81-89d3-726a-bf5b-e082e0f1ace2@DancesWithMice.info>
Message-ID: <SJ0PR14MB5864AC199F980EDD7CE22FBDB9AB9@SJ0PR14MB5864.namprd14.prod.outlook.com>

?
NB my PyCharm-settings grumble whenever I create an identifier which is
only used once (and perhaps, soon after it was established). I
understand the (space) optimisation, but prefer to trade that for
'readability'.
?
I haven?t seen that one. What I get is warnings about:

def is_adult( self )->bool:
    LEGAL_AGE_US = 21
    return LEGAL_AGE

It doesn?t like LEGAL_AGE_US being all caps if declared in a function.

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of dn via Python-list <python-list at python.org>
Date: Thursday, February 23, 2023 at 5:46 PM
To: python-list at python.org <python-list at python.org>
Subject: Re: Line continuation and comments
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

On 22/02/2023 21.49, Robert Latest via Python-list wrote:
> I found myself building a complicated logical condition with many ands and ors
> which I made more manageable by putting the various terms on individual lines
> and breaking them with the "\" line continuation character. In this context it
> would have been nice to be able to add comments to lines terms which of course
> isn't possible because the backslash must be the last character on the line.
>
> Question: If the Python syntax were changed to allow comments after line-ending
> backslashes, would it break any existing code? I can't think of an example.

Alternative to suggestions thus far: break the complex* logical
condition into 'labelled' components, and then combine those (likely
shorter) into the condition:

if person.is_adult and person.is_qualified and person.has_funds ...

which presumes that at some previous time we have, for example:

def is_adult( self )->bool:
     return 21 <= self.age <= 65

(not that I'd like to see those two 'magic-constants' in anyone's code,
but (hopefully) the idea has been conveyed...)


* "simple is better than..."

NB my PyCharm-settings grumble whenever I create an identifier which is
only used once (and perhaps, soon after it was established). I
understand the (space) optimisation, but prefer to trade that for
'readability'.
--
Regards,
=dn
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ghW7FUX8GJF79keLaMyaVewXcKw3jexuxF-QJh8h564QBAIoi2ez20tIl5fg762Rcfnh-XA4sG53CKt2NYgHpTWlyA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ghW7FUX8GJF79keLaMyaVewXcKw3jexuxF-QJh8h564QBAIoi2ez20tIl5fg762Rcfnh-XA4sG53CKt2NYgHpTWlyA$>

From PythonList at DancesWithMice.info  Thu Feb 23 21:01:17 2023
From: PythonList at DancesWithMice.info (dn)
Date: Fri, 24 Feb 2023 15:01:17 +1300
Subject: Line continuation and comments
In-Reply-To: <SJ0PR14MB5864AC199F980EDD7CE22FBDB9AB9@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d1ddbd81-89d3-726a-bf5b-e082e0f1ace2@DancesWithMice.info>
 <SJ0PR14MB5864AC199F980EDD7CE22FBDB9AB9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <b5e89b8a-529e-7f60-37c7-4a47f9e675e3@DancesWithMice.info>

On 24/02/2023 12.45, Weatherby,Gerard wrote:
> ?
> NB my PyCharm-settings grumble whenever I create an identifier which is
> only used once (and perhaps, soon after it was established). I
> understand the (space) optimisation, but prefer to trade that for
> 'readability'.
> ?
> 
> I haven?t seen that one. What I get is warnings about:
> 
> def is_adult( self )->bool:
>  ??? LEGAL_AGE_US = 21
>  ?? ?return LEGAL_AGE
> 
> It doesn?t like LEGAL_AGE_US being all caps if declared in a function.

Yes, I suffered this one too.

The rationale comes from PEP-008 (Constants):

Constants are usually defined on a module level and written in all 
capital letters with underscores separating words.


Today, I wasn't criticised for:
> NB my PyCharm-settings grumble whenever I create an identifier which is
> only used once (and perhaps, soon after it was established). I
> understand the (space) optimisation, but prefer to trade that for
> 'readability'.

Perhaps that came from AWS CodeWhisperer which I have since abandoned, 
or maybe from SonarLint (which I've just checked to discover it is not 
working properly...)

-- 
Regards,
=dn


From avi.e.gross at gmail.com  Thu Feb 23 21:20:52 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Thu, 23 Feb 2023 21:20:52 -0500
Subject: Line continuation and comments
In-Reply-To: <b5e89b8a-529e-7f60-37c7-4a47f9e675e3@DancesWithMice.info>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d1ddbd81-89d3-726a-bf5b-e082e0f1ace2@DancesWithMice.info>
 <SJ0PR14MB5864AC199F980EDD7CE22FBDB9AB9@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <b5e89b8a-529e-7f60-37c7-4a47f9e675e3@DancesWithMice.info>
Message-ID: <006401d947f6$9e435500$dac9ff00$@gmail.com>

Many "warnings" can safely be ignored.

The function as shown does not look right. I assume it is just an example, but a function that ignores the argument supplied is already a tad suspect.

Since it is SUGGESTED that the variable name "self" normally is used in a method for a class/instance, it is of course possible for it to set a variable called LEGAL_AGE_US to 21 and for no special reason, returns the age.

But my imagination is that a function called is_adult() should perhaps receive an age either as an argument, or an attribute of the current object and return True only if that age is greater than or equal to the legal age. Of course LEGAL_AGE_US may suggest a family of such functions specifying a legal age threshold for various countries or regions and all you need is the age between non-adult and adult. 

So one GUESS I have is that if this is a method, then you are seen not as setting a constant inside the function, where all-caps might be sensible but as setting an instance variable or changing it. A true constant might have been set when the class was designed or perhaps in __init__() or similar. 

I wonder how PyCharm would react if you used:

self.LEGAL_AGE_US = 21



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of dn via Python-list
Sent: Thursday, February 23, 2023 9:01 PM
To: python-list at python.org
Subject: Re: Line continuation and comments

On 24/02/2023 12.45, Weatherby,Gerard wrote:
> ?
> NB my PyCharm-settings grumble whenever I create an identifier which 
> is only used once (and perhaps, soon after it was established). I 
> understand the (space) optimisation, but prefer to trade that for 
> 'readability'.
> ?
> 
> I haven?t seen that one. What I get is warnings about:
> 
> def is_adult( self )->bool:
>      LEGAL_AGE_US = 21
>      return LEGAL_AGE
> 
> It doesn?t like LEGAL_AGE_US being all caps if declared in a function.

Yes, I suffered this one too.

The rationale comes from PEP-008 (Constants):

Constants are usually defined on a module level and written in all capital letters with underscores separating words.


Today, I wasn't criticised for:
> NB my PyCharm-settings grumble whenever I create an identifier which is
> only used once (and perhaps, soon after it was established). I
> understand the (space) optimisation, but prefer to trade that for
> 'readability'.

Perhaps that came from AWS CodeWhisperer which I have since abandoned, 
or maybe from SonarLint (which I've just checked to discover it is not 
working properly...)

-- 
Regards,
=dn

-- 
https://mail.python.org/mailman/listinfo/python-list


From PythonList at DancesWithMice.info  Thu Feb 23 22:12:10 2023
From: PythonList at DancesWithMice.info (dn)
Date: Fri, 24 Feb 2023 16:12:10 +1300
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
Message-ID: <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>

On 23/02/2023 09.05, Hen Hanna wrote:
> 
>            >  py   bug.py
>         Traceback (most recent call last):
>           File "C:\Usenet\bug.py", line 5, in <module>
>                       print( a + 12 )
>            TypeError: can only concatenate str (not "int") to str
> 
> 
> Why doesn't  Python (error msg) do the obvious thing and tell me
>              WHAT   the actual   (offending,  arg)  values are ?
> 
> In many cases, it'd help to know what string the var  A  had  ,   when the error occurred.
>               ------------  i wouldn't have to put      print(a)     just above,  to see.

In some ways, providing this information seems appropriate. Curiously, 
this does not even occur during an assert exception - despite the 
value/relationship being the whole point of using the command!

     x = 1
     assert x == 2

AssertionError (and that's it)

Then again, remember that exceptions can be 'caught'. So, such data 
would need to be added to the exception-instance. This could become 
quite costly.



What are the appropriate tools for the job?


Don't add an extra print(), use a debugger.

Not only does this allow you to breakpoint critical points in the code, 
but identifiers can be watch-ed and changes noted. The other handy 
feature is being able to correct the current erroneous value of the 
identifier and continue execution.

For us, memory-challenged coders, there is no need to remember to remove 
the print() again, afterwards.


The TypeError indicates a problem between the programmer's ears. What 
was thought to be a string or an integer was the opposite. This seems to 
be playing fast-and-loose with Python's dynamic-typing. To quote: "we're 
all adults here". Thus, I wouldn't recommend 're-cycling' an identifier 
to represent two different (types of) data-point in the same code - but 
there's nothing to stop you/anyone!

The other possibility is that it was an accident. Sounds more like 
something I would do, but... In this case, the tool which is your/my 
friend is typing. The IDE should catch most of the situations where an 
int would be used as an str, or v-v. Remember though, Python's typing is 
(a) not part of the language, and (b) probably won't help at run-time.


PS are you aware that there is a Python-Tutor list for the use of people 
learning Python?
-- 
Regards,
=dn

From torriem at gmail.com  Thu Feb 23 22:32:26 2023
From: torriem at gmail.com (Michael Torrie)
Date: Thu, 23 Feb 2023 20:32:26 -0700
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <59af3ca8-bad9-4f91-a631-f84e38bce02en@googlegroups.com>
References: <EAA85A55-F12A-4DE4-803D-D08643E782F4@barrys-emacs.org>
 <84b38a83-5113-481f-8eb1-720401908d24n@googlegroups.com>
 <mailman.1935.1677139044.20444.python-list@python.org>
 <59af3ca8-bad9-4f91-a631-f84e38bce02en@googlegroups.com>
Message-ID: <4566bab8-c7b4-31b0-93a4-b2b222654b50@gmail.com>

On 2/23/23 01:08, Hen Hanna wrote:
>  Python VM  is seeing an "int" object (123)   (and telling me that)   ...   so it should be easy to print that "int" object 
> What does  Python VM    know ?   and when does it know it ?
It knows there is an object and its name and type.  It knows this from
the first moment you create the object and bind a name to it.
> it seems like  it is being playful, teasing (or mean),    and   hiding  the ball from me

Sorry you aren't understanding.  Whenever you print() out an object,
python calls the object's __repr__() method to generate the string to
display.  For built-in objects this is obviously trivial. But if you
were dealing an object of some arbitrary class, there may not be a
__repr__() method which would cause an exception, or if the __repr__()
method itself raised an exception, you'd lose the original error message
and the stack trace would be all messed up and of no value to you.  Does
that make sense?  Remember that Python is a very dynamic language and
what might be common sense for a built-in type makes no sense at all for
a custom type.  Thus there's no consistent way for Python to print out
the information you think is so simple.

From cs at cskk.id.au  Thu Feb 23 22:50:39 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Fri, 24 Feb 2023 14:50:39 +1100
Subject: Line continuation and comments
In-Reply-To: <b5e89b8a-529e-7f60-37c7-4a47f9e675e3@DancesWithMice.info>
References: <b5e89b8a-529e-7f60-37c7-4a47f9e675e3@DancesWithMice.info>
Message-ID: <Y/g0D0Q+P8tXTGuY@cskk.homeip.net>

On 24/02/2023 12.45, Weatherby,Gerard wrote:
>>NB my PyCharm-settings grumble whenever I create an identifier which 
>>is
>>only used once (and perhaps, soon after it was established). I
>>understand the (space) optimisation, but prefer to trade that for
>>'readability'.

It isn't "space". Got an example for that warning? I don't use PyCharm, 
but the main linter warning I get is an _unused_ variable, eg:

     def f():
         x = 3

I set x and never use it. Likely brain fade on my part, and worth a 
warning.

On 24Feb2023 15:01, dn <PythonList at DancesWithMice.info> wrote:
>>I haven?t seen that one. What I get is warnings about:
>>
>>def is_adult( self )->bool:
>> ??? LEGAL_AGE_US = 21
>> ?? ?return LEGAL_AGE
>>
>>It doesn?t like LEGAL_AGE_US being all caps if declared in a function.
>
>Yes, I suffered this one too.
>
>The rationale comes from PEP-008 (Constants):
>
>Constants are usually defined on a module level and written in all 
>capital letters with underscores separating words.

Yeah. The above looks like a method. I'd probably have something like:

     class C:

         LEGAL_AGE_US = 21

         def is_adult(self) -> bool:
             return self.age >= self.LEGAL_AGE_US

Variables used (self). Constant a class attribute.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From cs at cskk.id.au  Thu Feb 23 22:58:04 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Fri, 24 Feb 2023 14:58:04 +1100
Subject: Error-Msg  Jeannie's  charming,  teasing ways
In-Reply-To: <3e837f1f-cf35-472b-8782-b90e6f8b91edn@googlegroups.com>
References: <3e837f1f-cf35-472b-8782-b90e6f8b91edn@googlegroups.com>
Message-ID: <Y/g1zOq7BUfuL+Zp@cskk.homeip.net>

On 23Feb2023 14:58, Hen Hanna <henhanna at gmail.com> wrote:
>Python's Error-Msg  genie  (Jeannie)  is cute and fickle...   She 
>sometimes teases me by not telling me what the VALUE of the   "int"  is
>                         ( "That's for me to know, and for you to find out!" )
>as in:
>                           TypeError: can only concatenate str (not 
>                           "int") to str

This is a runtime error< not a syntax error. Valid code using invalid 
values. It can only be seen at runtime.

>Other times, she (Jeannie)  gives me a helpful comment:
>ABCD (above and beyond... her job desc.)
>                              if (x=0):
>                                    ^^^
>SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?

Syntax error can be seen at compile time. Since Python's interpreted, 
the compilation step i usually implicit in trying to run things. But 
technically this happens before any of _your_ code runs.

The helpfulness of this warning is a _very_ recent upgrade, with Python 
3.11 I think, maybe 3.10.  The syntax errors used to be a lot less 
helpful.

>is there a Tool that can scan my code and tell me   such   (wink,wink)  
>type suggestions????

There are several type checking programs for Python, with mypy probably 
being the best known. I seem to recall seeing some mention of tools 
which will aid inferring types from partially types programmes, usually 
as an aid to completing the type annotations.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From avi.e.gross at gmail.com  Thu Feb 23 23:09:52 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Thu, 23 Feb 2023 23:09:52 -0500
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <4566bab8-c7b4-31b0-93a4-b2b222654b50@gmail.com>
References: <EAA85A55-F12A-4DE4-803D-D08643E782F4@barrys-emacs.org>
 <84b38a83-5113-481f-8eb1-720401908d24n@googlegroups.com>
 <mailman.1935.1677139044.20444.python-list@python.org>
 <59af3ca8-bad9-4f91-a631-f84e38bce02en@googlegroups.com>
 <4566bab8-c7b4-31b0-93a4-b2b222654b50@gmail.com>
Message-ID: <008601d94805$d89f1180$89dd3480$@gmail.com>

We have been supplying many possible reasons or consequences for why the
implementation of python does not do what the OP wants and even DEMANDS.

I am satisfied with knowing it was because they CHOSE NOT TO in some places
and maybe not in others. It is nice to see some possible reasons, but
something as simple as efficiency or needing to complicate the code in
something used regularly, might be enough for now.

But to comment on what Michael T. and Dave N. have been saying, newcomers
often have no clue of what can happen so their questions may sound quite
reasonable.

So what happens if you create a large data structure, so some operation that
fails, catch the error and save the variables involved in an exception and
throw that onward and perhaps the program keeps running? There is now a
pointer to the large data structure in the exception object, or even a copy.
If that exception is not discarded or garbage collected, it can remain in
memory indefinitely even if the original string was expected to be removed,
replaced, or garbage collected. Some modern features in R such as generators
will stay alive infinitely and retain their state in between calls for a
next item.

You can end up with memory leaks that are not trivial to solve or that may
mysteriously disappear when an iterable has finally been consumed and all
the storage it used or pointed at can be retrieved, as one example.

A more rational approach is to realize that python has multiple levels of
debugging and exceptions are one among many. They are not meant to solve the
entire problem but just enough to be helpful or point you in some direction.
Yes, they can do more.

And, FYI, I too pointed this person at the Tutor list and I see no sign they
care how many people they make waste their time with so many mainly gripes.
I personally now ignore any post by them.
-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Michael Torrie
Sent: Thursday, February 23, 2023 10:32 PM
To: python-list at python.org
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg)
values are ?

On 2/23/23 01:08, Hen Hanna wrote:
>  Python VM  is seeing an "int" object (123)   (and telling me that)   ...
so it should be easy to print that "int" object 
> What does  Python VM    know ?   and when does it know it ?
It knows there is an object and its name and type.  It knows this from the
first moment you create the object and bind a name to it.
> it seems like  it is being playful, teasing (or mean),    and   hiding
the ball from me

Sorry you aren't understanding.  Whenever you print() out an object, python
calls the object's __repr__() method to generate the string to display.  For
built-in objects this is obviously trivial. But if you were dealing an
object of some arbitrary class, there may not be a
__repr__() method which would cause an exception, or if the __repr__()
method itself raised an exception, you'd lose the original error message and
the stack trace would be all messed up and of no value to you.  Does that
make sense?  Remember that Python is a very dynamic language and what might
be common sense for a built-in type makes no sense at all for a custom type.
Thus there's no consistent way for Python to print out the information you
think is so simple.
--
https://mail.python.org/mailman/listinfo/python-list


From henhanna at gmail.com  Thu Feb 23 19:21:37 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Thu, 23 Feb 2023 16:21:37 -0800 (PST)
Subject: putting JUNK  at the end of a   [.py]    file
Message-ID: <aa3cd818-3cf4-496b-bfdd-e5b434cee71dn@googlegroups.com>


in a LaTeX file,       after the (1st)       \end{document}     line, 
      i can put any random Junk i want    (afterwards)   until the end of the file.


Is there a similar Method  for a    .py     file ?

Since i know of no such trick,  i sometimes put this (below) at the end of a .py file.



dummy= ("""              junk and more junk
                                      words in Dict
                            239  words in Dict
                    (((  notes or Code fragmetns  )))
             """ )



**    maybe i don't need the     dummy=     but it looks better.

From list1 at tompassin.net  Thu Feb 23 23:28:04 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Thu, 23 Feb 2023 23:28:04 -0500
Subject: Error-Msg Jeannie's charming, teasing ways
In-Reply-To: <Y/g1zOq7BUfuL+Zp@cskk.homeip.net>
References: <3e837f1f-cf35-472b-8782-b90e6f8b91edn@googlegroups.com>
 <Y/g1zOq7BUfuL+Zp@cskk.homeip.net>
Message-ID: <05724d5c-b4c7-13f4-d49c-01033fc7901d@tompassin.net>

On 2/23/2023 10:58 PM, Cameron Simpson wrote:
> On 23Feb2023 14:58, Hen Hanna <henhanna at gmail.com> wrote:
>> Python's Error-Msg? genie? (Jeannie)? is cute and fickle...?? She 
>> sometimes teases me by not telling me what the VALUE of the?? "int"? is
>> ??????????????????????? ( "That's for me to know, and for you to find 
>> out!" )
>> as in:
>> ????????????????????????? TypeError: can only concatenate str (not 
>> ????????????????????????? "int") to str
> 
> This is a runtime error< not a syntax error. Valid code using invalid 
> values. It can only be seen at runtime.
> 
>> Other times, she (Jeannie)? gives me a helpful comment:
>> ABCD (above and beyond... her job desc.)
>> ???????????????????????????? if (x=0):
>> ?????????????????????????????????? ^^^
>> SyntaxError: invalid syntax. Maybe you meant '==' or ':=' instead of '='?
> 
> Syntax error can be seen at compile time. Since Python's interpreted, 
> the compilation step i usually implicit in trying to run things. But 
> technically this happens before any of _your_ code runs.

It *is* a syntax error, and it *is* caught before execution.  It's a 
syntax error because the expression following the if (which doesn't need 
the parentheses) must be a boolean expression and "x = 0" is not one of 
those.  Python undergoes a compilation of sorts, down to the .pyc file, 
and it catches outright syntax errors in that phase.  There are other 
errors that can't be caught until runtime, and they might involve type 
errors (unless one uses some static analysis).

> The helpfulness of this warning is a _very_ recent upgrade, with Python 
> 3.11 I think, maybe 3.10.? The syntax errors used to be a lot less helpful.
> 
>> is there a Tool that can scan my code and tell me?? such?? (wink,wink) 
>> type suggestions????
> 
> There are several type checking programs for Python, with mypy probably 
> being the best known. I seem to recall seeing some mention of tools 
> which will aid inferring types from partially types programmes, usually 
> as an aid to completing the type annotations.
> 
> Cheers,
> Cameron Simpson <cs at cskk.id.au>


From list1 at tompassin.net  Fri Feb 24 00:16:33 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 24 Feb 2023 00:16:33 -0500
Subject: putting JUNK at the end of a [.py] file
In-Reply-To: <aa3cd818-3cf4-496b-bfdd-e5b434cee71dn@googlegroups.com>
References: <aa3cd818-3cf4-496b-bfdd-e5b434cee71dn@googlegroups.com>
Message-ID: <3832046e-184c-5ec2-9de7-2badabd8fa3d@tompassin.net>

On 2/23/2023 7:21 PM, Hen Hanna wrote:
> 
> in a LaTeX file,       after the (1st)       \end{document}     line,
>        i can put any random Junk i want    (afterwards)   until the end of the file.
> 
> 
> Is there a similar Method  for a    .py     file ?
> 
> Since i know of no such trick,  i sometimes put this (below) at the end of a .py file.
> 
> 
> 
> dummy= ("""              junk and more junk
>                                        words in Dict
>                              239  words in Dict
>                      (((  notes or Code fragmetns  )))
>               """ )
> 
> 
> 
> **    maybe i don't need the     dummy=     but it looks better.

You can just put it in triple quotes, no need to assign the string to a 
variable.  Or make each line a comment.


From greg.ewing at canterbury.ac.nz  Fri Feb 24 01:30:43 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 24 Feb 2023 19:30:43 +1300
Subject: semi colonic
In-Reply-To: <mailman.1950.1677186024.20444.python-list@python.org>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
 <9ef85a5e-05d8-8887-a714-91dd47634e0d@btinternet.com>
 <002001d947c5$2ad08ba0$8071a2e0$@gmail.com>
 <mailman.1950.1677186024.20444.python-list@python.org>
Message-ID: <k5r3sjF36njU1@mid.individual.net>

On 24/02/23 9:26 am, avi.e.gross at gmail.com wrote:
> Python One-Liners: Write Concise, Eloquent Python Like a Professional Illustrated Edition
> by Christian Mayer (Author)

I didn't know there were any Professional Illustrated Editions
writing Pythom. You learn something every day! :-)

-- 
Greg

From rosuav at gmail.com  Fri Feb 24 01:40:15 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 24 Feb 2023 17:40:15 +1100
Subject: semi colonic
In-Reply-To: <k5r3sjF36njU1@mid.individual.net>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
 <9ef85a5e-05d8-8887-a714-91dd47634e0d@btinternet.com>
 <002001d947c5$2ad08ba0$8071a2e0$@gmail.com>
 <mailman.1950.1677186024.20444.python-list@python.org>
 <k5r3sjF36njU1@mid.individual.net>
Message-ID: <CAPTjJmouzAALJhxN9KyosVumLQBiCVdmQmDHJ=ySYvsCervSfQ@mail.gmail.com>

On Fri, 24 Feb 2023 at 17:36, Greg Ewing via Python-list
<python-list at python.org> wrote:
>
> On 24/02/23 9:26 am, avi.e.gross at gmail.com wrote:
> > Python One-Liners: Write Concise, Eloquent Python Like a Professional Illustrated Edition
> > by Christian Mayer (Author)
>
> I didn't know there were any Professional Illustrated Editions
> writing Pythom. You learn something every day! :-)
>

"I don't want to be a moving picture in a book!" -- Bert, locomotive
on the Small Railway

ChrisA

From robin at reportlab.com  Fri Feb 24 04:04:22 2023
From: robin at reportlab.com (Robin Becker)
Date: Fri, 24 Feb 2023 09:04:22 +0000
Subject: lxml with python-3.12.0a5
In-Reply-To: <630af59b-fe9e-3bbc-7715-89ba3e27a26a@wichmann.us>
References: <a7091c28-61ae-9e5f-9737-0ff21106fe72@everest.reportlab.co.uk>
 <42284c49-430c-67d0-6dcf-e86f4d2041ea@wichmann.us>
 <630af59b-fe9e-3bbc-7715-89ba3e27a26a@wichmann.us>
Message-ID: <7fff412d-cf64-591a-9064-cbd0a184f706@everest.reportlab.co.uk>

On 23/02/2023 18:09, Mats Wichmann wrote:
........
>> I seem to always have trouble with lxml (which I know doesn't help).
>>
>> The cause would seem to be this:
>>
>> ???? GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292)
> 
> So it looks to me like cython was affected, and has patched themselves. It's possible regenerating with a patched cython 
> will clear up the build problem (something which the lxml project takes pains to tell you that you don't really want to 
> do :)
Thanks Mats,

I build a cython wheel from git latest and after installing into python-3.12.0a5 I used latest lxml source and

  python setup.py bdist_wheel --with-cython

which built without error. The installed lxml seems fine (at least for reportlab tests).
--
Robin Becker

From hjp-python at hjp.at  Fri Feb 24 09:02:21 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 24 Feb 2023 15:02:21 +0100
Subject: semi colonic
In-Reply-To: <002801d947c9$5cacb000$16061000$@gmail.com>
References: <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
 <k5oeg7FkujpU3@mid.individual.net>
 <63f7be30.020a0220.e3cd1.3e77@mx.google.com>
 <002801d947c9$5cacb000$16061000$@gmail.com>
Message-ID: <20230224140221.ro4umabs7w6hkfs5@hjp.at>

On 2023-02-23 15:56:54 -0500, avi.e.gross at gmail.com wrote:
> I am not sure it is fair to blame JSON for a design choice.

We can't blame JSON (it has no agency), but as you say, it it was a
choice. And we can absolutely blame Doug for making that choice!

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230224/8880d0dd/attachment.sig>

From mats at wichmann.us  Fri Feb 24 10:27:56 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Fri, 24 Feb 2023 08:27:56 -0700
Subject: putting JUNK at the end of a [.py] file
In-Reply-To: <3832046e-184c-5ec2-9de7-2badabd8fa3d@tompassin.net>
References: <aa3cd818-3cf4-496b-bfdd-e5b434cee71dn@googlegroups.com>
 <3832046e-184c-5ec2-9de7-2badabd8fa3d@tompassin.net>
Message-ID: <9a004322-2122-b3d0-b409-db27aac43815@wichmann.us>

On 2/23/23 22:16, Thomas Passin wrote:
> On 2/23/2023 7:21 PM, Hen Hanna wrote:
>>
>> in a LaTeX file,?????? after the (1st)?????? \end{document}???? line,
>> ?????? i can put any random Junk i want??? (afterwards)?? until the 
>> end of the file.
>>
>>
>> Is there a similar Method? for a??? .py???? file ?
>>
>> Since i know of no such trick,? i sometimes put this (below) at the 
>> end of a .py file.
>>
>>
>>
>> dummy= ("""????????????? junk and more junk
>> ?????????????????????????????????????? words in Dict
>> ???????????????????????????? 239? words in Dict
>> ???????????????????? (((? notes or Code fragmetns? )))
>> ????????????? """ )
>>
>>
>>
>> **??? maybe i don't need the???? dummy=???? but it looks better.
> 
> You can just put it in triple quotes, no need to assign the string to a 
> variable.? Or make each line a comment.
> 

Indeed, I work on a project that by convention puts editor instructions 
at the end of each file (which some might consider junk :-) ), like this:


# Local Variables:
# tab-width:4
# indent-tabs-mode:nil
# End:
# vim: set expandtab tabstop=4 shiftwidth=4:



From mats at wichmann.us  Fri Feb 24 10:33:38 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Fri, 24 Feb 2023 08:33:38 -0700
Subject: putting JUNK at the end of a [.py] file
In-Reply-To: <9a004322-2122-b3d0-b409-db27aac43815@wichmann.us>
References: <aa3cd818-3cf4-496b-bfdd-e5b434cee71dn@googlegroups.com>
 <3832046e-184c-5ec2-9de7-2badabd8fa3d@tompassin.net>
 <9a004322-2122-b3d0-b409-db27aac43815@wichmann.us>
Message-ID: <f4006ab2-a6c3-d3ac-4b1c-3867e810d7d7@wichmann.us>

On 2/24/23 08:27, Mats Wichmann wrote:

> 
> Indeed, I work on a project that by convention puts editor instructions 
> at the end of each file (which some might consider junk :-) ), like this:
> 
> 
> # Local Variables:
> # tab-width:4
> # indent-tabs-mode:nil
> # End:
> # vim: set expandtab tabstop=4 shiftwidth=4:

I should probably have added that I know the above is unnecessary, as 
there is now a standard for defining this stuff once, called 
EditorConfig, which many editors understand natively, and many more via 
plugins.

https://editorconfig.org/



From henhanna at gmail.com  Fri Feb 24 00:37:54 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Thu, 23 Feb 2023 21:37:54 -0800 (PST)
Subject: putting JUNK at the end of a [.py] file
In-Reply-To: <mailman.1965.1677215806.20444.python-list@python.org>
References: <3832046e-184c-5ec2-9de7-2badabd8fa3d@tompassin.net>
 <aa3cd818-3cf4-496b-bfdd-e5b434cee71dn@googlegroups.com>
 <mailman.1965.1677215806.20444.python-list@python.org>
Message-ID: <438bd389-7fec-49cf-8edb-d915a7fab15an@googlegroups.com>

On Thursday, February 23, 2023 at 9:17:05?PM UTC-8, Thomas Passin wrote:
> On 2/23/2023 7:21 PM, Hen Hanna wrote: 
> > 
> > in a LaTeX file, after the (1st) \end{document} line, 
> > i can put any random Junk i want (afterwards) until the end of the file. 
> > 
> > 
> > Is there a similar Method for a .py file ? 
> > 
> > Since i know of no such trick, i sometimes put this (below) at the end of a .py file. 
> > 
> > 
> > 
> > dummy= (""" junk and more junk 
> > words in Dict 
> > 239 words in Dict 
> > ((( notes or Code fragmetns ))) 
> >           """) 
> > 
> > 
> > 
> > ** maybe i don't need the          dummy=           but it looks better.


> You can just put it in triple quotes, no need to assign the string to a 
> variable. Or make each line a comment.


thanks for the   rare   concise comment   taht  also seems  devoid of insulting  attitude.

                                  (  i  still can't  tell one person from another, but
                                              i did notice that  one person sent me email 
                                                    that was  really long, insulting, and creepy )


i guess i  use    ("""           """)   because i wish   it  (this feature)   came in matching pairs.

and   put         dummy=        to remind myself    and to  tell the compiler,   obivously  this Var is unused to feel free to ignore it.

From loris.bennett at fu-berlin.de  Fri Feb 24 04:31:12 2023
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Fri, 24 Feb 2023 10:31:12 +0100
Subject: putting JUNK  at the end of a   [.py]    file
References: <aa3cd818-3cf4-496b-bfdd-e5b434cee71dn@googlegroups.com>
Message-ID: <87wn47tnof.fsf@hornfels.zedat.fu-berlin.de>

Hen Hanna <henhanna at gmail.com> writes:

> in a LaTeX file,       after the (1st)       \end{document}     line, 
>       i can put any random Junk i want    (afterwards)   until the end of the file.
>
>
> Is there a similar Method  for a    .py     file ?
>
> Since i know of no such trick,  i sometimes put this (below) at the end of a .py file.
>
>
>
> dummy= ("""              junk and more junk
>                                       words in Dict
>                             239  words in Dict
>                     (((  notes or Code fragmetns  )))
>              """ )
>
>
>
> **    maybe i don't need the     dummy=     but it looks better.

Apropos looking better:

In my newsreader with a fixed-width font, I find your postings rather
hard to read as there is a lot of, for me, inexplicable white space.

Is there a particular reason why your postings are formatted like this?

Cheers,

Loris

-- 
This signature is currently under constuction.

From list1 at tompassin.net  Fri Feb 24 12:03:07 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 24 Feb 2023 12:03:07 -0500
Subject: putting JUNK at the end of a [.py] file
In-Reply-To: <438bd389-7fec-49cf-8edb-d915a7fab15an@googlegroups.com>
References: <3832046e-184c-5ec2-9de7-2badabd8fa3d@tompassin.net>
 <aa3cd818-3cf4-496b-bfdd-e5b434cee71dn@googlegroups.com>
 <mailman.1965.1677215806.20444.python-list@python.org>
 <438bd389-7fec-49cf-8edb-d915a7fab15an@googlegroups.com>
Message-ID: <f6247f0f-87f6-d760-9590-df458cfb4f3b@tompassin.net>

On 2/24/2023 12:37 AM, Hen Hanna wrote:
> On Thursday, February 23, 2023 at 9:17:05?PM UTC-8, Thomas Passin wrote:
>> On 2/23/2023 7:21 PM, Hen Hanna wrote:
>>>
>>> in a LaTeX file, after the (1st) \end{document} line,
>>> i can put any random Junk i want (afterwards) until the end of the file.
>>>
>>>
>>> Is there a similar Method for a .py file ?
>>>
>>> Since i know of no such trick, i sometimes put this (below) at the end of a .py file.
>>>
>>>
>>>
>>> dummy= (""" junk and more junk
>>> words in Dict
>>> 239 words in Dict
>>> ((( notes or Code fragmetns )))
>>>            """)
>>>
>>>
>>>
>>> ** maybe i don't need the          dummy=           but it looks better.
> 
> 
>> You can just put it in triple quotes, no need to assign the string to a
>> variable. Or make each line a comment.
> 
> 
> thanks for the   rare   concise comment   taht  also seems  devoid of insulting  attitude.
> 
>                                    (  i  still can't  tell one person from another, but
>                                                i did notice that  one person sent me email
>                                                      that was  really long, insulting, and creepy )
> 
> 
> i guess i  use    ("""           """)   because i wish   it  (this feature)   came in matching pairs.
> 
> and   put         dummy=        to remind myself    and to  tell the compiler,   obivously  this Var is unused to feel free to ignore it.

The bare string will be evaluated but not assigned to any variable name. 
You don't need to tell the compiler anything.  As an example, here is a 
tiny program that runs:

"""A little test program to verify that a floating string is OK."""

MSG = 'this is a test'
print(MSG)

"""This is a free-floating message
that is just a note to myself.
"


From avi.e.gross at gmail.com  Fri Feb 24 12:28:09 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Fri, 24 Feb 2023 12:28:09 -0500
Subject: terse 
In-Reply-To: <k5r3sjF36njU1@mid.individual.net>
References: <20230222133239.e2ebfbffed03b64cea42e816@g{oogle}mail.com>
 <366acf23-ae0f-482e-997e-3f968aea7ce1n@googlegroups.com>
 <78349338-a6ec-52bc-9b64-4de7db275392@tompassin.net>
 <00d401d94721$f96ec7a0$ec4c56e0$@gmail.com>
 <48e4df64-fec4-cb84-e1de-2091f7c9b501@btinternet.com>
 <c3ce5be6-7be6-9fb7-fd73-5d6ca72d85a3@tompassin.net>
 <mailman.1930.1677118854.20444.python-list@python.org>
 <fc117ffb-7424-419f-a635-cce09d21e5fen@googlegroups.com>
 <9ef85a5e-05d8-8887-a714-91dd47634e0d@btinternet.com>
 <002001d947c5$2ad08ba0$8071a2e0$@gmail.com>
 <mailman.1950.1677186024.20444.python-list@python.org>
 <k5r3sjF36njU1@mid.individual.net>
Message-ID: <003301d94875$5d645410$182cfc30$@gmail.com>

Greg,

I do not advocate for writing extremely concise python as mentioned in that
book although I was quite interested and do use some of the methods. 

But I worry about what you focused in on. Everyone says a picture is worth a
thousand words. So when writing about python one-liners, you might shorten
some programs even more with a nice illustration!

But if that is an obstacle, perhaps the edition below is less illustrative.

https://www.amazon.com/gp/product/B07ZY7XMX8

Just a reminder. My point was not about a book or set of techniques. It was
that lots of Python features can be used quite effectively to reduce the
need for more lines of cluttered code or places you may be tempted to use
semicolons. Having said that, guess what some one-liner techniques use?

In my view, terseness is not a goal in and of itself. However, it has often
been said that the number of bugs in code often seems correlated with the
number of lines and code written using higher levels of coordination and
abstraction in a language that supports that, can often be written with
fewer lines and apparently even with fewer bugs.

There is also a well-known set of phenomena about us humans in that many of
us are wired to handle fairly small amounts in memory at a time, such as
perhaps 7.  People who can do substantially better are often using an
assortment of tricks like clumping. I mean most people can remember a
10-digit phone number for a while because they may chunk it into
xxx-yyy-abcd or something like that where xxx is just remembered as one unit
as a complete area code while abcd is remembered as 4 individual digits.

So languages that allow and encourage not so much terseness as variations
like chunking, meaning using lots of smaller well-named functions, objects
that encapsulate what your program logic is, and yes, terse but easy to
understand ways of doing things like loops and functional programming
methods, and that are extended by modules or packages with a well-defined
interface, can let you focus on the program at higher levels that do fit in
your memory and can be reasoned more easily and accurately.

So some one-liners are great. Others not so much. Can you imagine a list
comprehension with a dozen or so nested loops on one long line, including
regions grouped in parentheses and lots of "if" clauses? It may be one line
that wraps on your screen into many or has to be scrolled sideways. Yes, you
can rewrite it split across many lines. But at some point, it may be better
to refactor it using a functional style or writing the loops out explicitly.

In my experience, some one-liners are accompanied by paragraphs of comments
explaining them. And often they use tricks that are less efficient.

Avi


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Greg Ewing via Python-list
Sent: Friday, February 24, 2023 1:31 AM
To: python-list at python.org
Subject: Re: semi colonic

On 24/02/23 9:26 am, avi.e.gross at gmail.com wrote:
> Python One-Liners: Write Concise, Eloquent Python Like a Professional 
> Illustrated Edition by Christian Mayer (Author)

I didn't know there were any Professional Illustrated Editions writing
Pythom. You learn something every day! :-)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list


From hjp-python at hjp.at  Fri Feb 24 14:01:44 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 24 Feb 2023 20:01:44 +0100
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <f4aa2a4b-0cf9-415f-88dc-6d8cc6de8404n@googlegroups.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <f4aa2a4b-0cf9-415f-88dc-6d8cc6de8404n@googlegroups.com>
Message-ID: <20230224190144.t4nnbgnziud4kmzx@hjp.at>

On 2023-02-22 15:46:09 -0800, Hen Hanna wrote:
> On Wednesday, February 22, 2023 at 12:05:34 PM UTC-8, Hen Hanna wrote:
> > > py bug.py 
> > Traceback (most recent call last): 
> > File "C:\Usenet\bug.py", line 5, in <module> 
> > print( a + 12 ) 
> > TypeError: can only concatenate str (not "int") to str 
> > 
> > 
> > Why doesn't Python (error msg) do the obvious thing and tell me 
> > WHAT the actual (offending, arg) values are ? 
> > 
> > In many cases, it'd help to know what string the var A had , when the error occurred. 
> > ------------ i wouldn't have to put print(a) just above, to see. 
> > 
> > ( pypy doesn't do that either, but Python makes programming
> > (debugging) so easy that i hardly feel any inconvenience.)

That seems like a non-sequitur to me. If you hardly feel any
inconvenience, why argue so forcefully?
And why is pypy relevant here?


> i  see that my example   would be clearER  with this one-line  change:
> 
> 
>                   >  py   bug.py
> 
>                    Traceback (most recent call last):
> 
>                   File "C:\Usenet\bug.py", line 5, in <module>
>                                          map( Func,    fooBar(  X,  Y,  X +  Y  ))
>  
>                    TypeError: can only concatenate str (not "int") to str
> 
> 
> i hope that   NOW   a few of you  can  see this as a genuine,  (reasonable)  question.

That doesn't seem a better example to me. There is still only one
subexpression (X + Y) where that error can come from, so I know that X
is a str and Y is an int.

A better example would be something like 

    x = (a + b) * (c + d)

In this case it could be either (a + b) or (c + d) which caused the
error. But what I really want to know here is the names of the involved
variables, NOT the values. If the error message told me that the values
were 'foo' and 12.3, I still wouldn't be any wiser. The problem here of
course is that the operands aren't necessarily simple variables as in
this example - they may be arbitrarily complex expressions. However, it
might be sufficient to mark the operator which caused the exception:

|   ...
|   File "/home/hjp/tmp/./foo", line 4, in f
|     return (a + b) * (c + d)
|                         ^
| TypeError: can only concatenate str (not "int") to str

would tell me that (c + d) caused the problem and therefore that c must
be a str which it obviously shouldn't be.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230224/3b36c3f2/attachment.sig>

From hjp-python at hjp.at  Fri Feb 24 14:06:06 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 24 Feb 2023 20:06:06 +0100
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <4566bab8-c7b4-31b0-93a4-b2b222654b50@gmail.com>
References: <EAA85A55-F12A-4DE4-803D-D08643E782F4@barrys-emacs.org>
 <84b38a83-5113-481f-8eb1-720401908d24n@googlegroups.com>
 <mailman.1935.1677139044.20444.python-list@python.org>
 <59af3ca8-bad9-4f91-a631-f84e38bce02en@googlegroups.com>
 <4566bab8-c7b4-31b0-93a4-b2b222654b50@gmail.com>
Message-ID: <20230224190606.uemv56sbvalbaiql@hjp.at>

On 2023-02-23 20:32:26 -0700, Michael Torrie wrote:
> On 2/23/23 01:08, Hen Hanna wrote:
> >  Python VM  is seeing an "int" object (123)   (and telling me that)
> >  ...   so it should be easy to print that "int" object What does
> >  Python VM    know ?   and when does it know it ?
> It knows there is an object and its name and type.  It knows this from
> the first moment you create the object and bind a name to it.
> > it seems like  it is being playful, teasing (or mean),    and
> > hiding  the ball from me
> 
> Sorry you aren't understanding.  Whenever you print() out an object,
> python calls the object's __repr__() method to generate the string to
> display.  For built-in objects this is obviously trivial. But if you
> were dealing an object of some arbitrary class, there may not be a
> __repr__() method

Is this even possible? object has a __repr__ method, so all other
classes would inherit that if they don't define one themselves. I guess
it's possible to explicitely remove it ...

> which would cause an exception, or if the __repr__()
> method itself raised an exception,

Yup. That is possible and has happened to me several times - of course
always in a situation where I really needed that output ...

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230224/9bab0fc0/attachment.sig>

From hjp-python at hjp.at  Fri Feb 24 14:12:44 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 24 Feb 2023 20:12:44 +0100
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
Message-ID: <20230224191244.kn6quzr4x5muthln@hjp.at>

On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
> In some ways, providing this information seems appropriate. Curiously, this
> does not even occur during an assert exception - despite the
> value/relationship being the whole point of using the command!
> 
>     x = 1
>     assert x == 2
> 
> AssertionError (and that's it)

Pytest is great there. If an assertion in a test case fails it analyzes
the expression to give you various levels of details:

======================================== test session starts ========================================
platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /home/hjp/tmp/t
plugins: cov-3.0.0, anyio-3.6.1
collected 1 item

test_a.py F                                                                                   [100%]

============================================= FAILURES ==============================================
______________________________________________ test_a _______________________________________________

    def test_a():
        a = [1, 2, 3]
        b = {"a": a, "b": 2}

>       assert len(a) == len(b)
E       AssertionError: assert 3 == 2
E        +  where 3 = len([1, 2, 3])
E        +  and   2 = len({'a': [1, 2, 3], 'b': 2})

test_a.py:7: AssertionError
====================================== short test summary info ======================================
FAILED test_a.py::test_a - AssertionError: assert 3 == 2
========================================= 1 failed in 0.09s =========================================

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230224/8250bb13/attachment.sig>

From henhanna at gmail.com  Fri Feb 24 13:34:21 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Fri, 24 Feb 2023 10:34:21 -0800 (PST)
Subject: Find   6-letter words that are hidden (embedded)  within
Message-ID: <dd48cdc4-a57e-418d-b956-162a8d7030e8n@googlegroups.com>


  i just wrote a program,  which...
                    within    [FunFunPython]  
           finds:             (funny,    futon,    python)

                             ( 5- and 6- letter words )


(my program uses a Trie, but is pretty simple)



Maybe someone would show me
            how it's done using    itertools,   Permutations, etc.

 Wouldn't it get too slow for Letter-Seeds longer than 11 letters or so?


______________________
 
 Find   6-letter words that are hidden (embedded)  within each row of letters.
                              The letters are in the correct order.
 
 1.    JSOYOMFUBELR
 2.    SCDUARWDRLYE
 3.    DASNAGEFERTY
 4.    CLULOOTSCEHN
 5.    USENEARSEYNE

From Pancho.Jones at proton.me  Fri Feb 24 13:59:18 2023
From: Pancho.Jones at proton.me (Pancho)
Date: Fri, 24 Feb 2023 18:59:18 +0000
Subject: Find 6-letter words that are hidden (embedded) within
In-Reply-To: <dd48cdc4-a57e-418d-b956-162a8d7030e8n@googlegroups.com>
References: <dd48cdc4-a57e-418d-b956-162a8d7030e8n@googlegroups.com>
Message-ID: <ttb1e6$298tp$1@dont-email.me>

On 24/02/2023 18:34, Hen Hanna wrote:
> 
>    i just wrote a program,  which...
>                      within    [FunFunPython]
>             finds:             (funny,    futon,    python)
> 
>                               ( 5- and 6- letter words )
> 
> 
> (my program uses a Trie, but is pretty simple)
> 
> 
> 
> Maybe someone would show me
>              how it's done using    itertools,   Permutations, etc.
> 
>   Wouldn't it get too slow for Letter-Seeds longer than 11 letters or so?
> 

For performance, generally you sort the characters in a word to create a 
new word (character list) and compare the sorted character lists (which 
you can use a string for). Two anagrams will produce the same sorted 
character list.

The problem with going through all permutations is that the number of 
permutations tends to grow exponentially, so that is why you sort first.

So first take an English dictionary and build a python dictionary with 
the key being the character sorted list and the value being a list of
the English words that produce the sorted list, aka anagrams.

You then sort your specific word and test each subset of this sorted 
word (each combination of characters) against your python anagram 
dictionary.


From PythonList at DancesWithMice.info  Fri Feb 24 14:47:00 2023
From: PythonList at DancesWithMice.info (dn)
Date: Sat, 25 Feb 2023 08:47:00 +1300
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <20230224191244.kn6quzr4x5muthln@hjp.at>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
Message-ID: <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>

On 25/02/2023 08.12, Peter J. Holzer wrote:
> On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
>> In some ways, providing this information seems appropriate. Curiously, this
>> does not even occur during an assert exception - despite the
>> value/relationship being the whole point of using the command!
>>
>>      x = 1
>>      assert x == 2
>>
>> AssertionError (and that's it)
> 
> Pytest is great there. If an assertion in a test case fails it analyzes
> the expression to give you various levels of details:
> 
> ======================================== test session starts ========================================
> platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
> rootdir: /home/hjp/tmp/t
> plugins: cov-3.0.0, anyio-3.6.1
> collected 1 item
> 
> test_a.py F                                                                                   [100%]
> 
> ============================================= FAILURES ==============================================
> ______________________________________________ test_a _______________________________________________
> 
>      def test_a():
>          a = [1, 2, 3]
>          b = {"a": a, "b": 2}
> 
>>        assert len(a) == len(b)
> E       AssertionError: assert 3 == 2
> E        +  where 3 = len([1, 2, 3])
> E        +  and   2 = len({'a': [1, 2, 3], 'b': 2})
> 
> test_a.py:7: AssertionError
> ====================================== short test summary info ======================================
> FAILED test_a.py::test_a - AssertionError: assert 3 == 2
> ========================================= 1 failed in 0.09s =========================================

+1
and hence the tone of slight surprise in the observation - because only 
ever use assert within pytests, and as observed, pytest amplifies the 
report-back to provide actionable-intelligence. See also: earlier 
contribution about using a debugger.


That said, have observed coders 'graduating' from other languages, 
making wider use of assert - assumed to be more data (value) 
sanity-checks than typing, but ...

Do you use assert frequently?


The OP seems wedded to his?her ways, complaining that Python does not 
work the way it 'should'. In turn, gives rise to the impression that 
expounding the advantages of TDD, and thus anticipating such unit and 
integration error-possibilities, might be considered an insult or unhelpful.
(sigh!)

Personally, I struggled a bit to adapt from the more-strictured (if not 
more-structured) languages of my past, to Python - particularly the 
different philosophies or emphases of what happens at 'compile-time' cf 
'execution-time'; and how such required marked changes in attitudes to 
design, time-allocation, work-flow, and tool-set. Two related-activities 
which made the language-change more workable and unleashed greater than 
step-change advantage, were: increased use of TDD, and actively learning 
the facilities within Python-oriented IDEs.

-- 
Regards,
=dn

From hjp-python at hjp.at  Fri Feb 24 15:13:30 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 24 Feb 2023 21:13:30 +0100
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
 <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
Message-ID: <20230224201330.cmxf63cwq66eqsvd@hjp.at>

On 2023-02-25 08:47:00 +1300, dn via Python-list wrote:
> That said, have observed coders 'graduating' from other languages, making
> wider use of assert - assumed to be more data (value) sanity-checks than
> typing, but ...
> 
> Do you use assert frequently?

Not very often, but I do use it. Sometimes for its intended purpose
(i.e. to guard against bugs or wrong assumptions), sometimes just to
guard incomplete or sloppy code (e.g. browsing through some projects I
find
    assert len(data["structure"]["dimensions"]["observation"]) == 1
(incomplete code - I didn't bother to implement multiple observations)
and
    assert(header[0] == "Monat (JJJJMM)")
(the code below is sloppy. Instead of fixing it I just made the original
programmer's assumptions explicit)
and of course
    assert False
(this point should never be reached)).

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230224/94269a7a/attachment.sig>

From henhanna at gmail.com  Fri Feb 24 15:15:22 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Fri, 24 Feb 2023 12:15:22 -0800 (PST)
Subject: Find 6-letter words that are hidden (embedded) within
In-Reply-To: <dd48cdc4-a57e-418d-b956-162a8d7030e8n@googlegroups.com>
References: <dd48cdc4-a57e-418d-b956-162a8d7030e8n@googlegroups.com>
Message-ID: <60c3952d-a0cd-44b6-b762-43d68149b1e8n@googlegroups.com>

On Friday, February 24, 2023 at 10:34:31?AM UTC-8, Hen Hanna wrote:
> i just wrote a program, which... 
>                within [FunFunPython] 
>                finds: (funny, futon, python) 
> 
>                                ( 5- and 6- letter words ) 
> 
> 
> (my program uses a Trie, but is pretty simple) 
> 
> 
> 
> Maybe someone would show me 
>                   how it's done using itertools, Permutations, etc. 
> 
> Wouldn't it get too slow for Letter-Seeds longer than 11 letters or so? 
> 
> 
> ______________________ 
> 
> Find 6-letter words that are hidden (embedded) within each row of letters. 
>                                 The letters are in the correct order. 
> 
> 1. JSOYOMFUBELR 
> 2. SCDUARWDRLYE 
> 3. DASNAGEFERTY 
> 4. CLULOOTSCEHN 
> 5. USENEARSEYNE



> The letters are in the correct order. -------- So this problem is not about Anagraming.



it seems that 
https://docs.python.org/3/library/itertools.html
doesn't offer waht i want, so i wrote this (below) and it works.

                           Is there a better or faster way to do the same thing?


def subs( x ): 
    if x=='':   return  ['']
    x1= subs( x[1:] )
    return x1 + mapAdd(x[0] , x1)

def mapAdd( x, Ylis ):    return [ x+y for y in Ylis ]

print( subs('ab' ))
print( subs('abc' ))

From David.Raymond at tomtom.com  Fri Feb 24 16:16:17 2023
From: David.Raymond at tomtom.com (David Raymond)
Date: Fri, 24 Feb 2023 21:16:17 +0000
Subject: Find 6-letter words that are hidden (embedded) within
In-Reply-To: <60c3952d-a0cd-44b6-b762-43d68149b1e8n@googlegroups.com>
References: <dd48cdc4-a57e-418d-b956-162a8d7030e8n@googlegroups.com>
 <60c3952d-a0cd-44b6-b762-43d68149b1e8n@googlegroups.com>
Message-ID: <HE1PR0701MB2857ED82402A56D70CBC882D87A89@HE1PR0701MB2857.eurprd07.prod.outlook.com>

> Find 6-letter words that are hidden (embedded) within each row of letters. 
>                                 The letters are in the correct order. 
> 
> 1. JSOYOMFUBELR 
> 2. SCDUARWDRLYE 
> 3. DASNAGEFERTY 
> 4. CLULOOTSCEHN 
> 5. USENEARSEYNE

> The letters are in the correct order. -------- So this problem is not about Anagraming.


You can get every combination of 6 letters out of it with itertools.combinations like below.
Just implement the isWord function to return whether a string actually counts as a legit word or not.
12 choose 6 is only 924 combinations to check, so shouldn't be too bad to check them all.


def isWord(word):
    return True #Best left as an exercise to the reader

startWord = "JSOYOMFUBELR"
subLetterCount = 6

foundWords = set()

for letters in itertools.combinations(startWord, subLetterCount):
    word = "".join(letters)
    if word not in foundWords and isWord(word):
        print(word)
        foundWords.add(word)

From henhanna at gmail.com  Fri Feb 24 15:36:14 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Fri, 24 Feb 2023 12:36:14 -0800 (PST)
Subject: TypeError: can only concatenate str (not "int") to str
Message-ID: <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>

                     TypeError: can only concatenate str (not "int") to str

thanks for the comments,  ---   esp. 2 or 3 (?) ppl  who  directly addressed it or commented on it.


py   and pypy  don't tell me what the (offending)  int   is  

                           (even tho'  it'd be easy to show me the int, or number)

because  a Tuple, List... may produce  a long output and they don't want to deal with ways of abbreviating  it   (with ....... )

and because  User-defined  Class object may cause another error while printing it.



_________________________________________
                  TypeError: can only join an iterable

Here too it'd help me   if  pypy (Jeannie)  could tell me what the offending data object is.

From nntp.mbourne at spamgourmet.com  Fri Feb 24 16:04:12 2023
From: nntp.mbourne at spamgourmet.com (Mark Bourne)
Date: Fri, 24 Feb 2023 21:04:12 +0000
Subject: Line continuation and comments
In-Reply-To: <mailman.1952.1677186731.20444.python-list@python.org>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
 <002d01d947cb$7c262c70$74728550$@gmail.com>
 <mailman.1952.1677186731.20444.python-list@python.org>
Message-ID: <ttb8od$2ac41$1@dont-email.me>

Personally, I don't particularly like the way you have to put multiline 
strings on the far left (rather than aligned with the rest of the scope) 
to avoid getting spaces at the beginning of each line.  I find it makes 
it more difficult to see where the scope of the class/method/etc. 
actually ends, especially if there are multiple such strings.  It's not 
too bad for strings defined at the module level (outer scope) though, 
and of course for docstrings the extra spaces at the beginning of each 
line don't matter.

However, rather than using "+" to join strings as in your examples 
(which, as you suggest, is probably less efficient), I tend to use 
string literal concatenation which I gather is more efficient (treated 
as a single string at compile-time rather than joining separate strings 
at run-time).  See 
<https://docs.python.org/3/reference/lexical_analysis.html#string-literal-concatenation>.

For example:
       HelpText = ("Left click:             Open spam\n"
                   "Shift + Left click:     Cook spam\n"
                   "Right click:            Crack egg\n"
                   "Shift + Right click:    Fry egg\n")

The downside is having to put an explicit "\n" at the end of each line, 
but to me that's not as bad as having to align the content to the far left.

Getting a bit more on topic, use of backslashes in strings is a bit 
different to backslashes for line continuation anyway.  You could almost 
think of "\
(newline)" in a multiline string as being like an escape sequence 
meaning "don't actually put a newline character in the string here", in 
a similar way to "\n" meaning "put a newline character here" and "\t" 
meaning "put a tab character here".

Mark.


avi.e.gross at gmail.com wrote:
> Good example, Rob, of how some people make what I consider RELIGIOUS edicts that one can easily violate if one wishes and it makes lots of sense in your example.
> 
> Let me extend that. The goal was to store a character string consisting of multiple lines when printed that are all left-aligned. Had you written:
> 
>       HelpText = """
> Left click:             Open spam
> ...
> Shift + Right click:    Fry egg
> """
> Then it would begin with an extra carriage return you did not want. Your example also ends with a carriage return because you closed the quotes on another line, so a \ on the last line of text (or moving the quotes to the end of the line) would be a way of avoiding that.
> 
> Consider some alternatives I have seen that are in a sense ugly and may involve extra work for the interpreter unless it is byte compiled once.
> 
> def someFunc():
>       HelpText =
>       "Left click:             Open spam" + "\n" +
>       "Shift + Left click:     Cook spam" + "\n" +
>       ...
> 
> Or the variant of:
> HelpText =  "Left click:             Open spam\n"
> HelpText +=  " Shift + Left click:     Cook spam\n"
> ...
> 
> Or perhaps just dumping the multi-line text into a file beforehand and reading that into a string!
> 
> def someFunc():
> 
> The backslash is not looking like such a bad idea! LOL!
> 
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Rob Cliffe via Python-list
> Sent: Wednesday, February 22, 2023 2:08 PM
> To: python-list at python.org
> Subject: Re: Line continuation and comments
> 
> 
> 
> On 22/02/2023 15:23, Paul Bryan wrote:
>> Adding to this, there should be no reason now in recent versions of
>> Python to ever use line continuation. Black goes so far as to state
>> "backslashes are bad and should never be used":
>>
>> https://black.readthedocs.io/en/stable/the_black_code_style/future_sty
>> le.html#using-backslashes-for-with-statements
> 
> def someFunc():
>       HelpText = """\
> Left click:             Open spam
> Shift + Left click:     Cook spam
> Right click:            Crack egg
> Shift + Right click:    Fry egg
> """
> 
> The initial backslash aligns the first line with the others (in a fixed font of course).
> Best wishes
> Rob Cliffe
> --
> https://mail.python.org/mailman/listinfo/python-list
> 

From roel at roelschroeven.net  Fri Feb 24 16:32:40 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Fri, 24 Feb 2023 22:32:40 +0100
Subject: Line continuation and comments
In-Reply-To: <ttb8od$2ac41$1@dont-email.me>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
 <002d01d947cb$7c262c70$74728550$@gmail.com>
 <mailman.1952.1677186731.20444.python-list@python.org>
 <ttb8od$2ac41$1@dont-email.me>
Message-ID: <e88f91b8-5095-d20d-4cc1-60978478b651@roelschroeven.net>

Mark Bourne schreef op 24/02/2023 om 22:04:
> Personally, I don't particularly like the way you have to put multiline
> strings on the far left (rather than aligned with the rest of the scope)
> to avoid getting spaces at the beginning of each line.  I find it makes
> it more difficult to see where the scope of the class/method/etc.
> actually ends, especially if there are multiple such strings.  It's not
> too bad for strings defined at the module level (outer scope) though,
> and of course for docstrings the extra spaces at the beginning of each
> line don't matter.
A way around this is using textwrap.dedent() 
(https://docs.python.org/3.10/library/textwrap.html?highlight=dedent#textwrap.dedent). 
Example from the documentation:

 ??? def test():
 ??????? # end first line with \ to avoid the empty line!
 ??????? s = '''\
 ??????? hello
 ????????? world
 ??????? '''
 ??????? print(repr(s))????????? # prints '??? hello\n world\n??? '
 ??????? print(repr(dedent(s)))? # prints 'hello\n? world\n'


-- 
"Met een spitsvondig citaat bewijs je niets."
         -- Voltaire


From avi.e.gross at gmail.com  Fri Feb 24 17:35:06 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Fri, 24 Feb 2023 17:35:06 -0500
Subject: Evaluate once or every time
In-Reply-To: <ttb8od$2ac41$1@dont-email.me>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
 <002d01d947cb$7c262c70$74728550$@gmail.com>
 <mailman.1952.1677186731.20444.python-list@python.org>
 <ttb8od$2ac41$1@dont-email.me>
Message-ID: <002201d948a0$3e91a580$bbb4f080$@gmail.com>

Mark,

I was very interested in the point you made and have never thought much about string concatenation this way but adjacency is an operator worth using.

This message has a new subject line as it is not about line continuation or comments.

>From what you say, concatenation between visibly adjacent strings is done once when generating bytecode. Using a plus is supposed to be about the same but may indeed result in either an error if you use anything other than a string literal

bad = "hello " str(12)

or you must use something like a "+" to do the concatenation at each run time. Or, weirder, do it manually as : 

good = "hello ".__add__(str(12))

This may be no big deal in terms of efficiency but something to consider.

I have often stared in amazement at code like:

>>> mylist = "The quick brown fox jumps over the lazy dog".split()

>>> mylist
['The', 'quick', 'brown', 'fox', 'jumps', 'over', 'the', 'lazy', 'dog']

Or perhaps to make a list of vowels:

import string

vowels = list("aeiouAEIOU")
consonants = sorted(list(set(string.ascii_letters) - set(vowels)))

I mean couldn't you do this work in advance and do something like:

vowels = ['A', 'E', 'I', 'O', 'U', 'a', 'e', 'i', 'o', 'u']
consonants = ['B', 'C', 'D', 'F', 'G', 'H', 'J', 'K', 'L', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'V', 'W', 'X', 'Y', 'Z', 'b', 'c', 'd', 'f', 'g', 'h', 'j', 'k', 'l', 'm', 'n', 'p', 'q', 'r', 's', 't', 'v', 'w', 'x', 'y', 'z']

I assume this latter version would be set once no matter how often you run the unchanged program. YES, I am aware this may be bad practice for code you want to adapt for international use. 

But why be wasteful? I am currently reading a book on refactoring and will not share if it is illustrated, or if the above is a decent example as the book uses examples in JavaScript. ?



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Mark Bourne
Sent: Friday, February 24, 2023 4:04 PM
To: python-list at python.org
Subject: Re: Line continuation and comments

Personally, I don't particularly like the way you have to put multiline strings on the far left (rather than aligned with the rest of the scope) to avoid getting spaces at the beginning of each line.  I find it makes it more difficult to see where the scope of the class/method/etc. 
actually ends, especially if there are multiple such strings.  It's not too bad for strings defined at the module level (outer scope) though, and of course for docstrings the extra spaces at the beginning of each line don't matter.

However, rather than using "+" to join strings as in your examples (which, as you suggest, is probably less efficient), I tend to use string literal concatenation which I gather is more efficient (treated as a single string at compile-time rather than joining separate strings at run-time).  See <https://docs.python.org/3/reference/lexical_analysis.html#string-literal-concatenation>.

For example:
       HelpText = ("Left click:             Open spam\n"
                   "Shift + Left click:     Cook spam\n"
                   "Right click:            Crack egg\n"
                   "Shift + Right click:    Fry egg\n")

The downside is having to put an explicit "\n" at the end of each line, but to me that's not as bad as having to align the content to the far left.

Getting a bit more on topic, use of backslashes in strings is a bit different to backslashes for line continuation anyway.  You could almost think of "\ (newline)" in a multiline string as being like an escape sequence meaning "don't actually put a newline character in the string here", in a similar way to "\n" meaning "put a newline character here" and "\t" 
meaning "put a tab character here".

Mark.


avi.e.gross at gmail.com wrote:
> Good example, Rob, of how some people make what I consider RELIGIOUS edicts that one can easily violate if one wishes and it makes lots of sense in your example.
> 
> Let me extend that. The goal was to store a character string consisting of multiple lines when printed that are all left-aligned. Had you written:
> 
>       HelpText = """
> Left click:             Open spam
> ...
> Shift + Right click:    Fry egg
> """
> Then it would begin with an extra carriage return you did not want. Your example also ends with a carriage return because you closed the quotes on another line, so a \ on the last line of text (or moving the quotes to the end of the line) would be a way of avoiding that.
> 
> Consider some alternatives I have seen that are in a sense ugly and may involve extra work for the interpreter unless it is byte compiled once.
> 
> def someFunc():
>       HelpText =
>       "Left click:             Open spam" + "\n" +
>       "Shift + Left click:     Cook spam" + "\n" +
>       ...
> 
> Or the variant of:
> HelpText =  "Left click:             Open spam\n"
> HelpText +=  " Shift + Left click:     Cook spam\n"
> ...
> 
> Or perhaps just dumping the multi-line text into a file beforehand and reading that into a string!
> 
> def someFunc():
> 
> The backslash is not looking like such a bad idea! LOL!
> 
> -----Original Message-----
> From: Python-list 
> <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of 
> Rob Cliffe via Python-list
> Sent: Wednesday, February 22, 2023 2:08 PM
> To: python-list at python.org
> Subject: Re: Line continuation and comments
> 
> 
> 
> On 22/02/2023 15:23, Paul Bryan wrote:
>> Adding to this, there should be no reason now in recent versions of 
>> Python to ever use line continuation. Black goes so far as to state 
>> "backslashes are bad and should never be used":
>>
>> https://black.readthedocs.io/en/stable/the_black_code_style/future_st
>> y le.html#using-backslashes-for-with-statements
> 
> def someFunc():
>       HelpText = """\
> Left click:             Open spam
> Shift + Left click:     Cook spam
> Right click:            Crack egg
> Shift + Right click:    Fry egg
> """
> 
> The initial backslash aligns the first line with the others (in a fixed font of course).
> Best wishes
> Rob Cliffe
> --
> https://mail.python.org/mailman/listinfo/python-list
> 
--
https://mail.python.org/mailman/listinfo/python-list


From rosuav at gmail.com  Fri Feb 24 17:38:13 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 25 Feb 2023 09:38:13 +1100
Subject: Evaluate once or every time
In-Reply-To: <002201d948a0$3e91a580$bbb4f080$@gmail.com>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
 <002d01d947cb$7c262c70$74728550$@gmail.com>
 <mailman.1952.1677186731.20444.python-list@python.org>
 <ttb8od$2ac41$1@dont-email.me> <002201d948a0$3e91a580$bbb4f080$@gmail.com>
Message-ID: <CAPTjJmpbNP3OTJCcBL0dT4y-hfnV9bHp_hfEfTbWBKTva7+SHA@mail.gmail.com>

On Sat, 25 Feb 2023 at 09:36, <avi.e.gross at gmail.com> wrote:
> From what you say, concatenation between visibly adjacent strings is done once when generating bytecode. Using a plus is supposed to be about the same but may indeed result in either an error if you use anything other than a string literal
>
> bad = "hello " str(12)
>
> or you must use something like a "+" to do the concatenation at each run time. Or, weirder, do it manually as :
>

Abuttal is a syntactic feature. It's completely different from string
concatenation. The only similarity is that some forms of addition may
be constant-folded.

ChrisA

From mats at wichmann.us  Fri Feb 24 17:39:46 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Fri, 24 Feb 2023 15:39:46 -0700
Subject: Not receiving posts
In-Reply-To: <63f7d2f8.020a0220.6ce17.4bdc@mx.google.com>
References: <f2a406d7-f12c-a8e3-c133-6279e0cc923f@comcast.net>
 <63f7d2f8.020a0220.6ce17.4bdc@mx.google.com>
Message-ID: <542dc94a-7d15-ece8-81af-b606d1d0efea@wichmann.us>

On 2/23/23 13:56, Grant Edwards wrote:
> On 2023-02-23, Jim Byrnes <jf_byrnes at comcast.net> wrote:
> 
>> I have been reading the python-list for some time now. At first via
>> gemane and since it's demise via a subscription.
> 
> FWIW, gmane is still there, and still working fine. I read this list
> by pointing slrn at news.gmane.io

Well gmane did die, and caused changes and the story is weird even by 
internet standards.  And the founder eventually spilled....

https://lars.ingebrigtsen.no/2020/01/06/whatever-happened-to-news-gmane-org/

Not everything came back the same.


From PythonList at DancesWithMice.info  Fri Feb 24 18:12:13 2023
From: PythonList at DancesWithMice.info (dn)
Date: Sat, 25 Feb 2023 12:12:13 +1300
Subject: Line continuation and comments
In-Reply-To: <ttb8od$2ac41$1@dont-email.me>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
 <002d01d947cb$7c262c70$74728550$@gmail.com>
 <mailman.1952.1677186731.20444.python-list@python.org>
 <ttb8od$2ac41$1@dont-email.me>
Message-ID: <3b7e2f43-9ff1-fe30-3738-9c636ac63c32@DancesWithMice.info>

On 25/02/2023 10.04, Mark Bourne wrote:
> Personally, I don't particularly like the way you have to put multiline 
> strings on the far left (rather than aligned with the rest of the scope) 
> to avoid getting spaces at the beginning of each line.? I find it makes 
> it more difficult to see where the scope of the class/method/etc. 
> actually ends, especially if there are multiple such strings.? It's not 
> too bad for strings defined at the module level (outer scope) though, 
> and of course for docstrings the extra spaces at the beginning of each 
> line don't matter.
> 
> However, rather than using "+" to join strings as in your examples 
> (which, as you suggest, is probably less efficient), I tend to use 
> string literal concatenation which I gather is more efficient (treated 
> as a single string at compile-time rather than joining separate strings 
> at run-time).? See 
> <https://docs.python.org/3/reference/lexical_analysis.html#string-literal-concatenation>.
> 
> For example:
>  ????? HelpText = ("Left click:???????????? Open spam\n"
>  ????????????????? "Shift + Left click:???? Cook spam\n"
>  ????????????????? "Right click:??????????? Crack egg\n"
>  ????????????????? "Shift + Right click:??? Fry egg\n")
> 
> The downside is having to put an explicit "\n" at the end of each line, 
> but to me that's not as bad as having to align the content to the far left.

S\Possible solution to that:

HelpText = "\n".join( ["Left click:             Open spam",
...                   "Shift + Left click:     Cook spam",
...                   "Right click:            Crack egg",
...                   "Shift + Right click:    Fry egg"
...                   ]
...                   )

To PEP-008 Police:
Apologies for formatting - was working from original example

In this manner, the sub-strings may be placed wherever you see fit, and 
those pesky line-endings no-longer contribute to visual clutter (but 
maybe the brackets do...)

Another philosophy is to move awkward strings, such as the examples in 
this thread, 'out' of the mainline code and into a config-file (of sorts).

The PEP-008 Police are more likely to ignore transgressions in a 
'data-file' during code-review. So, you can format the code in whatever 
fashion you like...


Text for GUIs, report print-outs, etc, tends to be at the (tender) mercy 
of (opinionated) users. By putting such into a separate file, such 
changes can be made without (as much) risk of 'breaking' the code 
itself. Ultimately, if you ever have to expand the application to 
multi-tenancy, such will become necessary.

Using a separate file, you can consider if should write such as Python 
(import module) or use something else like JSON, YAML, ...
(and may God bless all who sail in her...).

YMMV!


> Getting a bit more on topic, use of backslashes in strings is a bit 
> different to backslashes for line continuation anyway.? You could almost 
> think of "\
> (newline)" in a multiline string as being like an escape sequence 
> meaning "don't actually put a newline character in the string here", in 
> a similar way to "\n" meaning "put a newline character here" and "\t" 
> meaning "put a tab character here".

Book title: "Don't Make Me Think"
(thoroughly recommended. Author: Steve Krug)

-- 
Regards,
=dn

From list1 at tompassin.net  Fri Feb 24 18:19:52 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 24 Feb 2023 18:19:52 -0500
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
 <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
Message-ID: <c995a27d-ce40-c639-e589-55a8367a6f46@tompassin.net>

On 2/24/2023 2:47 PM, dn via Python-list wrote:
> On 25/02/2023 08.12, Peter J. Holzer wrote:
>> On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
>>> In some ways, providing this information seems appropriate. 
>>> Curiously, this
>>> does not even occur during an assert exception - despite the
>>> value/relationship being the whole point of using the command!
>>>
>>> ???? x = 1
>>> ???? assert x == 2
>>>
>>> AssertionError (and that's it)

Sometimes you can use a second parameter to assert if you know what kind 
of error to expect:

 >>> a = [1,2,3]
 >>> b = [4,5]
 >>> assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
AssertionError: len(a): 3 != len(b): 2

With type errors, assert may actually give you the information needed:

 >>> c = {"a": a, "b": 2}
 >>> assert a > c
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: '>' not supported between instances of 'list' and 'dict'

So now we know that a is a list and c is a dictionary.

>> Pytest is great there. If an assertion in a test case fails it analyzes
>> the expression to give you various levels of details:
>>
>> ======================================== test session starts 
>> ========================================
>> platform linux -- Python 3.10.6, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
>> rootdir: /home/hjp/tmp/t
>> plugins: cov-3.0.0, anyio-3.6.1
>> collected 1 item
>>
>> test_a.py 
>> F?????????????????????????????????????????????????????????????????????????????????? [100%]
>>
>> ============================================= FAILURES 
>> ==============================================
>> ______________________________________________ test_a 
>> _______________________________________________
>>
>> ???? def test_a():
>> ???????? a = [1, 2, 3]
>> ???????? b = {"a": a, "b": 2}
>>
>>> ?????? assert len(a) == len(b)
>> E?????? AssertionError: assert 3 == 2
>> E??????? +? where 3 = len([1, 2, 3])
>> E??????? +? and?? 2 = len({'a': [1, 2, 3], 'b': 2})
>>
>> test_a.py:7: AssertionError
>> ====================================== short test summary info 
>> ======================================
>> FAILED test_a.py::test_a - AssertionError: assert 3 == 2
>> ========================================= 1 failed in 0.09s 
>> =========================================
> 
> +1
> and hence the tone of slight surprise in the observation - because only 
> ever use assert within pytests, and as observed, pytest amplifies the 
> report-back to provide actionable-intelligence. See also: earlier 
> contribution about using a debugger.
> 
> 
> That said, have observed coders 'graduating' from other languages, 
> making wider use of assert - assumed to be more data (value) 
> sanity-checks than typing, but ...
> 
> Do you use assert frequently?
> 
> 
> The OP seems wedded to his?her ways, complaining that Python does not 
> work the way it 'should'. In turn, gives rise to the impression that 
> expounding the advantages of TDD, and thus anticipating such unit and 
> integration error-possibilities, might be considered an insult or 
> unhelpful.
> (sigh!)
> 
> Personally, I struggled a bit to adapt from the more-strictured (if not 
> more-structured) languages of my past, to Python - particularly the 
> different philosophies or emphases of what happens at 'compile-time' cf 
> 'execution-time'; and how such required marked changes in attitudes to 
> design, time-allocation, work-flow, and tool-set. Two related-activities 
> which made the language-change more workable and unleashed greater than 
> step-change advantage, were: increased use of TDD, and actively learning 
> the facilities within Python-oriented IDEs.
> 


From list1 at tompassin.net  Fri Feb 24 18:42:39 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 24 Feb 2023 18:42:39 -0500
Subject: Evaluate once or every time
In-Reply-To: <002201d948a0$3e91a580$bbb4f080$@gmail.com>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
 <002d01d947cb$7c262c70$74728550$@gmail.com>
 <mailman.1952.1677186731.20444.python-list@python.org>
 <ttb8od$2ac41$1@dont-email.me> <002201d948a0$3e91a580$bbb4f080$@gmail.com>
Message-ID: <b17fec1f-2608-1d63-d887-c425c325fc34@tompassin.net>

On 2/24/2023 5:35 PM, avi.e.gross at gmail.com wrote:
> Mark,
> 
> I was very interested in the point you made and have never thought much about string concatenation this way but adjacency is an operator worth using.
> 
> This message has a new subject line as it is not about line continuation or comments.
> 
>  From what you say, concatenation between visibly adjacent strings is done once when generating bytecode. Using a plus is supposed to be about the same but may indeed result in either an error if you use anything other than a string literal
> 
> bad = "hello " str(12)
> 
> or you must use something like a "+" to do the concatenation at each run time. Or, weirder, do it manually as :
> 
> good = "hello ".__add__(str(12))
> 
> This may be no big deal in terms of efficiency but something to consider.
> 
> I have often stared in amazement at code like:
> 
>>>> mylist = "The quick brown fox jumps over the lazy dog".split()
> 
>>>> mylist
> ['The', 'quick', 'brown', 'fox', 'jumps', 'over', 'the', 'lazy', 'dog']

Why be amazed? The .split() version is way easier to type.  I do it all 
the time.  Usually these evaluations will be done only once, at compile 
time, so the cost in computing time is usually not important.  And, if 
you get right down to it, the first form is actually what you mean.  The 
second form is an implementation detail (you need to feed a list of 
words into a function).

> Or perhaps to make a list of vowels:
> 
> import string
> 
> vowels = list("aeiouAEIOU")
> consonants = sorted(list(set(string.ascii_letters) - set(vowels)))
> 
> I mean couldn't you do this work in advance and do something like:
> 
> vowels = ['A', 'E', 'I', 'O', 'U', 'a', 'e', 'i', 'o', 'u']
> consonants = ['B', 'C', 'D', 'F', 'G', 'H', 'J', 'K', 'L', 'M', 'N', 'P', 'Q', 'R', 'S', 'T', 'V', 'W', 'X', 'Y', 'Z', 'b', 'c', 'd', 'f', 'g', 'h', 'j', 'k', 'l', 'm', 'n', 'p', 'q', 'r', 's', 't', 'v', 'w', 'x', 'y', 'z']

I don't want to have to type these monsters, and in this form I'd be 
prone to making an error.  And they are hard to proofread.  Anyway, a 
list like this is typically used to assess whether a character is in one 
or the other group, and you can do it without turning the string into a 
list of characters:

VOWELS = 'aeiouAEIOU'
is_vowel = 'y' in VOWELS

If I really needed them to be in a list, I'd probably do a list 
comprehension:

VOWEL_LIST = [ch for ch in VOWELS]

Again, much easier to type and to read, closer to what is really meant, 
and less error-prone.

In the case of the consonants, even in string form it would be hard to 
proofread, so I'd be inclined to insert spaces for readability:

CONSONANTS = 'BCDFGH JKLMNP QRSTVWX YZ bcdfgh jklmnp qrstvwx yz'

If for some reason spaces wouldn't work, I'd delete them:
CONSONANTS  = CONSONANTS.replace(' ', '')

In concept this isn't much different from using the C pre-processor to 
pre-compute some things for you.

> 
> I assume this latter version would be set once no matter how often you run the unchanged program. YES, I am aware this may be bad practice for code you want to adapt for international use.
> 
> But why be wasteful? I am currently reading a book on refactoring and will not share if it is illustrated, or if the above is a decent example as the book uses examples in JavaScript. ?


From grant.b.edwards at gmail.com  Fri Feb 24 18:33:01 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 24 Feb 2023 15:33:01 -0800 (PST)
Subject: Not receiving posts
References: <f2a406d7-f12c-a8e3-c133-6279e0cc923f@comcast.net>
 <63f7d2f8.020a0220.6ce17.4bdc@mx.google.com>
 <542dc94a-7d15-ece8-81af-b606d1d0efea@wichmann.us>
Message-ID: <63f9492d.020a0220.843ba.019c@mx.google.com>

On 2023-02-24, Mats Wichmann <mats at wichmann.us> wrote:
> On 2/23/23 13:56, Grant Edwards wrote:
>> On 2023-02-23, Jim Byrnes <jf_byrnes at comcast.net> wrote:
>> 
>>> I have been reading the python-list for some time now. At first via
>>> gemane and since it's demise via a subscription.
>> 
>> FWIW, gmane is still there, and still working fine. I read this list
>> by pointing slrn at news.gmane.io
>
> Well gmane did die, and caused changes and the story is weird even by 
> internet standards.  And the founder eventually spilled....
>
> https://lars.ingebrigtsen.no/2020/01/06/whatever-happened-to-news-gmane-org/

Interesting! I had seen less detailed versions of that story
elsewhere, but never saw that.

> Not everything came back the same.

I do sometimes miss the old webUI. The search function was useful, and
it was a handy place to point people to when you wanted to reference
an old mailing list posting. But, I certainly don't have the time and
skills to re-create it, so I've got absolutely no right to complain.

After setting up and maintaining gmane.org for all those years,
there's no way Mr. Ingebrigtsen's positive karma balance has been
noticably reduced by the bumpy transition that resulted in gmane.io.




From henhanna at gmail.com  Fri Feb 24 18:49:56 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Fri, 24 Feb 2023 15:49:56 -0800 (PST)
Subject: Find 6-letter words that are hidden (embedded) within
In-Reply-To: <mailman.1978.1677273382.20444.python-list@python.org>
References: <AQHZSI8oJpzCo/KJKUmUJN1eDSBm067ekwzQ>
 <dd48cdc4-a57e-418d-b956-162a8d7030e8n@googlegroups.com>
 <HE1PR0701MB2857ED82402A56D70CBC882D87A89@HE1PR0701MB2857.eurprd07.prod.outlook.com>
 <60c3952d-a0cd-44b6-b762-43d68149b1e8n@googlegroups.com>
 <mailman.1978.1677273382.20444.python-list@python.org>
Message-ID: <51b64c46-1c3f-4bb9-887a-ed38338626a2n@googlegroups.com>

On Friday, February 24, 2023 at 1:18:28?PM UTC-8, David Raymond wrote:
> > Find 6-letter words that are hidden (embedded) within each row of letters. 
> > The letters are in the correct order. 
> > 
> > 1. JSOYOMFUBELR 
> > 2. SCDUARWDRLYE 
> > 3. DASNAGEFERTY 
> > 4. CLULOOTSCEHN 
> > 5. USENEARSEYNE 
> 
> > The letters are in the correct order. -------- So this problem is not about Anagraming.
> You can get every combination of 6 letters out of it with itertools.combinations like below. 
> Just implement the isWord function to return whether a string actually counts as a legit word or not. 
> 12 choose 6 is only 924 combinations to check, so shouldn't be too bad to check them all. 
> 
> 
> def isWord(word): 
> return True #Best left as an exercise to the reader 
> 
> startWord = "JSOYOMFUBELR" 
> subLetterCount = 6 
> 
> foundWords = set() 
> 
> for letters in itertools.combinations(startWord, subLetterCount): 
> word = "".join(letters) 
> if word not in foundWords and isWord(word): 
> print(word) 
> foundWords.add(word)



                       thank you...  i could use the Pset below, too.


# Factorial Python One-Liner
def fac(n):       return       reduce(lambda x, y: x * y, range(1, n+1))

for i in range(1,11):        print ( '\t', fac(i) )


# Power set Python One-Liner
def Pset(L):      return        reduce(lambda z, x: z + [y + [x] for y in z], L, [[]])

print( '\n',   Pset( [1,] ))
print( '\n',   Pset( [1,2,] ))
print( '\n',   Pset( [1,2,3] ))

From 2QdxY4RzWzUUiLuE at potatochowder.com  Fri Feb 24 19:00:17 2023
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Fri, 24 Feb 2023 19:00:17 -0500
Subject: Evaluate once or every time
In-Reply-To: <b17fec1f-2608-1d63-d887-c425c325fc34@tompassin.net>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
 <002d01d947cb$7c262c70$74728550$@gmail.com>
 <mailman.1952.1677186731.20444.python-list@python.org>
 <ttb8od$2ac41$1@dont-email.me>
 <002201d948a0$3e91a580$bbb4f080$@gmail.com>
 <b17fec1f-2608-1d63-d887-c425c325fc34@tompassin.net>
Message-ID: <Y/lPkdAB76S2g1No@anomaly>

On 2023-02-24 at 18:42:39 -0500,
Thomas Passin <list1 at tompassin.net> wrote:

> VOWELS = 'aeiouAEIOU'
> is_vowel = 'y' in VOWELS
> 
> If I really needed them to be in a list, I'd probably do a list
> comprehension:
> 
> VOWEL_LIST = [ch for ch in VOWELS]

Why use a comprehension when a simple loop will do?  ;-)

No.  Wait.  That's not what I meant.

Why use a comprehension when the constructor will do?

VOWEL_LIST = list(VOWELS)

From list1 at tompassin.net  Fri Feb 24 20:36:53 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Fri, 24 Feb 2023 20:36:53 -0500
Subject: Evaluate once or every time
In-Reply-To: <Y/lPkdAB76S2g1No@anomaly>
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <77dd3069-da38-bdee-df26-28c1a94450ad@btinternet.com>
 <002d01d947cb$7c262c70$74728550$@gmail.com>
 <mailman.1952.1677186731.20444.python-list@python.org>
 <ttb8od$2ac41$1@dont-email.me> <002201d948a0$3e91a580$bbb4f080$@gmail.com>
 <b17fec1f-2608-1d63-d887-c425c325fc34@tompassin.net>
 <Y/lPkdAB76S2g1No@anomaly>
Message-ID: <1c9b4b52-4dcc-d69c-7f8c-8fc8eff39c40@tompassin.net>

On 2/24/2023 7:00 PM, 2QdxY4RzWzUUiLuE at potatochowder.com wrote:
> On 2023-02-24 at 18:42:39 -0500,
> Thomas Passin <list1 at tompassin.net> wrote:
> 
>> VOWELS = 'aeiouAEIOU'
>> is_vowel = 'y' in VOWELS
>>
>> If I really needed them to be in a list, I'd probably do a list
>> comprehension:
>>
>> VOWEL_LIST = [ch for ch in VOWELS]
> 
> Why use a comprehension when a simple loop will do?  ;-)
> 
> No.  Wait.  That's not what I meant.
> 
> Why use a comprehension when the constructor will do?
> 
> VOWEL_LIST = list(VOWELS)

It just didn't occur to me, I think, that this would create a list of 
the string's characters.  Very nice!


From PythonList at DancesWithMice.info  Fri Feb 24 22:39:00 2023
From: PythonList at DancesWithMice.info (dn)
Date: Sat, 25 Feb 2023 16:39:00 +1300
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
References: <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
Message-ID: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>

On 25/02/2023 09.36, Hen Hanna wrote:
>                       TypeError: can only concatenate str (not "int") to str
> 
> thanks for the comments,  ---   esp. 2 or 3 (?) ppl  who  directly addressed it or commented on it.

If you haven't already, please review the Python Software Foundation's 
Code of Conduct https://www.python.org/psf/conduct/


> py   and pypy  don't tell me what the (offending)  int   is
> 
>                             (even tho'  it'd be easy to show me the int, or number)
> 
> because  a Tuple, List... may produce  a long output and they don't want to deal with ways of abbreviating  it   (with ....... )
> 
> and because  User-defined  Class object may cause another error while printing it.
> 
> 
> 
> _________________________________________
>                    TypeError: can only join an iterable
> 
> Here too it'd help me   if  pypy (Jeannie)  could tell me what the offending data object is.

Observation:
- python doesn't tell you what you want
- correspondents are not telling you what you want and/or how you want
There's a common theme developing...

Suggestion: if you want help, and at the frequency with which you've 
been posting recently, would it be better not to sound like a 'troll'?


Please understand everyone here is volunteering time, in a bid to 
provide you (and each-other) with benefit.

Does the FACT that you have failed to answer several questions from 
colleagues really entitle you to demand others only directly-address you 
(and only your questions)?

-- 
Regards,
=dn

From hjp-python at hjp.at  Sat Feb 25 01:13:07 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 25 Feb 2023 07:13:07 +0100
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <c995a27d-ce40-c639-e589-55a8367a6f46@tompassin.net>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
 <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
 <c995a27d-ce40-c639-e589-55a8367a6f46@tompassin.net>
Message-ID: <20230225061307.slgufll32u24ex6p@hjp.at>

On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> On 2/24/2023 2:47 PM, dn via Python-list wrote:
> > On 25/02/2023 08.12, Peter J. Holzer wrote:
> > > On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
> > > > In some ways, providing this information seems appropriate.
> > > > Curiously, this does not even occur during an assert exception -
> > > > despite the value/relationship being the whole point of using
> > > > the command!
> > > > 
> > > > ???? x = 1
> > > > ???? assert x == 2
> > > > 
> > > > AssertionError (and that's it)
> 
> Sometimes you can use a second parameter to assert if you know what kind of
> error to expect:
> 
> >>> a = [1,2,3]
> >>> b = [4,5]
> >>> assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> AssertionError: len(a): 3 != len(b): 2

Yup. That's very useful (but I tend to forget that).


> With type errors, assert may actually give you the information needed:
> 
> >>> c = {"a": a, "b": 2}
> >>> assert a > c
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: '>' not supported between instances of 'list' and 'dict'

Actually in this case it isn't assert which gives you the information,
it's evaluating the expression itself. You get the same error with just
    a > c
on a line by its own.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230225/3d88c481/attachment.sig>

From list1 at tompassin.net  Sat Feb 25 09:10:06 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 25 Feb 2023 09:10:06 -0500
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <20230225061307.slgufll32u24ex6p@hjp.at>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
 <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
 <c995a27d-ce40-c639-e589-55a8367a6f46@tompassin.net>
 <20230225061307.slgufll32u24ex6p@hjp.at>
Message-ID: <deeda902-7798-09ed-4ead-c209c5ccd139@tompassin.net>

On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
> On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
>> On 2/24/2023 2:47 PM, dn via Python-list wrote:
>>> On 25/02/2023 08.12, Peter J. Holzer wrote:
>>>> On 2023-02-24 16:12:10 +1300, dn via Python-list wrote:
>>>>> In some ways, providing this information seems appropriate.
>>>>> Curiously, this does not even occur during an assert exception -
>>>>> despite the value/relationship being the whole point of using
>>>>> the command!
>>>>>
>>>>>  ???? x = 1
>>>>>  ???? assert x == 2
>>>>>
>>>>> AssertionError (and that's it)
>>
>> Sometimes you can use a second parameter to assert if you know what kind of
>> error to expect:
>>
>>>>> a = [1,2,3]
>>>>> b = [4,5]
>>>>> assert len(a) == len(b), f'len(a): {len(a)} != len(b): {len(b)}'
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>> AssertionError: len(a): 3 != len(b): 2
> 
> Yup. That's very useful (but I tend to forget that).
> 
> 
>> With type errors, assert may actually give you the information needed:
>>
>>>>> c = {"a": a, "b": 2}
>>>>> assert a > c
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>> TypeError: '>' not supported between instances of 'list' and 'dict'
> 
> Actually in this case it isn't assert which gives you the information,
> it's evaluating the expression itself. You get the same error with just
>      a > c
> on a line by its own.

In some cases.  For my example with an explanatory string, you wouldn't 
want to write code like that after an ordinary line of code, at least 
not very often.  The assert statement allows it syntactically.


From hjp-python at hjp.at  Sat Feb 25 09:20:31 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 25 Feb 2023 15:20:31 +0100
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <deeda902-7798-09ed-4ead-c209c5ccd139@tompassin.net>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
 <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
 <c995a27d-ce40-c639-e589-55a8367a6f46@tompassin.net>
 <20230225061307.slgufll32u24ex6p@hjp.at>
 <deeda902-7798-09ed-4ead-c209c5ccd139@tompassin.net>
Message-ID: <20230225142031.fecczvepuu24eygb@hjp.at>

On 2023-02-25 09:10:06 -0500, Thomas Passin wrote:
> On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
> > On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> > > Sometimes you can use a second parameter to assert if you know what kind of
> > > error to expect:
[...]
> > > With type errors, assert may actually give you the information needed:
> > > 
> > > > > > c = {"a": a, "b": 2}
> > > > > > assert a > c
> > > Traceback (most recent call last):
> > >    File "<stdin>", line 1, in <module>
> > > TypeError: '>' not supported between instances of 'list' and 'dict'
> > 
> > Actually in this case it isn't assert which gives you the information,
> > it's evaluating the expression itself. You get the same error with just
> >      a > c
> > on a line by its own.
> 
> In some cases.  For my example with an explanatory string, you wouldn't want
> to write code like that after an ordinary line of code, at least not very
> often.  The assert statement allows it syntactically.

Yes, but if an error in the expression triggers an exception (as in this
case) the explanatory string will never be displayed.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230225/0385517a/attachment-0001.sig>

From skip.montanaro at gmail.com  Sat Feb 25 10:52:15 2023
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sat, 25 Feb 2023 09:52:15 -0600
Subject: Is there a more efficient threading lock?
Message-ID: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>

I have a multi-threaded program which calls out to a non-thread-safe
library (not mine) in a couple places. I guard against multiple
threads executing code there using threading.Lock. The code is
straightforward:

from threading import Lock

# Something in textblob and/or nltk doesn't play nice with no-gil, so just
# serialize all blobby accesses.
BLOB_LOCK = Lock()

def get_terms(text):
    with BLOB_LOCK:
        phrases = TextBlob(text, np_extractor=EXTRACTOR).noun_phrases
    for phrase in phrases:
        yield phrase

When I monitor the application using py-spy, that with statement is
consuming huge amounts of CPU. Does threading.Lock.acquire() sleep
anywhere? I didn't see anything obvious poking around in the C code
which implements this stuff. I'm no expert though, so could easily
have missed something.

Thx,

Skip

From hjp-python at hjp.at  Sat Feb 25 11:27:51 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 25 Feb 2023 17:27:51 +0100
Subject: Is there a more efficient threading lock?
In-Reply-To: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
Message-ID: <20230225162751.jowhqsvyfpo5wh5u@hjp.at>

On 2023-02-25 09:52:15 -0600, Skip Montanaro wrote:
> I have a multi-threaded program which calls out to a non-thread-safe
> library (not mine) in a couple places. I guard against multiple
> threads executing code there using threading.Lock. The code is
> straightforward:
> 
> from threading import Lock
> 
> # Something in textblob and/or nltk doesn't play nice with no-gil, so just
> # serialize all blobby accesses.
> BLOB_LOCK = Lock()
> 
> def get_terms(text):
>     with BLOB_LOCK:
>         phrases = TextBlob(text, np_extractor=EXTRACTOR).noun_phrases
>     for phrase in phrases:
>         yield phrase
> 
> When I monitor the application using py-spy, that with statement is
> consuming huge amounts of CPU.

Which OS is this?

> Does threading.Lock.acquire() sleep anywhere?

On Linux it calls futex(2), which does sleep if it can't get the lock
right away. (Of course if it does get the lock, it will return
immediately which may use a lot of CPU if you are calling it a lot.)

        hp


-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230225/5e388756/attachment.sig>

From list1 at tompassin.net  Sat Feb 25 11:48:37 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 25 Feb 2023 11:48:37 -0500
Subject: Is there a more efficient threading lock?
In-Reply-To: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
Message-ID: <5fec3cb2-52a2-758b-fda9-bcd6608d5681@tompassin.net>

On 2/25/2023 10:52 AM, Skip Montanaro wrote:
> I have a multi-threaded program which calls out to a non-thread-safe
> library (not mine) in a couple places. I guard against multiple
> threads executing code there using threading.Lock. The code is
> straightforward:
> 
> from threading import Lock
> 
> # Something in textblob and/or nltk doesn't play nice with no-gil, so just
> # serialize all blobby accesses.
> BLOB_LOCK = Lock()
> 
> def get_terms(text):
>      with BLOB_LOCK:
>          phrases = TextBlob(text, np_extractor=EXTRACTOR).noun_phrases
>      for phrase in phrases:
>          yield phrase
> 
> When I monitor the application using py-spy, that with statement is
> consuming huge amounts of CPU. Does threading.Lock.acquire() sleep
> anywhere? I didn't see anything obvious poking around in the C code
> which implements this stuff. I'm no expert though, so could easily
> have missed something.

I'm no expert on locks, but you don't usually want to keep a lock while 
some long-running computation goes on.  You want the computation to be 
done by a separate thread, put its results somewhere, and then notify 
the choreographing thread that the result is ready.

This link may be helpful -

https://anandology.com/blog/using-iterators-and-generators/


From hjp-python at hjp.at  Sat Feb 25 16:53:22 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 25 Feb 2023 22:53:22 +0100
Subject: Is there a more efficient threading lock?
In-Reply-To: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
Message-ID: <20230225215322.h5ap5trf2xhu6zkx@hjp.at>

On 2023-02-25 09:52:15 -0600, Skip Montanaro wrote:
> BLOB_LOCK = Lock()
> 
> def get_terms(text):
>     with BLOB_LOCK:
>         phrases = TextBlob(text, np_extractor=EXTRACTOR).noun_phrases
>     for phrase in phrases:
>         yield phrase
> 
> When I monitor the application using py-spy, that with statement is
> consuming huge amounts of CPU.

Another thought:

How accurate is py-spy? Is it possible that it assigns time actually
spent in 
    phrases = TextBlob(text, np_extractor=EXTRACTOR).noun_phrases
to
    with BLOB_LOCK:
?

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230225/c9a445f4/attachment.sig>

From skip.montanaro at gmail.com  Sat Feb 25 16:41:52 2023
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sat, 25 Feb 2023 15:41:52 -0600
Subject: Is there a more efficient threading lock?
In-Reply-To: <5fec3cb2-52a2-758b-fda9-bcd6608d5681@tompassin.net>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <5fec3cb2-52a2-758b-fda9-bcd6608d5681@tompassin.net>
Message-ID: <CANc-5Ux9fg6v0Gjtqi=g=MdB-OJtOeM3u6xPS0c3nyPKp88Acg@mail.gmail.com>

Thanks for the responses.

Peter wrote:

> Which OS is this?

MacOS Ventura 13.1, M1 MacBook Pro (eight cores).

Thomas wrote:

> I'm no expert on locks, but you don't usually want to keep a lock while
> some long-running computation goes on.  You want the computation to be
> done by a separate thread, put its results somewhere, and then notify
> the choreographing thread that the result is ready.

In this case I'm extracting the noun phrases from the body of an email
message (returned as a list). I have a collection of email messages
organized by month (typically 1000 to 3000 messages per month). I'm using
concurrent.futures.ThreadPoolExecutor() with the default number of workers (
os.cpu_count() * 1.5, or 12 threads on my system) to process each month, so
12 active threads at a time. Given that the process is pretty much CPU
bound, maybe reducing the number of workers to the CPU count would make
sense. Processing of each email message enters that with block once. That's
about as minimal as I can make it. I thought for a bit about pushing the
textblob stuff into a separate worker thread, but it wasn't obvious how to
set up queues to handle the communication between the threads created by
ThreadPoolExecutor() and the worker thread. Maybe I'll think about it
harder. (I have a related problem with SQLite, since an open database can't
be manipulated from multiple threads. That makes much of the program's
end-of-run processing single-threaded.)

> This link may be helpful -
>
> https://anandology.com/blog/using-iterators-and-generators/

I don't think that's where my problem is. The lock protects the generation
of the noun phrases. My loop which does the yielding operates outside of
that lock's control. The version of the code is my latest, in which I
tossed out a bunch of phrase-processing code (effectively dead end ideas
for processing the phrases). Replacing the for loop with a simple return
seems not to have any effect. In any case, the caller which uses the
phrases does a fair amount of extra work with the phrases, populating a
SQLite database, so I don't think the amount of time it takes to process a
single email message is dominated by the phrase generation.

Here's timeit output for the noun_phrases code:

% python -m timeit -s 'text = """`python -m timeit --help`""" ; from
textblob import TextBlob ; from textblob.np_extractors import
ConllExtractor ; ext = ConllExtractor() ; phrases = TextBlob(text,
np_extractor=ext).noun_phrases' 'phrases = TextBlob(text,
np_extractor=ext).noun_phrases'
5000 loops, best of 5: 98.7 usec per loop

I process the output of timeit's help message which looks to be about the
same length as a typical email message, certainly the same order of
magnitude. Also, note that I call it once in the setup to eliminate the
initial training of the ConllExtractor instance. I don't know if ~100us
qualifies as long running or not.

I'll keep messing with it.

Skip

From gweatherby at uchc.edu  Sat Feb 25 16:47:00 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Sat, 25 Feb 2023 21:47:00 +0000
Subject: Is there a more efficient threading lock?
In-Reply-To: <5fec3cb2-52a2-758b-fda9-bcd6608d5681@tompassin.net>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <5fec3cb2-52a2-758b-fda9-bcd6608d5681@tompassin.net>
Message-ID: <SJ0PR14MB5864580AD1A6C6C47442808CB9A99@SJ0PR14MB5864.namprd14.prod.outlook.com>


?I'm no expert on locks, but you don't usually want to keep a lock while
some long-running computation goes on.  You want the computation to be
done by a separate thread, put its results somewhere, and then notify
the choreographing thread that the result is ready.?

Maybe. There are so many possible threaded application designs I?d hesitate to make a general statement.

The threading.Lock.acquire method has flags for both a non-blocking attempt and a timeout, so a valid design could include a long-running computation with a main thread or event loop polling the thread. Or the thread could signal a main loop some other way.

I?ve written some code that coordinated threads by having a process talk to itself using a socket.socketpair. The advantage is that you can bundle multiple items (sockets, file handles, a polling timeout) into a select.select call which waits without consuming resources (at least on Linux) until
something interesting happens.



From gweatherby at uchc.edu  Sat Feb 25 16:58:18 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Sat, 25 Feb 2023 21:58:18 +0000
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <20230225142031.fecczvepuu24eygb@hjp.at>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
 <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
 <c995a27d-ce40-c639-e589-55a8367a6f46@tompassin.net>
 <20230225061307.slgufll32u24ex6p@hjp.at>
 <deeda902-7798-09ed-4ead-c209c5ccd139@tompassin.net>
 <20230225142031.fecczvepuu24eygb@hjp.at>
Message-ID: <SJ0PR14MB5864A06F01E55EE5453F16F4B9A99@SJ0PR14MB5864.namprd14.prod.outlook.com>

I only use asserts for things I know to be true. Nothing is harder to debug than when something you know to be true turns out to be? not True. Because I?ll check everything else instead of the cause of the bug.

In other words, a failing assert means I have a hole in my program logic.

For that use, the default behavior ?telling me which line the assert is on, is more than sufficient. Depending on the circumstance, I?ll re-run the code with a breakpoint or replace the assert with an informative f-string Exception.



From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Peter J. Holzer <hjp-python at hjp.at>
Date: Saturday, February 25, 2023 at 9:22 AM
To: python-list at python.org <python-list at python.org>
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-25 09:10:06 -0500, Thomas Passin wrote:
> On 2/25/2023 1:13 AM, Peter J. Holzer wrote:
> > On 2023-02-24 18:19:52 -0500, Thomas Passin wrote:
> > > Sometimes you can use a second parameter to assert if you know what kind of
> > > error to expect:
[...]
> > > With type errors, assert may actually give you the information needed:
> > >
> > > > > > c = {"a": a, "b": 2}
> > > > > > assert a > c
> > > Traceback (most recent call last):
> > >    File "<stdin>", line 1, in <module>
> > > TypeError: '>' not supported between instances of 'list' and 'dict'
> >
> > Actually in this case it isn't assert which gives you the information,
> > it's evaluating the expression itself. You get the same error with just
> >      a > c
> > on a line by its own.
>
> In some cases.  For my example with an explanatory string, you wouldn't want
> to write code like that after an ordinary line of code, at least not very
> often.  The assert statement allows it syntactically.

Yes, but if an error in the expression triggers an exception (as in this
case) the explanatory string will never be displayed.

        hp

--
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

From hjp-python at hjp.at  Sat Feb 25 17:19:43 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 25 Feb 2023 23:19:43 +0100
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <SJ0PR14MB5864A06F01E55EE5453F16F4B9A99@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
 <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
 <c995a27d-ce40-c639-e589-55a8367a6f46@tompassin.net>
 <20230225061307.slgufll32u24ex6p@hjp.at>
 <deeda902-7798-09ed-4ead-c209c5ccd139@tompassin.net>
 <20230225142031.fecczvepuu24eygb@hjp.at>
 <SJ0PR14MB5864A06F01E55EE5453F16F4B9A99@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <20230225221943.m5tkc3mr5ze3dupg@hjp.at>

On 2023-02-25 21:58:18 +0000, Weatherby,Gerard wrote:
> I only use asserts for things I know to be true.

Yeah, that's what assers are for. Or rather for things that you *think*
are true.

> In other words, a failing assert means I have a hole in my program
> logic.

Yes, if you include your assumptions in your definition of "logic".


> For that use, the default behavior ?telling me which line the assert
> is on, is more than sufficient. Depending on the circumstance, I?ll
> re-run the code with a breakpoint or replace the assert with an
> informative f-string Exception.

That may not always be practical. Things that we know (or think) are
true often have *are* true in most cases (otherwise we wouldn't think
so). So the case where the assumption fails may not be easily
reproducable and the more information you can get post-mortem the
better. For example, in C on Linux a failed assertion causes a core
dump. So you can inspect the complete state of the program.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230225/52084f98/attachment.sig>

From list1 at tompassin.net  Sat Feb 25 17:20:20 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sat, 25 Feb 2023 17:20:20 -0500
Subject: Is there a more efficient threading lock?
In-Reply-To: <CANc-5Ux9fg6v0Gjtqi=g=MdB-OJtOeM3u6xPS0c3nyPKp88Acg@mail.gmail.com>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <5fec3cb2-52a2-758b-fda9-bcd6608d5681@tompassin.net>
 <CANc-5Ux9fg6v0Gjtqi=g=MdB-OJtOeM3u6xPS0c3nyPKp88Acg@mail.gmail.com>
Message-ID: <b55da204-36b9-b05b-f478-f44e8d2576da@tompassin.net>

On 2/25/2023 4:41 PM, Skip Montanaro wrote:
> Thanks for the responses.
> 
> Peter wrote:
> 
>> Which OS is this?
> 
> MacOS Ventura 13.1, M1 MacBook Pro (eight cores).
> 
> Thomas wrote:
> 
>  > I'm no expert on locks, but you don't usually want to keep a lock while
>  > some long-running computation goes on.? You want the computation to be
>  > done by a separate thread, put its results somewhere, and then notify
>  > the choreographing thread that the result is ready.
> 
> In this case I'm extracting the noun phrases from the body of an email 
> message(returned as a list). I have a collection of email messages 
> organized by month(typically 1000 to 3000 messages per month). I'm using 
> concurrent.futures.ThreadPoolExecutor() with the default number of 
> workers (os.cpu_count() * 1.5, or 12 threads on my system)to process 
> each month, so 12 active threads at a time. Given that the process is 
> pretty much CPU bound, maybe reducing the number of workers to the CPU 
> count would make sense. Processing of each email message enters that 
> with block once.That's about as minimal as I can make it. I thought for 
> a bit about pushing the textblob stuff into a separate worker thread, 
> but it wasn't obvious how to set up queues to handle the communication 
> between the threads created by ThreadPoolExecutor()and the worker 
> thread. Maybe I'll think about it harder. (I have a related problem with 
> SQLite, since an open database can't be manipulated from multiple 
> threads. That makes much of the program's end-of-run processing 
> single-threaded.)

If the noun extractor is single-threaded (which I think you mentioned), 
no amount of parallel access is going to help.  The best you can do is 
to queue up requests so that as soon as the noun extractor returns from 
one call, it gets handed another blob.  The CPU will be busy all the 
time running the noun-extraction code.

If that's the case, you might just as well eliminate all the threads and 
just do it sequentially in the most obvious and simple manner.

It would possibly be worth while to try this approach out and see what 
happens to the CPU usage and overall computation time.

>  > This link may be helpful -
>  >
>  > https://anandology.com/blog/using-iterators-and-generators/ 
> <https://anandology.com/blog/using-iterators-and-generators/>
> 
> I don't think that's where my problem is. The lock protects the 
> generation of the noun phrases. My loop which does the yielding operates 
> outside of that lock's control. The version of the code is my latest, in 
> which I tossed out a bunch of phrase-processing code (effectively dead 
> end ideas for processing the phrases). Replacing the for loop with a 
> simple return seems not to have any effect. In any case, the caller 
> which uses the phrases does a fair amount of extra work with the 
> phrases, populating a SQLite database, so I don't think the amount of 
> time it takes to process a single email message is dominated by the 
> phrase generation.
> 
> Here's timeitoutput for the noun_phrases code:
> 
> % python -m timeit -s 'text = """`python -m timeit --help`""" ; from 
> textblob import TextBlob ; from textblob.np_extractors import 
> ConllExtractor ; ext = ConllExtractor() ; phrases = TextBlob(text, 
> np_extractor=ext).noun_phrases' 'phrases = TextBlob(text, 
> np_extractor=ext).noun_phrases'
> 5000 loops, best of 5: 98.7 usec per loop
> 
> I process the output of timeit's?help message which looks to be about 
> the same length as a typical email message, certainly the same order of 
> magnitude. Also, note that I call it once in the setup to eliminate the 
> initial training of the ConllExtractor?instance. I don't know if ~100us 
> qualifies as long running or not.
> 
> I'll keep messing with it.
> 
> Skip


From barry at barrys-emacs.org  Sat Feb 25 17:48:47 2023
From: barry at barrys-emacs.org (Barry)
Date: Sat, 25 Feb 2023 22:48:47 +0000
Subject: Is there a more efficient threading lock?
In-Reply-To: <b55da204-36b9-b05b-f478-f44e8d2576da@tompassin.net>
References: <b55da204-36b9-b05b-f478-f44e8d2576da@tompassin.net>
Message-ID: <24872A80-C54D-44DE-B8EC-A286D87908C5@barrys-emacs.org>

Re sqlite and threads. The C API can be compiled to be thread safe from my
Reading if the sqlite docs. What I have not checked is how python?s bundled sqlite
is compiled. There are claims python?s sqlite is not thread safe.

Barry



From greg.ewing at canterbury.ac.nz  Sat Feb 25 19:54:16 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 26 Feb 2023 13:54:16 +1300
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <87edqds96y.fsf@nightsong.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com>
Message-ID: <k5votpFpjveU1@mid.individual.net>

On 26/02/23 10:53 am, Paul Rubin wrote:
> I'm not on either list but the purpose of the tutor list is to shunt
> beginner questions away from the main list.

There's a fundamental problem with tutor lists. They rely on
experienced people, the ones capable of answering the questions,
to go out of their way to read the tutor list -- something that
is not of personal benefit to them.

Also, pointing people towards tutor lists, if not done carefully,
can give the impression of saying "newcomers are not welcome here".
That's not a message we want to send to Python newcomers at all.

-- 
Greg

From gweatherby at uchc.edu  Sat Feb 25 21:38:08 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Sun, 26 Feb 2023 02:38:08 +0000
Subject: Why doesn't Python (error msg) tell me WHAT the actual (arg)
 values are ?
In-Reply-To: <20230225221943.m5tkc3mr5ze3dupg@hjp.at>
References: <a016e5d0-06ad-419f-a3f0-81c7a9eedd71n@googlegroups.com>
 <a17001d2-ec07-b640-b611-95afea129150@DancesWithMice.info>
 <20230224191244.kn6quzr4x5muthln@hjp.at>
 <ad1af26c-d2df-1196-ce63-10d924dcba39@DancesWithMice.info>
 <c995a27d-ce40-c639-e589-55a8367a6f46@tompassin.net>
 <20230225061307.slgufll32u24ex6p@hjp.at>
 <deeda902-7798-09ed-4ead-c209c5ccd139@tompassin.net>
 <20230225142031.fecczvepuu24eygb@hjp.at>
 <SJ0PR14MB5864A06F01E55EE5453F16F4B9A99@SJ0PR14MB5864.namprd14.prod.outlook.com>
 <20230225221943.m5tkc3mr5ze3dupg@hjp.at>
Message-ID: <SJ0PR14MB5864C2CB8ACE6DB024BBBFBCB9AE9@SJ0PR14MB5864.namprd14.prod.outlook.com>

?So the case where the assumption fails may not be easily
reproducable and the more information you can get post-mortem the
better?

That?s true for rare corner cases or esoteric race conditions. Usually, when I see asserts it's just because I was just plain stupid.

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Peter J. Holzer <hjp-python at hjp.at>
Date: Saturday, February 25, 2023 at 5:21 PM
To: python-list at python.org <python-list at python.org>
Subject: Re: Why doesn't Python (error msg) tell me WHAT the actual (arg) values are ?
On 2023-02-25 21:58:18 +0000, Weatherby,Gerard wrote:
> I only use asserts for things I know to be true.

Yeah, that's what assers are for. Or rather for things that you *think*
are true.

> In other words, a failing assert means I have a hole in my program
> logic.

Yes, if you include your assumptions in your definition of "logic".


> For that use, the default behavior ?telling me which line the assert
> is on, is more than sufficient. Depending on the circumstance, I?ll
> re-run the code with a breakpoint or replace the assert with an
> informative f-string Exception.

That may not always be practical. Things that we know (or think) are
true often have *are* true in most cases (otherwise we wouldn't think
so). So the case where the assumption fails may not be easily
reproducable and the more information you can get post-mortem the
better. For example, in C on Linux a failed assertion causes a core
dump. So you can inspect the complete state of the program.

        hp

--
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

From gweatherby at uchc.edu  Sat Feb 25 21:44:28 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Sun, 26 Feb 2023 02:44:28 +0000
Subject: Programming by contract.
Message-ID: <SJ0PR14MB58646CF12A021158050E4C6DB9A99@SJ0PR14MB5864.namprd14.prod.outlook.com>

The discussion of asserts got me thinking about Programming by Contract. Back in the 90s, I had occasion to learn Eiffel programming language. ( https://en.wikipedia.org/wiki/Eiffel_(programming_language) The concepts are intriguing, although Eiffel itself had barriers to widespread adoption.

Reviewing the topic, I found Wikipedia mentioned some Python implementations. I kicked the tires of ?deal? and found it straightforward. I?m wondering if anyone on the list has experience using any of the Programming by Contract modules?

#!/usr/bin/env python3
import argparse
import deal

@deal.pre(lambda x: x > 7)
@deal.pre(lambda x: isinstance(x, int))
def func(x):
    print(x)


parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('--nodeal', action='store_true', help="Disable deal")
parser.add_argument('bad_choice', choices=('type', 'value'))
args = parser.parse_args()
if args.nodeal:
    deal.disable()
func(8)
if args.bad_choice == 'type':
    func("8")
if args.bad_choice == 'value':
    func(1)

From avi.e.gross at gmail.com  Sat Feb 25 22:52:51 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sat, 25 Feb 2023 22:52:51 -0500
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <k5votpFpjveU1@mid.individual.net>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
Message-ID: <007001d94995$cd082c80$67188580$@gmail.com>

Greg,

Yes, the forum should be open. The first requests from the person were
replied to politely.

At some point a pattern was emerging of lots of fairly irreverent posts by
someone who is having trouble shifting programming paradigms. The suggestion
was then made as a SUGGESTION by several people that "some" of their
questions might be better asked on the tutor list where others new to python
may have similar questions and can learn.

This forum has all kinds of people and of course many topics are of more
interest to some that others. Programming styles differ too and I note some
here reacted to a suggestion that maybe constants could be more efficiently
be initiated in ways that use less resources. Some insisted it makes more
sense to be able to type what you want more compactly. Yes, of course,
multiple ways are equally valid especially as now, efficiency is not seen as
a major goal.

The reality is that several mailing lists are intended to be used for
occasional questions and people who have more serious needs should be using
local resources or taking courses and reading books as their main learning
method. An occasional question is welcomed. A barrage is an position and a
barrage where most of the answers are ignored or claimed to be wrong, can
generate an "attitude" some of us find less than appealing.

I continue to believe that a programmers job is to learn how to use a
language well, or switch languages, and not to keep moaning why it does not
do what you want or expect. Many answers have suggested how the OP can solve
some issues and apparently that is not of interest to them and they just
keep complaining.

I speak for nobody except myself. As I have said, I have chosen to not
respond and become frustrated.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Greg Ewing via Python-list
Sent: Saturday, February 25, 2023 7:54 PM
To: python-list at python.org
Subject: Re: TypeError: can only concatenate str (not "int") to str

On 26/02/23 10:53 am, Paul Rubin wrote:
> I'm not on either list but the purpose of the tutor list is to shunt 
> beginner questions away from the main list.

There's a fundamental problem with tutor lists. They rely on experienced
people, the ones capable of answering the questions, to go out of their way
to read the tutor list -- something that is not of personal benefit to them.

Also, pointing people towards tutor lists, if not done carefully, can give
the impression of saying "newcomers are not welcome here".
That's not a message we want to send to Python newcomers at all.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list


From cs at cskk.id.au  Sat Feb 25 22:57:58 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Sun, 26 Feb 2023 14:57:58 +1100
Subject: Programming by contract.
In-Reply-To: <SJ0PR14MB58646CF12A021158050E4C6DB9A99@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <SJ0PR14MB58646CF12A021158050E4C6DB9A99@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <Y/rYxrZbLp6d/R7R@cskk.homeip.net>

On 26Feb2023 02:44, Weatherby,Gerard <gweatherby at uchc.edu> wrote:
>The discussion of asserts got me thinking about Programming by Contract. Back in the 90s, I had occasion to learn Eiffel programming language. ( https://en.wikipedia.org/wiki/Eiffel_(programming_language) The concepts are intriguing, although Eiffel itself had barriers to widespread adoption.
>
>Reviewing the topic, I found Wikipedia mentioned some Python implementations. I kicked the tires of ?deal? and found it straightforward. I?m wondering if anyone on the list has experience using any of the Programming by Contract modules?

I've been using the icontract package happily.

     from icontract import require, ensure

     @typechecked
     @require(
         lambda name: is_valid_basename(name),  # pylint: disable=unnecessary-lambda
         "name should be a clean file basename"
     )
     def TagSetClass(self, name: str) -> TaggedPath:
       ''' factory to create a `TaggedPath` from a `name`.
       '''
       fspath = joinpath(dirname(self.fspath), name)
       return TaggedPath(fspath, fstags=self.fstags)

     @ensure(lambda result: result == normpath(result))
     def keypath(self, fspath):
       ''' Compute the absolute path used to index a `TaggedPath` instance.

           This returns `realpath(fspath)` if `self.config.physical`,
           otherwise `abspath(fspath)`.
       '''
       return realpath(fspath) if self.config.physical else abspath(fspath)

You can stack the decorators just like deal.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From henhanna at gmail.com  Fri Feb 24 19:34:02 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Fri, 24 Feb 2023 16:34:02 -0800 (PST)
Subject: unpack, pack
Message-ID: <49b602dc-5d5b-45d7-b38e-d0d67ddebfe2n@googlegroups.com>

when i unpack  the string (or Seed)    USENEARSEYNE   , 
                 i get    words like           Sense,  Sears, ........, Snare, Unary, ....
 (sarsen, seesee, senary, serene, unease, uneasy, usneae, usneas)
 (sensyne)



How would i approach  the PACKING  function ?

e.g. ...   i want to find the string  X  that packs
                Apple,  Pie,  Apricot,  Banana,  Candy

                  (packing doesn't require an English dictionary)

and i want X to be short.              Thanks!

                              (is this in Vol.3 of Knuth?)

From henhanna at gmail.com  Sat Feb 25 16:12:47 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sat, 25 Feb 2023 13:12:47 -0800 (PST)
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <mailman.1989.1677296483.20444.python-list@python.org>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
Message-ID: <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>


> PS are you aware that there is a Python-Tutor list for the use of people learning Python?


is this guy (dn)  always this rude???        is he even more rude  on this Python-Tutor list ?

he must have a reputation (for being rude)...  


On Friday, February 24, 2023 at 7:41:44?PM UTC-8, dn wrote:
> On 25/02/2023 09.36, Hen Hanna wrote: 
> > TypeError: can only concatenate str (not "int") to str 
> > 
> > thanks for the comments, --- esp. 2 or 3 (?) ppl who directly addressed it or commented on it.
> If you haven't already, please review the Python Software Foundation's 
> Code of Conduct https://www.python.org/psf/conduct/
> > py and pypy don't tell me what the (offending) int is 
> > 
> > (even tho' it'd be easy to show me the int, or number) 
> > 
> > because a Tuple, List... may produce a long output and they don't want to deal with ways of abbreviating it (with ....... ) 
> > 
> > and because User-defined Class object may cause another error while printing it. 
> > 
> > 
> > 
> > _________________________________________ 
> > TypeError: can only join an iterable 
> > 
> > Here too it'd help me if pypy (Jeannie) could tell me what the offending data object is.
> Observation: 
> - python doesn't tell you what you want 
> - correspondents are not telling you what you want and/or how you want 
> There's a common theme developing... 
> 
> Suggestion: if you want help, and at the frequency with which you've 
> been posting recently, would it be better not to sound like a 'troll'? 
> 
> 
> Please understand everyone here is volunteering time, in a bid to 
> provide you (and each-other) with benefit. 
> 
> Does the FACT that you have failed to answer several questions from 
> colleagues really entitle you to demand others only directly-address you 
> (and only your questions)? 
> 
> -- 
> Regards, 
> =dn

From jon+usenet at unequivocal.eu  Sat Feb 25 16:24:19 2023
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Sat, 25 Feb 2023 21:24:19 -0000 (UTC)
Subject: Is there a more efficient threading lock?
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
Message-ID: <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>

On 2023-02-25, Paul Rubin <no.email at nospam.invalid> wrote:
> Skip Montanaro <skip.montanaro at gmail.com> writes:
>> from threading import Lock
>
> 1) you generally want to use RLock rather than Lock

Why?

> 2) I have generally felt that using locks at the app level at all is an
> antipattern.  The main way I've stayed sane in multi-threaded Python
> code is to have every mutable strictly owned by exactly one thread, pass
> values around using Queues, and have an event loop in each thread taking
> requests from Queues.
>
> 3) I didn't know that no-gil was a now thing and I'm used to having the
> GIL.  So I would have considered the multiprocessing module rather than
> threading, for something like this.

What does this mean? Are you saying the GIL has been removed?

From henhanna at gmail.com  Sat Feb 25 17:41:02 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sat, 25 Feb 2023 14:41:02 -0800 (PST)
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <87edqds96y.fsf@nightsong.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com>
Message-ID: <3f0c23e6-5dae-40a0-89d6-d4870cb84b00n@googlegroups.com>

On Saturday, February 25, 2023 at 1:54:14?PM UTC-8,     Paul Rubin    wrote:
> Hen Hanna <henh... at gmail.com> writes: 
> > is this guy (dn) always this rude??? is he even more rude on this 
> > Python-Tutor list ?

> I'm not on either list but the purpose of the tutor list is to shunt 
> beginner questions away from the main list. Yes your questions do seem 
> excessively frequent even here on Usenet. I have mostly been ignoring 
> them after seeing the first few.


not sure...  what you mean by "either" list....

this rude guy ('dn'  who writes poorly) is someone i want to see posting less (and fewer),

and (from what i've seen) ,   Paul Rubin  is someone i wish to see posting more.


From jon+usenet at unequivocal.eu  Sat Feb 25 18:45:36 2023
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Sat, 25 Feb 2023 23:45:36 -0000 (UTC)
Subject: Is there a more efficient threading lock?
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
Message-ID: <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>

On 2023-02-25, Paul Rubin <no.email at nospam.invalid> wrote:
> Jon Ribbens <jon+usenet at unequivocal.eu> writes:
>>> 1) you generally want to use RLock rather than Lock
>> Why?
>
> So that a thread that tries to acquire it twice doesn't block itself,
> etc.  Look at the threading lib docs for more info.

Yes, I know what the docs say, I was asking why you were making the
statement above. I haven't used Lock very often, but I've literally
never once in 25 years needed to use RLock. As you say, it's best
to keep the lock-protected code brief, so it's usually pretty
obvious that the code can't be re-entered.

>> What does this mean? Are you saying the GIL has been removed?
>
> Last I heard there was an experimental version of CPython with the GIL
> removed.  It is supposed to take less of a performance hit due to
> INCREF/DECREF than an earlier attempt some years back.  I don't know its
> current status.
>
> The GIL is an evil thing, but it has been around for so long that most
> of us have gotten used to it, and some user code actually relies on it.
> For example, with the GIL in place, a statement like "x += 1" is always
> atomic, I believe.  But, I think it is better to not have any shared
> mutables regardless.

I think it is the case that x += 1 is atomic but foo.x += 1 is not.
Any replacement for the GIL would have to keep the former at least,
plus the fact that you can do hundreds of things like list.append(foo)
which are all effectively atomic.

From henhanna at gmail.com  Sat Feb 25 20:12:07 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sat, 25 Feb 2023 17:12:07 -0800 (PST)
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <k5votpFpjveU1@mid.individual.net>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
Message-ID: <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>

On Saturday, February 25, 2023 at 4:54:36?PM UTC-8, Greg Ewing wrote:
> On 26/02/23 10:53 am, Paul Rubin wrote: 
> > I'm not on either list but the purpose of the tutor list is to shunt 
> >              beginner questions away from the main list.


> There's a fundamental problem with tutor lists. They rely on 
> experienced people, the ones capable of answering the questions, 
> to go out of their way to read the tutor list -- something that 
> is not of personal benefit to them. 
> 
> Also, pointing people towards tutor lists, if not done carefully, 
> can give the impression of saying "newcomers are not welcome here". 
> That's not a message we want to send to Python newcomers at all. 
> 
> -- 
> Greg



1.   i still think that  my original question  about   (not "int")   is  a good one.    
                  it may  seem like a  stupid      [beginner question] ,     but it really isn't.


2. the rude guy ('dn') hasn't  offered  a single word of comment   that's directly relevant to it.
               --------  but he did  offer related stuff  which he thinks i should be  [grateful] for


3.   it's pretty clear  that  the rude guy ('dn')  intended-intends  to insult me as a Newbie... that much is certain.



___________________________
 It'd be easy for the Error-Msg to include   the   VALUE   of the int.

                A   LISP   programmer knows the value of everything,
                                          but the cost of nothing.



>>>   TypeError: can only join an iterable

                      Here she's not even telling me the TYPE of the offending object.

From henhanna at gmail.com  Sat Feb 25 21:25:33 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sat, 25 Feb 2023 18:25:33 -0800 (PST)
Subject: 'dn' saying "Welcome (to the list)"
Message-ID: <cecd7d72-0b4d-4e01-9f94-b54bdc4bf007n@googlegroups.com>


On Thursday, March 4, 2021  dn wrote:
> Hi, and welcome to the list.


note that this is the very same rude guy ('dn')   who is apparently the rudest of them all.

note that this is the very same rude guy ('dn')   who wants to shunt me away to the   TUTOR   list.

          -----------  i guess i'd consider being TUTOR'ed by 'dn' (or someone like it)
                               if   i  can  be  paid    $1000 / hour  for  it.



Have others noticed this too?  (it's often the guy who says "Welcome" that....)


Whenever someone  says  "Welcome"  to me (in this way),  my alarm goes off.


--------- a good way to explain it (the Phenomenon)  is...   it's an indication that he's itching to treat me as a Newbie...  that he 's  salivating because he senses  an opportunity to   MANsplain   and   WHITEsplain   things to me.

From wlfraed at ix.netcom.com  Sat Feb 25 22:10:48 2023
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sat, 25 Feb 2023 22:10:48 -0500
Subject: Is there a more efficient threading lock?
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <5fec3cb2-52a2-758b-fda9-bcd6608d5681@tompassin.net>
 <CANc-5Ux9fg6v0Gjtqi=g=MdB-OJtOeM3u6xPS0c3nyPKp88Acg@mail.gmail.com>
 <mailman.1997.1677362017.20444.python-list@python.org>
Message-ID: <a3jlvhhm479ltklcp8u42t6qffmltokba9@4ax.com>

On Sat, 25 Feb 2023 15:41:52 -0600, Skip Montanaro
<skip.montanaro at gmail.com> declaimed the following:


>concurrent.futures.ThreadPoolExecutor() with the default number of workers (
>os.cpu_count() * 1.5, or 12 threads on my system) to process each month, so
>12 active threads at a time. Given that the process is pretty much CPU
>bound, maybe reducing the number of workers to the CPU count would make

	Unless things have improved a lot over the years, the GIL still limits
active threads to the equivalent of a single CPU. The OS may swap among
which CPU as it schedules system processes, but only one thread will be
running at any moment regardless of CPU count.

	Common wisdom is that Python threading works well for I/O bound
systems, where each thread spends most of its time waiting for some I/O
operation to complete -- thereby allowing the OS to schedule other threads.

	For CPU bound, use of the multiprocessing package may be more suited --
though you'll have to device a working IPC system transfer data to/from the
separate processes (no shared objects as possible with threads).


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From henhanna at gmail.com  Sat Feb 25 23:25:06 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sat, 25 Feb 2023 20:25:06 -0800 (PST)
Subject: creepy email...  Subj: Just to you.
Message-ID: <6431527d-0242-4ddb-8386-f4f04a874a51n@googlegroups.com>

           i received this    creepy email...  Subj: Just to you.



>>>    Just to you as I see little further reason to help you ...........

>>>    You have not taken hints that bashing the language as not doing what you expect is not the way to get help so I leave you to others.


i'll check (!)  if he actually said anything helpful or informative !

From s20201060 at htsps.edu.hk  Sat Feb 25 06:43:45 2023
From: s20201060 at htsps.edu.hk (outlook_F8D50C5B9E059743@outlook.com)
Date: Sat, 25 Feb 2023 19:43:45 +0800
Subject: =?utf-8?Q?=E5=9B=9E=E5=A4=8D:_Question_about_logging.config.dictConfig?=
In-Reply-To: <CAE9rwzO9e0i9XqKHiew25Z7tqBUL0gZ1oUXsz_Xh+mtgzzw_gw@mail.gmail.com>
References: <CAE9rwzMOXCEhsiqDnoGbzhLn1iM3P=PPsda9HrEgX2z_nqYTmA@mail.gmail.com>
 <20230208003156.mfhoozbkr2rv2dde@hjp.at>,
 <CAE9rwzO9e0i9XqKHiew25Z7tqBUL0gZ1oUXsz_Xh+mtgzzw_gw@mail.gmail.com>
Message-ID: <D3A31FEB-3D31-46B6-A192-FBC4B62CC464@hxcore.ol>

   ?

   ?

   ? Windows ?[1]????

   ?

   ???: [2]Ivan "Rambius" Ivanov
   ????: 2023?2?8? 23:29
   ???: [3]python-list at python.org
   ??: Re: Question about logging.config.dictConfig

   ?

   On Tue, Feb 7, 2023 at 7:35 PM Peter J. Holzer <hjp-python at hjp.at> wrote:

   >?

   > On 2023-02-07 17:58:26 -0500, Ivan "Rambius" Ivanov wrote:

   > > I am trying to configure my loggers using dictConfig, but they do not

   > > print anything. Here are more details.

   > [...]

   > > from myloggingconf import configure_logging

   > >

   > > logger = logging.getLogger(os.path.basename(__file__))

   > [...]

   > >

   > > def main():

   > >???? configure_logging()

   > >???? dosmth()

   > [...]

   > > def configure_logging():

   > >???? config = {

   > >???????? 'version': 1,

   > >???????? 'disable_existing_loggers': True,

   >?

   >?????????? 'disable_existing_loggers': False,

   ?

   Thank you! That helped a lot!

   ?

   Regards

   rambius

   ?

   >?

   > I think I just found out why I've been cargo-culting this line since my

   > early Django days ;-)-

   >?

   > If you call getLogger very early (as you do), you don't want it disabled

   > by a later call to dictConfig().

   >?

   >?

   > >???????? 'formatters': {

   > >???????????? 'default': {

   >   >????????????????   'fmt':   '%(asctime)s   %(levelname)-7s   %(name)s
   %(funcName)s %(message)s'

   >?????????????????? 'format'

   > >???????????? }

   > >???????? },

   > [...]

   > >???? }

   > >???? logging.config.dictConfig(config)

   > >

   > > When I run uselog.py it prints nothing. I am wondering what is wrong

   > > with the second configuration.

   >?

   > See above.

   >?

   >???????? hp

   >?

   > --

   >??? _? | Peter J. Holzer??? | Story must make more sense than reality.

   > |_|_) |??????????????????? |

   > | |?? | hjp at hjp.at???????? |??? -- Charles Stross, "Creative writing

   > __/?? | http://www.hjp.at/ |?????? challenge!"

   > --

   > https://mail.python.org/mailman/listinfo/python-list

   ?

   ?

   ?

   --

   Tangra Mega Rock: http://www.radiotangra.com

   --

   https://mail.python.org/mailman/listinfo/python-list

   ?

References

   Visible links
   1. https://go.microsoft.com/fwlink/?LinkId=550986
   2. mailto:rambiusparkisanius at gmail.com
   3. mailto:python-list at python.org

From ofekshulberg at gmail.com  Sat Feb 25 18:58:35 2023
From: ofekshulberg at gmail.com (ofek shulberg)
Date: Sat, 25 Feb 2023 15:58:35 -0800 (PST)
Subject: No module named _socket, on windows
In-Reply-To: <mailman.426.1262632756.28905.python-list@python.org>
References: <fc76bf3a-7f7e-4c8d-899c-27197cdc9573@u41g2000yqe.googlegroups.com>
 <mailman.426.1262632756.28905.python-list@python.org>
Message-ID: <d418119f-cef8-44bd-87ed-8ea02c3baa80n@googlegroups.com>

On Monday, January 4, 2010 at 9:19:21?PM UTC+2, Gabriel Genellina wrote:
> En Mon, 04 Jan 2010 14:24:22 -0300, louisJ <loui... at gmail.com> escribi?:
> > I installed python 2.6 (from python.org) for windows XP, and then
> > Pylab.
> > When I type "import pylab" in a python shell it shows the error:
> >
> > ImportError: No module named _socket
> Open the Python command line, type the following lines, and tell us what 
> happens:
> Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit 
> (Intel)] on
> win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import socket
> >>> socket._socket
> <module '_socket' from 'd:\apps\python26\DLLs\_socket.pyd'>
> 
> -- 
> Gabriel Genellina


hi Gabriel,
i have the same problem and also pip is not installed for some reason, what can i do?

From rosuav at gmail.com  Sun Feb 26 00:35:50 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 26 Feb 2023 16:35:50 +1100
Subject: Is there a more efficient threading lock?
In-Reply-To: <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
Message-ID: <CAPTjJmr994eHyMB34j70gfXOchYDkGPinWspNyJk4K9txATtyQ@mail.gmail.com>

On Sun, 26 Feb 2023 at 16:16, Jon Ribbens via Python-list
<python-list at python.org> wrote:
>
> On 2023-02-25, Paul Rubin <no.email at nospam.invalid> wrote:
> > The GIL is an evil thing, but it has been around for so long that most
> > of us have gotten used to it, and some user code actually relies on it.
> > For example, with the GIL in place, a statement like "x += 1" is always
> > atomic, I believe.  But, I think it is better to not have any shared
> > mutables regardless.
>
> I think it is the case that x += 1 is atomic but foo.x += 1 is not.
> Any replacement for the GIL would have to keep the former at least,
> plus the fact that you can do hundreds of things like list.append(foo)
> which are all effectively atomic.

The GIL is most assuredly *not* an evil thing. If you think it's so
evil, go ahead and remove it, because we'll clearly be better off
without it, right?

As it turns out, most GIL-removal attempts have had a fairly nasty
negative effect on performance. The GIL is a huge performance boost.

As to what is atomic and what is not... it's complicated, as always.
Suppose that x (or foo.x) is a custom type:

class Thing:
    def __iadd__(self, other):
        print("Hi, I'm being added onto!")
        self.increment_by(other)
        return self

Then no, neither of these is atomic, although if the increment itself
is, it probably won't matter. As far as I know, the only way that it
would be at all different for x+=1 and foo.x+=1 would be if the
__iadd__ method both mutates and returns something other than self,
which is quite unusual. (Most incrementing is done by either
constructing a new object to return, or mutating the existing one, but
not a hybrid.)

Consider this:

import threading
d = {0:0, 1:0, 2:0, 3:0}
def thrd():
    for _ in range(10000):
        d[0] += 1
        d[1] += 1
        d[2] += 1
        d[3] += 1

threads = [threading.Thread(target=thrd) for _ in range(50)]
for t in threads: t.start()
for t in threads: t.join()
print(d)

Is this code guaranteed to result in 500000 in every slot in the
dictionary? What if you replace the dictionary with a four-element
list? Do you need a GIL for this, or some other sort of lock? What
exactly is it that is needed? To answer that question, let's look at
exactly what happens in the disassembly:

>>> def thrd():
...     d[0] += 1
...     d[1] += 1
...
>>> import dis
>>> dis.dis(thrd)
  1           0 RESUME                   0

  2           2 LOAD_GLOBAL              0 (d)
             14 LOAD_CONST               1 (0)
             16 COPY                     2
             18 COPY                     2
             20 BINARY_SUBSCR
             30 LOAD_CONST               2 (1)
             32 BINARY_OP               13 (+=)
             36 SWAP                     3
             38 SWAP                     2
             40 STORE_SUBSCR

  3          44 LOAD_GLOBAL              0 (d)
             56 LOAD_CONST               2 (1)
             58 COPY                     2
             60 COPY                     2
             62 BINARY_SUBSCR
             72 LOAD_CONST               2 (1)
             74 BINARY_OP               13 (+=)
             78 SWAP                     3
             80 SWAP                     2
             82 STORE_SUBSCR
             86 LOAD_CONST               0 (None)
             88 RETURN_VALUE
>>>

(Your exact disassembly may differ, this was on CPython 3.12.)
Crucially, note these three instructions that occur in each block:
BINARY_SUBSCR, BINARY_OP, and STORE_SUBSCR. Those are a lookup
(retrieving the value of d[0]), the actual addition (adding one to the
value), and a store (putting the result back into d[0]). So it's
actually not guaranteed to be atomic; it would be perfectly reasonable
to interrupt that sequence and have something else do another
subscript.

Here's the equivalent with just incrementing a global:

>>> def thrd():
...     x += 1
...
>>> dis.dis(thrd)
  1           0 RESUME                   0

  2           2 LOAD_FAST_CHECK          0 (x)
              4 LOAD_CONST               1 (1)
              6 BINARY_OP               13 (+=)
             10 STORE_FAST               0 (x)
             12 LOAD_CONST               0 (None)
             14 RETURN_VALUE
>>>

The exact same sequence: load, add, store. Still not atomic.

General takeaway: The GIL is a performance feature, not a magic
solution, and certainly not an evil beast that must be slain at any
cost. Attempts to remove it always have to provide equivalent
protection in some other way. But the protection you think you have
might not be what you actually have.

ChrisA

From rosuav at gmail.com  Sun Feb 26 00:50:38 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 26 Feb 2023 16:50:38 +1100
Subject: Is there a more efficient threading lock?
In-Reply-To: <a3jlvhhm479ltklcp8u42t6qffmltokba9@4ax.com>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <5fec3cb2-52a2-758b-fda9-bcd6608d5681@tompassin.net>
 <CANc-5Ux9fg6v0Gjtqi=g=MdB-OJtOeM3u6xPS0c3nyPKp88Acg@mail.gmail.com>
 <mailman.1997.1677362017.20444.python-list@python.org>
 <a3jlvhhm479ltklcp8u42t6qffmltokba9@4ax.com>
Message-ID: <CAPTjJmpMu+KDByZ16AfvKLAHSYBiNFrpsrXfXNsrAEqTbD77hg@mail.gmail.com>

On Sun, 26 Feb 2023 at 16:27, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>
> On Sat, 25 Feb 2023 15:41:52 -0600, Skip Montanaro
> <skip.montanaro at gmail.com> declaimed the following:
>
>
> >concurrent.futures.ThreadPoolExecutor() with the default number of workers (
> >os.cpu_count() * 1.5, or 12 threads on my system) to process each month, so
> >12 active threads at a time. Given that the process is pretty much CPU
> >bound, maybe reducing the number of workers to the CPU count would make
>
>         Unless things have improved a lot over the years, the GIL still limits
> active threads to the equivalent of a single CPU. The OS may swap among
> which CPU as it schedules system processes, but only one thread will be
> running at any moment regardless of CPU count.

Specifically, a single CPU core *executing Python bytecode*. There are
quite a few libraries that release the GIL during computation. Here's
a script that's quite capable of saturating my CPU entirely - in fact,
typing this email is glitchy due to lack of resources:

import threading
import bcrypt
results = [0, 0]
def thrd():
    for _ in range(10):
        ok = bcrypt.checkpw(b"password",
b'$2b$15$DGDXMb2zvPotw1rHFouzyOVzSopiLIUSedO5DVGQ1GblAd6L6I8/6')
        results[ok] += 1

threads = [threading.Thread(target=thrd) for _ in range(100)]
for t in threads: t.start()
for t in threads: t.join()
print(results)

I have four cores eight threads, and yeah, my CPU's not exactly the
latest and greatest (i7 6700k - it was quite good some years ago, but
outstripped now), but feel free to crank the numbers if you want to.

I'm pretty sure bcrypt won't use more than one CPU core for a single
hashpw/checkpw call, but by releasing the GIL during the hard number
crunching, it allows easy parallelization. Same goes for numpy work,
or anything else that can be treated as a separate operation.

So it's more accurate to say that only one CPU core can be
*manipulating Python objects* at a time, although it's hard to pin
down exactly what that means, making it easier to say that there can
only be one executing Python bytecode; it should be possible for any
function call into a C library to be a point where other threads can
take over (most notably, any sort of I/O, but also these kinds of
things).

As mentioned, GIL-removal has been under discussion at times, most
recently (and currently) with PEP 703
https://peps.python.org/pep-0703/ - and the benefits in multithreaded
applications always have to be factored against quite significant
performance penalties. It's looking like PEP 703's proposal has the
least-bad performance measurements of any GILectomy I've seen so far,
showing 10% worse performance on average (possibly able to be reduced
to 5%). As it happens, a GIL just makes sense when you want pure, raw
performance, and it's only certain workloads that suffer under it.

ChrisA

From rosuav at gmail.com  Sun Feb 26 00:52:04 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 26 Feb 2023 16:52:04 +1100
Subject: 'dn' saying "Welcome (to the list)"
In-Reply-To: <cecd7d72-0b4d-4e01-9f94-b54bdc4bf007n@googlegroups.com>
References: <cecd7d72-0b4d-4e01-9f94-b54bdc4bf007n@googlegroups.com>
Message-ID: <CAPTjJmrEGDkMT-Jec4OcrcMoAikEHkG_F_ORR-cie-NHUSiFew@mail.gmail.com>

On Sun, 26 Feb 2023 at 16:23, Hen Hanna <henhanna at gmail.com> wrote:
>
>
> On Thursday, March 4, 2021  dn wrote:
> > Hi, and welcome to the list.
>
>
> note that this is the very same rude guy ('dn')   who is apparently the rudest of them all.
>
> note that this is the very same rude guy ('dn')   who wants to shunt me away to the   TUTOR   list.
>
>           -----------  i guess i'd consider being TUTOR'ed by 'dn' (or someone like it)
>                                if   i  can  be  paid    $1000 / hour  for  it.
>
>
>
> Have others noticed this too?  (it's often the guy who says "Welcome" that....)
>
>
> Whenever someone  says  "Welcome"  to me (in this way),  my alarm goes off.
>
>
> --------- a good way to explain it (the Phenomenon)  is...   it's an indication that he's itching to treat me as a Newbie...  that he 's  salivating because he senses  an opportunity to   MANsplain   and   WHITEsplain   things to me.
>

Fine, I won't say "welcome to the list" to anyone in the future.
Instead, I'll say "why don't you go and solve your own problems by
searching the web".

That's WAY more friendly, right? We should all start doing that.

It's true what they say - no good deed goes unpunished.

ChrisA

From 2QdxY4RzWzUUiLuE at potatochowder.com  Sun Feb 26 04:08:51 2023
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Sun, 26 Feb 2023 04:08:51 -0500
Subject: No module named _socket, on windows
In-Reply-To: <d418119f-cef8-44bd-87ed-8ea02c3baa80n@googlegroups.com>
References: <fc76bf3a-7f7e-4c8d-899c-27197cdc9573@u41g2000yqe.googlegroups.com>
 <mailman.426.1262632756.28905.python-list@python.org>
 <d418119f-cef8-44bd-87ed-8ea02c3baa80n@googlegroups.com>
Message-ID: <Y/sho1aS2kbqLq8N@anomaly>

On 2023-02-25 at 15:58:35 -0800,
ofek shulberg <ofekshulberg at gmail.com> wrote:

> On Monday, January 4, 2010 at 9:19:21?PM UTC+2, Gabriel Genellina wrote:
> > En Mon, 04 Jan 2010 14:24:22 -0300, louisJ <loui... at gmail.com> escribi?:
> > > I installed python 2.6 (from python.org) for windows XP, and then
> > > Pylab.
> > > When I type "import pylab" in a python shell it shows the error:
> > >
> > > ImportError: No module named _socket
> > Open the Python command line, type the following lines, and tell us what 
> > happens:
> > Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit 
> > (Intel)] on
> > win32
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> import socket
> > >>> socket._socket
> > <module '_socket' from 'd:\apps\python26\DLLs\_socket.pyd'>
> > 
> > -- 
> > Gabriel Genellina
> 
> 
> hi Gabriel,
> i have the same problem and also pip is not installed for some reason, what can i do?

Gabriel's post is thirteen years old and against a truly ancient version
of Python (on Windows XP, no less).  Please repost your question.
Include the OS and the version of Python you're using, and a copy/paste
of the error(s) you're receiving.

From learn2program at gmail.com  Sun Feb 26 04:14:51 2023
From: learn2program at gmail.com (Alan Gauld)
Date: Sun, 26 Feb 2023 09:14:51 +0000
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <k5votpFpjveU1@mid.individual.net>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
Message-ID: <ac239ea7-c02f-4af8-88ee-cc8a62b53fae@yahoo.co.uk>

On 26/02/2023 00:54, Greg Ewing via Python-list wrote:
> On 26/02/23 10:53 am, Paul Rubin wrote:
>> I'm not on either list but the purpose of the tutor list is to shunt
>> beginner questions away from the main list.

I'm not sure that's why we set it up but it is
certainly a large part of our remit. But protecting newbies
from overly complex responses and covering wider topics
(beyond pure Pyhon) is also a large part of our purpose.

> There's a fundamental problem with tutor lists. They rely on
> experienced people, the ones capable of answering the questions,
> to go out of their way to read the tutor list -- something that
> is not of personal benefit to them.

In practice, the "tutors" tend to be split between folks
who inhabit both lists and those who only interact on the tutor
list. eg. I lurk here and only occasionally partake.

But the problem with this particular thread is that, if directed
to the tutor list, the OP would simply be told that "that's the
way Python works". The tutor list is not for discussing
language enhancements etc. It is purely about answering questions
on how to use the language (and standard library) as it exists.
(We also cover beginner questions about programming in general.)

So this thread is most definitely in the right place IMHO.

-- 
Alan G
Tutor list moderator



From barry at barrys-emacs.org  Sun Feb 26 06:53:26 2023
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 26 Feb 2023 11:53:26 +0000
Subject: Is there a more efficient threading lock?
In-Reply-To: <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
Message-ID: <2c44aec3-bc68-b78e-dfe9-a4e6259d024e@barrys-emacs.org>


On 25/02/2023 23:45, Jon Ribbens via Python-list wrote:
> I think it is the case that x += 1 is atomic but foo.x += 1 is not.

No that is not true, and has never been true.

:>>> def x(a):
:... ???a += 1
:...
:>>>
:>>> dis.dis(x)
 ?1 ??????????0 RESUME ??????????????????0

 ?2 ??????????2 LOAD_FAST ???????????????0 (a)
 ?????????????4 LOAD_CONST ??????????????1 (1)
 ?????????????6 BINARY_OP ??????????????13 (+=)
 ????????????10 STORE_FAST ??????????????0 (a)
 ????????????12 LOAD_CONST ??????????????0 (None)
 ????????????14 RETURN_VALUE
:>>>

As you can see there are 4 byte code ops executed.

Python's eval loop can switch to another thread between any of them.

Its is not true that the GIL provides atomic operations in python.

Barry


From list1 at tompassin.net  Sun Feb 26 09:40:27 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 26 Feb 2023 09:40:27 -0500
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
Message-ID: <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>

On 2/25/2023 8:12 PM, Hen Hanna wrote:
> 2. the rude guy ('dn') hasn't  offered  a single word of comment   that's directly relevant to it.
> >              --------  but he did  offer related stuff  which he 
thinks i should be  [grateful] for

Please let's stop the ad hominem messages. If someone doesn't like a 
particular person's messages, send them to spam or don't read them.  If 
too many people start to get too rude or personal, remind the whole list 
of the guidelines for respectful participation.  If you feel that 
someone's contribution was especially helpful in part because it was 
concise and to the point, you could say that.


From avi.e.gross at gmail.com  Sun Feb 26 12:34:10 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sun, 26 Feb 2023 12:34:10 -0500
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <ac239ea7-c02f-4af8-88ee-cc8a62b53fae@yahoo.co.uk>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ac239ea7-c02f-4af8-88ee-cc8a62b53fae@yahoo.co.uk>
Message-ID: <004801d94a08$891663c0$9b432b40$@gmail.com>

Alan,

Good tack. By not welcoming someone who is paranoid about being welcomed you
are clearly the right kind of welcoming!

Kidding aside, you have a point about one of the barrage of messages
probably not getting a great answer on your tutor forum. It is the MANY
messages often about fairly simple aspects of python, taken together, that
lead to the conclusion that this person is fairly new to python and still
thinking about things from a lifetime of experience using other languages.

I will say that at this point, it does not matter where they post as I
cannot imagine anyone having to pay them $1,000/hour for the privilege of
trying to tutor them.

There are topics raised that can be informative and lead to good discussions
amicably and as far as I can tell, many agree it would be nice if some
"error" messages provided more detail and perhaps some eventually will. But
as has been pointed out, these messages are only a small part of the python
environment and lots of other tools are typically used to debug that do
allow access to all kinds of details at breakpoints. 

I think many would be satisfied with some of the answers provided here and
note, FEW OR NONE OF US here (or am I wrong) are necessarily in a position
to make changes like this to the current or next versions of python. We are
all users who take what we get and work with it or look for a way around
things. The example used did not strike me as hard to figure out which of
X/Y was an int/str and what their values were. More time is wasted demanding
and debating a feature that is not there rather than solving the problem in
other ways.

In the interest of civility, I find removing myself sometimes works well. We
are volunteers and I don't need to volunteer to help any particular person
who does not seem to appreciate it. And if a forum fills up with nonsense so
the signal is hard to find amid the noise, why bother contributing?

Avi

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Alan Gauld
Sent: Sunday, February 26, 2023 4:15 AM
To: python-list at python.org
Subject: Re: TypeError: can only concatenate str (not "int") to str

On 26/02/2023 00:54, Greg Ewing via Python-list wrote:
> On 26/02/23 10:53 am, Paul Rubin wrote:
>> I'm not on either list but the purpose of the tutor list is to shunt 
>> beginner questions away from the main list.

I'm not sure that's why we set it up but it is certainly a large part of our
remit. But protecting newbies from overly complex responses and covering
wider topics (beyond pure Pyhon) is also a large part of our purpose.

> There's a fundamental problem with tutor lists. They rely on 
> experienced people, the ones capable of answering the questions, to go 
> out of their way to read the tutor list -- something that is not of 
> personal benefit to them.

In practice, the "tutors" tend to be split between folks who inhabit both
lists and those who only interact on the tutor list. eg. I lurk here and
only occasionally partake.

But the problem with this particular thread is that, if directed to the
tutor list, the OP would simply be told that "that's the way Python works".
The tutor list is not for discussing language enhancements etc. It is purely
about answering questions on how to use the language (and standard library)
as it exists.
(We also cover beginner questions about programming in general.)

So this thread is most definitely in the right place IMHO.

--
Alan G
Tutor list moderator


-- 
https://mail.python.org/mailman/listinfo/python-list


From skip.montanaro at gmail.com  Sun Feb 26 12:53:45 2023
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sun, 26 Feb 2023 11:53:45 -0600
Subject: Is there a more efficient threading lock?
In-Reply-To: <2c44aec3-bc68-b78e-dfe9-a4e6259d024e@barrys-emacs.org>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
 <2c44aec3-bc68-b78e-dfe9-a4e6259d024e@barrys-emacs.org>
Message-ID: <CANc-5UyC_=R2YzrFkdYYvGnvzt=oOAKrgHMtF_kmk57G9y3mqg@mail.gmail.com>

Thanks for the various replies. The program originally started out
single-threaded. I wandered down the multi-threaded path to see if I could
get a performance boost using Sam Gross's NoGIL fork
<https://github.com/colesbury/nogil-3.12>. I was pretty sure the GIL would
limit multi-threading performance on a stock Python interpreter. When I
first switched to threads, I didn't have a lock around the one or two
places which called out to the TextBlob <https://textblob.readthedocs.io/>/NLTK
stuff. The use of threading.Lock was the obvious simplest choice, and it
solved the crash I saw without it. I'm still thinking about using queues to
communicate between the email processing threads and the TextBlob & SQLite
processing stuff.

I had been doing a bit of pre- and post-processing of the default TextBlob
noun phrase generation, but I wasn't happy with it, so I decided to
experiment with an alternate noun phrase extractor
<https://textblob.readthedocs.io/en/dev/api_reference.html?highlight=ConllExtractor#textblob.en.np_extractors.ConllExtractor>.
I was happier with that, so ripped out most of the ad hoc stuff I was
doing. While doing this code surgery, I moved back to 3.11 to have a more
trusty Python interpreter. (I've yet to encounter a problem with NoGIL,
just cutting back on moving parts, and wasn't seeing any obvious
performance gains.)

As for SQLite and multi-threading, I figured if the core devs hadn't yet
gotten around to making it available then it probably wasn't
straightforward. I wasn't willing to tackle that.

So, I'll keep messing around. It's all just for fun
<https://www.smontanaro.net/CR> anyway.

Skip

From henhanna at gmail.com  Sun Feb 26 00:53:12 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sat, 25 Feb 2023 21:53:12 -0800 (PST)
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <k5oftlFlct6U1@mid.individual.net>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
Message-ID: <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>

On Wednesday, February 22, 2023 at 10:38:00?PM UTC-8, Greg Ewing wrote:
> On 23/02/23 9:37 am, Hen Hanna wrote: 
> > for the first several weeks... whenever i used Python... all i could think of....was -------- this is really Lisp (inside) with a thin veil of Java/Pascal syntax.......... 
> > 
> > ----- that everything is first converted (macro-expanded) into (intermediated) Lisp code, and then.........
> I once toyed with the idea of implementing a Python compiler 
> by translating it into Scheme and then feeding it to a Scheme 
> compiler. 
> 
> But I quickly realised that, although Scheme and Python are 
> both dynamically-typed languages, Python is way *more* dynamic 
> than Scheme. 
> 
> So without doing some very serious static analysis, the best 
> I could do would be just putting together calls to runtime 
> support routines that implement all the dynamic dispatching 
> that Python does for its operators, etc., and the result 
> wouldn't be much better than an interpreter. 
> 
> There are some similarities between Python and Lisp-family 
> languages, but really Python is its own thing. 
> 
> -- 
> Greg



   Scope (and extent ?) of   variables is one reminder that  Python is not Lisp

?????for    i     in      range(5):??    print( i )
                     .........
?????print( i )

ideally, after the FOR loop is done,  the (local) var  i should also disappear.
(this almost caused a bug for me)


Maybe in a future  ver. of Python,   it will be just like:

                     (dotimes? (i   5)?  (print   i))


From henhanna at gmail.com  Sun Feb 26 00:57:34 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sat, 25 Feb 2023 21:57:34 -0800 (PST)
Subject: 'dn' saying "Welcome (to the list)"
In-Reply-To: <mailman.2010.1677390737.20444.python-list@python.org>
References: <CAPTjJmrEGDkMT-Jec4OcrcMoAikEHkG_F_ORR-cie-NHUSiFew@mail.gmail.com>
 <cecd7d72-0b4d-4e01-9f94-b54bdc4bf007n@googlegroups.com>
 <mailman.2010.1677390737.20444.python-list@python.org>
Message-ID: <2e0dd81b-1f96-48ba-9b8c-6c66c2484cb5n@googlegroups.com>

On Saturday, February 25, 2023 at 9:52:35?PM UTC-8, Chris Angelico wrote:
> On Sun, 26 Feb 2023 at 16:23, Hen Hanna <henh... at gmail.com> wrote: 
> > 
> > 
> > On Thursday, March 4, 2021 dn wrote: 
> > > Hi, and welcome to the list. 
> > 
> > 
> > note that this is the very same rude guy ('dn') who is apparently the rudest of them all. 
> > 
> > note that this is the very same rude guy ('dn') who wants to shunt me away to the TUTOR list. 
> > 
> > ----------- i guess i'd consider being TUTOR'ed by 'dn' (or someone like it) 
> > if i can be paid $1000 / hour for it. 
> > 
> > 
> > 
> > Have others noticed this too? (it's often the guy who says "Welcome" that....) 
> > 
> > 
> > Whenever someone says "Welcome" to me (in this way), my alarm goes off. 
> > 
> > 
> > --------- a good way to explain it (the Phenomenon) is... it's an indication that he's itching to treat me as a Newbie... that he 's salivating because he senses an opportunity to MANsplain and WHITEsplain things to me. 
> >
> Fine, I won't say "welcome to the list" to anyone in the future. 


the rule of thumb is...          [Don't say it ;  show it ]

Maybe 10% of the time,  ppl who say  [Welcome to ...]   are actually good (welcoming) ppl.

From henhanna at gmail.com  Sun Feb 26 02:45:01 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sat, 25 Feb 2023 23:45:01 -0800 (PST)
Subject: one Liner:  Lisprint(x)  --> (a, b, c) instead of ['a', 'b', 'c']
Message-ID: <439d9918-210a-49c7-b601-5289257dd407n@googlegroups.com>


def Lisprint(x):  print( ' (' + ', '.join(x) + ')' , '\n')

a=    ' a b c ?  def f x if zero? x 0 1 '
a +=  ' A B C !   just an example '
x= a.split()

print(x)
Lisprint(x)

['a', 'b', 'c', '?', 'def', 'f', 'x', 'if', 'zero?', 'x', '0', '1', 'A', 'B', 'C', '!', 'just', 'an', 'example']

 (a, b, c, ?, def, f, x, if, zero?, x, 0, 1, A, B, C, !, just, an, example)



From henhanna at gmail.com  Sun Feb 26 04:54:02 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sun, 26 Feb 2023 01:54:02 -0800 (PST)
Subject: one Liner: Lisprint(x) --> (a, b, c) instead of ['a', 'b', 'c']
In-Reply-To: <439d9918-210a-49c7-b601-5289257dd407n@googlegroups.com>
References: <439d9918-210a-49c7-b601-5289257dd407n@googlegroups.com>
Message-ID: <a1fb44ad-45a3-49e4-9954-147317f0b86en@googlegroups.com>

On Saturday, February 25, 2023 at 11:45:12?PM UTC-8, Hen Hanna wrote:
> def Lisprint(x): print( ' (' + ', '.join(x) + ')' , '\n') 
> 
> a= ' a b c ? def f x if zero? x 0 1 ' 
> a += ' A B C ! just an example ' 
> x= a.split() 
> 
> print(x) 
> Lisprint(x) 
> 
> ['a', 'b', 'c', '?', 'def', 'f', 'x', 'if', 'zero?', 'x', '0', '1', 'A', 'B', 'C', '!', 'just', 'an', 'example'] 
> 
> (a, b, c, ?, def, f, x, if, zero?, x, 0, 1, A, B, C, !, just, an, example)


For nested lists....          impossible to improve   upon  P.Norvig's  code


def Lisprint(x):           print(lispstr(x))

def lispstr(exp):
    "Convert a Python object back into a Lisp-readable string."
    if isinstance(exp, list):
        return '(' + ' '.join(map(lispstr, exp)) + ')' 
    else:
        return str(exp)

a=    ' a b c '
x= a.split()
x +=  [['a', 'b', 'c']]
x +=  x

print(x) 
Lisprint(x) 

['a', 'b', 'c', ['a', 'b', 'c'], 'a', 'b', 'c', ['a', 'b', 'c']]

(a b c (a b c) a b c (a b c))


              ----------   Without the commas,   the visual difference (concision)  is  striking !

From jon+usenet at unequivocal.eu  Sun Feb 26 11:09:44 2023
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Sun, 26 Feb 2023 16:09:44 -0000 (UTC)
Subject: Is there a more efficient threading lock?
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
 <CAPTjJmr994eHyMB34j70gfXOchYDkGPinWspNyJk4K9txATtyQ@mail.gmail.com>
 <mailman.2008.1677389764.20444.python-list@python.org>
Message-ID: <slrntvn128.6q7u.jon+usenet@raven.unequivocal.eu>

On 2023-02-26, Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, 26 Feb 2023 at 16:16, Jon Ribbens via Python-list
><python-list at python.org> wrote:
>> On 2023-02-25, Paul Rubin <no.email at nospam.invalid> wrote:
>> > The GIL is an evil thing, but it has been around for so long that most
>> > of us have gotten used to it, and some user code actually relies on it.
>> > For example, with the GIL in place, a statement like "x += 1" is always
>> > atomic, I believe.  But, I think it is better to not have any shared
>> > mutables regardless.
>>
>> I think it is the case that x += 1 is atomic but foo.x += 1 is not.
>> Any replacement for the GIL would have to keep the former at least,
>> plus the fact that you can do hundreds of things like list.append(foo)
>> which are all effectively atomic.
>
> The GIL is most assuredly *not* an evil thing. If you think it's so
> evil, go ahead and remove it, because we'll clearly be better off
> without it, right?

If you say so. I said nothing whatsoever about the GIL being evil.

> As it turns out, most GIL-removal attempts have had a fairly nasty
> negative effect on performance. The GIL is a huge performance boost.
>
> As to what is atomic and what is not... it's complicated, as always.
> Suppose that x (or foo.x) is a custom type:

Yes, sure, you can make x += 1 not work even single-threaded if you
make custom types which override basic operations. I'm talking about
when you're dealing with simple atomic built-in types such as integers.

> Here's the equivalent with just incrementing a global:
>
>>>> def thrd():
> ...     x += 1
> ...
>>>> dis.dis(thrd)
>   1           0 RESUME                   0
>
>   2           2 LOAD_FAST_CHECK          0 (x)
>               4 LOAD_CONST               1 (1)
>               6 BINARY_OP               13 (+=)
>              10 STORE_FAST               0 (x)
>              12 LOAD_CONST               0 (None)
>              14 RETURN_VALUE
>>>>
>
> The exact same sequence: load, add, store. Still not atomic.

And yet, it appears that *something* changed between Python 2
and Python 3 such that it *is* atomic:

    import sys, threading
    class Foo:
        x = 0
    foo = Foo()
    y = 0
    def thrd():
        global y
        for _ in range(10000):
            foo.x += 1
            y += 1
    threads = [threading.Thread(target=thrd) for _ in range(50)]
    for t in threads: t.start()
    for t in threads: t.join()
    print(sys.version)
    print(foo.x, y)

2.7.5 (default, Jun 28 2022, 15:30:04)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
(64489, 59854)

3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0]
500000 500000


From jon+usenet at unequivocal.eu  Sun Feb 26 11:11:11 2023
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Sun, 26 Feb 2023 16:11:11 -0000 (UTC)
Subject: Is there a more efficient threading lock?
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
 <2c44aec3-bc68-b78e-dfe9-a4e6259d024e@barrys-emacs.org>
 <mailman.2013.1677412409.20444.python-list@python.org>
Message-ID: <slrntvn14v.6q7u.jon+usenet@raven.unequivocal.eu>

On 2023-02-26, Barry Scott <barry at barrys-emacs.org> wrote:
> On 25/02/2023 23:45, Jon Ribbens via Python-list wrote:
>> I think it is the case that x += 1 is atomic but foo.x += 1 is not.
>
> No that is not true, and has never been true.
>
>:>>> def x(a):
>:... ???a += 1
>:...
>:>>>
>:>>> dis.dis(x)
>  ?1 ??????????0 RESUME ??????????????????0
>
>  ?2 ??????????2 LOAD_FAST ???????????????0 (a)
>  ?????????????4 LOAD_CONST ??????????????1 (1)
>  ?????????????6 BINARY_OP ??????????????13 (+=)
>  ????????????10 STORE_FAST ??????????????0 (a)
>  ????????????12 LOAD_CONST ??????????????0 (None)
>  ????????????14 RETURN_VALUE
>:>>>
>
> As you can see there are 4 byte code ops executed.
>
> Python's eval loop can switch to another thread between any of them.
>
> Its is not true that the GIL provides atomic operations in python.

That's oversimplifying to the point of falsehood (just as the opposite
would be too). And: see my other reply in this thread just now - if the
GIL isn't making "x += 1" atomic, something else is.

From henhanna at gmail.com  Sun Feb 26 11:56:28 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sun, 26 Feb 2023 08:56:28 -0800 (PST)
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <mailman.2014.1677422439.20444.python-list@python.org>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
 <mailman.2014.1677422439.20444.python-list@python.org>
Message-ID: <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>


On Sunday, February 26, 2023 at 6:41:01?AM UTC-8, Thomas Passin wrote:
> On 2/25/2023 8:12 PM, Hen Hanna wrote: 
> > 2. the rude guy ('dn') hasn't offered a single word of comment that's directly relevant to it. 
> > > -------- but he did offer related stuff which he   thinks i should be [grateful] for


> Please let's stop the ad hominem messages. If someone doesn't like a 
> particular person's messages, send them to spam or don't read them. If 
> too many people start to get too rude or personal, remind the whole list 
> of the guidelines for respectful participation. If you feel that 
> someone's contribution was especially helpful in part because it was 
> concise and to the point, you could say that.


yes.   let's stop...


If you  (Thomas Passin)   feel that  someone's contribution was especially helpful in part because it was 
 concise and to the point, you could say that.
                        and pls don't hesitate to reproduce such a comment.




i'm sure a few others were also rude, but  it was the rudest of them all ('dn')  that   told me to read some [Code of Conduct] document.   -------- Do not EVER  do that  again.


Another rude guy who "asked"  me why i write in a Hard-to-read way....  Why don't you at least   also make a comment that's  On-Topic  besides  the insulting remark ?




so far,  i think  Paul Rubin's post (in another thread) was esp. concise, informative, --- but he's also made a comment about       'shunting'  beginners  (questions)     to a concentration camp, and sounded  a bit  like a cold-hearted (or warm-hearted)  Nazi  officer / scientist.


I think you can speed this up by building two sets and intersecting them:
================================================================
from itertools import combinations

ww = "JSOYOMFUBELR SCDUARWDRLYE DASNAGEFERTY CLULOOTSCEHN USENEARSEYNE".split()
ss = set(''.join(s) for w in ww for s in combinations(w.lower(),6))
d6 = set(d.strip().lower() for d in open('/usr/share/dict/words') if len(d)==7)
print(ss & d6)


i too often use  short Var names   x, xx,     w,ww,    s,ss ..........
                       and also long ones.

From henhanna at gmail.com  Sun Feb 26 12:17:46 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sun, 26 Feb 2023 09:17:46 -0800 (PST)
Subject: Vim (To move text-Lines between files) ___ :ab wt w! ~/temp.vi ___
 :ab rt r ~/temp.vi
Message-ID: <7c8f53bf-3e31-4a36-9cc9-294ce95e0308n@googlegroups.com>

within Vim,  do 
                             :smile

___________________________

To move text-Lines between files --- i do this (below)....   Maybe there's a better (or more standard) way,  but i've been doing this for 30+ years, so i'll prob.  keep doing it.


i have these in my .vimrc file.

                     :ab   qq   q!

                       :ab   wt   w!   ~/temp.vi
                      :ab   rt   r    ~/temp.vi
So i can ..

[1] from one file, do

                :.wt            --  write 1 line to temp file
                :.,.+5 wt     --  write 6 lines to temp file
                :.,'a wt         --  write up to (and including) mark A  to temp file

[2] optionally edit the temp file

[3] from another file, do the following to read (in) the temp file

                  :rt



____________________

> Yes your questions do seem excessively frequent even here on Usenet.  
> I have mostly been ignoring them after seeing the first few.


--------  what do you know about USENET ?....  i'm pretty sure i've been posting to USENET  longer than you ...

i'm also pretty sure that i've been programming longer than you... (but you may be faster and better.)

From henhanna at gmail.com  Sun Feb 26 13:09:57 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sun, 26 Feb 2023 10:09:57 -0800 (PST)
Subject: Rob Cliffe  should stop sending me rude email messages.
Message-ID: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>


Rob Cliffe should stop sending me rude email messages.


At the very least,   Rob Cliffe should stop sending me ANY email messages, if he doesn't intend to an email i've sent him.


From bowman at montana.com  Sun Feb 26 13:28:23 2023
From: bowman at montana.com (rbowman)
Date: 26 Feb 2023 18:28:23 GMT
Subject: Vim (To move text-Lines between files) ___ :ab wt w! ~/temp.vi
 ___ :ab rt r ~/temp.vi
References: <7c8f53bf-3e31-4a36-9cc9-294ce95e0308n@googlegroups.com>
Message-ID: <k61mm7F444uU1@mid.individual.net>

On Sun, 26 Feb 2023 09:17:46 -0800 (PST), Hen Hanna wrote:


> To move text-Lines between files --- i do this (below)....   Maybe
> there's a better (or more standard) way,  but i've been doing this for
> 30+ years, so i'll prob.  keep doing it.
>

You can use the buffers. 

"a yy  will add the current line to buffer a.

"A 5 yy  will add 5 lines to buffer a. Note the use of case.

"a p  will write the contents of buffer a to the other file. 

Note that buffer a does not interfere with using a for a bookmark. In 
other words if you've marked an area with 'm a',  "a y'a  will put the 
text from the current position to the bookmark in buffer a.

Also note that "* p  will insert the contents of the clipboard or copy the 
text to the clipboard. I use that if I have files open in two different 
gvim instances.


From henhanna at gmail.com  Sun Feb 26 13:49:59 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sun, 26 Feb 2023 10:49:59 -0800 (PST)
Subject: Vim (To move text-Lines between files) ___ :ab wt w! ~/temp.vi
 ___ :ab rt r ~/temp.vi
In-Reply-To: <k61mm7F444uU1@mid.individual.net>
References: <7c8f53bf-3e31-4a36-9cc9-294ce95e0308n@googlegroups.com>
 <k61mm7F444uU1@mid.individual.net>
Message-ID: <63e6a869-33f4-4abe-8287-fa7c3589885en@googlegroups.com>

On Sunday, February 26, 2023 at 10:28:41?AM UTC-8, rbowman wrote:
> On Sun, 26 Feb 2023 09:17:46 -0800 (PST), Hen Hanna wrote: 
> 
> 
> > To move text-Lines between files --- i do this (below).... Maybe 
> > there's a better (or more standard) way, but i've been doing this for 
> > 30+ years, so i'll prob. keep doing it. 
> >
> You can use the buffers. 
> 
> "a yy will add the current line to buffer a. 
> 
> "A 5 yy will add 5 lines to buffer a. Note the use of case. 
> 
> "a p will write the contents of buffer a to the other file. 
> 
> Note that buffer a does not interfere with using a for a bookmark. In 
> other words if you've marked an area with 'm a', "a y'a will put the 
> text from the current position to the bookmark in buffer a. 
> 
> Also note that "* p will insert the contents of the clipboard or copy the 
> text to the clipboard. I use that if I have files open in two different 
> gvim instances.


thank you... that seems to work...    i dont like to split  the screen   (into  Panes) in Vim

               Select the text in visual mode, then press y to "yank" it into the buffer (copy)
                               or d to "delete" it into the buffer (cut).

               Then you can :split <new file name> to split your vim window up, 
                          and press p to paste in the yanked text. Write the file as normal.

               To close the split again, pass the split you want to close :q .



_________________________________
USENET  Nazi  said:
                    Yes your questions do seem excessively frequent even here on Usenet yes.

Usenet-Nazis think they own the Usenet.    ---------     i'm   so  GLAD  that   they don't ! ! !

**   The Usenet-Nazi  as in .......... "The Soup Nazi" is the 116th episode of the NBC sitcom Seinfeld, which was the sixth episode of the seventh season. ...............

From henhanna at gmail.com  Sun Feb 26 14:09:38 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sun, 26 Feb 2023 11:09:38 -0800 (PST)
Subject: Rob Cliffe should stop sending me rude email messages.
In-Reply-To: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
References: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
Message-ID: <c5eb1276-7017-44d0-8768-936c99170068n@googlegroups.com>


[I replied to the list, but for some reason my posts don't always get through, so I'm replying to you separately.]


               i guess....   if you have anything useful to say....  you can do that again.
                    Your comments   about (not "int")  (whch didn't get through, apparently)  
                               were     among the few that were  relevant.


On Sunday, February 26, 2023 at 10:10:09?AM UTC-8, Hen Hanna wrote:
> Rob Cliffe should stop sending me rude email messages. 
> 
> 
> At the very least, Rob Cliffe should stop sending me ANY email messages, if he doesn't intend to an email i've sent him.



At the very least,    Rob Cliffe    should stop sending me
                  ANY email messages,  
                    if he doesn't intend to respond to an email i've sent him.

                                 (he failed to respond to   2 (or 3)   i've sent him)



_________________________________
USENET  Nazi  said:
                    Yes your questions do seem excessively frequent even here on Usenet yes.

this puts me in such a difficult position....   i feel i have to  post a few more just to show that i'm not folloiwng the Nazi's  advice (or suggestion)   or  those of this    Rob Cliffe 

From henhanna at gmail.com  Sun Feb 26 14:19:37 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sun, 26 Feb 2023 11:19:37 -0800 (PST)
Subject: Re-indenting  Python code in Vim
Message-ID: <79c5fee3-5b69-413e-a127-c2e32a76e664n@googlegroups.com>


Re-indenting Lisp code is          =%
and it works  really well !

                                  (define (foo x)
                                         (dotimes (i 100)
                        (bar bar x)))



----------- it doesn't work as well for Python code.
(doing         5==            at the 1st line Re-indents only 3 lines)


               for c in string.ascii_lowercase:
  for   x    in   Dict[tuple(sorted(cw))] :
        print(cw, x , end=' ')
        print('other stuff',   x , end=' ')
        PRSW=True

From henhanna at gmail.com  Sun Feb 26 16:07:19 2023
From: henhanna at gmail.com (Hen Hanna)
Date: Sun, 26 Feb 2023 13:07:19 -0800 (PST)
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <87v8qaxyna.fsf@nightsong.com>
References: <87v8qaxyna.fsf@nightsong.com>
Message-ID: <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>

On Monday, August 29, 2022 at 7:18:22?PM UTC-7, Paul Rubin wrote:
> Just because. 
> 
> from math import gcd 

> def fizz(n: int) -> str: 
>        match gcd(n, 15): 
>               case 3: return "Fizz" 
>               case 5: return "Buzz" 
>               case 15: return "FizzBuzz" 
>               case _: return str(n) 
> 
> for i in range(1,101): 
>             print(fizz(i))


is there any reason to prefer    "    over    '   ?

From larry.martell at gmail.com  Sun Feb 26 18:55:46 2023
From: larry.martell at gmail.com (Larry Martell)
Date: Sun, 26 Feb 2023 15:55:46 -0800
Subject: Rob Cliffe should stop sending me rude email messages.
In-Reply-To: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
References: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
Message-ID: <CACwCsY74E-962ZyCggr1=VNx4Roek+g6yyq=PYqFgv66KUosew@mail.gmail.com>

On Sun, Feb 26, 2023 at 3:49 PM Hen Hanna <henhanna at gmail.com> wrote:

>
> Rob Cliffe should stop sending me rude email messages.


You should stop spamming this lists with with meaningless posts.

From skip.montanaro at gmail.com  Sun Feb 26 19:20:02 2023
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sun, 26 Feb 2023 18:20:02 -0600
Subject: Is there a more efficient threading lock?
In-Reply-To: <slrntvn128.6q7u.jon+usenet@raven.unequivocal.eu>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
 <CAPTjJmr994eHyMB34j70gfXOchYDkGPinWspNyJk4K9txATtyQ@mail.gmail.com>
 <mailman.2008.1677389764.20444.python-list@python.org>
 <slrntvn128.6q7u.jon+usenet@raven.unequivocal.eu>
Message-ID: <CANc-5Uy2sZKCaJV95xsGgwBx1Xt-TeUFDy7G7VB0mHYh=-1s+w@mail.gmail.com>

> And yet, it appears that *something* changed between Python 2 and Python
3 such that it *is* atomic:

I haven't looked, but something to check in the source is opcode
prediction. It's possible that after the BINARY_OP executes, opcode
prediction jumps straight to the STORE_FAST opcode, avoiding the transfer
to the top of the virtual machine loop. That would (I think) avoid checks
related to GIL release and thread switches.

I don't guarantee that's what's going on, and even if I'm correct, I don't
think you can rely on it.

Skip

From skip.montanaro at gmail.com  Sun Feb 26 19:28:24 2023
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sun, 26 Feb 2023 18:28:24 -0600
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
Message-ID: <CANc-5Uz4SraTyBeodPBagWsNLJh0i04_NBnwuA9yV=t0Upg+wQ@mail.gmail.com>

>
> is there any reason to prefer    "    over    '   ?
>

Not really. As an old C programmer for many years I used double
quotes"around "strings" and single word around 'c'haracters, because that's
what I was used to. (This was long before triple quoted strings appeared in
the language.)

Aside: Given all the various ways to quote strings for display, it irks me
a bit to see repr() still use single quotes in all cases, which requires
display of single quotes to be escaped. (In similar fashion, it would be a
minor improvement in my mind if the repr() code used raw strings where they
would simplify the display.)

Skip

>

From skip.montanaro at gmail.com  Sun Feb 26 19:32:34 2023
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sun, 26 Feb 2023 18:32:34 -0600
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <CANc-5Uz4SraTyBeodPBagWsNLJh0i04_NBnwuA9yV=t0Upg+wQ@mail.gmail.com>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <CANc-5Uz4SraTyBeodPBagWsNLJh0i04_NBnwuA9yV=t0Upg+wQ@mail.gmail.com>
Message-ID: <CANc-5UymFLDbgyMJsAhRJZOY0AEqjX4Esqe=ZHqAC6v5WwK41g@mail.gmail.com>

Dang auto-correct... Should read

... double quotes around "strings" and single quotes around 'c'haracters ...

On Sun, Feb 26, 2023, 6:28 PM Skip Montanaro <skip.montanaro at gmail.com>
wrote:

> is there any reason to prefer    "    over    '   ?
>>
>
> Not really. As an old C programmer for many years I used double
> quotes"around "strings" and single word around 'c'haracters, because that's
> what I was used to. (This was long before triple quoted strings appeared in
> the language.)
>
> Aside: Given all the various ways to quote strings for display, it irks me
> a bit to see repr() still use single quotes in all cases, which requires
> display of single quotes to be escaped. (In similar fashion, it would be a
> minor improvement in my mind if the repr() code used raw strings where they
> would simplify the display.)
>
> Skip
>
>>

From rosuav at gmail.com  Sun Feb 26 20:25:28 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 27 Feb 2023 12:25:28 +1100
Subject: Is there a more efficient threading lock?
In-Reply-To: <slrntvn128.6q7u.jon+usenet@raven.unequivocal.eu>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
 <CAPTjJmr994eHyMB34j70gfXOchYDkGPinWspNyJk4K9txATtyQ@mail.gmail.com>
 <mailman.2008.1677389764.20444.python-list@python.org>
 <slrntvn128.6q7u.jon+usenet@raven.unequivocal.eu>
Message-ID: <CAPTjJmphugMr7Pzq+8bE5NZOfDk-ifx0rNiTXgjy9CkYi9Sccg@mail.gmail.com>

On Mon, 27 Feb 2023 at 10:42, Jon Ribbens via Python-list
<python-list at python.org> wrote:
>
> On 2023-02-26, Chris Angelico <rosuav at gmail.com> wrote:
> > On Sun, 26 Feb 2023 at 16:16, Jon Ribbens via Python-list
> ><python-list at python.org> wrote:
> >> On 2023-02-25, Paul Rubin <no.email at nospam.invalid> wrote:
> >> > The GIL is an evil thing, but it has been around for so long that most
> >> > of us have gotten used to it, and some user code actually relies on it.
> >> > For example, with the GIL in place, a statement like "x += 1" is always
> >> > atomic, I believe.  But, I think it is better to not have any shared
> >> > mutables regardless.
> >>
> >> I think it is the case that x += 1 is atomic but foo.x += 1 is not.
> >> Any replacement for the GIL would have to keep the former at least,
> >> plus the fact that you can do hundreds of things like list.append(foo)
> >> which are all effectively atomic.
> >
> > The GIL is most assuredly *not* an evil thing. If you think it's so
> > evil, go ahead and remove it, because we'll clearly be better off
> > without it, right?
>
> If you say so. I said nothing whatsoever about the GIL being evil.

You didn't, but I was also responding to Paul's description that the
GIL "is an evil thing". Apologies if that wasn't clear.

> Yes, sure, you can make x += 1 not work even single-threaded if you
> make custom types which override basic operations. I'm talking about
> when you're dealing with simple atomic built-in types such as integers.
>
> > Here's the equivalent with just incrementing a global:
> >
> >>>> def thrd():
> > ...     x += 1
> > ...
> >>>> dis.dis(thrd)
> >   1           0 RESUME                   0
> >
> >   2           2 LOAD_FAST_CHECK          0 (x)
> >               4 LOAD_CONST               1 (1)
> >               6 BINARY_OP               13 (+=)
> >              10 STORE_FAST               0 (x)
> >              12 LOAD_CONST               0 (None)
> >              14 RETURN_VALUE
> >>>>
> >
> > The exact same sequence: load, add, store. Still not atomic.
>
> And yet, it appears that *something* changed between Python 2
> and Python 3 such that it *is* atomic:

I don't think that's a guarantee. You might be unable to make it
break, but that doesn't mean it's dependable.

In any case, it's not the GIL that's doing this. It might be a quirk
of the current implementation of the core evaluation loop, or it might
be something unrelated, but whatever it is, removing the GIL wouldn't
change that; and it's certainly no different whether it's a global or
an attribute of an object.

ChrisA

From python at mrabarnett.plus.com  Sun Feb 26 20:40:17 2023
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 27 Feb 2023 01:40:17 +0000
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
 <mailman.2014.1677422439.20444.python-list@python.org>
 <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
Message-ID: <4cf7ff4e-fce2-c58f-2e44-1d20fdf62c78@mrabarnett.plus.com>

On 2023-02-26 16:56, Hen Hanna wrote:
> 
> On Sunday, February 26, 2023 at 6:41:01?AM UTC-8, Thomas Passin wrote:
>> On 2/25/2023 8:12 PM, Hen Hanna wrote: 
>> > 2. the rude guy ('dn') hasn't offered a single word of comment that's directly relevant to it. 
>> > > -------- but he did offer related stuff which he   thinks i should be [grateful] for
> 
> 
>> Please let's stop the ad hominem messages. If someone doesn't like a 
>> particular person's messages, send them to spam or don't read them. If 
>> too many people start to get too rude or personal, remind the whole list 
>> of the guidelines for respectful participation. If you feel that 
>> someone's contribution was especially helpful in part because it was 
>> concise and to the point, you could say that.
> 
> 
> yes.   let's stop...
> 
> 
> If you  (Thomas Passin)   feel that  someone's contribution was especially helpful in part because it was
>   concise and to the point, you could say that.
>                          and pls don't hesitate to reproduce such a comment.
> 
> 
> 
> 
> i'm sure a few others were also rude, but  it was the rudest of them all ('dn')  that   told me to read some [Code of Conduct] document.   -------- Do not EVER  do that  again.
> 
> 
> Another rude guy who "asked"  me why i write in a Hard-to-read way....  Why don't you at least   also make a comment that's  On-Topic  besides  the insulting remark ?
> 
> 
> 
> 
> so far,  i think  Paul Rubin's post (in another thread) was esp. concise, informative, --- but he's also made a comment about       'shunting'  beginners  (questions)     to a concentration camp, and sounded  a bit  like a cold-hearted (or warm-hearted)  Nazi  officer / scientist.
> 
Oh dear. An example of Godwin's Law.

From rosuav at gmail.com  Sun Feb 26 20:46:00 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 27 Feb 2023 12:46:00 +1100
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <4cf7ff4e-fce2-c58f-2e44-1d20fdf62c78@mrabarnett.plus.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
 <mailman.2014.1677422439.20444.python-list@python.org>
 <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
 <4cf7ff4e-fce2-c58f-2e44-1d20fdf62c78@mrabarnett.plus.com>
Message-ID: <CAPTjJmo8A9544c2Wo+B-OLBfgCYfvc=0ib4hwhxnm=1ZYxuLow@mail.gmail.com>

On Mon, 27 Feb 2023 at 12:44, MRAB <python at mrabarnett.plus.com> wrote:
> Oh dear. An example of Godwin's Law.

Yeah, is that finally enough to get this user banned already?

ChrisA

From larry.martell at gmail.com  Sun Feb 26 21:09:44 2023
From: larry.martell at gmail.com (Larry Martell)
Date: Sun, 26 Feb 2023 18:09:44 -0800
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <CAPTjJmo8A9544c2Wo+B-OLBfgCYfvc=0ib4hwhxnm=1ZYxuLow@mail.gmail.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
 <mailman.2014.1677422439.20444.python-list@python.org>
 <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
 <4cf7ff4e-fce2-c58f-2e44-1d20fdf62c78@mrabarnett.plus.com>
 <CAPTjJmo8A9544c2Wo+B-OLBfgCYfvc=0ib4hwhxnm=1ZYxuLow@mail.gmail.com>
Message-ID: <CACwCsY7vFyoEYeRsJJsM7165ZtxfANJOUDfzJuA=rTKEvgSmHQ@mail.gmail.com>

On Sun, Feb 26, 2023 at 5:46 PM Chris Angelico <rosuav at gmail.com> wrote:

> On Mon, 27 Feb 2023 at 12:44, MRAB <python at mrabarnett.plus.com> wrote:
> > Oh dear. An example of Godwin's Law.
>
> Yeah, is that finally enough to get this user banned ?


I hope so

>

From list1 at tompassin.net  Sun Feb 26 21:54:50 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Sun, 26 Feb 2023 21:54:50 -0500
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <4cf7ff4e-fce2-c58f-2e44-1d20fdf62c78@mrabarnett.plus.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
 <mailman.2014.1677422439.20444.python-list@python.org>
 <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
 <4cf7ff4e-fce2-c58f-2e44-1d20fdf62c78@mrabarnett.plus.com>
Message-ID: <cd79d674-07fb-7bea-0543-a9e8c2312e9e@tompassin.net>

On 2/26/2023 8:40 PM, MRAB wrote:
> On 2023-02-26 16:56, Hen Hanna wrote:
>>
>> On Sunday, February 26, 2023 at 6:41:01?AM UTC-8, Thomas Passin wrote:
>>> On 2/25/2023 8:12 PM, Hen Hanna wrote: > 2. the rude guy ('dn') 
>>> hasn't offered a single word of comment that's directly relevant to 
>>> it. > > -------- but he did offer related stuff which he?? thinks i 
>>> should be [grateful] for
>>
>>
>>> Please let's stop the ad hominem messages. If someone doesn't like a 
>>> particular person's messages, send them to spam or don't read them. 
>>> If too many people start to get too rude or personal, remind the 
>>> whole list of the guidelines for respectful participation. If you 
>>> feel that someone's contribution was especially helpful in part 
>>> because it was concise and to the point, you could say that.
>>
>>
>> yes.?? let's stop...
>>
>>
>> If you? (Thomas Passin)?? feel that? someone's contribution was 
>> especially helpful in part because it was
>> ? concise and to the point, you could say that.
>> ???????????????????????? and pls don't hesitate to reproduce such a 
>> comment.
>>
>>
>>
>>
>> i'm sure a few others were also rude, but? it was the rudest of them 
>> all ('dn')? that?? told me to read some [Code of Conduct] document.   
>> -------- Do not EVER? do that? again.
>>
>>
>> Another rude guy who "asked"? me why i write in a Hard-to-read 
>> way....? Why don't you at least?? also make a comment that's  
>> On-Topic? besides? the insulting remark ?
>>
>>
>>
>>
>> so far,? i think? Paul Rubin's post (in another thread) was esp. 
>> concise, informative, --- but he's also made a comment about       
>> 'shunting'? beginners? (questions)???? to a concentration camp, and 
>> sounded? a bit? like a cold-hearted (or warm-hearted)? Nazi? officer / 
>> scientist.
>>
> Oh dear. An example of Godwin's Law.

+1 Nicely put!


From avi.e.gross at gmail.com  Sun Feb 26 22:05:33 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sun, 26 Feb 2023 22:05:33 -0500
Subject: one Liner: Lisprint(x) --> (a, b, c) instead of ['a', 'b', 'c']
In-Reply-To: <a1fb44ad-45a3-49e4-9954-147317f0b86en@googlegroups.com>
References: <439d9918-210a-49c7-b601-5289257dd407n@googlegroups.com>
 <a1fb44ad-45a3-49e4-9954-147317f0b86en@googlegroups.com>
Message-ID: <001801d94a58$5e0cb990$1a262cb0$@gmail.com>

I so rarely need to save a list in python in a form acceptable to LISP but here is a go with no visible recursion needed.

>>> nested = [1, 2, [3, 4, [5, 6, 7], 8], 9]

>>> print(nested)
[1, 2, [3, 4, [5, 6, 7], 8], 9]

# Just converting to a tuple does not change nested lists
>>> print(tuple(nested))
(1, 2, [3, 4, [5, 6, 7], 8], 9)

# But a function that typographically replaces [] with () needs no recursion
>>> def p2b(nested_list): return repr(nested_list).replace('[','(').replace(']',')')

>>> print(p2b(nested))
(1, 2, (3, 4, (5, 6, 7), 8), 9)

People who speak python well do not necessarily lisp.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Hen Hanna
Sent: Sunday, February 26, 2023 4:54 AM
To: python-list at python.org
Subject: Re: one Liner: Lisprint(x) --> (a, b, c) instead of ['a', 'b', 'c']

On Saturday, February 25, 2023 at 11:45:12?PM UTC-8, Hen Hanna wrote:
> def Lisprint(x): print( ' (' + ', '.join(x) + ')' , '\n') 
> 
> a= ' a b c ? def f x if zero? x 0 1 ' 
> a += ' A B C ! just an example ' 
> x= a.split() 
> 
> print(x) 
> Lisprint(x) 
> 
> ['a', 'b', 'c', '?', 'def', 'f', 'x', 'if', 'zero?', 'x', '0', '1', 'A', 'B', 'C', '!', 'just', 'an', 'example'] 
> 
> (a, b, c, ?, def, f, x, if, zero?, x, 0, 1, A, B, C, !, just, an, example)


For nested lists....          impossible to improve   upon  P.Norvig's  code


def Lisprint(x):           print(lispstr(x))

def lispstr(exp):
    "Convert a Python object back into a Lisp-readable string."
    if isinstance(exp, list):
        return '(' + ' '.join(map(lispstr, exp)) + ')' 
    else:
        return str(exp)

a=    ' a b c '
x= a.split()
x +=  [['a', 'b', 'c']]
x +=  x

print(x) 
Lisprint(x) 

['a', 'b', 'c', ['a', 'b', 'c'], 'a', 'b', 'c', ['a', 'b', 'c']]

(a b c (a b c) a b c (a b c))


              ----------   Without the commas,   the visual difference (concision)  is  striking !
-- 
https://mail.python.org/mailman/listinfo/python-list


From knomenet at gmail.com  Sun Feb 26 22:19:55 2023
From: knomenet at gmail.com (Michael Speer)
Date: Sun, 26 Feb 2023 22:19:55 -0500
Subject: Is there a more efficient threading lock?
In-Reply-To: <slrntvn14v.6q7u.jon+usenet@raven.unequivocal.eu>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
 <2c44aec3-bc68-b78e-dfe9-a4e6259d024e@barrys-emacs.org>
 <mailman.2013.1677412409.20444.python-list@python.org>
 <slrntvn14v.6q7u.jon+usenet@raven.unequivocal.eu>
Message-ID: <CABpSurAQcQ3d-XE0far6W+kv3=DGCBftpK+ZpA+vebGX-LGV7A@mail.gmail.com>

I wanted to provide an example that your claimed atomicity is simply wrong,
but I found there is something different in the 3.10+ cpython
implementations.

I've tested the code at the bottom of this message using a few docker
python images, and it appears there is a difference starting in 3.10.0

python3.8
EXPECTED 2560000000
ACTUAL   84533137
python:3.9
EXPECTED 2560000000
ACTUAL   95311773
python:3.10 (.8)
EXPECTED 2560000000
ACTUAL   2560000000

just to see if there was a specific sub-version of 3.10 that added it
python:3.10.0
EXPECTED 2560000000
ACTUAL   2560000000

nope, from the start of 3.10 this is happening

the only difference in the bytecode I see is 3.10 adds SETUP_LOOP and
POP_BLOCK around the for loop

I don't see anything different in the long c code that I would expect would
cause this.

AFAICT the inplace add is null for longs and so should revert to the
long_add that always creates a new integer in x_add

another test
python:3.11
EXPECTED 2560000000
ACTUAL   2560000000

I'm not sure where the difference is at the moment. I didn't see anything
in the release notes given a quick glance.

I do agree that you shouldn't depend on this unless you find a written
guarantee of the behavior, as it is likely an implementation quirk of some
kind

--[code]--

import threading

UPDATES = 10000000
THREADS = 256

vv = 0

def update_x_times( xx ):
    for _ in range( xx ):
        global vv
        vv += 1

def main():
    tts = []
    for _ in range( THREADS ):
        tts.append( threading.Thread( target = update_x_times, args =
(UPDATES,) ) )

    for tt in tts:
        tt.start()

    for tt in tts:
        tt.join()

    print( 'EXPECTED', UPDATES * THREADS )
    print( 'ACTUAL  ', vv )

if __name__ == '__main__':
    main()

On Sun, Feb 26, 2023 at 6:35?PM Jon Ribbens via Python-list <
python-list at python.org> wrote:

> On 2023-02-26, Barry Scott <barry at barrys-emacs.org> wrote:
> > On 25/02/2023 23:45, Jon Ribbens via Python-list wrote:
> >> I think it is the case that x += 1 is atomic but foo.x += 1 is not.
> >
> > No that is not true, and has never been true.
> >
> >:>>> def x(a):
> >:...    a += 1
> >:...
> >:>>>
> >:>>> dis.dis(x)
> >   1           0 RESUME                   0
> >
> >   2           2 LOAD_FAST                0 (a)
> >               4 LOAD_CONST               1 (1)
> >               6 BINARY_OP               13 (+=)
> >              10 STORE_FAST               0 (a)
> >              12 LOAD_CONST               0 (None)
> >              14 RETURN_VALUE
> >:>>>
> >
> > As you can see there are 4 byte code ops executed.
> >
> > Python's eval loop can switch to another thread between any of them.
> >
> > Its is not true that the GIL provides atomic operations in python.
>
> That's oversimplifying to the point of falsehood (just as the opposite
> would be too). And: see my other reply in this thread just now - if the
> GIL isn't making "x += 1" atomic, something else is.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From avi.e.gross at gmail.com  Sun Feb 26 22:36:23 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Sun, 26 Feb 2023 22:36:23 -0500
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
Message-ID: <002401d94a5c$aa20c020$fe624060$@gmail.com>

Only sometimes.

Is it an insult to suggest the question about what quotes to use is quite basic? Python has a wide variety of ways to make a string and if you have text that contains one kind of quote, you can nest it in the other kind. Otherwise, it really does not matter.

And, yes, there are triply quoted strings as well as raw and formatted but to know about these things, you might have to read a manual.

Since you won't, I provided an answer. The answer is that for the meaningless Fizzbuzz homework type of problem which is just ASCII text, it does not matter at all which kind of quote you use as long as what you use matches itself at the end of the string and as long as you use the ASCII versions, not the ones you might make in programs like WORD that have a pair for each.

Oh, by the way, people here use lots of editors to deal with their code including versions derived from vi and emacs and MANY others, so many people here need to be told why you are asking some of your editing questions that do not at first seem to relate. We strive to focus here a bit more on using the language than on how to make your editor do tricks.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Hen Hanna
Sent: Sunday, February 26, 2023 4:07 PM
To: python-list at python.org
Subject: Re: Python 3.10 Fizzbuzz

On Monday, August 29, 2022 at 7:18:22?PM UTC-7, Paul Rubin wrote:
> Just because. 
> 
> from math import gcd 

> def fizz(n: int) -> str: 
>        match gcd(n, 15): 
>               case 3: return "Fizz" 
>               case 5: return "Buzz" 
>               case 15: return "FizzBuzz" 
>               case _: return str(n) 
> 
> for i in range(1,101): 
>             print(fizz(i))


is there any reason to prefer    "    over    '   ?
-- 
https://mail.python.org/mailman/listinfo/python-list


From knomenet at gmail.com  Mon Feb 27 01:26:59 2023
From: knomenet at gmail.com (Michael Speer)
Date: Mon, 27 Feb 2023 01:26:59 -0500
Subject: Is there a more efficient threading lock?
In-Reply-To: <CABpSurAQcQ3d-XE0far6W+kv3=DGCBftpK+ZpA+vebGX-LGV7A@mail.gmail.com>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
 <2c44aec3-bc68-b78e-dfe9-a4e6259d024e@barrys-emacs.org>
 <mailman.2013.1677412409.20444.python-list@python.org>
 <slrntvn14v.6q7u.jon+usenet@raven.unequivocal.eu>
 <CABpSurAQcQ3d-XE0far6W+kv3=DGCBftpK+ZpA+vebGX-LGV7A@mail.gmail.com>
Message-ID: <CABpSurA30nUOiVkVUnXafHhqJkvCOVCkKGOgXUGF+kzrENO3+A@mail.gmail.com>

https://stackoverflow.com/questions/69993959/python-threads-difference-for-3-10-and-others

https://github.com/python/cpython/commit/4958f5d69dd2bf86866c43491caf72f774ddec97

it's a quirk of implementation. the scheduler currently only checks if it
needs to release the gil after the POP_JUMP_IF_FALSE, POP_JUMP_IF_TRUE,
JUMP_ABSOLUTE, CALL_METHOD, CALL_FUNCTION, CALL_FUNCTION_KW, and
CALL_FUNCTION_EX opcodes.

>>> import code
>>> import dis
>>> dis.dis( code.update_x_times )
 10           0 LOAD_GLOBAL              0 (range)
              2 LOAD_FAST                0 (xx)
              4 CALL_FUNCTION            1
##### GIL CAN RELEASE HERE #####
              6 GET_ITER
        >>    8 FOR_ITER                 6 (to 22)
             10 STORE_FAST               1 (_)
 12          12 LOAD_GLOBAL              1 (vv)
             14 LOAD_CONST               1 (1)
             16 INPLACE_ADD
             18 STORE_GLOBAL             1 (vv)
             20 JUMP_ABSOLUTE            4 (to 8)
##### GIL CAN RELEASE HERE (after JUMP_ABSOLUTE points the instruction
counter back to FOR_ITER, but before the interpreter actually jumps to
FOR_ITER again) #####
 10     >>   22 LOAD_CONST               0 (None)
             24 RETURN_VALUE
>>>

due to this, this section:
 12          12 LOAD_GLOBAL              1 (vv)
             14 LOAD_CONST               1 (1)
             16 INPLACE_ADD
             18 STORE_GLOBAL             1 (vv)

is effectively locked/atomic on post-3.10 interpreters, though this is
neither portable nor guaranteed to stay that way into the future


On Sun, Feb 26, 2023 at 10:19?PM Michael Speer <knomenet at gmail.com> wrote:

> I wanted to provide an example that your claimed atomicity is simply
> wrong, but I found there is something different in the 3.10+ cpython
> implementations.
>
> I've tested the code at the bottom of this message using a few docker
> python images, and it appears there is a difference starting in 3.10.0
>
> python3.8
> EXPECTED 2560000000
> ACTUAL   84533137
> python:3.9
> EXPECTED 2560000000
> ACTUAL   95311773
> python:3.10 (.8)
> EXPECTED 2560000000
> ACTUAL   2560000000
>
> just to see if there was a specific sub-version of 3.10 that added it
> python:3.10.0
> EXPECTED 2560000000
> ACTUAL   2560000000
>
> nope, from the start of 3.10 this is happening
>
> the only difference in the bytecode I see is 3.10 adds SETUP_LOOP and
> POP_BLOCK around the for loop
>
> I don't see anything different in the long c code that I would expect
> would cause this.
>
> AFAICT the inplace add is null for longs and so should revert to the
> long_add that always creates a new integer in x_add
>
> another test
> python:3.11
> EXPECTED 2560000000
> ACTUAL   2560000000
>
> I'm not sure where the difference is at the moment. I didn't see anything
> in the release notes given a quick glance.
>
> I do agree that you shouldn't depend on this unless you find a written
> guarantee of the behavior, as it is likely an implementation quirk of some
> kind
>
> --[code]--
>
> import threading
>
> UPDATES = 10000000
> THREADS = 256
>
> vv = 0
>
> def update_x_times( xx ):
>     for _ in range( xx ):
>         global vv
>         vv += 1
>
> def main():
>     tts = []
>     for _ in range( THREADS ):
>         tts.append( threading.Thread( target = update_x_times, args =
> (UPDATES,) ) )
>
>     for tt in tts:
>         tt.start()
>
>     for tt in tts:
>         tt.join()
>
>     print( 'EXPECTED', UPDATES * THREADS )
>     print( 'ACTUAL  ', vv )
>
> if __name__ == '__main__':
>     main()
>
> On Sun, Feb 26, 2023 at 6:35?PM Jon Ribbens via Python-list <
> python-list at python.org> wrote:
>
>> On 2023-02-26, Barry Scott <barry at barrys-emacs.org> wrote:
>> > On 25/02/2023 23:45, Jon Ribbens via Python-list wrote:
>> >> I think it is the case that x += 1 is atomic but foo.x += 1 is not.
>> >
>> > No that is not true, and has never been true.
>> >
>> >:>>> def x(a):
>> >:...    a += 1
>> >:...
>> >:>>>
>> >:>>> dis.dis(x)
>> >   1           0 RESUME                   0
>> >
>> >   2           2 LOAD_FAST                0 (a)
>> >               4 LOAD_CONST               1 (1)
>> >               6 BINARY_OP               13 (+=)
>> >              10 STORE_FAST               0 (a)
>> >              12 LOAD_CONST               0 (None)
>> >              14 RETURN_VALUE
>> >:>>>
>> >
>> > As you can see there are 4 byte code ops executed.
>> >
>> > Python's eval loop can switch to another thread between any of them.
>> >
>> > Its is not true that the GIL provides atomic operations in python.
>>
>> That's oversimplifying to the point of falsehood (just as the opposite
>> would be too). And: see my other reply in this thread just now - if the
>> GIL isn't making "x += 1" atomic, something else is.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>

From rosuav at gmail.com  Mon Feb 27 01:37:32 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 27 Feb 2023 17:37:32 +1100
Subject: Is there a more efficient threading lock?
In-Reply-To: <CABpSurA30nUOiVkVUnXafHhqJkvCOVCkKGOgXUGF+kzrENO3+A@mail.gmail.com>
References: <CANc-5Uz1GwFrw9Rw134A3maCs-FK06VXFygJD6W7NFGEKy20pA@mail.gmail.com>
 <mailman.1993.1677340365.20444.python-list@python.org>
 <87mt51sfc6.fsf@nightsong.com>
 <slrntvkv43.6q7u.jon+usenet@raven.unequivocal.eu>
 <87ilfps9at.fsf@nightsong.com>
 <slrntvl7d0.6q7u.jon+usenet@raven.unequivocal.eu>
 <2c44aec3-bc68-b78e-dfe9-a4e6259d024e@barrys-emacs.org>
 <mailman.2013.1677412409.20444.python-list@python.org>
 <slrntvn14v.6q7u.jon+usenet@raven.unequivocal.eu>
 <CABpSurAQcQ3d-XE0far6W+kv3=DGCBftpK+ZpA+vebGX-LGV7A@mail.gmail.com>
 <CABpSurA30nUOiVkVUnXafHhqJkvCOVCkKGOgXUGF+kzrENO3+A@mail.gmail.com>
Message-ID: <CAPTjJmrwqUQqL=Kn599ojb-MvBnJgxubfwT0kWVxUaCiwXWdrg@mail.gmail.com>

On Mon, 27 Feb 2023 at 17:28, Michael Speer <knomenet at gmail.com> wrote:
>
> https://github.com/python/cpython/commit/4958f5d69dd2bf86866c43491caf72f774ddec97
>
> it's a quirk of implementation. the scheduler currently only checks if it
> needs to release the gil after the POP_JUMP_IF_FALSE, POP_JUMP_IF_TRUE,
> JUMP_ABSOLUTE, CALL_METHOD, CALL_FUNCTION, CALL_FUNCTION_KW, and
> CALL_FUNCTION_EX opcodes.
>

Oh now that is VERY interesting. It's a quirk of implementation, yes,
but there's a reason for it; a bug being solved. The underlying
guarantee about __exit__ should be considered to be defined behaviour,
meaning that the precise quirk might not be relevant even though the
bug has to remain fixed in all future versions. But I'd also note here
that, if it can be absolutely 100% guaranteed that the GIL will be
released and signals checked on a reasonable interval, there's no
particular reason to state that signals are checked after every single
Python bytecode. (See the removed comment about empty loops, which
would have been a serious issue and is probably why the backward jump
rule exists.)

So it wouldn't be too hard for a future release of Python to mandate
atomicity of certain specific operations. Obviously it'd require
buy-in from other implementations, but it would be rather convenient
if, subject to some very tight rules like "only when adding integers
onto core data types" etc, a simple statement like "x.y += 1" could
actually be guaranteed to take place atomically.

Though it's still probably not as useful as you might hope. In C, if I
can do "int id = counter++;" atomically, it would guarantee me a new
ID that no other thread could ever have. But in Python, that increment
operation doesn't give you the result, so all it's really useful for
is statistics on operations done. Still, that in itself could be of
value in quite a few situations.

In any case, though, this isn't something to depend upon at the moment.

ChrisA

From roel at roelschroeven.net  Mon Feb 27 03:51:18 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Mon, 27 Feb 2023 09:51:18 +0100
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
 <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
Message-ID: <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>

Op 26/02/2023 om 6:53 schreef Hen Hanna:
> > There are some similarities between Python and Lisp-family 
> > languages, but really Python is its own thing. 
>
>
>     Scope (and extent ?) of   variables is one reminder that  Python is not Lisp
>
> ?????for    i     in      range(5):??    print( i )
>                       .........
> ?????print( i )
>
> ideally, after the FOR loop is done,  the (local) var  i should also disappear.
> (this almost caused a bug for me)
I wouldn't say "i *should* also disappear". There is no big book of 
programming language design with rules like that that all languages have 
to follow. Different languages have different behavior. In some 
languages, for/if/while statements introduce a new scope, in other 
languages they don't. In Python, they don't. I won't say one is better 
than the other; they're just different.

-- 
"Most of us, when all is said and done, like what we like and make up
reasons for it afterwards."
         -- Soren F. Petersen


From inhahe at gmail.com  Mon Feb 27 03:56:22 2023
From: inhahe at gmail.com (inhahe)
Date: Mon, 27 Feb 2023 03:56:22 -0500
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
 <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
 <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
Message-ID: <CAPeSgWrRXcjKS8FPhXNZQDffT2UhUFwaooSU9LfU92s1Q=KPCA@mail.gmail.com>

On Mon, Feb 27, 2023 at 3:52?AM Roel Schroeven <roel at roelschroeven.net>
wrote:

> Op 26/02/2023 om 6:53 schreef Hen Hanna:
> > > There are some similarities between Python and Lisp-family
> > > languages, but really Python is its own thing.
> >
> >
> >     Scope (and extent ?) of   variables is one reminder that  Python is
> not Lisp
> >
> > for    i     in      range(5):     print( i )
> >                       .........
> > print( i )
> >
> > ideally, after the FOR loop is done,  the (local) var  i should also
> disappear.
> > (this almost caused a bug for me)
> I wouldn't say "i *should* also disappear". There is no big book of
> programming language design with rules like that that all languages have
> to follow. Different languages have different behavior. In some
> languages, for/if/while statements introduce a new scope, in other
> languages they don't. In Python, they don't. I won't say one is better
> than the other; they're just different.
>
> --
>
>
I'm not sure, but I think I remember this was actually a bug in the
interpreter, and presumably they didn't fix it because they didn't want to
break backward compatibility?

From inhahe at gmail.com  Mon Feb 27 04:05:56 2023
From: inhahe at gmail.com (inhahe)
Date: Mon, 27 Feb 2023 04:05:56 -0500
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <CAPeSgWrRXcjKS8FPhXNZQDffT2UhUFwaooSU9LfU92s1Q=KPCA@mail.gmail.com>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
 <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
 <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
 <CAPeSgWrRXcjKS8FPhXNZQDffT2UhUFwaooSU9LfU92s1Q=KPCA@mail.gmail.com>
Message-ID: <CAPeSgWp-jkzGYAfFCL86KSchaGR0Gq9MDjiOApig_6Da+0sghA@mail.gmail.com>

On Mon, Feb 27, 2023 at 3:56?AM inhahe <inhahe at gmail.com> wrote:

>
>
> On Mon, Feb 27, 2023 at 3:52?AM Roel Schroeven <roel at roelschroeven.net>
> wrote:
>
>> Op 26/02/2023 om 6:53 schreef Hen Hanna:
>> > > There are some similarities between Python and Lisp-family
>> > > languages, but really Python is its own thing.
>> >
>> >
>> >     Scope (and extent ?) of   variables is one reminder that  Python is
>> not Lisp
>> >
>> > for    i     in      range(5):     print( i )
>> >                       .........
>> > print( i )
>> >
>> > ideally, after the FOR loop is done,  the (local) var  i should also
>> disappear.
>> > (this almost caused a bug for me)
>> I wouldn't say "i *should* also disappear". There is no big book of
>> programming language design with rules like that that all languages have
>> to follow. Different languages have different behavior. In some
>> languages, for/if/while statements introduce a new scope, in other
>> languages they don't. In Python, they don't. I won't say one is better
>> than the other; they're just different.
>>
>> --
>>
>>
> I'm not sure, but I think I remember this was actually a bug in the
> interpreter, and presumably they didn't fix it because they didn't want to
> break backward compatibility?
>
>
Maybe I'm thinking of a variable scope leak after list comprehensions.

From roel at roelschroeven.net  Mon Feb 27 04:07:54 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Mon, 27 Feb 2023 10:07:54 +0100
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <CAPeSgWrRXcjKS8FPhXNZQDffT2UhUFwaooSU9LfU92s1Q=KPCA@mail.gmail.com>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
 <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
 <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
 <CAPeSgWrRXcjKS8FPhXNZQDffT2UhUFwaooSU9LfU92s1Q=KPCA@mail.gmail.com>
Message-ID: <fc7551f9-8470-6c1f-26fb-1431a6330a1d@roelschroeven.net>

Op 27/02/2023 om 9:56 schreef inhahe:
> On Mon, Feb 27, 2023 at 3:52?AM Roel Schroeven <roel at roelschroeven.net>
> wrote:
>
> > Op 26/02/2023 om 6:53 schreef Hen Hanna:
> > > > There are some similarities between Python and Lisp-family
> > > > languages, but really Python is its own thing.
> > >
> > >
> > >     Scope (and extent ?) of   variables is one reminder that  Python is
> > not Lisp
> > >
> > > for    i     in      range(5):     print( i )
> > >                       .........
> > > print( i )
> > >
> > > ideally, after the FOR loop is done,  the (local) var  i should also
> > disappear.
> > > (this almost caused a bug for me)
> > I wouldn't say "i *should* also disappear". There is no big book of
> > programming language design with rules like that that all languages have
> > to follow. Different languages have different behavior. In some
> > languages, for/if/while statements introduce a new scope, in other
> > languages they don't. In Python, they don't. I won't say one is better
> > than the other; they're just different.
> >
> > --
> >
> >
> I'm not sure, but I think I remember this was actually a bug in the
> interpreter, and presumably they didn't fix it because they didn't want to
> break backward compatibility?
I'm guessing you're thinking about variables leaking out of list 
comprehensions. I seem to remember (but I could be wrong) it was a 
design mistake rather than a bug in the code, but in any case it's been 
fixed now (in the 2 to 3 transition, I think).

For loops (and while loops, and if statements) not introducing a new 
scope is a deliberate decision and is not subject to change.

-- 
"Ever since I learned about confirmation bias, I've been seeing
it everywhere."
         -- Jon Ronson


From Karsten.Hilbert at gmx.net  Mon Feb 27 06:32:05 2023
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Mon, 27 Feb 2023 12:32:05 +0100
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
 <mailman.2014.1677422439.20444.python-list@python.org>
 <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
Message-ID: <Y/yUteNK5tJFrigG@hermes.hilbert.loc>

Am Sun, Feb 26, 2023 at 08:56:28AM -0800 schrieb Hen Hanna:

> so far,  i think  Paul Rubin's post (in another thread) was
> esp. concise, informative, --- but he's also made a comment
> about       'shunting'  beginners  (questions)     to a
> concentration camp, and sounded  a bit  like a cold-hearted
> (or warm-hearted)  Nazi  officer / scientist.

Now, I have a lot of sympathy -- not least from a
professional point of view -- and see quite some leeway for
people acting neuro-atypically, but the last line of the
above really isn't necessary to be read on this list.

Best,
Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B

From greg.ewing at canterbury.ac.nz  Mon Feb 27 07:41:07 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Feb 2023 01:41:07 +1300
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <mailman.2034.1677488879.20444.python-list@python.org>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
 <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
 <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
 <CAPeSgWrRXcjKS8FPhXNZQDffT2UhUFwaooSU9LfU92s1Q=KPCA@mail.gmail.com>
 <fc7551f9-8470-6c1f-26fb-1431a6330a1d@roelschroeven.net>
 <mailman.2034.1677488879.20444.python-list@python.org>
Message-ID: <k63mn5FdiqjU1@mid.individual.net>

On 27/02/23 10:07 pm, Roel Schroeven wrote:
> I'm guessing you're thinking about variables leaking out of list 
> comprehensions. I seem to remember (but I could be wrong) it was a 
> design mistake rather than a bug in the code, but in any case it's been 
> fixed now (in the 2 to 3 transition, I think).

The semantics of list comprehensions was originally defined
in terms of nested for loops. A consequence was that the loop
variables ended up in the local scope just as with ordinary for
loops. Later it was decided to change that.

-- 
Greg

From torriem at gmail.com  Mon Feb 27 10:22:14 2023
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 27 Feb 2023 08:22:14 -0700
Subject: Rob Cliffe should stop sending me rude email messages.
In-Reply-To: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
References: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
Message-ID: <b891d90b-6075-2b39-5323-0ee544ea8aa1@gmail.com>

I've been putting off sending this message for days, but the list noise
level is now to the point that it has to be said.

Often it is better to contact someone directly and privately rather than
publicly embarrass them by calling them out. You've made it clear,
however, that publicly calling you out is necessary. No doubt you will
think my post rude as well, even though the tone is moderate and
deliberate.  Sometimes things need to said and others need to be asked
to make changes. That's just part of communication in real, grown-up life.

Everyone that's responded to you has patiently attempted to answer your
questions and engage with you despite your unorthodox and very difficult
communications style.  I can assure you that not one person who's
replied to you has been rude or insulting, yet I cannot say the same
about your own disparaging comments in reply.  The only ad homimems I've
seen have come from you.

We are frustrated and exasperated with your unwillingness to read,
learn, and understand, yes, definitely!  Although your posts are quite a
bit less frustrating than those trying to turn Python into Java.  We can
probably handle trying to turn Python into LISP! :)

Do you understand why your posts have been causing frustration?  This is
an existing community that you've chosen to join.  Many of the people
you've insulted here, including dn have been participating and helpfully
contributing to this list for many years.

Please stop posting messages about how you think people have been rude
to you. Besides being off-topic they are simply false.

From mats at wichmann.us  Mon Feb 27 11:01:26 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Mon, 27 Feb 2023 09:01:26 -0700
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
Message-ID: <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>

On 2/26/23 14:07, Hen Hanna wrote:
> On Monday, August 29, 2022 at 7:18:22?PM UTC-7, Paul Rubin wrote:
>> Just because.
>>
>> from math import gcd
> 
>> def fizz(n: int) -> str:
>>         match gcd(n, 15):
>>                case 3: return "Fizz"
>>                case 5: return "Buzz"
>>                case 15: return "FizzBuzz"
>>                case _: return str(n)
>>
>> for i in range(1,101):
>>              print(fizz(i))
> 
> 
> is there any reason to prefer    "    over    '   ?

If you intend to run Black on your code to ensure consistent formatting, 
you may as well learn to prefer double quotes, because it's going to 
convert single to double (or: don't learn, and set your IDE to "convert 
on save" and don't think about it...)

As has already been mentioned, syntactically there is no difference.




From list1 at tompassin.net  Mon Feb 27 11:08:22 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 27 Feb 2023 11:08:22 -0500
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
Message-ID: <25df924e-8e07-9b21-f75d-c9e194f915b9@tompassin.net>

On 2/27/2023 11:01 AM, Mats Wichmann wrote:
> On 2/26/23 14:07, Hen Hanna wrote:
>> On Monday, August 29, 2022 at 7:18:22?PM UTC-7, Paul Rubin wrote:
>>> Just because.
>>>
>>> from math import gcd
>>
>>> def fizz(n: int) -> str:
>>> ??????? match gcd(n, 15):
>>> ?????????????? case 3: return "Fizz"
>>> ?????????????? case 5: return "Buzz"
>>> ?????????????? case 15: return "FizzBuzz"
>>> ?????????????? case _: return str(n)
>>>
>>> for i in range(1,101):
>>> ???????????? print(fizz(i))
>>
>>
>> is there any reason to prefer??? "??? over??? '?? ?
> 
> If you intend to run Black on your code to ensure consistent formatting, 
> you may as well learn to prefer double quotes, because it's going to 
> convert single to double (or: don't learn, and set your IDE to "convert 
> on save" and don't think about it...)
> 
> As has already been mentioned, syntactically there is no difference.

I prefer single quotes because they are easier to type.


From grant.b.edwards at gmail.com  Mon Feb 27 11:17:28 2023
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 27 Feb 2023 08:17:28 -0800 (PST)
Subject: Rob Cliffe should stop sending me rude email messages.
References: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
 <b891d90b-6075-2b39-5323-0ee544ea8aa1@gmail.com>
Message-ID: <63fcd798.020a0220.4e8d2.3b9c@mx.google.com>

On 2023-02-27, Michael Torrie <torriem at gmail.com> wrote:

> I've been putting off sending this message for days, but the list noise
> level is now to the point that it has to be said.

Ah, I've finially realized why some of those threads have seemed so
disjointed to me. Years ago, I plonked all posts which are (like Hen
Hanna's) submitted via Googole Groups.

I highly recommend it.

FWIW, here's the "score" rule for doing that with srln:

    Score:: =-9999
      Message-ID: .*googlegroups.com

--
Grant

From boblatest at yahoo.com  Mon Feb 27 10:23:27 2023
From: boblatest at yahoo.com (Robert Latest)
Date: 27 Feb 2023 15:23:27 GMT
Subject: Line continuation and comments
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <mailman.1915.1677079386.20444.python-list@python.org>
Message-ID: <k6407fFf08sU1@mid.individual.net>

Paul Bryan wrote:
> Adding to this, there should be no reason now in recent versions of
> Python to ever use line continuation. Black goes so far as to state
> "backslashes are bad and should never be used":
>
> https://black.readthedocs.io/en/stable/the_black_code_style/
  future_style.html#using-backslashes-for-with-statements

Then I wonder how Mr. Black would go about these long "dot chaining"
expressions that packages like pandas and sqlalchemy require.

From boblatest at yahoo.com  Mon Feb 27 10:24:12 2023
From: boblatest at yahoo.com (Robert Latest)
Date: 27 Feb 2023 15:24:12 GMT
Subject: Line continuation and comments
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
Message-ID: <k6408sFf08sU2@mid.individual.net>

Edmondo Giovannozzi wrote:
> Il giorno mercoled? 22 febbraio 2023 alle 09:50:14 UTC+1 Robert Latest ha
> scritto:
>> I found myself building a complicated logical condition with many ands and
>> ors which I made more manageable by putting the various terms on individual
>> lines and breaking them with the "\" line continuation character. In this
>> context it would have been nice to be able to add comments to lines terms
>> which of course isn't possible because the backslash must be the last
>> character on the line. 
>> 
>> Question: If the Python syntax were changed to allow comments after
>> line-ending 
>> backslashes, would it break any existing code? I can't think of an example.
>
> Well you can if you use parenthesis like in:
> x = 5
> a = (x > 3 and
> #     x < 21 or
>      x > 100
>      )
> You don't need the "\" to continue a line in this case

I like that. Never thought of it.

From boblatest at yahoo.com  Mon Feb 27 10:35:03 2023
From: boblatest at yahoo.com (Robert Latest)
Date: 27 Feb 2023 15:35:03 GMT
Subject: Line continuation and comments
References: <k5m39hF9sj0U1@mid.individual.net>
 <d87b5e5b-e2c1-40df-bd8a-e765c623692cn@googlegroups.com>
 <70d1a03582ddfec7336412d61c1a874150a05771.camel@anode.ca>
 <mailman.1915.1677079386.20444.python-list@python.org>
 <k6407fFf08sU1@mid.individual.net>
Message-ID: <k640t7Ff08sU3@mid.individual.net>

Robert Latest wrote:
> Paul Bryan wrote:
>> Adding to this, there should be no reason now in recent versions of
>> Python to ever use line continuation. Black goes so far as to state
>> "backslashes are bad and should never be used":
>>
>> https://black.readthedocs.io/en/stable/the_black_code_style/
>   future_style.html#using-backslashes-for-with-statements
>
> Then I wonder how Mr. Black would go about these long "dot chaining"
> expressions that packages like pandas and sqlalchemy require.

Just found out that parentheses work there, too.

From gweatherby at uchc.edu  Mon Feb 27 10:31:28 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Mon, 27 Feb 2023 15:31:28 +0000
Subject: Python type system
Message-ID: <SJ0PR14MB5864C4006519AC9B16CD7CA4B9AF9@SJ0PR14MB5864.namprd14.prod.outlook.com>

When I first started transitioning to Python as a Perl replacement, with my Java/C++ baggage, I thought Pythnon had some loosey-goosey type system. I thought int() and str() were casts, not constructors. I now realize Python has a great strong type system. Duck typing. If it walks like a duck, quacks like a duck, it?s probably a duck. (C++ added the auto keyword at some point to get analogous behavior).

Beyond Python, I?ve also found duck typing useful in real life. If it walks like a troll, quacks like a troll ?



From ethan at stoneleaf.us  Mon Feb 27 12:35:51 2023
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 27 Feb 2023 09:35:51 -0800
Subject: Hen Hanna & google groups
Message-ID: <abb274f6-9783-6183-f7d3-abe5606c3dd8@stoneleaf.us>

Greetings, all!

As has been stated, Hen Hanna is posting through Google Groups, over which the Python List moderators have zero control.

The only thing we can do, and which has now been done, is not allow those posts in to the Python List.

--
~Ethan~
Moderator

From list1 at tompassin.net  Mon Feb 27 13:06:00 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 27 Feb 2023 13:06:00 -0500
Subject: Hen Hanna & google groups
In-Reply-To: <abb274f6-9783-6183-f7d3-abe5606c3dd8@stoneleaf.us>
References: <abb274f6-9783-6183-f7d3-abe5606c3dd8@stoneleaf.us>
Message-ID: <1b726c49-01e4-e776-7a66-2a9d1ee49b5c@tompassin.net>

On 2/27/2023 12:35 PM, Ethan Furman wrote:
> Greetings, all!
> 
> As has been stated, Hen Hanna is posting through Google Groups, over 
> which the Python List moderators have zero control.
> 
> The only thing we can do, and which has now been done, is not allow 
> those posts in to the Python List.
> 
> -- 
> ~Ethan~
> Moderator

Thank you.


From rosuav at gmail.com  Mon Feb 27 13:07:45 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 28 Feb 2023 05:07:45 +1100
Subject: Python type system
In-Reply-To: <SJ0PR14MB5864C4006519AC9B16CD7CA4B9AF9@SJ0PR14MB5864.namprd14.prod.outlook.com>
References: <SJ0PR14MB5864C4006519AC9B16CD7CA4B9AF9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Message-ID: <CAPTjJmrtnQCf=OezOxVNs-2bxRXRiK4XAZc2w5yZ4eZj45MyUA@mail.gmail.com>

On Tue, 28 Feb 2023 at 03:29, Weatherby,Gerard <gweatherby at uchc.edu> wrote:
> Beyond Python, I?ve also found duck typing useful in real life. If it walks like a troll, quacks like a troll ?
>
... regenerates limbs like a troll...

ChrisA

From avi.e.gross at gmail.com  Mon Feb 27 13:40:42 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 13:40:42 -0500
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
 <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
 <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
Message-ID: <002801d94ada$ff7fecc0$fe7fc640$@gmail.com>

I am not a big fan of religions or philosophies that say a road to salvation is for the "I" to disappear.

But on a more serious note, as Roel said, there is NO RULE being violated unless the documentation of the language says it is supposed to do something different.

There are many excellent reasons to keep the final value of a loop variable around. On the other hand, there are also many good reasons to make such variables be totally kept within the context of the loop so they can mask a variable with the same name only temporarily within the loop.

Neither choice is wrong as long as it is applied consistently.

Now, having said that, does python allow you to in some way over-ride the behavior?

Well, first, you can simply choose an odd name like local______loopy___variable that is not used elsewhere in your code, except perhaps in the next loop downstream where it is re-initialized.

You can also use "del Variable" or reset it to null or something in every way you can exit the loop such as before a break or in an "else" clause if it bothers you.

inhahe <inhahe at gmail.com> made the point that this may not have been the original intent for python and may be a sort of bug that it is too late to fix. Perhaps so, but insisting it be changed now is far from a simple request as I bet some people depend on the feature. True, it could be straightforward to recode any existing loops to update a secondary variable at the top of each loop that is declared before the loop and persists after the loop. 

Alas, that might force some to use the dreaded semicolon!

Final note is to look at something like the "with" statement in python as a context manager where it normally allows the resource to be closed or removed at the end. Of course you can set up an object that does not do the expected closure and preserves something, but generally what is wanted is to make sure the context exits gracefully.

Avi

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Roel Schroeven
Sent: Monday, February 27, 2023 3:51 AM
To: python-list at python.org
Subject: Re: it seems like a few weeks ago... but actually it was more like 30 years ago that i was programming in C, and

Op 26/02/2023 om 6:53 schreef Hen Hanna:
> > There are some similarities between Python and Lisp-family 
> > languages, but really Python is its own thing.
>
>
>     Scope (and extent ?) of   variables is one reminder that  Python is not Lisp
>
> ?????for    i     in      range(5):??    print( i )
>                       .........
> ?????print( i )
>
> ideally, after the FOR loop is done,  the (local) var  i should also disappear.
> (this almost caused a bug for me)
I wouldn't say "i *should* also disappear". There is no big book of programming language design with rules like that that all languages have to follow. Different languages have different behavior. In some languages, for/if/while statements introduce a new scope, in other languages they don't. In Python, they don't. I won't say one is better than the other; they're just different.

--
"Most of us, when all is said and done, like what we like and make up reasons for it afterwards."
         -- Soren F. Petersen

--
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Mon Feb 27 14:15:36 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 14:15:36 -0500
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <Y/yUteNK5tJFrigG@hermes.hilbert.loc>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
 <mailman.2014.1677422439.20444.python-list@python.org>
 <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
 <Y/yUteNK5tJFrigG@hermes.hilbert.loc>
Message-ID: <005d01d94adf$df482530$9dd86f90$@gmail.com>

Karsten,

There are limits to the disruption a group should tolerate even from people
who may need some leeway.

I wonder if Hen Hanna has any idea that some of the people he is saying this
to lost most of their family in the Holocaust and had parents who barely
survived passing through multiple concentration camps, I doubt he would
change his words or attitude in the slightest as some of his other gems
indicate a paranoid view of the world at best.

It is disproportionate to call everyone a Nazi at the slightest imagined
slight. But we are not here in this forum to discuss world affairs or
politics or how to replace python with the same language they have been
using and likely abusing. Like every resource, it is best used as intended
and that tends to mean not treating all the recipients as being willing to
receive every thought you have had since breakfast followed by demanding
everyone stop responding to him privately or in public or disagreeing in any
way.

I apologize for my part in even bothering to try to help him as it clearly
is a thankless task and a huge waste of andwidth.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Karsten Hilbert
Sent: Monday, February 27, 2023 6:32 AM
To: python-list at python.org
Subject: Re: TypeError: can only concatenate str (not "int") to str

Am Sun, Feb 26, 2023 at 08:56:28AM -0800 schrieb Hen Hanna:

> so far,  i think  Paul Rubin's post (in another thread) was esp. 
> concise, informative, --- but he's also made a comment
> about       'shunting'  beginners  (questions)     to a
> concentration camp, and sounded  a bit  like a cold-hearted (or 
> warm-hearted)  Nazi  officer / scientist.

Now, I have a lot of sympathy -- not least from a professional point of view
-- and see quite some leeway for people acting neuro-atypically, but the
last line of the above really isn't necessary to be read on this list.

Best,
Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Mon Feb 27 14:48:29 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 27 Feb 2023 14:48:29 -0500
Subject: TypeError: can only concatenate str (not "int") to str
In-Reply-To: <005d01d94adf$df482530$9dd86f90$@gmail.com>
References: <0245c58b-3af4-91fd-2012-e03c382e4ba3@DancesWithMice.info>
 <f1f4c115-9743-437f-b2c4-fdf43b8dae31n@googlegroups.com>
 <mailman.1989.1677296483.20444.python-list@python.org>
 <8912032a-2894-4ce6-bd25-0fbcc07b3a76n@googlegroups.com>
 <87edqds96y.fsf@nightsong.com> <k5votpFpjveU1@mid.individual.net>
 <ee387dac-c6da-a45a-9f3f-e61c29484937@tompassin.net>
 <063cb1f3-f328-4722-aaa1-b11a0fad7f8an@googlegroups.com>
 <mailman.2014.1677422439.20444.python-list@python.org>
 <35033fa4-cb8d-4595-9b34-884a3b9661e2n@googlegroups.com>
 <Y/yUteNK5tJFrigG@hermes.hilbert.loc>
 <005d01d94adf$df482530$9dd86f90$@gmail.com>
Message-ID: <27a2e7a5-a4ed-de2c-bcb9-f56f8650cdb1@tompassin.net>

On 2/27/2023 2:15 PM, avi.e.gross at gmail.com wrote:
> Karsten,
> 
> There are limits to the disruption a group should tolerate even from people
> who may need some leeway.
> 
> I wonder if Hen Hanna has any idea that some of the people he is saying this
> to lost most of their family in the Holocaust and had parents who barely
> survived passing through multiple concentration camps, I doubt he would
> change his words or attitude in the slightest as some of his other gems
> indicate a paranoid view of the world at best.

This whole paragraph is about a person with speculation on his views. 
Please let's deprecate writing about people this way :)

> 
> It is disproportionate to call everyone a Nazi at the slightest imagined
> slight. But we are not here in this forum to discuss world affairs or
> politics or how to replace python with the same language they have been
> using and likely abusing. Like every resource, it is best used as intended
> and that tends to mean not treating all the recipients as being willing to
> receive every thought you have had since breakfast followed by demanding
> everyone stop responding to him privately or in public or disagreeing in any
> way.
> 
> I apologize for my part in even bothering to try to help him as it clearly
> is a thankless task and a huge waste of andwidth.

Again, this paragraph is commentary and speculation about another 
poster. It's so easy to write things like this without even realizing 
we're doing it (I'm not immune).

> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
> Behalf Of Karsten Hilbert
> Sent: Monday, February 27, 2023 6:32 AM
> To: python-list at python.org
> Subject: Re: TypeError: can only concatenate str (not "int") to str
> 
> Am Sun, Feb 26, 2023 at 08:56:28AM -0800 schrieb Hen Hanna:
> 
>> so far,  i think  Paul Rubin's post (in another thread) was esp.
>> concise, informative, --- but he's also made a comment
>> about       'shunting'  beginners  (questions)     to a
>> concentration camp, and sounded  a bit  like a cold-hearted (or
>> warm-hearted)  Nazi  officer / scientist.
> 
> Now, I have a lot of sympathy -- not least from a professional point of view
> -- and see quite some leeway for people acting neuro-atypically, but the
> last line of the above really isn't necessary to be read on this list.
> 
> Best,
> Karsten
> --
> GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
> --
> https://mail.python.org/mailman/listinfo/python-list
> 


From bowman at montana.com  Mon Feb 27 15:16:43 2023
From: bowman at montana.com (rbowman)
Date: 27 Feb 2023 20:16:43 GMT
Subject: Python 3.10 Fizzbuzz
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <25df924e-8e07-9b21-f75d-c9e194f915b9@tompassin.net>
 <mailman.2038.1677514114.20444.python-list@python.org>
Message-ID: <k64hdbFhbvkU3@mid.individual.net>

On Mon, 27 Feb 2023 11:08:22 -0500, Thomas Passin wrote:

> I prefer single quotes because they are easier to type.

There is that. JavaScript makes me lazy and C# slaps my knuckles with a 
steel edged ruler. 

From bowman at montana.com  Mon Feb 27 15:20:44 2023
From: bowman at montana.com (rbowman)
Date: 27 Feb 2023 20:20:44 GMT
Subject: Python 3.10 Fizzbuzz
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
Message-ID: <k64hksFhbvkU4@mid.individual.net>

On Mon, 27 Feb 2023 09:01:26 -0700, Mats Wichmann wrote:


> If you intend to run Black on your code to ensure consistent formatting,
> you may as well learn to prefer double quotes, because it's going to
> convert single to double (or: don't learn, and set your IDE to "convert
> on save" and don't think about it...)

I'd never heard of Black.

"By using Black, you agree to cede control over minutiae of hand-
formatting. In return, Black gives you speed, determinism, and freedom 
from pycodestyle nagging about formatting. You will save time and mental 
energy for more important matters."

Somehow I don't think we would get along very well. I'm a little on the 
opinionated side myself.

From ethan at stoneleaf.us  Mon Feb 27 16:04:06 2023
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 27 Feb 2023 13:04:06 -0800
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <k64hksFhbvkU4@mid.individual.net>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
Message-ID: <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>

On 2/27/23 12:20, rbowman wrote:

 > "By using Black, you agree to cede control over minutiae of hand-
 > formatting. In return, Black gives you speed, determinism, and freedom
 > from pycodestyle nagging about formatting. You will save time and mental
 > energy for more important matters."
 >
 > Somehow I don't think we would get along very well. I'm a little on the
 > opinionated side myself.

I personally cannot stand Black.  It feels like every major choice it makes (and some minor ones) are exactly the 
opposite of the choice I make.

--
~Ethan~

From jenkris at tutanota.com  Mon Feb 27 18:11:10 2023
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 28 Feb 2023 00:11:10 +0100 (CET)
Subject: How to escape strings for re.finditer?
Message-ID: <NPK8DYA--3-9@tutanota.com>

When matching a string against a longer string, where both strings have spaces in them, we need to escape the spaces.? 

This works (no spaces):

import re
example = 'abcdefabcdefabcdefg'
find_string = "abc"
for match in re.finditer(find_string, example):
??? print(match.start(), match.end())

That gives me the start and end character positions, which is what I want.?

However, this does not work:

import re
example = re.escape('X - cty_degrees + 1 + qq')
find_string = re.escape('cty_degrees + 1')
for match in re.finditer(find_string, example):
??? print(match.start(), match.end())

I?ve tried several other attempts based on my reseearch, but still no match.?

I don?t have much experience with regex, so I hoped a reg-expert might help.?

Thanks,

Jen


From oscar.j.benjamin at gmail.com  Mon Feb 27 18:42:03 2023
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Mon, 27 Feb 2023 23:42:03 +0000
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
Message-ID: <CAHVvXxTCtu6ugipcB1B-5v4r7=ZRhJgDWOffEhxGXH9RUfuiQw@mail.gmail.com>

On Mon, 27 Feb 2023 at 21:06, Ethan Furman <ethan at stoneleaf.us> wrote:
>
> On 2/27/23 12:20, rbowman wrote:
>
>  > "By using Black, you agree to cede control over minutiae of hand-
>  > formatting. In return, Black gives you speed, determinism, and freedom
>  > from pycodestyle nagging about formatting. You will save time and mental
>  > energy for more important matters."
>  >
>  > Somehow I don't think we would get along very well. I'm a little on the
>  > opinionated side myself.
>
> I personally cannot stand Black.  It feels like every major choice it makes (and some minor ones) are exactly the
> opposite of the choice I make.

I agree partially. There are two types of decisions black makes:

1. Leave the code alone because it seems okay or make small modifications.
2. Reformat the code because it violates some generic rule (like line
too long or something).

I've recently tried Black and mostly for my code it seems to go with 1
(code looks okay). There might be some minor changes like double vs
single quotes but I really don't care about those. In that sense me
and Black seem to agree.

However I have also reviewed code where it is clear that the author
has used black and their code came under case 2. In that case Black
seems to produce awful things. What I can't understand is someone
accepting the awful rewrite rather than just fixing the code. Treating
Black almost like a linter makes sense to me but accepting the
rewrites that it offers for bad code does not.

--
Oscar

From python at mrabarnett.plus.com  Mon Feb 27 18:45:45 2023
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 27 Feb 2023 23:45:45 +0000
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPK8DYA--3-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com>
Message-ID: <22223151-d4fe-792b-c73f-319586efcdd9@mrabarnett.plus.com>

On 2023-02-27 23:11, Jen Kris via Python-list wrote:
> When matching a string against a longer string, where both strings have spaces in them, we need to escape the spaces.
> 
> This works (no spaces):
> 
> import re
> example = 'abcdefabcdefabcdefg'
> find_string = "abc"
> for match in re.finditer(find_string, example):
>  ??? print(match.start(), match.end())
> 
> That gives me the start and end character positions, which is what I want.
> 
> However, this does not work:
> 
> import re
> example = re.escape('X - cty_degrees + 1 + qq')
> find_string = re.escape('cty_degrees + 1')
> for match in re.finditer(find_string, example):
>  ??? print(match.start(), match.end())
> 
> I?ve tried several other attempts based on my reseearch, but still no match.
> 
> I don?t have much experience with regex, so I hoped a reg-expert might help.
> 
You need to escape only the pattern, not the string you're searching.

From greg.ewing at canterbury.ac.nz  Mon Feb 27 18:48:40 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Feb 2023 12:48:40 +1300
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <mailman.2044.1677523248.20444.python-list@python.org>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
 <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
 <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
 <002801d94ada$ff7fecc0$fe7fc640$@gmail.com>
 <mailman.2044.1677523248.20444.python-list@python.org>
Message-ID: <k64tqrFjga2U1@mid.individual.net>

On 28/02/23 7:40 am, avi.e.gross at gmail.com wrote:
> inhahe <inhahe at gmail.com> made the point that this may not have been the original intent for python and may be a sort of bug that it is too late to fix.

Guido has publically stated that it was a deliberate design choice.
The merits of that design choice can be debated, but it wasn't a
bug or an accident.

-- 
Greg

From cs at cskk.id.au  Mon Feb 27 18:54:43 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 28 Feb 2023 10:54:43 +1100
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPK8DYA--3-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com>
Message-ID: <Y/1Cw5olV2jzubj7@cskk.homeip.net>

On 28Feb2023 00:11, Jen Kris <jenkris at tutanota.com> wrote:
>When matching a string against a longer string, where both strings have spaces in them, we need to escape the spaces.?
>
>This works (no spaces):
>
>import re
>example = 'abcdefabcdefabcdefg'
>find_string = "abc"
>for match in re.finditer(find_string, example):
>??? print(match.start(), match.end())
>
>That gives me the start and end character positions, which is what I want.?
>
>However, this does not work:
>
>import re
>example = re.escape('X - cty_degrees + 1 + qq')
>find_string = re.escape('cty_degrees + 1')
>for match in re.finditer(find_string, example):
>??? print(match.start(), match.end())
>
>I?ve tried several other attempts based on my reseearch, but still no 
>match.?

You need to print those strings out. You're escaping the _example_ 
string, which would make it:

     X - cty_degrees \+ 1 \+ qq

because `+` is a special character in regexps and so `re.escape` escapes 
it. But you don't want to mangle the string you're searching! After all, 
the text above does not contain the string `cty_degrees + 1`.

My secondary question is: if you're escaping the thing you're searching 
_for_, then you're effectively searching for a _fixed_ string, not a 
pattern/regexp. So why on earth are you using regexps to do your 
searching?

The `str` type has a `find(substring)` function. Just use that! It'll be 
faster and the code simpler!

Cheers,
Cameron Simpson <cs at cskk.id.au>

From greg.ewing at canterbury.ac.nz  Mon Feb 27 18:54:09 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Feb 2023 12:54:09 +1300
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <mailman.2038.1677514114.20444.python-list@python.org>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <25df924e-8e07-9b21-f75d-c9e194f915b9@tompassin.net>
 <mailman.2038.1677514114.20444.python-list@python.org>
Message-ID: <k64u55FjhokU1@mid.individual.net>

On 28/02/23 5:08 am, Thomas Passin wrote:
> On 2/27/2023 11:01 AM, Mats Wichmann wrote:
>> If you intend to run Black on your code to ensure consistent 
>> formatting, you may as well learn to prefer double quotes, because 
>> it's going to convert single to double
> 
> I prefer single quotes because they are easier to type.

I tend to use the convention of double quotes for strings seen
by the outside world, and single quotes for internal constants
(such as enumerated types that happen to be represented by
strings).

I guess this means I can't use Black. :-(

-- 
Greg


From rob.cliffe at btinternet.com  Mon Feb 27 18:55:36 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Mon, 27 Feb 2023 23:55:36 +0000
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
Message-ID: <749b4375-9dd4-99e1-19bc-dbc90fc068bc@btinternet.com>



On 27/02/2023 21:04, Ethan Furman wrote:
> On 2/27/23 12:20, rbowman wrote:
>
> > "By using Black, you agree to cede control over minutiae of hand-
> > formatting. In return, Black gives you speed, determinism, and freedom
> > from pycodestyle nagging about formatting. You will save time and 
> mental
> > energy for more important matters."
> >
> > Somehow I don't think we would get along very well. I'm a little on the
> > opinionated side myself.
>
> I personally cannot stand Black.? It feels like every major choice it 
> makes (and some minor ones) are exactly the opposite of the choice I 
> make.
>
> -- 
> ~Ethan~
I've never tried Black or any other code formatter, but I'm sure we 
wouldn't get on.

From jenkris at tutanota.com  Mon Feb 27 18:57:22 2023
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 28 Feb 2023 00:57:22 +0100 (CET)
Subject: How to escape strings for re.finditer?
In-Reply-To: <22223151-d4fe-792b-c73f-319586efcdd9@mrabarnett.plus.com>
References: <NPK8DYA--3-9@tutanota.com>
 <22223151-d4fe-792b-c73f-319586efcdd9@mrabarnett.plus.com>
Message-ID: <NPKInVt--3-9@tutanota.com>

Yes, that's it.? I don't know how long it would have taken to find that detail with research through the voluminous re documentation.? Thanks very much.? 

Feb 27, 2023, 15:47 by python at mrabarnett.plus.com:

> On 2023-02-27 23:11, Jen Kris via Python-list wrote:
>
>> When matching a string against a longer string, where both strings have spaces in them, we need to escape the spaces.
>>
>> This works (no spaces):
>>
>> import re
>> example = 'abcdefabcdefabcdefg'
>> find_string = "abc"
>> for match in re.finditer(find_string, example):
>>  ??? print(match.start(), match.end())
>>
>> That gives me the start and end character positions, which is what I want.
>>
>> However, this does not work:
>>
>> import re
>> example = re.escape('X - cty_degrees + 1 + qq')
>> find_string = re.escape('cty_degrees + 1')
>> for match in re.finditer(find_string, example):
>>  ??? print(match.start(), match.end())
>>
>> I?ve tried several other attempts based on my reseearch, but still no match.
>>
>> I don?t have much experience with regex, so I hoped a reg-expert might help.
>>
> You need to escape only the pattern, not the string you're searching.
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From avi.e.gross at gmail.com  Mon Feb 27 19:06:04 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 19:06:04 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <22223151-d4fe-792b-c73f-319586efcdd9@mrabarnett.plus.com>
References: <NPK8DYA--3-9@tutanota.com>
 <22223151-d4fe-792b-c73f-319586efcdd9@mrabarnett.plus.com>
Message-ID: <00db01d94b08$735397a0$59fac6e0$@gmail.com>

MRAB makes a valid point. The regular expression compiled is only done on the pattern you are looking for and it it contains anything that might be a command, such as an ^ at the start or [12] in middle, you want that converted so NONE OF THAT is one. It will be compiled to something that looks for an ^, including later in the string, and look for a real [ then a real 1 and a real 2 and a real ], not for one of the choices of 1 or 2. 

Your example was 'cty_degrees + 1' which can have a subtle bug introduced. The special character is "+" which means match greedily as many copies of the previous entity as possible. In this case, the previous entity was a single space. So the regular expression will match 'cty degrees' then match the single space it sees because it sees a space followed ny a plus  then not looking for a plus, hits a plus and fails. If your example is rewritten in whatever way re.escape uses, it might be 'cty_degrees \+ 1' and then it should work fine.

But converting what you are searching for just breaks that as the result will have a '\+" whish is being viewed as two unrelated symbols and the backslash breaks the match from going further.



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of MRAB
Sent: Monday, February 27, 2023 6:46 PM
To: python-list at python.org
Subject: Re: How to escape strings for re.finditer?

On 2023-02-27 23:11, Jen Kris via Python-list wrote:
> When matching a string against a longer string, where both strings have spaces in them, we need to escape the spaces.
> 
> This works (no spaces):
> 
> import re
> example = 'abcdefabcdefabcdefg'
> find_string = "abc"
> for match in re.finditer(find_string, example):
>      print(match.start(), match.end())
> 
> That gives me the start and end character positions, which is what I want.
> 
> However, this does not work:
> 
> import re
> example = re.escape('X - cty_degrees + 1 + qq') find_string = 
> re.escape('cty_degrees + 1') for match in re.finditer(find_string, 
> example):
>      print(match.start(), match.end())
> 
> I?ve tried several other attempts based on my reseearch, but still no match.
> 
> I don?t have much experience with regex, so I hoped a reg-expert might help.
> 
You need to escape only the pattern, not the string you're searching.
--
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Mon Feb 27 19:08:28 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 19:08:28 -0500
Subject: it seems like a few weeks ago... but actually it was more like 30
 years ago that i was programming in C, and
In-Reply-To: <k64tqrFjga2U1@mid.individual.net>
References: <30067ec7-a71d-40b1-8a22-0b2be06c702bn@googlegroups.com>
 <k5oftlFlct6U1@mid.individual.net>
 <8207930a-c724-433b-8c5c-ef214aa81a24n@googlegroups.com>
 <420edcbf-593c-2c4f-768b-325a9cf2d796@roelschroeven.net>
 <002801d94ada$ff7fecc0$fe7fc640$@gmail.com>
 <mailman.2044.1677523248.20444.python-list@python.org>
 <k64tqrFjga2U1@mid.individual.net>
Message-ID: <00dd01d94b08$c9035280$5b09f780$@gmail.com>

Yes, Greg, you are correct. After I posted, I encountered a later message
that suggested it was list comprehensions that had accidentally left a
variable behind in a context when theoretically you got ALL you asked for in
the resulting list, so it fixed eventually.

You live and learn till you don't.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Greg Ewing via Python-list
Sent: Monday, February 27, 2023 6:49 PM
To: python-list at python.org
Subject: Re: it seems like a few weeks ago... but actually it was more like
30 years ago that i was programming in C, and

On 28/02/23 7:40 am, avi.e.gross at gmail.com wrote:
> inhahe <inhahe at gmail.com> made the point that this may not have been the
original intent for python and may be a sort of bug that it is too late to
fix.

Guido has publically stated that it was a deliberate design choice.
The merits of that design choice can be debated, but it wasn't a bug or an
accident.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list


From jenkris at tutanota.com  Mon Feb 27 19:13:32 2023
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 28 Feb 2023 01:13:32 +0100 (CET)
Subject: How to escape strings for re.finditer?
In-Reply-To: <Y/1Cw5olV2jzubj7@cskk.homeip.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
Message-ID: <NPKLE0_--3-9@tutanota.com>


I went to the re module because the specified string may appear more than once in the string (in the code I'm writing).? For example:? 

a = "X - abc_degree + 1 + qq + abc_degree + 1"
?b = "abc_degree + 1"
?q = a.find(b)

print(q)
4

So it correctly finds the start of the first instance, but not the second one.? The re code finds both instances.? If I knew that the substring occurred only once then the str.find would be best.? 

I changed my re code after MRAB's comment, it now works.? 

Thanks much.? 

Jen


Feb 27, 2023, 15:56 by cs at cskk.id.au:

> On 28Feb2023 00:11, Jen Kris <jenkris at tutanota.com> wrote:
>
>> When matching a string against a longer string, where both strings have spaces in them, we need to escape the spaces.?
>>
>> This works (no spaces):
>>
>> import re
>> example = 'abcdefabcdefabcdefg'
>> find_string = "abc"
>> for match in re.finditer(find_string, example):
>> ??? print(match.start(), match.end())
>>
>> That gives me the start and end character positions, which is what I want.?
>>
>> However, this does not work:
>>
>> import re
>> example = re.escape('X - cty_degrees + 1 + qq')
>> find_string = re.escape('cty_degrees + 1')
>> for match in re.finditer(find_string, example):
>> ??? print(match.start(), match.end())
>>
>> I?ve tried several other attempts based on my reseearch, but still no match.?
>>
>
> You need to print those strings out. You're escaping the _example_ string, which would make it:
>
>  X - cty_degrees \+ 1 \+ qq
>
> because `+` is a special character in regexps and so `re.escape` escapes it. But you don't want to mangle the string you're searching! After all, the text above does not contain the string `cty_degrees + 1`.
>
> My secondary question is: if you're escaping the thing you're searching _for_, then you're effectively searching for a _fixed_ string, not a pattern/regexp. So why on earth are you using regexps to do your searching?
>
> The `str` type has a `find(substring)` function. Just use that! It'll be faster and the code simpler!
>
> Cheers,
> Cameron Simpson <cs at cskk.id.au>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From avi.e.gross at gmail.com  Mon Feb 27 19:34:49 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 19:34:49 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPKLE0_--3-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com>
Message-ID: <00e201d94b0c$77a05fb0$66e11f10$@gmail.com>

Just FYI, Jen, there are times a sledgehammer works but perhaps is not the only way. These days people worry less about efficiency and more about programmer time and education and that can be fine.

But it you looked at methods available in strings or in some other modules, your situation is quite common. Some may use another RE front end called finditer().

I am NOT suggesting you do what I say next, but imagine writing a loop that takes a substring of what you are searching for of the same length as your search string. Near the end, it stops as there is too little left.

You can now simply test your searched for string against that substring for equality and it tends to return rapidly when they are not equal early on.

Your loop would return whatever data structure or results you want such as that it matched it three times at offsets a, b and c.

But do you allow overlaps? If not, your loop needs to skip len(search_str) after a match.

What you may want to consider is another form of pre-processing. Do you care if "abc_degree + 1" has missing or added spaces at the tart or end or anywhere in middle as in " abc_degree +1"?

Do you care if stuff is a different case like "Abc_Degree + 1"?

Some such searches can be done if both the pattern and searched string are first converted to a canonical format that maps to the same output. But that complicates things a bit and you may to display what you match differently.

And are you also willing to match this: "myabc_degree + 1"?

When using a crafter RE there is a way to ask for a word boundary so abc will only be matched if before that is a space or the start of the string and not "my".

So this may be a case where you can solve an easy version with the chance it can be fooled or overengineer it. If you are allowing the user to type in what to search for, as many programs including editors, do, you will often find such false positives unless the user knows RE syntax and applies it and you do not escape it. I have experienced havoc when doing a careless global replace that matched more than I expected, including making changes in comments or constant strings rather than just the name of a function. Adding a paren is helpful as is not replacing them all but one at a time and skipping any that are not wanted.

Good luck.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Jen Kris via Python-list
Sent: Monday, February 27, 2023 7:14 PM
To: Cameron Simpson <cs at cskk.id.au>
Cc: Python List <python-list at python.org>
Subject: Re: How to escape strings for re.finditer?


I went to the re module because the specified string may appear more than once in the string (in the code I'm writing).  For example:  

a = "X - abc_degree + 1 + qq + abc_degree + 1"
 b = "abc_degree + 1"
 q = a.find(b)

print(q)
4

So it correctly finds the start of the first instance, but not the second one.  The re code finds both instances.  If I knew that the substring occurred only once then the str.find would be best.  

I changed my re code after MRAB's comment, it now works.  

Thanks much.  

Jen


Feb 27, 2023, 15:56 by cs at cskk.id.au:

> On 28Feb2023 00:11, Jen Kris <jenkris at tutanota.com> wrote:
>
>> When matching a string against a longer string, where both strings 
>> have spaces in them, we need to escape the spaces.
>>
>> This works (no spaces):
>>
>> import re
>> example = 'abcdefabcdefabcdefg'
>> find_string = "abc"
>> for match in re.finditer(find_string, example):
>>     print(match.start(), match.end())
>>
>> That gives me the start and end character positions, which is what I 
>> want.
>>
>> However, this does not work:
>>
>> import re
>> example = re.escape('X - cty_degrees + 1 + qq') find_string = 
>> re.escape('cty_degrees + 1') for match in re.finditer(find_string, 
>> example):
>>     print(match.start(), match.end())
>>
>> I?ve tried several other attempts based on my reseearch, but still no 
>> match.
>>
>
> You need to print those strings out. You're escaping the _example_ string, which would make it:
>
>  X - cty_degrees \+ 1 \+ qq
>
> because `+` is a special character in regexps and so `re.escape` escapes it. But you don't want to mangle the string you're searching! After all, the text above does not contain the string `cty_degrees + 1`.
>
> My secondary question is: if you're escaping the thing you're searching _for_, then you're effectively searching for a _fixed_ string, not a pattern/regexp. So why on earth are you using regexps to do your searching?
>
> The `str` type has a `find(substring)` function. Just use that! It'll be faster and the code simpler!
>
> Cheers,
> Cameron Simpson <cs at cskk.id.au>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

-- 
https://mail.python.org/mailman/listinfo/python-list


From jenkris at tutanota.com  Mon Feb 27 19:39:57 2023
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 28 Feb 2023 01:39:57 +0100 (CET)
Subject: How to escape strings for re.finditer?
In-Reply-To: <CA+HdcZjg-QvLBw5JAkHkkCpCiYX8GoQMOfjDvbd0BG4nftfBRw@mail.gmail.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com>
 <CA+HdcZjg-QvLBw5JAkHkkCpCiYX8GoQMOfjDvbd0BG4nftfBRw@mail.gmail.com>
Message-ID: <NPKSY7e--3-9@tutanota.com>


string.count() only tells me there are N instances of the string; it does not say where they begin and end, as does re.finditer.? 

Feb 27, 2023, 16:20 by bobmellowood at gmail.com:

> Would string.count() work for you then?
>
> On Mon, Feb 27, 2023 at 5:16?PM Jen Kris via Python-list <> python-list at python.org> > wrote:
>
>>
>> I went to the re module because the specified string may appear more than once in the string (in the code I'm writing).? For example:?
>>  
>>  a = "X - abc_degree + 1 + qq + abc_degree + 1"
>>  ?b = "abc_degree + 1"
>>  ?q = a.find(b)
>>  
>>  print(q)
>>  4
>>  
>>  So it correctly finds the start of the first instance, but not the second one.? The re code finds both instances.? If I knew that the substring occurred only once then the str.find would be best.? 
>>  
>>  I changed my re code after MRAB's comment, it now works.? 
>>  
>>  Thanks much.? 
>>  
>>  Jen
>>  
>>  
>>  Feb 27, 2023, 15:56 by >> cs at cskk.id.au>> :
>>  
>>  > On 28Feb2023 00:11, Jen Kris <>> jenkris at tutanota.com>> > wrote:
>>  >
>>  >> When matching a string against a longer string, where both strings have spaces in them, we need to escape the spaces.?
>>  >>
>>  >> This works (no spaces):
>>  >>
>>  >> import re
>>  >> example = 'abcdefabcdefabcdefg'
>>  >> find_string = "abc"
>>  >> for match in re.finditer(find_string, example):
>>  >> ??? print(match.start(), match.end())
>>  >>
>>  >> That gives me the start and end character positions, which is what I want.?
>>  >>
>>  >> However, this does not work:
>>  >>
>>  >> import re
>>  >> example = re.escape('X - cty_degrees + 1 + qq')
>>  >> find_string = re.escape('cty_degrees + 1')
>>  >> for match in re.finditer(find_string, example):
>>  >> ??? print(match.start(), match.end())
>>  >>
>>  >> I?ve tried several other attempts based on my reseearch, but still no match.?
>>  >>
>>  >
>>  > You need to print those strings out. You're escaping the _example_ string, which would make it:
>>  >
>>  >? X - cty_degrees \+ 1 \+ qq
>>  >
>>  > because `+` is a special character in regexps and so `re.escape` escapes it. But you don't want to mangle the string you're searching! After all, the text above does not contain the string `cty_degrees + 1`.
>>  >
>>  > My secondary question is: if you're escaping the thing you're searching _for_, then you're effectively searching for a _fixed_ string, not a pattern/regexp. So why on earth are you using regexps to do your searching?
>>  >
>>  > The `str` type has a `find(substring)` function. Just use that! It'll be faster and the code simpler!
>>  >
>>  > Cheers,
>>  > Cameron Simpson <>> cs at cskk.id.au>> >
>>  > -- 
>>  > >> https://mail.python.org/mailman/listinfo/python-list
>>  >
>>  
>>  -- 
>>  >> https://mail.python.org/mailman/listinfo/python-list
>>
>
>
> -- 
> **** Listen to my CD at > http://www.mellowood.ca/music/cedars>  ****
> Bob van der Poel ** Wynndel, British Columbia, CANADA **
> EMAIL: > bob at mellowood.ca
> WWW:?? > http://www.mellowood.ca
>


From cs at cskk.id.au  Mon Feb 27 19:33:47 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 28 Feb 2023 11:33:47 +1100
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPKLE0_--3-9@tutanota.com>
References: <NPKLE0_--3-9@tutanota.com>
Message-ID: <Y/1L63McJV7ixmte@cskk.homeip.net>

On 28Feb2023 01:13, Jen Kris <jenkris at tutanota.com> wrote:
>I went to the re module because the specified string may appear more 
>than once in the string (in the code I'm writing).

Sure, but writing a `finditer` for plain `str` is pretty easy 
(untested):

     pos = 0
     while True:
         found = s.find(substring, pos)
         if found < 0:
             break
         start = found
         end = found + len(substring)
         ... do whatever with start and end ...
         pos = end

Many people go straight to the `re` module whenever they're looking for 
strings. It is often cryptic error prone overkill. Just something to 
keep in mind.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From cs at cskk.id.au  Mon Feb 27 19:36:45 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 28 Feb 2023 11:36:45 +1100
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPKInVt--3-9@tutanota.com>
References: <NPKInVt--3-9@tutanota.com>
Message-ID: <Y/1MnamF6SJio1JX@cskk.homeip.net>

On 28Feb2023 00:57, Jen Kris <jenkris at tutanota.com> wrote:
>Yes, that's it.? I don't know how long it would have taken to find that 
>detail with research through the voluminous re documentation.? Thanks 
>very much.?

You find things like this by printing out the strings you're actually 
working with. Not the original strings, but the strings when you're 
invoking `finditer` i.e. in your case, escaped strings.

Then you might have seen that what you were searching no longer 
contained what you were searching for.

Don't underestimate the value of the debugging print call. It lets you 
see what your programme is actually working with, instead of what you 
thought it was working with.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From syedarslanali404 at gmail.com  Mon Feb 27 19:51:29 2023
From: syedarslanali404 at gmail.com (Arslan Mehmood)
Date: Mon, 27 Feb 2023 16:51:29 -0800 (PST)
Subject: How to fix this issue
Message-ID: <47d350be-ab36-4e69-afed-13539f600665n@googlegroups.com>

How I can remove python terminl, its again and again open during working in python. Please help me to resolve this issue.
Python 3.11.1 (tags/v3.11.1:a7a450f, Dec  6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

From jenkris at tutanota.com  Mon Feb 27 20:36:26 2023
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 28 Feb 2023 02:36:26 +0100 (CET)
Subject: How to escape strings for re.finditer?
In-Reply-To: <Y/1L63McJV7ixmte@cskk.homeip.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
Message-ID: <NPKeTUo--3-9@tutanota.com>


I haven't tested it either but it looks like it would work.? But for this case I prefer the relative simplicity of:

example = 'X - abc_degree + 1 + qq + abc_degree + 1'
find_string = re.escape('abc_degree + 1')
for match in re.finditer(find_string, example):
??? print(match.start(), match.end())

4 18
26 40

I don't insist on terseness for its own sake, but it's cleaner this way.? 

Jen


Feb 27, 2023, 16:55 by cs at cskk.id.au:

> On 28Feb2023 01:13, Jen Kris <jenkris at tutanota.com> wrote:
>
>> I went to the re module because the specified string may appear more than once in the string (in the code I'm writing).
>>
>
> Sure, but writing a `finditer` for plain `str` is pretty easy (untested):
>
>  pos = 0
>  while True:
>  found = s.find(substring, pos)
>  if found < 0:
>  break
>  start = found
>  end = found + len(substring)
>  ... do whatever with start and end ...
>  pos = end
>
> Many people go straight to the `re` module whenever they're looking for strings. It is often cryptic error prone overkill. Just something to keep in mind.
>
> Cheers,
> Cameron Simpson <cs at cskk.id.au>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From avi.e.gross at gmail.com  Mon Feb 27 20:56:00 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 20:56:00 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPKSY7e--3-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com>
 <CA+HdcZjg-QvLBw5JAkHkkCpCiYX8GoQMOfjDvbd0BG4nftfBRw@mail.gmail.com>
 <NPKSY7e--3-9@tutanota.com>
Message-ID: <010401d94b17$cf0802c0$6d180840$@gmail.com>

Jen,

What you just described is why that tool is not the right tool for the job, albeit it may help you confirm if whatever method you choose does work correctly and finds the same number of matches.

Sometimes you simply do some searching and roll your own.

Consider this code using a sort of list comprehension feature:

>>> short = "hello world"
>>> longer =  "hello world is how many programs start for novices but some use hello world! to show how happy they are to say hello world"

>>> short in longer
True
>>> howLong = len(short)

>>> res = [(offset, offset + howLong)  for offset  in range(len(longer)) if longer.startswith(short, offset)]
>>> res
[(0, 11), (64, 75), (111, 122)]
>>> len(res)
3

I could do a bit more but it seems to work. Did I get the offsets right? Checking:

>>> print( [ longer[res[index][0]:res[index][1]] for index in range(len(res))])
['hello world', 'hello world', 'hello world']

Seems to work but thrown together quickly so can likely be done much nicer.

But as noted, the above has flaws such as matching overlaps like:

>>> short = "good good"
>>> longer = "A good good good but not douple plus good good good goody"
>>> howLong = len(short)
>>> res = [(offset, offset + howLong)  for offset  in range(len(longer)) if longer.startswith(short, offset)]
>>> res
[(2, 11), (7, 16), (37, 46), (42, 51), (47, 56)]

It matched five times as sometimes we had three of four good in a row. Some other method might match only three.

What some might do can get long and you clearly want one answer and not tutorials. For example, people can make a loop that finds a match and either sabotages the area by replacing or deleting it, or keeps track and searched again on a substring offset from the beginning. 

When you do not find a tool, consider making one. You can take (better) code than I show above and make it info a function and now you have a tool. Even better, you can make it return whatever you want.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Jen Kris via Python-list
Sent: Monday, February 27, 2023 7:40 PM
To: Bob van der Poel <bobmellowood at gmail.com>
Cc: Python List <python-list at python.org>
Subject: Re: How to escape strings for re.finditer?


string.count() only tells me there are N instances of the string; it does not say where they begin and end, as does re.finditer.  

Feb 27, 2023, 16:20 by bobmellowood at gmail.com:

> Would string.count() work for you then?
>
> On Mon, Feb 27, 2023 at 5:16?PM Jen Kris via Python-list <> python-list at python.org> > wrote:
>
>>
>> I went to the re module because the specified string may appear more 
>> than once in the string (in the code I'm writing).  For example:
>>  
>>  a = "X - abc_degree + 1 + qq + abc_degree + 1"
>>   b = "abc_degree + 1"
>>   q = a.find(b)
>>  
>>  print(q)
>>  4
>>  
>>  So it correctly finds the start of the first instance, but not the 
>> second one.  The re code finds both instances.  If I knew that the substring occurred only once then the str.find would be best.
>>  
>>  I changed my re code after MRAB's comment, it now works.
>>  
>>  Thanks much.
>>  
>>  Jen
>>  
>>  
>>  Feb 27, 2023, 15:56 by >> cs at cskk.id.au>> :
>>  
>>  > On 28Feb2023 00:11, Jen Kris <>> jenkris at tutanota.com>> > wrote:
>>  >
>>  >> When matching a string against a longer string, where both 
>> strings have spaces in them, we need to escape the spaces.  >>  >> 
>> This works (no spaces):
>>  >>
>>  >> import re
>>  >> example = 'abcdefabcdefabcdefg'
>>  >> find_string = "abc"
>>  >> for match in re.finditer(find_string, example):
>>  >>     print(match.start(), match.end())  >>  >> That gives me the 
>> start and end character positions, which is what I want.
>>  >>
>>  >> However, this does not work:
>>  >>
>>  >> import re
>>  >> example = re.escape('X - cty_degrees + 1 + qq')  >> find_string = 
>> re.escape('cty_degrees + 1')  >> for match in 
>> re.finditer(find_string, example):
>>  >>     print(match.start(), match.end())  >>  >> I?ve tried several 
>> other attempts based on my reseearch, but still no match.
>>  >>
>>  >
>>  > You need to print those strings out. You're escaping the _example_ string, which would make it:
>>  >
>>  >  X - cty_degrees \+ 1 \+ qq
>>  >
>>  > because `+` is a special character in regexps and so `re.escape` escapes it. But you don't want to mangle the string you're searching! After all, the text above does not contain the string `cty_degrees + 1`.
>>  >
>>  > My secondary question is: if you're escaping the thing you're searching _for_, then you're effectively searching for a _fixed_ string, not a pattern/regexp. So why on earth are you using regexps to do your searching?
>>  >
>>  > The `str` type has a `find(substring)` function. Just use that! It'll be faster and the code simpler!
>>  >
>>  > Cheers,
>>  > Cameron Simpson <>> cs at cskk.id.au>> >  > --  > >> 
>> https://mail.python.org/mailman/listinfo/python-list
>>  >
>>  
>>  --
>>  >> https://mail.python.org/mailman/listinfo/python-list
>>
>
>
> --
> **** Listen to my CD at > http://www.mellowood.ca/music/cedars>  **** 
> Bob van der Poel ** Wynndel, British Columbia, CANADA **
> EMAIL: > bob at mellowood.ca
> WWW:   > http://www.mellowood.ca
>

--
https://mail.python.org/mailman/listinfo/python-list


From torriem at gmail.com  Mon Feb 27 21:08:07 2023
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 27 Feb 2023 19:08:07 -0700
Subject: Rob Cliffe should stop sending me rude email messages.
In-Reply-To: <63fcd798.020a0220.4e8d2.3b9c@mx.google.com>
References: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
 <b891d90b-6075-2b39-5323-0ee544ea8aa1@gmail.com>
 <63fcd798.020a0220.4e8d2.3b9c@mx.google.com>
Message-ID: <3ed9ecd0-31bb-0293-9d4d-1099b45cf744@gmail.com>

On 2/27/23 09:17, Grant Edwards wrote:
> On 2023-02-27, Michael Torrie <torriem at gmail.com> wrote:
> 
>> I've been putting off sending this message for days, but the list noise
>> level is now to the point that it has to be said.
> 
> Ah, I've finially realized why some of those threads have seemed so
> disjointed to me. Years ago, I plonked all posts which are (like Hen
> Hanna's) submitted via Googole Groups.
> 
> I highly recommend it.
> 
> FWIW, here's the "score" rule for doing that with srln:
> 
>     Score:: =-9999
>       Message-ID: .*googlegroups.com

Thanks for the tip and reminder.  I'll add that to my gmail filter.


From avi.e.gross at gmail.com  Mon Feb 27 21:16:01 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 21:16:01 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPKeTUo--3-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com>
Message-ID: <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>

Jen,

Can you see what SOME OF US see as ASCII text? We can help you better if we get code that can be copied and run as-is.

 What you sent is not terse. It is wrong. It will not run on any python interpreter because you somehow lost a carriage return and indent.

This is what you sent:

example = 'X - abc_degree + 1 + qq + abc_degree + 1'
find_string = re.escape('abc_degree + 1') for match in re.finditer(find_string, example):
    print(match.start(), match.end())

This is code indentedproperly:

example = 'X - abc_degree + 1 + qq + abc_degree + 1'
find_string = re.escape('abc_degree + 1') 
for match in re.finditer(find_string, example):
    print(match.start(), match.end())

Of course I am sure you wrote and ran code more like the latter version but somewhere in your copy/paste process, ....

And, just for fun, since there is nothing wrong with your code, this minor change is terser:

>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
...     print(match.start(), match.end())
... 
...     
4 18
26 40

But note once you use regular expressions, and not in your case, you might match multiple things that are far from the same such as matching two repeated words of any kind in any case including "and and" and "so so" or finding words that have multiple doubled letter as in the  stereotypical bookkeeper. In those cases, you may want even more than offsets but also show the exact text that matched or even show some characters before and/or after for context.


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Jen Kris via Python-list
Sent: Monday, February 27, 2023 8:36 PM
To: Cameron Simpson <cs at cskk.id.au>
Cc: Python List <python-list at python.org>
Subject: Re: How to escape strings for re.finditer?


I haven't tested it either but it looks like it would work.  But for this case I prefer the relative simplicity of:

example = 'X - abc_degree + 1 + qq + abc_degree + 1'
find_string = re.escape('abc_degree + 1') for match in re.finditer(find_string, example):
    print(match.start(), match.end())

4 18
26 40

I don't insist on terseness for its own sake, but it's cleaner this way.  

Jen


Feb 27, 2023, 16:55 by cs at cskk.id.au:

> On 28Feb2023 01:13, Jen Kris <jenkris at tutanota.com> wrote:
>
>> I went to the re module because the specified string may appear more than once in the string (in the code I'm writing).
>>
>
> Sure, but writing a `finditer` for plain `str` is pretty easy (untested):
>
>  pos = 0
>  while True:
>  found = s.find(substring, pos)
>  if found < 0:
>  break
>  start = found
>  end = found + len(substring)
>  ... do whatever with start and end ...
>  pos = end
>
> Many people go straight to the `re` module whenever they're looking for strings. It is often cryptic error prone overkill. Just something to keep in mind.
>
> Cheers,
> Cameron Simpson <cs at cskk.id.au>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

-- 
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Mon Feb 27 21:21:28 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 21:21:28 -0500
Subject: XXX XXX  should stop sending me rude email messages.
In-Reply-To: <3ed9ecd0-31bb-0293-9d4d-1099b45cf744@gmail.com>
References: <ebad7eee-8d6a-4e2a-888f-93832b38b288n@googlegroups.com>
 <b891d90b-6075-2b39-5323-0ee544ea8aa1@gmail.com>
 <63fcd798.020a0220.4e8d2.3b9c@mx.google.com>
 <3ed9ecd0-31bb-0293-9d4d-1099b45cf744@gmail.com>
Message-ID: <012301d94b1b$5dc17520$19445f60$@gmail.com>

Michael,

Although I appreciate much of what you say, I ask humbly and politely that
we change the Subject line for messages like this one. HH is out of range
for now, albeit I think he can still read what we say.

But keeping the name Michael Torrie in the subject line, should be sort of
XXX rated.

And I mean especially in this case where we have no idea what "he" was
writing but it was a private message not intended for the group.

Just kidding BTW. But since YOU are Michael Torrie, I guess you can
broadcast a request to stop mailing yourself if that pleases. I will just no
longer bring your name forth in this context. Feel free to send me email, if
the need arises.

Avi

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Michael Torrie
Sent: Monday, February 27, 2023 9:08 PM
To: python-list at python.org
Subject: Re: Rob Cliffe should stop sending me rude email messages.

On 2/27/23 09:17, Grant Edwards wrote:
> On 2023-02-27, Michael Torrie <torriem at gmail.com> wrote:
> 
>> I've been putting off sending this message for days, but the list 
>> noise level is now to the point that it has to be said.
> 
> Ah, I've finially realized why some of those threads have seemed so 
> disjointed to me. Years ago, I plonked all posts which are (like Hen
> Hanna's) submitted via Googole Groups.
> 
> I highly recommend it.
> 
> FWIW, here's the "score" rule for doing that with srln:
> 
>     Score:: =-9999
>       Message-ID: .*googlegroups.com

Thanks for the tip and reminder.  I'll add that to my gmail filter.

--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Mon Feb 27 21:44:26 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Mon, 27 Feb 2023 21:44:26 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
Message-ID: <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>

On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
> And, just for fun, since there is nothing wrong with your code, this minor change is terser:
> 
>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
> ...     print(match.start(), match.end())
> ...
> ...
> 4 18
> 26 40

Just for more fun :) -

Without knowing how general your expressions will be, I think the 
following version is very readable, certainly more readable than regexes:

example = 'X - abc_degree + 1 + qq + abc_degree + 1'
KEY = 'abc_degree + 1'

for i in range(len(example)):
     if example[i:].startswith(KEY):
         print(i, i + len(KEY))
# prints:
4 18
26 40

If you may have variable numbers of spaces around the symbols, OTOH, the 
whole situation changes and then regexes would almost certainly be the 
best approach.  But the regular expression strings would become harder 
to read.

From avi.e.gross at gmail.com  Mon Feb 27 22:47:57 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Mon, 27 Feb 2023 22:47:57 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
Message-ID: <012701d94b27$724c96d0$56e5c470$@gmail.com>

I think by now we have given all that is needed by the OP but Dave's answer
strikes me as being able to be a tad faster as a while loop if you are
searching  larger corpus such as an entire ebook or all books as you can do
on books.google.com

I think I mentioned earlier that some assumptions need to apply. The text
needs to be something like an ASCII encoding or seen as code points rather
than bytes. We assume a match should move forward by the length of the
match. And, clearly, there cannot be a match too close to the end.

So a while loop would begin with a variable set to zero to mark the current
location of the search. The condition for repeating the loop is that this
variable is less than or equal to len(searched_text) - len(key)

In the loop, each comparison is done the same way as David uses, or anything
similar enough but the twist is a failure increments the variable by 1 while
success increments by len(key).

Will this make much difference? It might as the simpler algorithm counts
overlapping matches and wastes some time hunting where perhaps it shouldn't.

And, of course, if you made something like this into a search function, you
can easily add features such as asking that you only return the first N
matches or the next N, simply by making it a generator.
So tying this into an earlier discussion, do you want the LAST match info
visible when the While loop has completed? If it was available, it opens up
possibilities for running the loop again but starting from where you left
off.



-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Thomas Passin
Sent: Monday, February 27, 2023 9:44 PM
To: python-list at python.org
Subject: Re: How to escape strings for re.finditer?

On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
> And, just for fun, since there is nothing wrong with your code, this minor
change is terser:
> 
>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
> ...     print(match.start(), match.end())
> ...
> ...
> 4 18
> 26 40

Just for more fun :) -

Without knowing how general your expressions will be, I think the following
version is very readable, certainly more readable than regexes:

example = 'X - abc_degree + 1 + qq + abc_degree + 1'
KEY = 'abc_degree + 1'

for i in range(len(example)):
     if example[i:].startswith(KEY):
         print(i, i + len(KEY))
# prints:
4 18
26 40

If you may have variable numbers of spaces around the symbols, OTOH, the
whole situation changes and then regexes would almost certainly be the best
approach.  But the regular expression strings would become harder to read.
--
https://mail.python.org/mailman/listinfo/python-list


From greg.ewing at canterbury.ac.nz  Tue Feb 28 01:24:45 2023
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Feb 2023 19:24:45 +1300
Subject: one Liner: Lisprint(x) --> (a, b, c) instead of ['a', 'b', 'c']
In-Reply-To: <275cd907-6dd5-4553-bf2f-b3b048837687n@googlegroups.com>
References: <AQHaiTQYqU3iuXhS3Bvoh045JVHRGwKjNmXHrsppDSA=>
 <439d9918-210a-49c7-b601-5289257dd407n@googlegroups.com>
 <001801d94a58$5e0cb990$1a262cb0$@gmail.com>
 <a1fb44ad-45a3-49e4-9954-147317f0b86en@googlegroups.com>
 <mailman.2026.1677467141.20444.python-list@python.org>
 <275cd907-6dd5-4553-bf2f-b3b048837687n@googlegroups.com>
Message-ID: <k65l1dFmo3jU1@mid.individual.net>

On 28/02/23 4:24 pm, Hen Hanna wrote:
>                            is it poss. to peek at the Python-list's  messages without joining ?

It's mirrored to the comp.lang.python usenet group, or
you can read it through gmane with a news client.

-- 
Greg

From roel at roelschroeven.net  Tue Feb 28 04:33:20 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Tue, 28 Feb 2023 10:33:20 +0100
Subject: How to escape strings for re.finditer?
In-Reply-To: <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
Message-ID: <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>

Op 28/02/2023 om 3:44 schreef Thomas Passin:
> On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
>> And, just for fun, since there is nothing wrong with your code, this 
>> minor change is terser:
>>
>>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
>> ...???? print(match.start(), match.end())
>> ...
>> ...
>> 4 18
>> 26 40
>
> Just for more fun :) -
>
> Without knowing how general your expressions will be, I think the 
> following version is very readable, certainly more readable than regexes:
>
> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
> KEY = 'abc_degree + 1'
>
> for i in range(len(example)):
> ??? if example[i:].startswith(KEY):
> ??????? print(i, i + len(KEY))
> # prints:
> 4 18
> 26 40
I think it's often a good idea to use a standard library function 
instead of rolling your own. The issue becomes less clear-cut when the 
standard library doesn't do exactly what you need (as here, where 
re.finditer() uses regular expressions while the use case only uses 
simple search strings). Ideally there would be a str.finditer() method 
we could use, but in the absence of that I think we still need to 
consider using the almost-but-not-quite fitting re.finditer().

Two reasons:

(1) I think it's clearer: the name tells us what it does (though of 
course we could solve this in a hand-written version by wrapping it in a 
suitably named function).

(2) Searching for a string in another string, in a performant way, is 
not as simple as it first appears. Your version works correctly, but 
slowly. In some situations it doesn't matter, but in other cases it 
will. For better performance, string searching algorithms jump ahead 
either when they found a match or when they know for sure there isn't a 
match for some time (see e.g. the Boyer?Moore string-search algorithm). 
You could write such a more efficient algorithm, but then it becomes 
more complex and more error-prone. Using a well-tested existing function 
becomes quite attractive.

To illustrate the difference performance, I did a simple test (using the 
paragraph above is test text):

 ??? import re
 ??? import timeit

 ??? def using_re_finditer(key, text):
 ??????? matches = []
 ??????? for match in re.finditer(re.escape(key), text):
 ??????????? matches.append((match.start(), match.end()))
 ??????? return matches


 ??? def using_simple_loop(key, text):
 ??????? matches = []
 ??????? for i in range(len(text)):
 ??????????? if text[i:].startswith(key):
 ??????????????? matches.append((i, i + len(key)))
 ??????? return matches


 ??? CORPUS = """Searching for a string in another string, in a 
performant way, is
 ??? not as simple as it first appears. Your version works correctly, 
but slowly.
 ??? In some situations it doesn't matter, but in other cases it will. 
For better
 ??? performance, string searching algorithms jump ahead either when 
they found a
 ??? match or when they know for sure there isn't a match for some time 
(see e.g.
 ??? the Boyer?Moore string-search algorithm). You could write such a more
 ??? efficient algorithm, but then it becomes more complex and more 
error-prone.
 ??? Using a well-tested existing function becomes quite attractive."""
 ??? KEY = 'in'
 ??? print('using_simple_loop:', 
timeit.repeat(stmt='using_simple_loop(KEY, CORPUS)', globals=globals(), 
number=1000))
 ??? print('using_re_finditer:', 
timeit.repeat(stmt='using_re_finditer(KEY, CORPUS)', globals=globals(), 
number=1000))

This does 5 runs of 1000 repetitions each, and reports the time in 
seconds for each of those runs.
Result on my machine:

 ??? using_simple_loop: [0.13952950000020792, 0.13063130000000456, 
0.12803450000001249, 0.13186180000002423, 0.13084610000032626]
 ??? using_re_finditer: [0.003861400000005233, 0.004061900000124297, 
0.003478999999970256, 0.003413100000216218, 0.0037320000001273]

We find that in this test re.finditer() is more than 30 times faster 
(despite the overhead of regular expressions.

While speed isn't everything in programming, with such a large 
difference in performance and (to me) no real disadvantages of using 
re.finditer(), I would prefer re.finditer() over writing my own.

-- 
"The saddest aspect of life right now is that science gathers knowledge
faster than society gathers wisdom."
         -- Isaac Asimov


From list1 at tompassin.net  Tue Feb 28 08:35:35 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 28 Feb 2023 08:35:35 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
Message-ID: <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>

On 2/28/2023 4:33 AM, Roel Schroeven wrote:
> Op 28/02/2023 om 3:44 schreef Thomas Passin:
>> On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
>>> And, just for fun, since there is nothing wrong with your code, this 
>>> minor change is terser:
>>>
>>>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
>>> ...???? print(match.start(), match.end())
>>> ...
>>> ...
>>> 4 18
>>> 26 40
>>
>> Just for more fun :) -
>>
>> Without knowing how general your expressions will be, I think the 
>> following version is very readable, certainly more readable than regexes:
>>
>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>> KEY = 'abc_degree + 1'
>>
>> for i in range(len(example)):
>> ??? if example[i:].startswith(KEY):
>> ??????? print(i, i + len(KEY))
>> # prints:
>> 4 18
>> 26 40
> I think it's often a good idea to use a standard library function 
> instead of rolling your own. The issue becomes less clear-cut when the 
> standard library doesn't do exactly what you need (as here, where 
> re.finditer() uses regular expressions while the use case only uses 
> simple search strings). Ideally there would be a str.finditer() method 
> we could use, but in the absence of that I think we still need to 
> consider using the almost-but-not-quite fitting re.finditer().
> 
> Two reasons:
> 
> (1) I think it's clearer: the name tells us what it does (though of 
> course we could solve this in a hand-written version by wrapping it in a 
> suitably named function).
> 
> (2) Searching for a string in another string, in a performant way, is 
> not as simple as it first appears. Your version works correctly, but 
> slowly. In some situations it doesn't matter, but in other cases it 
> will. For better performance, string searching algorithms jump ahead 
> either when they found a match or when they know for sure there isn't a 
> match for some time (see e.g. the Boyer?Moore string-search algorithm). 
> You could write such a more efficient algorithm, but then it becomes 
> more complex and more error-prone. Using a well-tested existing function 
> becomes quite attractive.

Sure, it all depends on what the real task will be.  That's why I wrote 
"Without knowing how general your expressions will be". For the example 
string, it's unlikely that speed will be a factor, but who knows what 
target strings and keys will turn up in the future?

> To illustrate the difference performance, I did a simple test (using the 
> paragraph above is test text):
> 
>  ??? import re
>  ??? import timeit
> 
>  ??? def using_re_finditer(key, text):
>  ??????? matches = []
>  ??????? for match in re.finditer(re.escape(key), text):
>  ??????????? matches.append((match.start(), match.end()))
>  ??????? return matches
> 
> 
>  ??? def using_simple_loop(key, text):
>  ??????? matches = []
>  ??????? for i in range(len(text)):
>  ??????????? if text[i:].startswith(key):
>  ??????????????? matches.append((i, i + len(key)))
>  ??????? return matches
> 
> 
>  ??? CORPUS = """Searching for a string in another string, in a 
> performant way, is
>  ??? not as simple as it first appears. Your version works correctly, 
> but slowly.
>  ??? In some situations it doesn't matter, but in other cases it will. 
> For better
>  ??? performance, string searching algorithms jump ahead either when 
> they found a
>  ??? match or when they know for sure there isn't a match for some time 
> (see e.g.
>  ??? the Boyer?Moore string-search algorithm). You could write such a more
>  ??? efficient algorithm, but then it becomes more complex and more 
> error-prone.
>  ??? Using a well-tested existing function becomes quite attractive."""
>  ??? KEY = 'in'
>  ??? print('using_simple_loop:', 
> timeit.repeat(stmt='using_simple_loop(KEY, CORPUS)', globals=globals(), 
> number=1000))
>  ??? print('using_re_finditer:', 
> timeit.repeat(stmt='using_re_finditer(KEY, CORPUS)', globals=globals(), 
> number=1000))
> 
> This does 5 runs of 1000 repetitions each, and reports the time in 
> seconds for each of those runs.
> Result on my machine:
> 
>  ??? using_simple_loop: [0.13952950000020792, 0.13063130000000456, 
> 0.12803450000001249, 0.13186180000002423, 0.13084610000032626]
>  ??? using_re_finditer: [0.003861400000005233, 0.004061900000124297, 
> 0.003478999999970256, 0.003413100000216218, 0.0037320000001273]
> 
> We find that in this test re.finditer() is more than 30 times faster 
> (despite the overhead of regular expressions.
> 
> While speed isn't everything in programming, with such a large 
> difference in performance and (to me) no real disadvantages of using 
> re.finditer(), I would prefer re.finditer() over writing my own.
> 


From roel at roelschroeven.net  Tue Feb 28 10:05:47 2023
From: roel at roelschroeven.net (Roel Schroeven)
Date: Tue, 28 Feb 2023 16:05:47 +0100
Subject: How to escape strings for re.finditer?
In-Reply-To: <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
 <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
Message-ID: <d9c55bbd-6502-c9fb-41fc-9fc15a22691c@roelschroeven.net>

Op 28/02/2023 om 14:35 schreef Thomas Passin:
> On 2/28/2023 4:33 AM, Roel Schroeven wrote:
>> [...]
>> (2) Searching for a string in another string, in a performant way, is 
>> not as simple as it first appears. Your version works correctly, but 
>> slowly. In some situations it doesn't matter, but in other cases it 
>> will. For better performance, string searching algorithms jump ahead 
>> either when they found a match or when they know for sure there isn't 
>> a match for some time (see e.g. the Boyer?Moore string-search 
>> algorithm). You could write such a more efficient algorithm, but then 
>> it becomes more complex and more error-prone. Using a well-tested 
>> existing function becomes quite attractive.
>
> Sure, it all depends on what the real task will be.? That's why I 
> wrote "Without knowing how general your expressions will be". For the 
> example string, it's unlikely that speed will be a factor, but who 
> knows what target strings and keys will turn up in the future?
On hindsight I think it was overthinking things a bit. "It all depends 
on what the real task will be" you say, and indeed I think that should 
be the main conclusion here.

-- 
"Man had always assumed that he was more intelligent than dolphins because
he had achieved so much ? the wheel, New York, wars and so on ? whilst all
the dolphins had ever done was muck about in the water having a good time.
But conversely, the dolphins had always believed that they were far more
intelligent than man ? for precisely the same reasons."
         -- Douglas Adams


From list1 at tompassin.net  Tue Feb 28 10:41:11 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 28 Feb 2023 10:41:11 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <d9c55bbd-6502-c9fb-41fc-9fc15a22691c@roelschroeven.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
 <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
 <d9c55bbd-6502-c9fb-41fc-9fc15a22691c@roelschroeven.net>
Message-ID: <6bc201c0-c4a7-060f-8c16-87e9efa63388@tompassin.net>

On 2/28/2023 10:05 AM, Roel Schroeven wrote:
> Op 28/02/2023 om 14:35 schreef Thomas Passin:
>> On 2/28/2023 4:33 AM, Roel Schroeven wrote:
>>> [...]
>>> (2) Searching for a string in another string, in a performant way, is 
>>> not as simple as it first appears. Your version works correctly, but 
>>> slowly. In some situations it doesn't matter, but in other cases it 
>>> will. For better performance, string searching algorithms jump ahead 
>>> either when they found a match or when they know for sure there isn't 
>>> a match for some time (see e.g. the Boyer?Moore string-search 
>>> algorithm). You could write such a more efficient algorithm, but then 
>>> it becomes more complex and more error-prone. Using a well-tested 
>>> existing function becomes quite attractive.
>>
>> Sure, it all depends on what the real task will be.? That's why I 
>> wrote "Without knowing how general your expressions will be". For the 
>> example string, it's unlikely that speed will be a factor, but who 
>> knows what target strings and keys will turn up in the future?
> On hindsight I think it was overthinking things a bit. "It all depends 
> on what the real task will be" you say, and indeed I think that should 
> be the main conclusion here.


It is interesting, though, how pre-processing the search pattern can 
improve search times if you can afford the pre-processing.  Here's a 
paper on rapidly finding matches when there may be up to one misspelled 
character.  It's easy enough to implement, though in Python you can't 
take the additional step of tuning it to stay in cache.

https://Robert.Muth.Org/Papers/1996-Approx-Multi.Pdf


From jenkris at tutanota.com  Tue Feb 28 12:57:52 2023
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 28 Feb 2023 18:57:52 +0100 (CET)
Subject: How to escape strings for re.finditer?
In-Reply-To: <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
Message-ID: <NPOA6-y--3-9@tutanota.com>

The code I sent is correct, and it runs here.? Maybe you received it with a carriage return removed, but on my copy after posting, it is correct:

example = 'X - abc_degree + 1 + qq + abc_degree + 1'
?find_string = re.escape('abc_degree + 1')
?for match in re.finditer(find_string, example):
???? print(match.start(), match.end())

One question:? several people have made suggestions other than regex (not your terser example with regex you shown below).? Is there a reason why regex is not preferred to, for example, a list comp?? Performance?? Reliability?? 



? 


Feb 27, 2023, 18:16 by avi.e.gross at gmail.com:

> Jen,
>
> Can you see what SOME OF US see as ASCII text? We can help you better if we get code that can be copied and run as-is.
>
>  What you sent is not terse. It is wrong. It will not run on any python interpreter because you somehow lost a carriage return and indent.
>
> This is what you sent:
>
> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
> find_string = re.escape('abc_degree + 1') for match in re.finditer(find_string, example):
>  print(match.start(), match.end())
>
> This is code indentedproperly:
>
> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
> find_string = re.escape('abc_degree + 1') 
> for match in re.finditer(find_string, example):
>  print(match.start(), match.end())
>
> Of course I am sure you wrote and ran code more like the latter version but somewhere in your copy/paste process, ....
>
> And, just for fun, since there is nothing wrong with your code, this minor change is terser:
>
>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
>>>>
> ...     print(match.start(), match.end())
> ... 
> ... 
> 4 18
> 26 40
>
> But note once you use regular expressions, and not in your case, you might match multiple things that are far from the same such as matching two repeated words of any kind in any case including "and and" and "so so" or finding words that have multiple doubled letter as in the  stereotypical bookkeeper. In those cases, you may want even more than offsets but also show the exact text that matched or even show some characters before and/or after for context.
>
>
> -----Original Message-----
> From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Jen Kris via Python-list
> Sent: Monday, February 27, 2023 8:36 PM
> To: Cameron Simpson <cs at cskk.id.au>
> Cc: Python List <python-list at python.org>
> Subject: Re: How to escape strings for re.finditer?
>
>
> I haven't tested it either but it looks like it would work.  But for this case I prefer the relative simplicity of:
>
> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
> find_string = re.escape('abc_degree + 1') for match in re.finditer(find_string, example):
>  print(match.start(), match.end())
>
> 4 18
> 26 40
>
> I don't insist on terseness for its own sake, but it's cleaner this way. 
>
> Jen
>
>
> Feb 27, 2023, 16:55 by cs at cskk.id.au:
>
>> On 28Feb2023 01:13, Jen Kris <jenkris at tutanota.com> wrote:
>>
>>> I went to the re module because the specified string may appear more than once in the string (in the code I'm writing).
>>>
>>
>> Sure, but writing a `finditer` for plain `str` is pretty easy (untested):
>>
>>  pos = 0
>>  while True:
>>  found = s.find(substring, pos)
>>  if found < 0:
>>  break
>>  start = found
>>  end = found + len(substring)
>>  ... do whatever with start and end ...
>>  pos = end
>>
>> Many people go straight to the `re` module whenever they're looking for strings. It is often cryptic error prone overkill. Just something to keep in mind.
>>
>> Cheers,
>> Cameron Simpson <cs at cskk.id.au>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From jenkris at tutanota.com  Tue Feb 28 13:07:42 2023
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 28 Feb 2023 19:07:42 +0100 (CET)
Subject: How to escape strings for re.finditer?
In-Reply-To: <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
Message-ID: <NPOCM6N--B-9@tutanota.com>


Using str.startswith is a cool idea in this case.? But is it better than regex for performance or reliability?? Regex syntax is not a model of simplicity, but in my simple case it's not too difficult.? 


Feb 27, 2023, 18:52 by list1 at tompassin.net:

> On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
>
>> And, just for fun, since there is nothing wrong with your code, this minor change is terser:
>>
>>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
>>>>>
>> ...     print(match.start(), match.end())
>> ...
>> ...
>> 4 18
>> 26 40
>>
>
> Just for more fun :) -
>
> Without knowing how general your expressions will be, I think the following version is very readable, certainly more readable than regexes:
>
> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
> KEY = 'abc_degree + 1'
>
> for i in range(len(example)):
>  if example[i:].startswith(KEY):
>  print(i, i + len(KEY))
> # prints:
> 4 18
> 26 40
>
> If you may have variable numbers of spaces around the symbols, OTOH, the whole situation changes and then regexes would almost certainly be the best approach.  But the regular expression strings would become harder to read.
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From jenkris at tutanota.com  Tue Feb 28 13:16:54 2023
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 28 Feb 2023 19:16:54 +0100 (CET)
Subject: How to escape strings for re.finditer?
In-Reply-To: <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
 <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
Message-ID: <NPOESrJ--3-9@tutanota.com>


I wrote my previous message before reading this.? Thank you for the test you ran -- it answers the question of performance.? You show that re.finditer is 30x faster, so that certainly recommends that over a simple loop, which introduces looping overhead.? 


Feb 28, 2023, 05:44 by list1 at tompassin.net:

> On 2/28/2023 4:33 AM, Roel Schroeven wrote:
>
>> Op 28/02/2023 om 3:44 schreef Thomas Passin:
>>
>>> On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
>>>
>>>> And, just for fun, since there is nothing wrong with your code, this minor change is terser:
>>>>
>>>>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
>>>>>>>
>>>> ...???? print(match.start(), match.end())
>>>> ...
>>>> ...
>>>> 4 18
>>>> 26 40
>>>>
>>>
>>> Just for more fun :) -
>>>
>>> Without knowing how general your expressions will be, I think the following version is very readable, certainly more readable than regexes:
>>>
>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>> KEY = 'abc_degree + 1'
>>>
>>> for i in range(len(example)):
>>> ??? if example[i:].startswith(KEY):
>>> ??????? print(i, i + len(KEY))
>>> # prints:
>>> 4 18
>>> 26 40
>>>
>> I think it's often a good idea to use a standard library function instead of rolling your own. The issue becomes less clear-cut when the standard library doesn't do exactly what you need (as here, where re.finditer() uses regular expressions while the use case only uses simple search strings). Ideally there would be a str.finditer() method we could use, but in the absence of that I think we still need to consider using the almost-but-not-quite fitting re.finditer().
>>
>> Two reasons:
>>
>> (1) I think it's clearer: the name tells us what it does (though of course we could solve this in a hand-written version by wrapping it in a suitably named function).
>>
>> (2) Searching for a string in another string, in a performant way, is not as simple as it first appears. Your version works correctly, but slowly. In some situations it doesn't matter, but in other cases it will. For better performance, string searching algorithms jump ahead either when they found a match or when they know for sure there isn't a match for some time (see e.g. the Boyer?Moore string-search algorithm). You could write such a more efficient algorithm, but then it becomes more complex and more error-prone. Using a well-tested existing function becomes quite attractive.
>>
>
> Sure, it all depends on what the real task will be.  That's why I wrote "Without knowing how general your expressions will be". For the example string, it's unlikely that speed will be a factor, but who knows what target strings and keys will turn up in the future?
>
>> To illustrate the difference performance, I did a simple test (using the paragraph above is test text):
>>
>>  ??? import re
>>  ??? import timeit
>>
>>  ??? def using_re_finditer(key, text):
>>  ??????? matches = []
>>  ??????? for match in re.finditer(re.escape(key), text):
>>  ??????????? matches.append((match.start(), match.end()))
>>  ??????? return matches
>>
>>
>>  ??? def using_simple_loop(key, text):
>>  ??????? matches = []
>>  ??????? for i in range(len(text)):
>>  ??????????? if text[i:].startswith(key):
>>  ??????????????? matches.append((i, i + len(key)))
>>  ??????? return matches
>>
>>
>>  ??? CORPUS = """Searching for a string in another string, in a performant way, is
>>  ??? not as simple as it first appears. Your version works correctly, but slowly.
>>  ??? In some situations it doesn't matter, but in other cases it will. For better
>>  ??? performance, string searching algorithms jump ahead either when they found a
>>  ??? match or when they know for sure there isn't a match for some time (see e.g.
>>  ??? the Boyer?Moore string-search algorithm). You could write such a more
>>  ??? efficient algorithm, but then it becomes more complex and more error-prone.
>>  ??? Using a well-tested existing function becomes quite attractive."""
>>  ??? KEY = 'in'
>>  ??? print('using_simple_loop:', timeit.repeat(stmt='using_simple_loop(KEY, CORPUS)', globals=globals(), number=1000))
>>  ??? print('using_re_finditer:', timeit.repeat(stmt='using_re_finditer(KEY, CORPUS)', globals=globals(), number=1000))
>>
>> This does 5 runs of 1000 repetitions each, and reports the time in seconds for each of those runs.
>> Result on my machine:
>>
>>  ??? using_simple_loop: [0.13952950000020792, 0.13063130000000456, 0.12803450000001249, 0.13186180000002423, 0.13084610000032626]
>>  ??? using_re_finditer: [0.003861400000005233, 0.004061900000124297, 0.003478999999970256, 0.003413100000216218, 0.0037320000001273]
>>
>> We find that in this test re.finditer() is more than 30 times faster (despite the overhead of regular expressions.
>>
>> While speed isn't everything in programming, with such a large difference in performance and (to me) no real disadvantages of using re.finditer(), I would prefer re.finditer() over writing my own.
>>
>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From jon+usenet at unequivocal.eu  Tue Feb 28 11:48:29 2023
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Tue, 28 Feb 2023 16:48:29 -0000 (UTC)
Subject: How to escape strings for re.finditer?
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
 <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
 <d9c55bbd-6502-c9fb-41fc-9fc15a22691c@roelschroeven.net>
 <6bc201c0-c4a7-060f-8c16-87e9efa63388@tompassin.net>
 <mailman.2071.1677599475.20444.python-list@python.org>
Message-ID: <slrntvsc2t.6q7u.jon+usenet@raven.unequivocal.eu>

On 2023-02-28, Thomas Passin <list1 at tompassin.net> wrote:
> On 2/28/2023 10:05 AM, Roel Schroeven wrote:
>> Op 28/02/2023 om 14:35 schreef Thomas Passin:
>>> On 2/28/2023 4:33 AM, Roel Schroeven wrote:
>>>> [...]
>>>> (2) Searching for a string in another string, in a performant way, is 
>>>> not as simple as it first appears. Your version works correctly, but 
>>>> slowly. In some situations it doesn't matter, but in other cases it 
>>>> will. For better performance, string searching algorithms jump ahead 
>>>> either when they found a match or when they know for sure there isn't 
>>>> a match for some time (see e.g. the Boyer?Moore string-search 
>>>> algorithm). You could write such a more efficient algorithm, but then 
>>>> it becomes more complex and more error-prone. Using a well-tested 
>>>> existing function becomes quite attractive.
>>>
>>> Sure, it all depends on what the real task will be.? That's why I 
>>> wrote "Without knowing how general your expressions will be". For the 
>>> example string, it's unlikely that speed will be a factor, but who 
>>> knows what target strings and keys will turn up in the future?
>> On hindsight I think it was overthinking things a bit. "It all depends 
>> on what the real task will be" you say, and indeed I think that should 
>> be the main conclusion here.
>
> It is interesting, though, how pre-processing the search pattern can 
> improve search times if you can afford the pre-processing.  Here's a 
> paper on rapidly finding matches when there may be up to one misspelled 
> character.  It's easy enough to implement, though in Python you can't 
> take the additional step of tuning it to stay in cache.
>
> https://Robert.Muth.Org/Papers/1996-Approx-Multi.Pdf

You've somehow title-cased that URL. The correct URL is:

https://robert.muth.org/Papers/1996-approx-multi.pdf

From list1 at tompassin.net  Tue Feb 28 13:30:32 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 28 Feb 2023 13:30:32 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPOCM6N--B-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <NPOCM6N--B-9@tutanota.com>
Message-ID: <4c28e6fe-c030-cdf4-cd43-95ae1010d24a@tompassin.net>

On 2/28/2023 1:07 PM, Jen Kris wrote:
> 
> Using str.startswith is a cool idea in this case.? But is it better than 
> regex for performance or reliability?? Regex syntax is not a model of 
> simplicity, but in my simple case it's not too difficult.

The trouble is that we don't know what your case really is.  If you are 
talking about a short pattern like your example and a small text to 
search, and you don't need to do it too often, then my little code 
example is probably ideal. Reliability wouldn't be an issue, and 
performance would not be relevant.  If your case is going to be much 
larger, called many times in a loop, or be much more complicated in some 
other way, then a regex or some other approach is likely to be much faster.


> Feb 27, 2023, 18:52 by list1 at tompassin.net:
> 
>     On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
> 
>         And, just for fun, since there is nothing wrong with your code,
>         this minor change is terser:
> 
>                     example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>                     for match in re.finditer(re.escape('abc_degree + 1')
>                     , example):
> 
>         ... print(match.start(), match.end())
>         ...
>         ...
>         4 18
>         26 40
> 
> 
>     Just for more fun :) -
> 
>     Without knowing how general your expressions will be, I think the
>     following version is very readable, certainly more readable than
>     regexes:
> 
>     example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>     KEY = 'abc_degree + 1'
> 
>     for i in range(len(example)):
>     if example[i:].startswith(KEY):
>     print(i, i + len(KEY))
>     # prints:
>     4 18
>     26 40
> 
>     If you may have variable numbers of spaces around the symbols, OTOH,
>     the whole situation changes and then regexes would almost certainly
>     be the best approach. But the regular expression strings would
>     become harder to read.
>     -- 
>     https://mail.python.org/mailman/listinfo/python-list
> 
> 


From avi.e.gross at gmail.com  Tue Feb 28 13:31:22 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 28 Feb 2023 13:31:22 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
Message-ID: <003b01d94ba2$dbd33f20$9379bd60$@gmail.com>

Roel,

You make some good points. One to consider is that when you ask a regular expression matcher to search using something that uses NO regular expression features, much of the complexity disappears and what it creates is probably similar enough to what you get with a string search except that loops and all are written as something using fast functions probably written in C. 

That is one reason the roll your own versions have a disadvantage unless you roll your own in a similar way by writing a similar C function.

Nobody has shown us what really should be out there of a simple but fast text search algorithm that does a similar job and it may still be out there, but as you point out, perhaps it is not needed as long as people just use the re version.

Avi

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Roel Schroeven
Sent: Tuesday, February 28, 2023 4:33 AM
To: python-list at python.org
Subject: Re: How to escape strings for re.finditer?

Op 28/02/2023 om 3:44 schreef Thomas Passin:
> On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
>> And, just for fun, since there is nothing wrong with your code, this 
>> minor change is terser:
>>
>>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
>> ...     print(match.start(), match.end()) ...
>> ...
>> 4 18
>> 26 40
>
> Just for more fun :) -
>
> Without knowing how general your expressions will be, I think the 
> following version is very readable, certainly more readable than regexes:
>
> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
> KEY = 'abc_degree + 1'
>
> for i in range(len(example)):
>     if example[i:].startswith(KEY):
>         print(i, i + len(KEY))
> # prints:
> 4 18
> 26 40
I think it's often a good idea to use a standard library function instead of rolling your own. The issue becomes less clear-cut when the standard library doesn't do exactly what you need (as here, where
re.finditer() uses regular expressions while the use case only uses simple search strings). Ideally there would be a str.finditer() method we could use, but in the absence of that I think we still need to consider using the almost-but-not-quite fitting re.finditer().

Two reasons:

(1) I think it's clearer: the name tells us what it does (though of course we could solve this in a hand-written version by wrapping it in a suitably named function).

(2) Searching for a string in another string, in a performant way, is not as simple as it first appears. Your version works correctly, but slowly. In some situations it doesn't matter, but in other cases it will. For better performance, string searching algorithms jump ahead either when they found a match or when they know for sure there isn't a match for some time (see e.g. the Boyer?Moore string-search algorithm). 
You could write such a more efficient algorithm, but then it becomes more complex and more error-prone. Using a well-tested existing function becomes quite attractive.

To illustrate the difference performance, I did a simple test (using the paragraph above is test text):

     import re
     import timeit

     def using_re_finditer(key, text):
         matches = []
         for match in re.finditer(re.escape(key), text):
             matches.append((match.start(), match.end()))
         return matches


     def using_simple_loop(key, text):
         matches = []
         for i in range(len(text)):
             if text[i:].startswith(key):
                 matches.append((i, i + len(key)))
         return matches


     CORPUS = """Searching for a string in another string, in a performant way, is
     not as simple as it first appears. Your version works correctly, but slowly.
     In some situations it doesn't matter, but in other cases it will. 
For better
     performance, string searching algorithms jump ahead either when they found a
     match or when they know for sure there isn't a match for some time (see e.g.
     the Boyer?Moore string-search algorithm). You could write such a more
     efficient algorithm, but then it becomes more complex and more error-prone.
     Using a well-tested existing function becomes quite attractive."""
     KEY = 'in'
     print('using_simple_loop:',
timeit.repeat(stmt='using_simple_loop(KEY, CORPUS)', globals=globals(),
number=1000))
     print('using_re_finditer:',
timeit.repeat(stmt='using_re_finditer(KEY, CORPUS)', globals=globals(),
number=1000))

This does 5 runs of 1000 repetitions each, and reports the time in seconds for each of those runs.
Result on my machine:

     using_simple_loop: [0.13952950000020792, 0.13063130000000456, 0.12803450000001249, 0.13186180000002423, 0.13084610000032626]
     using_re_finditer: [0.003861400000005233, 0.004061900000124297, 0.003478999999970256, 0.003413100000216218, 0.0037320000001273]

We find that in this test re.finditer() is more than 30 times faster (despite the overhead of regular expressions.

While speed isn't everything in programming, with such a large difference in performance and (to me) no real disadvantages of using re.finditer(), I would prefer re.finditer() over writing my own.

--
"The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom."
         -- Isaac Asimov

--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Tue Feb 28 13:23:25 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 28 Feb 2023 13:23:25 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPOA6-y--3-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <NPOA6-y--3-9@tutanota.com>
Message-ID: <ac511719-da62-4c49-d265-415bf431cfe9@tompassin.net>

On 2/28/2023 12:57 PM, Jen Kris via Python-list wrote:
> The code I sent is correct, and it runs here.? Maybe you received it with a carriage return removed, but on my copy after posting, it is correct:
> 
> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>  ?find_string = re.escape('abc_degree + 1')
>  ?for match in re.finditer(find_string, example):
>  ???? print(match.start(), match.end())
> 
> One question:? several people have made suggestions other than regex (not your terser example with regex you shown below).? Is there a reason why regex is not preferred to, for example, a list comp?? Performance?? Reliability?

"Some people, when confronted with a problem, think 'I know, I'll use 
regular expressions.' Now they have two problems."

- 
https://blog.codinghorror.com/regular-expressions-now-you-have-two-problems/

Of course, if you actually read the blog post in the link, there's more 
to it than that...

> 
> Feb 27, 2023, 18:16 by avi.e.gross at gmail.com:
> 
>> Jen,
>>
>> Can you see what SOME OF US see as ASCII text? We can help you better if we get code that can be copied and run as-is.
>>
>>   What you sent is not terse. It is wrong. It will not run on any python interpreter because you somehow lost a carriage return and indent.
>>
>> This is what you sent:
>>
>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>> find_string = re.escape('abc_degree + 1') for match in re.finditer(find_string, example):
>>   print(match.start(), match.end())
>>
>> This is code indentedproperly:
>>
>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>> find_string = re.escape('abc_degree + 1')
>> for match in re.finditer(find_string, example):
>>   print(match.start(), match.end())
>>
>> Of course I am sure you wrote and ran code more like the latter version but somewhere in your copy/paste process, ....
>>
>> And, just for fun, since there is nothing wrong with your code, this minor change is terser:
>>
>>>>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>>>>> for match in re.finditer(re.escape('abc_degree + 1') , example):
>>>>>
>> ...     print(match.start(), match.end())
>> ...
>> ...
>> 4 18
>> 26 40
>>
>> But note once you use regular expressions, and not in your case, you might match multiple things that are far from the same such as matching two repeated words of any kind in any case including "and and" and "so so" or finding words that have multiple doubled letter as in the  stereotypical bookkeeper. In those cases, you may want even more than offsets but also show the exact text that matched or even show some characters before and/or after for context.
>>
>>
>> -----Original Message-----
>> From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Jen Kris via Python-list
>> Sent: Monday, February 27, 2023 8:36 PM
>> To: Cameron Simpson <cs at cskk.id.au>
>> Cc: Python List <python-list at python.org>
>> Subject: Re: How to escape strings for re.finditer?
>>
>>
>> I haven't tested it either but it looks like it would work.  But for this case I prefer the relative simplicity of:
>>
>> example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>> find_string = re.escape('abc_degree + 1') for match in re.finditer(find_string, example):
>>   print(match.start(), match.end())
>>
>> 4 18
>> 26 40
>>
>> I don't insist on terseness for its own sake, but it's cleaner this way.
>>
>> Jen
>>
>>
>> Feb 27, 2023, 16:55 by cs at cskk.id.au:
>>
>>> On 28Feb2023 01:13, Jen Kris <jenkris at tutanota.com> wrote:
>>>
>>>> I went to the re module because the specified string may appear more than once in the string (in the code I'm writing).
>>>>
>>>
>>> Sure, but writing a `finditer` for plain `str` is pretty easy (untested):
>>>
>>>   pos = 0
>>>   while True:
>>>   found = s.find(substring, pos)
>>>   if found < 0:
>>>   break
>>>   start = found
>>>   end = found + len(substring)
>>>   ... do whatever with start and end ...
>>>   pos = end
>>>
>>> Many people go straight to the `re` module whenever they're looking for strings. It is often cryptic error prone overkill. Just something to keep in mind.
>>>
>>> Cheers,
>>> Cameron Simpson <cs at cskk.id.au>
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>>>
>>
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>>
> 


From garyfallidis at gmail.com  Tue Feb 28 13:48:56 2023
From: garyfallidis at gmail.com (Eleftherios Garyfallidis)
Date: Tue, 28 Feb 2023 13:48:56 -0500
Subject: ANN: DIPY 1.6.0
Message-ID: <CAN6JmWM5L6FtqMeFQbeoB=4HKwBLnOBaeS+=fE2xycqV_d2d+Q@mail.gmail.com>

Hello all,


We are excited to announce a new release of DIPY: DIPY 1.6.0 is out from
the oven!


In addition, registration for the oceanic DIPY workshop 2023 (April 24-28)
is now open! Our comprehensive program is designed to equip you with the
skills and knowledge needed to master the latest techniques and tools in
structural and diffusion imaging.  An intense hands-on experience in Santa
Monica, Los Angeles! See the exquisite program here
<https://dipy.org/workshops/dipy-workshop-2023#Schedule> for this highly
anticipated event <https://dipy.org/workshops/dipy-workshop-2023>.



DIPY 1.6.0 (Monday, 16 January 2023)

The release 1.6.0 received contributions from 22 developers (the full
release notes are at:
https://dipy.org/documentation/1.6.0./release_notes/release1.6/).


Thank you all for your contributions and feedback!


Please click here
<https://dipy.org/documentation/1.6.0./api_changes/#dipy-1-6-0-changes> to
check 1.6.0 API changes.


Highlights of 1.6.0 release include:

   -

   NF: Unbiased groupwise linear bundle registration added.
   -

   NF: MAP+ constraints added.
   -

   Generalized PCA to less than 3 spatial dims.
   -

   Added positivity constraints to QTI.
   -

   New functionality to apply Symmetric Diffeomorphic Registration to
   points/streamlines.
   -

   New Human Connectome Project (HCP) data fetcher added.
   -

   New Healthy Brain Network (HBN) data fetcher added.
   -

   Multiple Workflows updated (DTIFlow, LPCAFlow, MPPCA) and added
   (RUMBAFlow).
   -

   Ability to handle VTP files.
   -

   Large codebase cleaning.
   -

   Large documentation update.
   -

   Closed 75 issues and merged 41 pull requests.

To upgrade or install <http://dipy.org/release0.10.html> DIPY


Run the following command in your terminal:
<http://dipy.org/release0.10.html>


pip install --upgrade dipy


or


conda install -c conda-forge dipy


This version of DIPY depends on nibabel (3.0.0+).

For visualization you need FURY (0.8.0+).


Please support us by citing DIPY in your papers using the following
DOI: 10.3389/fninf.2014.00008
<https://www.ncbi.nlm.nih.gov/pubmed/24600385>


Questions or suggestions?



For any questions go to https://dipy.org, or send an e-mail to
dipy at python.org <neuroimaging at python.org>

We also have an instant messaging service and chat room available at
https://gitter.im/dipy/dipy

Finally, a new forum is available at
https://github.com/dipy/dipy/discussions


Have a wonderful time using the new version.


On behalf of the DIPY developers,

Eleftherios Garyfallidis, Ariel Rokem, Serge Koudoro

https://dipy.org/contributors

From list1 at tompassin.net  Tue Feb 28 14:17:30 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 28 Feb 2023 14:17:30 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <slrntvsc2t.6q7u.jon+usenet@raven.unequivocal.eu>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
 <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
 <d9c55bbd-6502-c9fb-41fc-9fc15a22691c@roelschroeven.net>
 <6bc201c0-c4a7-060f-8c16-87e9efa63388@tompassin.net>
 <mailman.2071.1677599475.20444.python-list@python.org>
 <slrntvsc2t.6q7u.jon+usenet@raven.unequivocal.eu>
Message-ID: <9c5a71e7-b9fe-356f-74bf-edec17f0b1f6@tompassin.net>

On 2/28/2023 11:48 AM, Jon Ribbens via Python-list wrote:
> On 2023-02-28, Thomas Passin <list1 at tompassin.net> wrote:
...
>>
>> It is interesting, though, how pre-processing the search pattern can
>> improve search times if you can afford the pre-processing.  Here's a
>> paper on rapidly finding matches when there may be up to one misspelled
>> character.  It's easy enough to implement, though in Python you can't
>> take the additional step of tuning it to stay in cache.
>>
>> https://Robert.Muth.Org/Papers/1996-Approx-Multi.Pdf
> 
> You've somehow title-cased that URL. The correct URL is:
> 
> https://robert.muth.org/Papers/1996-approx-multi.pdf

Thanks, not sure how that happened ...


From PythonList at DancesWithMice.info  Tue Feb 28 14:21:40 2023
From: PythonList at DancesWithMice.info (dn)
Date: Wed, 1 Mar 2023 08:21:40 +1300
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <749b4375-9dd4-99e1-19bc-dbc90fc068bc@btinternet.com>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
 <749b4375-9dd4-99e1-19bc-dbc90fc068bc@btinternet.com>
Message-ID: <14355d71-81af-422b-9d80-68768cd9bb2e@DancesWithMice.info>

On 28/02/2023 12.55, Rob Cliffe via Python-list wrote:
> 
> 
> On 27/02/2023 21:04, Ethan Furman wrote:
>> On 2/27/23 12:20, rbowman wrote:
>>
>> > "By using Black, you agree to cede control over minutiae of hand-
>> > formatting. In return, Black gives you speed, determinism, and freedom
>> > from pycodestyle nagging about formatting. You will save time and 
>> mental
>> > energy for more important matters."
>> >
>> > Somehow I don't think we would get along very well. I'm a little on the
>> > opinionated side myself.
>>
>> I personally cannot stand Black.? It feels like every major choice it 
>> makes (and some minor ones) are exactly the opposite of the choice I 
>> make.
>>
>> -- 
>> ~Ethan~
> I've never tried Black or any other code formatter, but I'm sure we 
> wouldn't get on.

Does this suggest, that because Black doesn't respect other people's 
opinions and feelings, that it wouldn't meet the PSF's Code of Conduct?

-- 
Regards,
=dn

From rosuav at gmail.com  Tue Feb 28 14:29:47 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 1 Mar 2023 06:29:47 +1100
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <14355d71-81af-422b-9d80-68768cd9bb2e@DancesWithMice.info>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
 <749b4375-9dd4-99e1-19bc-dbc90fc068bc@btinternet.com>
 <14355d71-81af-422b-9d80-68768cd9bb2e@DancesWithMice.info>
Message-ID: <CAPTjJmq_012kSkUxJoZHyXLzF+kkQvu1pPKUQW2Q1t4m8yR6xg@mail.gmail.com>

On Wed, 1 Mar 2023 at 06:25, dn via Python-list <python-list at python.org> wrote:
>
> On 28/02/2023 12.55, Rob Cliffe via Python-list wrote:
> >
> >
> > On 27/02/2023 21:04, Ethan Furman wrote:
> >> On 2/27/23 12:20, rbowman wrote:
> >>
> >> > "By using Black, you agree to cede control over minutiae of hand-
> >> > formatting. In return, Black gives you speed, determinism, and freedom
> >> > from pycodestyle nagging about formatting. You will save time and
> >> mental
> >> > energy for more important matters."
> >> >
> >> > Somehow I don't think we would get along very well. I'm a little on the
> >> > opinionated side myself.
> >>
> >> I personally cannot stand Black.  It feels like every major choice it
> >> makes (and some minor ones) are exactly the opposite of the choice I
> >> make.
> >>
> >> --
> >> ~Ethan~
> > I've never tried Black or any other code formatter, but I'm sure we
> > wouldn't get on.
>
> Does this suggest, that because Black doesn't respect other people's
> opinions and feelings, that it wouldn't meet the PSF's Code of Conduct?
>

Yes, so if Black ever posts on this list, it will probably get banned...

ChrisA

From David.Raymond at tomtom.com  Tue Feb 28 14:40:10 2023
From: David.Raymond at tomtom.com (David Raymond)
Date: Tue, 28 Feb 2023 19:40:10 +0000
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPOESrJ--3-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
 <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
 <NPOESrJ--3-9@tutanota.com>
Message-ID: <HE1PR0701MB2857612CE117BAC1263ACFAC87AC9@HE1PR0701MB2857.eurprd07.prod.outlook.com>

> I wrote my previous message before reading this.? Thank you for the test you ran -- it answers the question of performance.? You show that re.finditer is 30x faster, so that certainly recommends that over a simple loop, which introduces looping overhead.? 

>>  ??? def using_simple_loop(key, text):
>>  ??????? matches = []
>>  ??????? for i in range(len(text)):
>>  ??????????? if text[i:].startswith(key):
>>  ??????????????? matches.append((i, i + len(key)))
>>  ??????? return matches
>>
>>  ??? using_simple_loop: [0.13952950000020792, 0.13063130000000456, 0.12803450000001249, 0.13186180000002423, 0.13084610000032626]
>>  ??? using_re_finditer: [0.003861400000005233, 0.004061900000124297, 0.003478999999970256, 0.003413100000216218, 0.0037320000001273]


With a slight tweak to the simple loop code using .find() it becomes a third faster than the RE version though.


def using_simple_loop2(key, text):
    matches = []
    keyLen = len(key)
    start = 0
    while (foundSpot := text.find(key, start)) > -1:
        start = foundSpot + keyLen
        matches.append((foundSpot, start))
    return matches


using_simple_loop: [0.1732664997689426, 0.1601669997908175, 0.15792609984055161, 0.1573973000049591, 0.15759290009737015]
using_re_finditer: [0.003412699792534113, 0.0032823001965880394, 0.0033694999292492867, 0.003354900050908327, 0.0033336998894810677]
using_simple_loop2: [0.00256159994751215, 0.0025471001863479614, 0.0025424999184906483, 0.0025831996463239193, 0.0025555999018251896]

From Karsten.Hilbert at gmx.net  Tue Feb 28 14:43:36 2023
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Tue, 28 Feb 2023 20:43:36 +0100
Subject: Aw: Re: Python 3.10 Fizzbuzz
In-Reply-To: <14355d71-81af-422b-9d80-68768cd9bb2e@DancesWithMice.info>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
 <749b4375-9dd4-99e1-19bc-dbc90fc068bc@btinternet.com>
 <14355d71-81af-422b-9d80-68768cd9bb2e@DancesWithMice.info>
Message-ID: <trinity-fb3a9260-f644-41cf-9820-cca521feac22-1677613416220@3c-app-gmx-bap53>

> > I've never tried Black or any other code formatter, but I'm sure we
> > wouldn't get on.
>
> Does this suggest, that because Black doesn't respect other people's
> opinions and feelings, that it wouldn't meet the PSF's Code of Conduct?

That much depends on The Measure Of A Man.

Karsten

From avi.e.gross at gmail.com  Tue Feb 28 15:25:05 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 28 Feb 2023 15:25:05 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPOA6-y--3-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <NPOA6-y--3-9@tutanota.com>
Message-ID: <00d001d94bb2$bea422b0$3bec6810$@gmail.com>

Jen,

 

I had no doubt the code you ran was indented properly or it would not work.

 

I am merely letting you know that somewhere in the process of copying the code or the transition between mailers, my version is messed up. It happens to be easy for me to fix but I sometimes see garbled code I then simply ignore.

 

At times what may help is to leave blank lines that python ignores but also keeps the line rearrangements minimal.

 

On to your real question.

 

In my OPINION, there are many interesting questions that can get in the way of just getting a working solution. Some may be better in some abstract way but except for big projects it often hardly matters.

 

So regex is one thing or more a cluster of things and a list comp is something completely different. They are both tools you can use and abuse or lose.

 

The distinction I believe we started with was how to find a fixed string inside another fixed string in as many places as needed and perhaps return offset info. So this can be solved in too many ways using a side of python focused on pure text. As discussed, solutions can include explicit loops such as ?for? and ?while? and their syntactic sugar cousin of a list comp. Not mentioned yet are other techniques like a recursive function that finds the first and passes on the rest of the string to itself to find the rest, or various functional programming techniques that may do sort of hidden loops. YOU DO NOT NEED ALL OF THEM but it can be interesting to learn.

 

Regex is a completely different universe that is a bit more of MORE. If I ask you for a ride to the grocery store, I might expect you to show up with a car and not a James Bond vehicle that also is a boat, submarine, airplane, and maybe spaceship. Well, Regex is the latter. And in your case, it is this complexity that meant you had to convert your text so it will not see what it considers commands or hints.

 

In normal use, put a bit too simply, it wants a carefully crafted pattern to be spelled out and it weaves an often complex algorithm it then sort of compiles that represents the understanding of what you asked for. The simplest pattern is to match EXACTLY THIS. That is your case.

 

A more complex pattern may say to match Boston OR Chicago followed by any amount of whitespace then a number of digits between 3 and 5 and then should not be followed by something specific. Oh, and by the way, save selected parts in parentheses to be accessed as \1 or \2 so I can ask you to do things like match a word followed by itself. It goes on and on. 

 

Be warned RE is implemented now all over the place including outside the usual UNIX roots and there are somewhat different versions. For your need, it does not matter.

 

The compiled monstrosity though can be fairly fast and might be a tad hard for you to write by yourself as a bunch of if statements nested that are  weirdly matching various patterns with some look ahead or look behind. 

 

What you are being told is that despite this being way more than you asked for, it not only works but is fairly fast when doing the simple thing you asked for. That may be why a text version you are looking for is hard to find.

 

I am not clear what exactly the rest of your project is about but my guess is your first priority is completing it decently and not to try umpteen methods and compare them. Not today. Of course if the working version is slow and you profile it and find this part seems to be holding it back, it may be worth examining.

 

 

From: Jen Kris <jenkris at tutanota.com> 
Sent: Tuesday, February 28, 2023 12:58 PM
To: avi.e.gross at gmail.com
Cc: 'Python List' <python-list at python.org>
Subject: RE: How to escape strings for re.finditer?

 

The code I sent is correct, and it runs here.  Maybe you received it with a carriage return removed, but on my copy after posting, it is correct:

 

example = 'X - abc_degree + 1 + qq + abc_degree + 1'

 find_string = re.escape('abc_degree + 1')

 for match in re.finditer(find_string, example):

     print(match.start(), match.end())

 

One question:  several people have made suggestions other than regex (not your terser example with regex you shown below).  Is there a reason why regex is not preferred to, for example, a list comp?  Performance?  Reliability?  

 

 

 

  

 

 

Feb 27, 2023, 18:16 by avi.e.gross at gmail.com <mailto:avi.e.gross at gmail.com> :

Jen,

 

Can you see what SOME OF US see as ASCII text? We can help you better if we get code that can be copied and run as-is.

 

What you sent is not terse. It is wrong. It will not run on any python interpreter because you somehow lost a carriage return and indent.

 

This is what you sent:

 

example = 'X - abc_degree + 1 + qq + abc_degree + 1'

find_string = re.escape('abc_degree + 1') for match in re.finditer(find_string, example):

print(match.start(), match.end())

 

This is code indentedproperly:

 

example = 'X - abc_degree + 1 + qq + abc_degree + 1'

find_string = re.escape('abc_degree + 1') 

for match in re.finditer(find_string, example):

print(match.start(), match.end())

 

Of course I am sure you wrote and ran code more like the latter version but somewhere in your copy/paste process, ....

 

And, just for fun, since there is nothing wrong with your code, this minor change is terser:

example = 'X - abc_degree + 1 + qq + abc_degree + 1'

for match in re.finditer(re.escape('abc_degree + 1') , example):

... print(match.start(), match.end())

... 

... 

4 18

26 40

 

But note once you use regular expressions, and not in your case, you might match multiple things that are far from the same such as matching two repeated words of any kind in any case including "and and" and "so so" or finding words that have multiple doubled letter as in the stereotypical bookkeeper. In those cases, you may want even more than offsets but also show the exact text that matched or even show some characters before and/or after for context.

 

 

-----Original Message-----

From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org <mailto:python-list-bounces+avi.e.gross=gmail.com at python.org> > On Behalf Of Jen Kris via Python-list

Sent: Monday, February 27, 2023 8:36 PM

To: Cameron Simpson <cs at cskk.id.au <mailto:cs at cskk.id.au> >

Cc: Python List <python-list at python.org <mailto:python-list at python.org> >

Subject: Re: How to escape strings for re.finditer?

 

 

I haven't tested it either but it looks like it would work. But for this case I prefer the relative simplicity of:

 

example = 'X - abc_degree + 1 + qq + abc_degree + 1'

find_string = re.escape('abc_degree + 1') for match in re.finditer(find_string, example):

print(match.start(), match.end())

 

4 18

26 40

 

I don't insist on terseness for its own sake, but it's cleaner this way. 

 

Jen

 

 

Feb 27, 2023, 16:55 by cs at cskk.id.au <mailto:cs at cskk.id.au> :

On 28Feb2023 01:13, Jen Kris <jenkris at tutanota.com <mailto:jenkris at tutanota.com> > wrote:

I went to the re module because the specified string may appear more than once in the string (in the code I'm writing).

 

Sure, but writing a `finditer` for plain `str` is pretty easy (untested):

 

pos = 0

while True:

found = s.find(substring, pos)

if found < 0:

break

start = found

end = found + len(substring)

... do whatever with start and end ...

pos = end

 

Many people go straight to the `re` module whenever they're looking for strings. It is often cryptic error prone overkill. Just something to keep in mind.

 

Cheers,

Cameron Simpson <cs at cskk.id.au <mailto:cs at cskk.id.au> >

--

https://mail.python.org/mailman/listinfo/python-list

 

-- 

https://mail.python.org/mailman/listinfo/python-list

 


From avi.e.gross at gmail.com  Tue Feb 28 15:40:26 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 28 Feb 2023 15:40:26 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <4c28e6fe-c030-cdf4-cd43-95ae1010d24a@tompassin.net>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <NPOCM6N--B-9@tutanota.com>
 <4c28e6fe-c030-cdf4-cd43-95ae1010d24a@tompassin.net>
Message-ID: <00eb01d94bb4$e3bad8d0$ab308a70$@gmail.com>

This message is more for Thomas than Jen,

You made me think of what happens in fairly large cases. What happens if I ask you to search a thousand pages looking for your name? 

One solution might be to break the problem into parts that can be run in independent threads or processes and perhaps across different CPU's or on many machines at once. Think of it as a variant on a merge sort where each chunk returns where it found one or more items and then those are gathered together and merged upstream.

The problem is you cannot just randomly divide the text.  Any matches across a divide are lost. So if you know you are searching for "Thomas Passin" you need an overlap big enough to hold enough of that size. It would not be made as something like a pure binary tree and if the choices made included variant sizes in what might match, you would get duplicates. So the merging part would obviously have to eventually remove those.

I have often wondered how Google and other such services are able to find millions of things in hardly any time and arguably never show most of them as who looks past a few pages/screens?

I think much of that may involve other techniques including quite a bit of pre-indexing. But they also seem to enlist lots of processors that each do the search on a subset of the problem space and combine and prioritize.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of Thomas Passin
Sent: Tuesday, February 28, 2023 1:31 PM
To: python-list at python.org
Subject: Re: How to escape strings for re.finditer?

On 2/28/2023 1:07 PM, Jen Kris wrote:
> 
> Using str.startswith is a cool idea in this case.  But is it better 
> than regex for performance or reliability?  Regex syntax is not a 
> model of simplicity, but in my simple case it's not too difficult.

The trouble is that we don't know what your case really is.  If you are talking about a short pattern like your example and a small text to search, and you don't need to do it too often, then my little code example is probably ideal. Reliability wouldn't be an issue, and performance would not be relevant.  If your case is going to be much larger, called many times in a loop, or be much more complicated in some other way, then a regex or some other approach is likely to be much faster.


> Feb 27, 2023, 18:52 by list1 at tompassin.net:
> 
>     On 2/27/2023 9:16 PM, avi.e.gross at gmail.com wrote:
> 
>         And, just for fun, since there is nothing wrong with your code,
>         this minor change is terser:
> 
>                     example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>                     for match in re.finditer(re.escape('abc_degree + 1')
>                     , example):
> 
>         ... print(match.start(), match.end())
>         ...
>         ...
>         4 18
>         26 40
> 
> 
>     Just for more fun :) -
> 
>     Without knowing how general your expressions will be, I think the
>     following version is very readable, certainly more readable than
>     regexes:
> 
>     example = 'X - abc_degree + 1 + qq + abc_degree + 1'
>     KEY = 'abc_degree + 1'
> 
>     for i in range(len(example)):
>     if example[i:].startswith(KEY):
>     print(i, i + len(KEY))
>     # prints:
>     4 18
>     26 40
> 
>     If you may have variable numbers of spaces around the symbols, OTOH,
>     the whole situation changes and then regexes would almost certainly
>     be the best approach. But the regular expression strings would
>     become harder to read.
>     -- 
>     https://mail.python.org/mailman/listinfo/python-list
> 
> 

--
https://mail.python.org/mailman/listinfo/python-list


From mats at wichmann.us  Tue Feb 28 15:52:00 2023
From: mats at wichmann.us (Mats Wichmann)
Date: Tue, 28 Feb 2023 13:52:00 -0700
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <CAHVvXxTCtu6ugipcB1B-5v4r7=ZRhJgDWOffEhxGXH9RUfuiQw@mail.gmail.com>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
 <CAHVvXxTCtu6ugipcB1B-5v4r7=ZRhJgDWOffEhxGXH9RUfuiQw@mail.gmail.com>
Message-ID: <050ca75d-aef6-8536-3182-76f23533858a@wichmann.us>

On 2/27/23 16:42, Oscar Benjamin wrote:
> On Mon, 27 Feb 2023 at 21:06, Ethan Furman <ethan at stoneleaf.us> wrote:
>>
>> On 2/27/23 12:20, rbowman wrote:
>>
>>   > "By using Black, you agree to cede control over minutiae of hand-
>>   > formatting. In return, Black gives you speed, determinism, and freedom
>>   > from pycodestyle nagging about formatting. You will save time and mental
>>   > energy for more important matters."
>>   >
>>   > Somehow I don't think we would get along very well. I'm a little on the
>>   > opinionated side myself.
>>
>> I personally cannot stand Black.  It feels like every major choice it makes (and some minor ones) are exactly the
>> opposite of the choice I make.
> 
> I agree partially. There are two types of decisions black makes:
> 
> 1. Leave the code alone because it seems okay or make small modifications.
> 2. Reformat the code because it violates some generic rule (like line
> too long or something).
> 
> I've recently tried Black and mostly for my code it seems to go with 1
> (code looks okay). There might be some minor changes like double vs
> single quotes but I really don't care about those. In that sense me
> and Black seem to agree.
> 
> However I have also reviewed code where it is clear that the author
> has used black and their code came under case 2. In that case Black
> seems to produce awful things. What I can't understand is someone
> accepting the awful rewrite rather than just fixing the code. Treating
> Black almost like a linter makes sense to me but accepting the
> rewrites that it offers for bad code does not.

The amount of disagreement you see here and elsewhere are exactly why 
Black  is like it is - virtually without options.  It doesn't aim to 
solve the challenge of producing The Most Beautiful Code Layout, for 
*you*, or even for any moderately sized group of programmers.  Instead 
it's to remove the bickering:
1. we choose to use black for a project.
2. black runs automatically
3. there is now no need to spend cycles thinking about code-style 
aspects in reviews, or when we're making changes, because black makes 
sure the code aligns with the chosen style (1).

Many teams find the removal of this potential disagreement valuable - 
there's plenty of more important stuff to spend time on. If as an 
individual user, not trying to conform to style choices of a project, it 
doesn't appeal, there's no need to fuss with it.

One can certainly pick a different code style, and make sure it's 
captured in the rules for one of the several more flexible formatting 
tools (for example, I *used* to use yapf pretty regularly, and had that 
tuned as I wanted)

From avi.e.gross at gmail.com  Tue Feb 28 16:05:19 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 28 Feb 2023 16:05:19 -0500
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <14355d71-81af-422b-9d80-68768cd9bb2e@DancesWithMice.info>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
 <749b4375-9dd4-99e1-19bc-dbc90fc068bc@btinternet.com>
 <14355d71-81af-422b-9d80-68768cd9bb2e@DancesWithMice.info>
Message-ID: <00ef01d94bb8$5d71e030$1855a090$@gmail.com>

Dave, 

Is it rude to name something "black" to make it hard for some of us to remind them of the rules or claim that our personal style is so often the opposite that it should be called "white" or at least shade of gray?

The usual kidding aside, I have no idea what it was called black but in all seriousness this is not a black and white issue. Opinions may differ when a language provides many valid options on how to write code. If someone wants to standardize and impose some decisions, fine. But other may choose their own variant and take their chances.

I, for example, like certain features in many languages where if I am only doing one short line of code, I prefer to skip the fanfare. Consider an (non-python)

If (condition) {
    print(5)
}

Who needs that nonsense? If the language allows it:

If (condition) print(5)

Or in python:

If condition: print(5)

Rather than a multi-line version.

But will I always use the short version? Nope. If I expect to add code later, might as well start with the multi-line form. If the line gets too long, ditto. And, quite importantly, if editing other people's code, I look around and follow their lead.

There often is no (one) right way to do things but there often are many wrong ways. Tools like black (which I know nothing detailed about) can be helpful. But I have experience times when I wrote carefully crafted code (as it happens in R inside the RSTUDIO editor) and selected a region and asked it to reformat, and gasped at how it ruined my neatly arranged code. I just wanted the bit I had added to be formatted the same as the rest already was, not a complete re-make. Luckily, there is an undo. 

There must be some parameterized tools out there that let you set up a profile of your own personal preferences that help keep your code in your own preferred format, and re-arrange it after you have done some editing like copying from somewhere else so it fits together.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On Behalf Of dn via Python-list
Sent: Tuesday, February 28, 2023 2:22 PM
To: python-list at python.org
Subject: Re: Python 3.10 Fizzbuzz

On 28/02/2023 12.55, Rob Cliffe via Python-list wrote:
> 
> 
> On 27/02/2023 21:04, Ethan Furman wrote:
>> On 2/27/23 12:20, rbowman wrote:
>>
>> > "By using Black, you agree to cede control over minutiae of hand- 
>> > formatting. In return, Black gives you speed, determinism, and 
>> > freedom from pycodestyle nagging about formatting. You will save 
>> > time and
>> mental
>> > energy for more important matters."
>> >
>> > Somehow I don't think we would get along very well. I'm a little on 
>> > the opinionated side myself.
>>
>> I personally cannot stand Black.  It feels like every major choice it 
>> makes (and some minor ones) are exactly the opposite of the choice I 
>> make.
>>
>> --
>> ~Ethan~
> I've never tried Black or any other code formatter, but I'm sure we 
> wouldn't get on.

Does this suggest, that because Black doesn't respect other people's opinions and feelings, that it wouldn't meet the PSF's Code of Conduct?

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Tue Feb 28 16:16:54 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 28 Feb 2023 16:16:54 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <HE1PR0701MB2857612CE117BAC1263ACFAC87AC9@HE1PR0701MB2857.eurprd07.prod.outlook.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
 <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
 <NPOESrJ--3-9@tutanota.com>
 <HE1PR0701MB2857612CE117BAC1263ACFAC87AC9@HE1PR0701MB2857.eurprd07.prod.outlook.com>
Message-ID: <00f301d94bb9$fbea9cb0$f3bfd610$@gmail.com>

David,

Your results suggest we need to be reminded that lots depends on other
factors. There are multiple versions/implementations of python out there
including some written in C but also other underpinnings. Each can often
have sections of pure python code replaced carefully with libraries of
compiled code, or not. So your results will vary.

Just as an example, assume you derive a type of your own as a subclass of
str and you over-ride the find method by writing it in pure python using
loops and maybe add a few bells and whistles. If you used your improved
algorithm using this variant of str, might it not be quite a bit slower?
Imagine how much slower if your improvement also implemented caching and
logging and the option of ignoring case which are not really needed here.

This type of thing can happen in many other scenarios and some module may be
shared that is slow and a while later is updated but not everyone installs
the update so performance stats can vary wildly. 

Some people advocate using some functional programming tactics, in various
languages, partially because the more general loops are SLOW. But that is
largely because some of the functional stuff is a compiled function that
hides the loops inside a faster environment than the interpreter.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of David Raymond
Sent: Tuesday, February 28, 2023 2:40 PM
To: python-list at python.org
Subject: RE: How to escape strings for re.finditer?

> I wrote my previous message before reading this.? Thank you for the test
you ran -- it answers the question of performance.? You show that
re.finditer is 30x faster, so that certainly recommends that over a simple
loop, which introduces looping overhead.? 

>>  ??? def using_simple_loop(key, text):
>>  ??????? matches = []
>>  ??????? for i in range(len(text)):
>>  ??????????? if text[i:].startswith(key):
>>  ??????????????? matches.append((i, i + len(key)))
>>  ??????? return matches
>>
>>  ??? using_simple_loop: [0.13952950000020792, 0.13063130000000456,
0.12803450000001249, 0.13186180000002423, 0.13084610000032626]
>>  ??? using_re_finditer: [0.003861400000005233, 0.004061900000124297,
0.003478999999970256, 0.003413100000216218, 0.0037320000001273]


With a slight tweak to the simple loop code using .find() it becomes a third
faster than the RE version though.


def using_simple_loop2(key, text):
    matches = []
    keyLen = len(key)
    start = 0
    while (foundSpot := text.find(key, start)) > -1:
        start = foundSpot + keyLen
        matches.append((foundSpot, start))
    return matches


using_simple_loop: [0.1732664997689426, 0.1601669997908175,
0.15792609984055161, 0.1573973000049591, 0.15759290009737015]
using_re_finditer: [0.003412699792534113, 0.0032823001965880394,
0.0033694999292492867, 0.003354900050908327, 0.0033336998894810677]
using_simple_loop2: [0.00256159994751215, 0.0025471001863479614,
0.0025424999184906483, 0.0025831996463239193, 0.0025555999018251896]
-- 
https://mail.python.org/mailman/listinfo/python-list


From avi.e.gross at gmail.com  Tue Feb 28 16:33:19 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 28 Feb 2023 16:33:19 -0500
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <trinity-fb3a9260-f644-41cf-9820-cca521feac22-1677613416220@3c-app-gmx-bap53>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
 <749b4375-9dd4-99e1-19bc-dbc90fc068bc@btinternet.com>
 <14355d71-81af-422b-9d80-68768cd9bb2e@DancesWithMice.info>
 <trinity-fb3a9260-f644-41cf-9820-cca521feac22-1677613416220@3c-app-gmx-bap53>
Message-ID: <00f801d94bbc$46fa6490$d4ef2db0$@gmail.com>

Karsten,

Would it be OK if we paused this discussion a day till February is History?

Sarcasm aside, I repeat, the word black has many unrelated meanings as
presumably this case includes. And for those who do not keep close track of
the local US nonsense, February has for some reason been dedicated to be a
National Black History Month.

Can software violate a code for human conduct? The recent AI news suggests
it does! LOL!

But you know, if you hire a program to tell you if your code passes a
designated series of tests and it just points out where they did not, and
suggest changes that may put you in alignment, that by itself is not
abusive. But if you did not ask for their opinion, yes, it can be annoying
as being unsolicited.

Humans can be touchy and lose context. I have people in my life who
magically ignore my carefully thought-out phrases like "If ..." by acting as
if I had said something rather than IF something. Worse, they hear
abstractions too concretely. I might be discussing COVID and saying "If
COVID was a lethal as it used to be ..." and they interject BUT IT ISN'T.
OK, listen again. I am abstract and trying to make a point. The fact that
you think it isn't is nice to note but hardly relevant to a WHAT IF
question.

So a program designed by programmers, a few of whom are not well known for
how they interact with humans but who nonetheless insist on designed user
interfaces by themselves, may well come across negatively. The reality is
humans vary tremendously and one may appreciate feedback as a way to improve
and get out of the red and the other will assume it is a put down that
leaves them black and blue, even when the words are the same.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Karsten Hilbert
Sent: Tuesday, February 28, 2023 2:44 PM
To: PythonList at DancesWithMice.info
Cc: python-list at python.org
Subject: Aw: Re: Python 3.10 Fizzbuzz

> > I've never tried Black or any other code formatter, but I'm sure we 
> > wouldn't get on.
>
> Does this suggest, that because Black doesn't respect other people's 
> opinions and feelings, that it wouldn't meet the PSF's Code of Conduct?

That much depends on The Measure Of A Man.

Karsten
--
https://mail.python.org/mailman/listinfo/python-list


From list1 at tompassin.net  Tue Feb 28 16:26:04 2023
From: list1 at tompassin.net (Thomas Passin)
Date: Tue, 28 Feb 2023 16:26:04 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <HE1PR0701MB2857612CE117BAC1263ACFAC87AC9@HE1PR0701MB2857.eurprd07.prod.outlook.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <13943d05-ccf0-588a-e0e0-9b3dca4b7e0a@roelschroeven.net>
 <a48a755b-bd08-793f-1511-640b7e337600@tompassin.net>
 <NPOESrJ--3-9@tutanota.com>
 <HE1PR0701MB2857612CE117BAC1263ACFAC87AC9@HE1PR0701MB2857.eurprd07.prod.outlook.com>
Message-ID: <9ee17200-be0c-83aa-2010-8028e74eb044@tompassin.net>

On 2/28/2023 2:40 PM, David Raymond wrote:
> With a slight tweak to the simple loop code using .find() it becomes a third faster than the RE version though.
> 
> 
> def using_simple_loop2(key, text):
>      matches = []
>      keyLen = len(key)
>      start = 0
>      while (foundSpot := text.find(key, start)) > -1:
>          start = foundSpot + keyLen
>          matches.append((foundSpot, start))
>      return matches
> 
> 
> using_simple_loop: [0.1732664997689426, 0.1601669997908175, 0.15792609984055161, 0.1573973000049591, 0.15759290009737015]
> using_re_finditer: [0.003412699792534113, 0.0032823001965880394, 0.0033694999292492867, 0.003354900050908327, 0.0033336998894810677]
> using_simple_loop2: [0.00256159994751215, 0.0025471001863479614, 0.0025424999184906483, 0.0025831996463239193, 0.0025555999018251896]

On my system the difference is way bigger than that:

KEY = '''it doesn't matter, but in other cases it will.'''

using_simple_loop2: [0.0004955999902449548, 0.0004844000213779509, 
0.0004862999776378274, 0.0004800999886356294, 0.0004792999825440347]

using_re_finditer: [0.002840900036972016, 0.0028330000350251794, 
0.002701299963518977, 0.0028105000383220613, 0.0029977999511174858]

Shorter keys show the least differential:

KEY = 'in'

using_simple_loop2: [0.001983499969355762, 0.0019614999764598906, 
0.0019617999787442386, 0.002027600014116615, 0.0020669000223279]

using_re_finditer: [0.002787900040857494, 0.0027620999608188868, 
0.0027723999810405076, 0.002776700013782829, 0.002946800028439611]

Brilliant!

Python 3.10.9
Windows 10 AMD64 (build 10.0.19044) SP0


From oscar.j.benjamin at gmail.com  Tue Feb 28 16:42:11 2023
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Tue, 28 Feb 2023 21:42:11 +0000
Subject: Python 3.10 Fizzbuzz
In-Reply-To: <050ca75d-aef6-8536-3182-76f23533858a@wichmann.us>
References: <87v8qaxyna.fsf@nightsong.com>
 <b22897de-91ea-4454-a34d-02da0c41875fn@googlegroups.com>
 <c092cb77-12a3-cf29-115c-1386e8a005d4@wichmann.us>
 <mailman.2037.1677513701.20444.python-list@python.org>
 <k64hksFhbvkU4@mid.individual.net>
 <e3d8b25e-4546-faaa-c219-8524093b5838@stoneleaf.us>
 <CAHVvXxTCtu6ugipcB1B-5v4r7=ZRhJgDWOffEhxGXH9RUfuiQw@mail.gmail.com>
 <050ca75d-aef6-8536-3182-76f23533858a@wichmann.us>
Message-ID: <CAHVvXxTj6_FiCmEVsPMq9u=oMmqFwzT13LWNTu_vkp5XLC63iw@mail.gmail.com>

On Tue, 28 Feb 2023 at 20:55, Mats Wichmann <mats at wichmann.us> wrote:
>
> On 2/27/23 16:42, Oscar Benjamin wrote:
> > On Mon, 27 Feb 2023 at 21:06, Ethan Furman <ethan at stoneleaf.us> wrote:
> >>
> >> On 2/27/23 12:20, rbowman wrote:
> >>
> >>   > "By using Black, you agree to cede control over minutiae of hand-
> >>   > formatting. In return, Black gives you speed, determinism, and freedom
> >>   > from pycodestyle nagging about formatting. You will save time and mental
> >>   > energy for more important matters."
> >>   >
> >>   > Somehow I don't think we would get along very well. I'm a little on the
> >>   > opinionated side myself.
> >>
> >> I personally cannot stand Black.  It feels like every major choice it makes (and some minor ones) are exactly the
> >> opposite of the choice I make.
> >
> > I agree partially. There are two types of decisions black makes:
> >
> > 1. Leave the code alone because it seems okay or make small modifications.
> > 2. Reformat the code because it violates some generic rule (like line
> > too long or something).
> >
> > I've recently tried Black and mostly for my code it seems to go with 1
> > (code looks okay). There might be some minor changes like double vs
> > single quotes but I really don't care about those. In that sense me
> > and Black seem to agree.
> >
> > However I have also reviewed code where it is clear that the author
> > has used black and their code came under case 2. In that case Black
> > seems to produce awful things. What I can't understand is someone
> > accepting the awful rewrite rather than just fixing the code. Treating
> > Black almost like a linter makes sense to me but accepting the
> > rewrites that it offers for bad code does not.
>
> The amount of disagreement you see here and elsewhere are exactly why
> Black  is like it is - virtually without options.  It doesn't aim to
> solve the challenge of producing The Most Beautiful Code Layout, for
> *you*, or even for any moderately sized group of programmers.  Instead
> it's to remove the bickering:
> 1. we choose to use black for a project.
> 2. black runs automatically
> 3. there is now no need to spend cycles thinking about code-style
> aspects in reviews, or when we're making changes, because black makes
> sure the code aligns with the chosen style (1).

The problem is that although Black runs automatically it doesn't solve
the code problems automatically. Instead it takes something
questionable and produces something worse. If Black just rejected the
author's code and told them to write something better then they
probably would produce something better than what Black produces.

The limitation of Black is that it only reformats but usually at the
point when it does that the option of reformatting is not really the
thing that needs doing. Instead the right option is something like
introducing a new variable to split one statement into two but Black
just goes ahead and reformats without considering that option.

I'm fine with not arguing about what kinds of quotes to use but that
doesn't mean that I'll accept any output from Black without arguing
about the code being improved.

--
Oscar

From cs at cskk.id.au  Tue Feb 28 16:58:59 2023
From: cs at cskk.id.au (Cameron Simpson)
Date: Wed, 1 Mar 2023 08:58:59 +1100
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPOA6-y--3-9@tutanota.com>
References: <NPOA6-y--3-9@tutanota.com>
Message-ID: <Y/55Ixk7UwBeYuz7@cskk.homeip.net>

On 28Feb2023 18:57, Jen Kris <jenkris at tutanota.com> wrote:
>One question:? several people have made suggestions other than regex 
>(not your terser example with regex you shown below).? Is there a 
>reason why regex is not preferred to, for example, a list comp?? 

These are different things; I'm not sure a comparison is meaningful.

>Performance?? Reliability??

Regexps are:
- cryptic and error prone (you can make them more readable, but the 
   notation is deliberately both terse and powerful, which means that 
   small changes can have large effects in behaviour); the "error prone" 
   part does not mean that a regexp is unreliable, but that writing one 
   which is _correct_ for your task can be difficult, and also difficult 
   to debug
- have a compile step, which slows things down
- can be slower to execute as well, as a regexp does a bunch of 
   housekeeping for you

The more complex the tool the more... indirection between your solution 
using that tool and the smallest thing which needs to be done, and often 
the slower the solution. This isn't absolute;  there are times for the 
complex tool.

Common opinion here is often that if you're doing simple fixed-string 
things such as your task, which was finding instances of a fixed string, 
just use the existing str methods. You'll end up writing what you need 
directly and overtly.

I've a personal maxim that one should use the "smallest" tool which 
succinctly solves the problem. I usually use it to choose a programming 
language (eg sed vs awk vs shell vs python in loose order of problem 
difficulty), but it applies also to choosing tools within a language.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From barry at barrys-emacs.org  Tue Feb 28 18:04:39 2023
From: barry at barrys-emacs.org (Barry)
Date: Tue, 28 Feb 2023 23:04:39 +0000
Subject: Is there a more efficient threading lock?
In-Reply-To: <CAPTjJmrwqUQqL=Kn599ojb-MvBnJgxubfwT0kWVxUaCiwXWdrg@mail.gmail.com>
References: <CAPTjJmrwqUQqL=Kn599ojb-MvBnJgxubfwT0kWVxUaCiwXWdrg@mail.gmail.com>
Message-ID: <AC4038D1-3871-4F15-97CA-F2A62E2A8E0B@barrys-emacs.org>

> Though it's still probably not as useful as you might hope. In C, if I
> can do "int id = counter++;" atomically, it would guarantee me a new
> ID that no other thread could ever have.

C does not have to do that atomically. In fact it is free to use lots of instructions to build the int value. And some compilers indeed do, the linux kernel folks see this in gcc generated code.

I understand you have to use the new atomics features.

Barry



From hjp-python at hjp.at  Tue Feb 28 19:01:42 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Wed, 1 Mar 2023 01:01:42 +0100
Subject: How to escape strings for re.finditer?
In-Reply-To: <00d001d94bb2$bea422b0$3bec6810$@gmail.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com>
 <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <NPOA6-y--3-9@tutanota.com>
 <00d001d94bb2$bea422b0$3bec6810$@gmail.com>
Message-ID: <20230301000142.mvehc6lay2nhv42d@hjp.at>

On 2023-02-28 15:25:05 -0500, avi.e.gross at gmail.com wrote:
> Jen,
> 
>  
> 
> I had no doubt the code you ran was indented properly or it would not work.
> 
>  
> 
> I am merely letting you know that somewhere in the process of copying
> the code or the transition between mailers, my version is messed up.

The problem seems to be at your end. Jen's code looks ok here.

The content type is text/plain, no format=flowed or anything which would
affect the interpretation of line endings. However, after
base64-decoding it only contains unix-style LF line endings, not CRLF
line endings. That might throw your mailer off, but I have no idea why
it would join only some lines but not others.

> It happens to be easy for me to fix but I sometimes see garbled code I
> then simply ignore.

Truth to be told, that's one reason why I rarely read your mails to the
end. The long lines and the triple-spaced paragraphs make it just too
uncomfortable.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230301/6a4bbfd8/attachment.sig>

From hjp-python at hjp.at  Tue Feb 28 19:13:28 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Wed, 1 Mar 2023 01:13:28 +0100
Subject: Cryptic software announcements (was: ANN: DIPY 1.6.0)
In-Reply-To: <CAN6JmWM5L6FtqMeFQbeoB=4HKwBLnOBaeS+=fE2xycqV_d2d+Q@mail.gmail.com>
References: <CAN6JmWM5L6FtqMeFQbeoB=4HKwBLnOBaeS+=fE2xycqV_d2d+Q@mail.gmail.com>
Message-ID: <20230301001328.iw4ranneqvo6o3r5@hjp.at>

[This isn't specifically about DIPY, I've noticed the same thing in
other announcements]

On 2023-02-28 13:48:56 -0500, Eleftherios Garyfallidis wrote:
> Hello all,
> 
> 
> We are excited to announce a new release of DIPY: DIPY 1.6.0 is out from
> the oven!

That's nice, but what is DIPY?


> In addition, registration for the oceanic DIPY workshop 2023 (April 24-28)
> is now open! Our comprehensive program is designed to equip you with the
> skills and knowledge needed to master the latest techniques and tools in
> structural and diffusion imaging.

Ok, so since the workshop is about ".., tools in structural and
diffusion imaging", DIPY is probably such a tool.

However, without this incidental announcement I wouldn't have any idea
what it is or if it would be worth my time clicking at any of the links.


I think it would be a good idea if software announcements would include
a single paragraph (or maybe just a single sentence) summarizing what
the software is and does.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230301/f436be34/attachment.sig>

From hjp-python at hjp.at  Tue Feb 28 19:25:58 2023
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Wed, 1 Mar 2023 01:25:58 +0100
Subject: How to escape strings for re.finditer?
In-Reply-To: <20230301000142.mvehc6lay2nhv42d@hjp.at>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com>
 <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <NPOA6-y--3-9@tutanota.com>
 <00d001d94bb2$bea422b0$3bec6810$@gmail.com>
 <20230301000142.mvehc6lay2nhv42d@hjp.at>
Message-ID: <20230301002558.xhatsupowar7tjiu@hjp.at>

On 2023-03-01 01:01:42 +0100, Peter J. Holzer wrote:
> On 2023-02-28 15:25:05 -0500, avi.e.gross at gmail.com wrote:
> > It happens to be easy for me to fix but I sometimes see garbled code I
> > then simply ignore.
> 
> Truth to be told, that's one reason why I rarely read your mails to the
> end. The long lines and the triple-spaced paragraphs make it just too
> uncomfortable.

Hmm, since I was now paying a bit more attention to formatting problems
I saw that only about half of your messages have those long lines
although all seem to be sent with the same mailer. Don't know what's
going on there.

        hp


-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20230301/171e1999/attachment.sig>

From gweatherby at uchc.edu  Tue Feb 28 20:01:48 2023
From: gweatherby at uchc.edu (Weatherby,Gerard)
Date: Wed, 1 Mar 2023 01:01:48 +0000
Subject: How to escape strings for re.finditer?
In-Reply-To: <NPOCM6N--B-9@tutanota.com>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <a2d81a9a-16c9-2abb-78a0-aefec65a7734@tompassin.net>
 <NPOCM6N--B-9@tutanota.com>
Message-ID: <SJ0PR14MB586411D42220E8AFEFDE5050B9AD9@SJ0PR14MB5864.namprd14.prod.outlook.com>

Regex is fine if it works for you. The critiques ? ?difficult to read? ?are subjective. Unless the code is in a section that has been profiled to be a bottleneck, I don?t sweat performance at this level.

For me, using code that has already been written and vetted is the preferred approach to writing new code I have to test and maintain. I use an online regex tester, https://pythex.org, to get the syntax write before copying pasting it into my code.

From: Python-list <python-list-bounces+gweatherby=uchc.edu at python.org> on behalf of Jen Kris via Python-list <python-list at python.org>
Date: Tuesday, February 28, 2023 at 1:11 PM
To: Thomas Passin <list1 at tompassin.net>
Cc: python-list at python.org <python-list at python.org>
Subject: Re: How to escape strings for re.finditer?
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Using str.startswith is a cool idea in this case.  But is it better than regex for performance or reliability?  Regex syntax is not a model of simplicity, but in my simple case it's not too difficult.



From rosuav at gmail.com  Tue Feb 28 20:58:54 2023
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 1 Mar 2023 12:58:54 +1100
Subject: Is there a more efficient threading lock?
In-Reply-To: <AC4038D1-3871-4F15-97CA-F2A62E2A8E0B@barrys-emacs.org>
References: <CAPTjJmrwqUQqL=Kn599ojb-MvBnJgxubfwT0kWVxUaCiwXWdrg@mail.gmail.com>
 <AC4038D1-3871-4F15-97CA-F2A62E2A8E0B@barrys-emacs.org>
Message-ID: <CAPTjJmqJs3MaSmrgA_WbjTT57dxGRZ-kPZABKeNyfOQh32HuvQ@mail.gmail.com>

On Wed, 1 Mar 2023 at 10:04, Barry <barry at barrys-emacs.org> wrote:
>
> > Though it's still probably not as useful as you might hope. In C, if I
> > can do "int id = counter++;" atomically, it would guarantee me a new
> > ID that no other thread could ever have.
>
> C does not have to do that atomically. In fact it is free to use lots of instructions to build the int value. And some compilers indeed do, the linux kernel folks see this in gcc generated code.
>
> I understand you have to use the new atomics features.
>

Yeah, I didn't have a good analogy so I went with a hypothetical.  The
atomicity would be more useful in that context as it would give
lock-free ID generation, which doesn't work in Python.

ChrisA

From avi.e.gross at gmail.com  Tue Feb 28 21:19:05 2023
From: avi.e.gross at gmail.com (avi.e.gross at gmail.com)
Date: Tue, 28 Feb 2023 21:19:05 -0500
Subject: How to escape strings for re.finditer?
In-Reply-To: <20230301002558.xhatsupowar7tjiu@hjp.at>
References: <NPK8DYA--3-9@tutanota.com> <Y/1Cw5olV2jzubj7@cskk.homeip.net>
 <NPKLE0_--3-9@tutanota.com> <Y/1L63McJV7ixmte@cskk.homeip.net>
 <NPKeTUo--3-9@tutanota.com> <012101d94b1a$9a743440$cf5c9cc0$@gmail.com>
 <NPOA6-y--3-9@tutanota.com> <00d001d94bb2$bea422b0$3bec6810$@gmail.com>
 <20230301000142.mvehc6lay2nhv42d@hjp.at>
 <20230301002558.xhatsupowar7tjiu@hjp.at>
Message-ID: <000801d94be4$33119fc0$9934df40$@gmail.com>

Peter,

Nobody here would appreciate it if I tested it by sending out multiple
copies of each email to see if the same message wraps differently.

I am using a fairly standard mailer in Outlook that interfaces with gmail
and I could try mailing directly from gmail but apparently there are
systemic problems and I experience other complaints when sending directly
from AOL mail too. 

So, if some people don't read me, I can live with that. I mean the right
people, LOL!

Or did I get that wrong?

I do appreciate the feedback. Ironically, when I politely shared how someone
else's email was displaying on my screen, it seems I am equally causing
similar issues for others.

An interesting question is whether any of us reading the archived copies see
different things including with various browsers:

https://mail.python.org/pipermail/python-list/

I am not sure which letters from me had the anomalies you mention but
spot-checking a few of them showed a normal display when I use Chrome.

But none of this is really a python issue except insofar as you never know
what functionality in the network was written for in python.

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com at python.org> On
Behalf Of Peter J. Holzer
Sent: Tuesday, February 28, 2023 7:26 PM
To: python-list at python.org
Subject: Re: How to escape strings for re.finditer?

On 2023-03-01 01:01:42 +0100, Peter J. Holzer wrote:
> On 2023-02-28 15:25:05 -0500, avi.e.gross at gmail.com wrote:
> > It happens to be easy for me to fix but I sometimes see garbled code 
> > I then simply ignore.
> 
> Truth to be told, that's one reason why I rarely read your mails to 
> the end. The long lines and the triple-spaced paragraphs make it just 
> too uncomfortable.

Hmm, since I was now paying a bit more attention to formatting problems I
saw that only about half of your messages have those long lines although all
seem to be sent with the same mailer. Don't know what's going on there.

        hp


-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"


From rob.cliffe at btinternet.com  Mon Feb 27 22:24:17 2023
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Tue, 28 Feb 2023 03:24:17 +0000
Subject: Packing Problem
Message-ID: <d9a7e11f-abc5-6519-4e4a-f74435aa8d8e@btinternet.com>

I found Hen Hanna's "packing" problem to be an intriguing one: Given a 
list of words:
 ??? ['APPLE', 'PIE', 'APRICOT', 'BANANA', 'CANDY']
find a string (in general non-unique) as short as possible which 
contains the letters of each of these words, in order, as a subsequence.
It struck me as being rather hard for a homework problem, unless I'm 
missing something blindingly obvious.
Here is what I came up with (I could have done with 
removeprefix/removesuffix but I'm stuck on Python 3.8 for now ?):

# Pack.py
def Pack(Words):
 ??? if not Words:
 ??????? return ''
 ??? # The method is to build up the result by adding letters at the 
beginning
 ??? # and working forward, and by adding letters at the end, working 
backwards,
 ??? # meanwhile shrinking the words in the list.
 ??? Words = list(Words) # Don't mutate the original
 ??? Initial = ''
 ??? Final = ''
 ??? while True:
 ??????? AnyProgress = False
 ??????? # It is safe to add an initial letter (of one or more of the 
words) if
 ??????? # EITHER??? There is no word that contains it as
 ??????? #???????????? a non-initial letter but not as an initial letter.
 ??????? #? OR?????? All words start with it.
 ??????? while True:
 ??????????? FirstLetters = ''.join(w[0] for w in Words)
 ??????????? FirstLetters = [ ch for ch in FirstLetters if
 ??????????????? all(w[0]==ch for w in Words)
 ??????????????? or not any(ch in w[1:] and w[0]!=ch for w in Words) ]
 ??????????? if not FirstLetters:
 ??????????????? break
 ??????????? AnyProgress = True
 ??????????? ch = FirstLetters[0]??? # Pick one
 ??????????? Initial += ch?????????? # Build up the answer from the 
beginning
 ??????????? Words = [ (w[1:] if w[0]==ch else w) for w in Words ]
 ??????????? Words = [ w for w in Words if w != '']
 ??????????? if not Words:
 ??????????????? return Initial + Final
 ??????? # It is safe to add a final letter (of one or more of the words) of
 ??????? # EITHER??? There is no word that contains it as
 ??????? #???????????? a non-final letter but not as a final letter.
 ??????? #? OR?????? All words end with it.
 ??????? while True:
 ??????????? LastLetters = ''.join(w[-1] for w in Words)
 ??????????? LastLetters = [ ch for ch in LastLetters if
 ??????????????? all(w[-1]==ch for w in Words)
 ??????????????? or not any(ch in w[:-1] and w[-1]!=ch for w in Words) ]
 ??????????? if not LastLetters:
 ??????????????? break
 ??????????? AnyProgress = True
 ??????????? ch = LastLetters[0]???? # Pick one
 ??????????? Final = ch + Final????? # Build up the answer from the end
 ??????????? Words = [ (w[:-1] if w[-1]==ch else w) for w in Words ]
 ??????????? Words = [ w for w in Words if w != '']
 ??????????? if not Words:
 ??????????????? return Initial + Final
 ??????? if not AnyProgress:
 ??????????? break
 ??? # Try all the possibilities for the next letter to add at the 
beginning,
 ??? # with recursive calls, and pick one that gives a shortest answer:
 ??? BestResult = None
 ??? for ch in set(w[0] for w in Words):
 ??????????? Words2 = list( (w[1:] if w[0] == ch else w) for w in Words )
 ??????????? Words2 = [ w for w in Words2 if w != '' ]
 ??????????? res = ch + Pack(Words2)
 ??????????? if BestResult is None or len(res) < len(BestResult):
 ??????????????? BestResult = res
 ??? return Initial + BestResult + Final

print(Pack(['APPLE', 'PIE', 'APRICOT', 'BANANA', 'CANDY']))

The output:
BAPPRICNANADYOTLE
which has the same length as the answer I came up with trying to solve 
it with my unaided brain, which may or may not be reassuring ?,
and also contains a much-needed BRANDY.
I expect there are simpler and more efficient solutions.
Best wishes
Rob Cliffe