From barry at barrys-emacs.org  Tue Feb  1 12:05:52 2022
From: barry at barrys-emacs.org (Barry)
Date: Tue, 1 Feb 2022 17:05:52 +0000
Subject: http.client and dns lookups
In-Reply-To: <25080.12886.84182.252875@ixdm.fritz.box>
References: <25080.12886.84182.252875@ixdm.fritz.box>
Message-ID: <D9D4F283-BBCC-480A-922C-ABE0CF3740B6@barrys-emacs.org>



> On 31 Jan 2022, at 19:22, Dieter Maurer <dieter at handshake.de> wrote:
> 
> ?Michael Welle wrote at 2022-1-30 09:18 +0100:
>> ...
> The machine this is running on regularly switches
>> its network configuration without restarting the Python application. Now
>> it turns out that the application is still using an old, outdated dns
>> server after such a network configuration switch.
> 
> It is unlikely that Python performs the host -> IP address translation
> itself. Almost surely, the translation is delegated to functions
> of the underlying C runtime library, e.g. "gethostbyname".

And glibc does not have the ability to reload the /etc/resolv.conf I recall.
If you rely on that you will need to change to using something like dnsmasq or systemd-resolved so that resolv.conf uses localhost.

Barry


> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From jenkris at tutanota.com  Tue Feb  1 15:14:31 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 1 Feb 2022 21:14:31 +0100 (CET)
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
Message-ID: <Muqv9er--3-2@tutanota.com>

I am using multiprocesssing.shared_memory to pass data between NASM and Python.? The shared memory is created in NASM before Python is called.? Python connects to the shm:? shm_00 = shared_memory.SharedMemory(name='shm_object_00',create=False).? 

I have used shared memory at other points in this project to pass text data from Python back to NASM with no problems.? But now this time I need to pass a 32-bit integer (specifically 32,894) from NASM to Python.?

First I convert the integer to bytes in a C program linked into NASM:

??? unsigned char bytes[4]
??? unsigned long int_to_convert = 32894;

??? bytes[0] = (int_to_convert >> 24) & 0xFF;
??? bytes[1] = (int_to_convert >> 16) & 0xFF;
??? bytes[2] = (int_to_convert >> 8) & 0xFF;
??? bytes[3] = int_to_convert & 0xFF;
??? memcpy(outbuf, bytes, 4);

where outbuf is a pointer to the shared memory.? On return from C to NASM, I verify that the first four bytes of the shared memory contain what I want, and they are 0, 0, -128, 126 which is binary 00000000 00000000 10000000 01111110, and that's correct (32,894).?

Next I send a message to Python through a FIFO to read the data from shared memory.? Python uses the following code to read the first four bytes of the shared memory:

??????? byte_val = shm_00.buf[:4]
??????? print(shm_00.buf[0])
??????? print(shm_00.buf[1])
??????? print(shm_00.buf[2])
??????? print(shm_00.buf[3])

But the bytes show as 40 39 96 96, which is exactly what the first four bytes of this shared memory contained before I called C to overwrite them with the bytes 0, 0, -128, 126.? So Python does not see the updated bytes, and naturally int.from_bytes(byte_val, "little") does not return the result I want.?

I know that Python refers to shm00.buf, using the buffer protocol.? Is that the reason that Python can't see the data that has been updated by another language??

So my question is, how can I alter the data in shared memory in a non-Python language to pass back to Python??

Thanks,

Jen


From barry at barrys-emacs.org  Tue Feb  1 17:00:35 2022
From: barry at barrys-emacs.org (Barry)
Date: Tue, 1 Feb 2022 22:00:35 +0000
Subject: http.client and dns lookups
In-Reply-To: <vmaocix7f2.ln2@news.c0t0d0s0.de>
References: <vmaocix7f2.ln2@news.c0t0d0s0.de>
Message-ID: <0F5894EA-A511-4174-B987-F4FDFE407B8B@barrys-emacs.org>



> On 1 Feb 2022, at 19:21, Michael Welle <mwe012008 at gmx.net> wrote:
> 
> ?Hello,
> 
> Barry <barry at barrys-emacs.org> writes:
> 
>>>> On 31 Jan 2022, at 19:22, Dieter Maurer <dieter at handshake.de> wrote:
>>> 
>>> ?Michael Welle wrote at 2022-1-30 09:18 +0100:
>>>> ...
>>> The machine this is running on regularly switches
>>>> its network configuration without restarting the Python application. Now
>>>> it turns out that the application is still using an old, outdated dns
>>>> server after such a network configuration switch.
>>> 
>>> It is unlikely that Python performs the host -> IP address translation
>>> itself. Almost surely, the translation is delegated to functions
>>> of the underlying C runtime library, e.g. "gethostbyname".
>> 
>> And glibc does not have the ability to reload the /etc/resolv.conf I recall.
>> If you rely on that you will need to change to using something like
>> dnsmasq or systemd-resolved so that resolv.conf uses localhost.
> hmmm, I don't know if that's all. For a test I just changed
> /etc/resolv.conf manually while the application was sleeping. After
> returning from sleep the application knew the change I had made in
> resolv.conf and, interestingly, an exception (name resolution failure,
> which is correct with the change I made) surfaces in the application.
> That doesn't happen when the 'real' issue occurs. Attaching strace to
> the Python process I can see that resolv.conf is stat'ed and open'ed. I
> guess now I'm more confused than before ;). There must be an additional
> condition that I'm missing.

That is good to know, last time I hit this was on centos 6 a little while ago.

Barry

> 
> Thanks for helping
> hmw
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From barry at barrys-emacs.org  Tue Feb  1 17:20:40 2022
From: barry at barrys-emacs.org (Barry)
Date: Tue, 1 Feb 2022 22:20:40 +0000
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <Muqv9er--3-2@tutanota.com>
References: <Muqv9er--3-2@tutanota.com>
Message-ID: <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>



> On 1 Feb 2022, at 20:26, Jen Kris via Python-list <python-list at python.org> wrote:
> 
> ?I am using multiprocesssing.shared_memory to pass data between NASM and Python.  The shared memory is created in NASM before Python is called.  Python connects to the shm:  shm_00 = shared_memory.SharedMemory(name='shm_object_00',create=False).  
> 
> I have used shared memory at other points in this project to pass text data from Python back to NASM with no problems.  But now this time I need to pass a 32-bit integer (specifically 32,894) from NASM to Python. 
> 
> First I convert the integer to bytes in a C program linked into NASM:
> 
>     unsigned char bytes[4]
>     unsigned long int_to_convert = 32894;
> 
>     bytes[0] = (int_to_convert >> 24) & 0xFF;
>     bytes[1] = (int_to_convert >> 16) & 0xFF;
>     bytes[2] = (int_to_convert >> 8) & 0xFF;
>     bytes[3] = int_to_convert & 0xFF;
>     memcpy(outbuf, bytes, 4);
> 
> where outbuf is a pointer to the shared memory.  On return from C to NASM, I verify that the first four bytes of the shared memory contain what I want, and they are 0, 0, -128, 126 which is binary 00000000 00000000 10000000 01111110, and that's correct (32,894). 
> 
> Next I send a message to Python through a FIFO to read the data from shared memory.  Python uses the following code to read the first four bytes of the shared memory:
> 
>         byte_val = shm_00.buf[:4]
>         print(shm_00.buf[0])
>         print(shm_00.buf[1])
>         print(shm_00.buf[2])
>         print(shm_00.buf[3])
> 
> But the bytes show as 40 39 96 96, which is exactly what the first four bytes of this shared memory contained before I called C to overwrite them with the bytes 0, 0, -128, 126.  So Python does not see the updated bytes, and naturally int.from_bytes(byte_val, "little") does not return the result I want. 
> 
> I know that Python refers to shm00.buf, using the buffer protocol.  Is that the reason that Python can't see the data that has been updated by another language? 
> 
> So my question is, how can I alter the data in shared memory in a non-Python language to pass back to Python? 

Maybe you need to use a memory barrier to force the data to be seen by another cpu?
Maybe use shm lock operation to sync both sides?
Googling I see people talking about using stdatomic.h for this.

But I am far from clear what you would need to do.

Barry

> 
> Thanks,
> 
> Jen
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From jenkris at tutanota.com  Tue Feb  1 18:40:22 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Wed, 2 Feb 2022 00:40:22 +0100 (CET)
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
Message-ID: <MureHAB----2@tutanota.com>

Barry, thanks for your reply.? 

On the theory that it is not yet possible to pass data from a non-Python language to Python with multiprocessing.shared_memory, I bypassed the problem by attaching 4 bytes to my FIFO pipe message from NASM to Python:

byte_val = v[10:14]

where v is the message read from the FIFO.? Then:

 breakup = int.from_bytes(byte_val, "big")
print("this is breakup " + str(breakup))

Python prints:? this is breakup 32894

Note that I had to switch from little endian to big endian.? Python is little endian by default, but in this case it's big endian.? 

However, if anyone on this list knows how to pass data from a non-Python language to Python in multiprocessing.shared_memory please let me (and the list) know.? 

Thanks.? 


Feb 1, 2022, 14:20 by barry at barrys-emacs.org:

>
>
>> On 1 Feb 2022, at 20:26, Jen Kris via Python-list <python-list at python.org> wrote:
>>
>> ?I am using multiprocesssing.shared_memory to pass data between NASM and Python.  The shared memory is created in NASM before Python is called.  Python connects to the shm:  shm_00 = shared_memory.SharedMemory(name='shm_object_00',create=False). 
>>
>> I have used shared memory at other points in this project to pass text data from Python back to NASM with no problems.  But now this time I need to pass a 32-bit integer (specifically 32,894) from NASM to Python. 
>>
>> First I convert the integer to bytes in a C program linked into NASM:
>>
>>  unsigned char bytes[4]
>>  unsigned long int_to_convert = 32894;
>>
>>  bytes[0] = (int_to_convert >> 24) & 0xFF;
>>  bytes[1] = (int_to_convert >> 16) & 0xFF;
>>  bytes[2] = (int_to_convert >> 8) & 0xFF;
>>  bytes[3] = int_to_convert & 0xFF;
>>  memcpy(outbuf, bytes, 4);
>>
>> where outbuf is a pointer to the shared memory.  On return from C to NASM, I verify that the first four bytes of the shared memory contain what I want, and they are 0, 0, -128, 126 which is binary 00000000 00000000 10000000 01111110, and that's correct (32,894). 
>>
>> Next I send a message to Python through a FIFO to read the data from shared memory.  Python uses the following code to read the first four bytes of the shared memory:
>>
>>  byte_val = shm_00.buf[:4]
>>  print(shm_00.buf[0])
>>  print(shm_00.buf[1])
>>  print(shm_00.buf[2])
>>  print(shm_00.buf[3])
>>
>> But the bytes show as 40 39 96 96, which is exactly what the first four bytes of this shared memory contained before I called C to overwrite them with the bytes 0, 0, -128, 126.  So Python does not see the updated bytes, and naturally int.from_bytes(byte_val, "little") does not return the result I want. 
>>
>> I know that Python refers to shm00.buf, using the buffer protocol.  Is that the reason that Python can't see the data that has been updated by another language? 
>>
>> So my question is, how can I alter the data in shared memory in a non-Python language to pass back to Python?
>>
>
> Maybe you need to use a memory barrier to force the data to be seen by another cpu?
> Maybe use shm lock operation to sync both sides?
> Googling I see people talking about using stdatomic.h for this.
>
> But I am far from clear what you would need to do.
>
> Barry
>
>>
>> Thanks,
>>
>> Jen
>>
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>>


From barry at barrys-emacs.org  Wed Feb  2 07:52:22 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Wed, 2 Feb 2022 12:52:22 +0000
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <MureHAB----2@tutanota.com>
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
 <MureHAB----2@tutanota.com>
Message-ID: <B1B7170F-7A66-45F9-8B4A-E39078D3BE10@barrys-emacs.org>



> On 1 Feb 2022, at 23:40, Jen Kris <jenkris at tutanota.com> wrote:
> 
> Barry, thanks for your reply.  
> 
> On the theory that it is not yet possible to pass data from a non-Python language to Python with multiprocessing.shared_memory, I bypassed the problem by attaching 4 bytes to my FIFO pipe message from NASM to Python:
> 
> byte_val = v[10:14]
> 
> where v is the message read from the FIFO.  Then:
> 
> breakup = int.from_bytes(byte_val, "big")
> print("this is breakup " + str(breakup))
> 
> Python prints:  this is breakup 32894
> 
> Note that I had to switch from little endian to big endian.  Python is little endian by default, but in this case it's big endian.  
> 
> However, if anyone on this list knows how to pass data from a non-Python language to Python in multiprocessing.shared_memory please let me (and the list) know.  

By using the struct module you can control for endian of the data.

Barry

> 
> Thanks.  
> 
> 
> Feb 1, 2022, 14:20 by barry at barrys-emacs.org:
> 
> On 1 Feb 2022, at 20:26, Jen Kris via Python-list <python-list at python.org> wrote:
> 
> ?I am using multiprocesssing.shared_memory to pass data between NASM and Python. The shared memory is created in NASM before Python is called. Python connects to the shm: shm_00 = shared_memory.SharedMemory(name='shm_object_00',create=False). 
> 
> I have used shared memory at other points in this project to pass text data from Python back to NASM with no problems. But now this time I need to pass a 32-bit integer (specifically 32,894) from NASM to Python. 
> 
> First I convert the integer to bytes in a C program linked into NASM:
> 
> unsigned char bytes[4]
> unsigned long int_to_convert = 32894;
> 
> bytes[0] = (int_to_convert >> 24) & 0xFF;
> bytes[1] = (int_to_convert >> 16) & 0xFF;
> bytes[2] = (int_to_convert >> 8) & 0xFF;
> bytes[3] = int_to_convert & 0xFF;
> memcpy(outbuf, bytes, 4);
> 
> where outbuf is a pointer to the shared memory. On return from C to NASM, I verify that the first four bytes of the shared memory contain what I want, and they are 0, 0, -128, 126 which is binary 00000000 00000000 10000000 01111110, and that's correct (32,894). 
> 
> Next I send a message to Python through a FIFO to read the data from shared memory. Python uses the following code to read the first four bytes of the shared memory:
> 
> byte_val = shm_00.buf[:4]
> print(shm_00.buf[0])
> print(shm_00.buf[1])
> print(shm_00.buf[2])
> print(shm_00.buf[3])
> 
> But the bytes show as 40 39 96 96, which is exactly what the first four bytes of this shared memory contained before I called C to overwrite them with the bytes 0, 0, -128, 126. So Python does not see the updated bytes, and naturally int.from_bytes(byte_val, "little") does not return the result I want. 
> 
> I know that Python refers to shm00.buf, using the buffer protocol. Is that the reason that Python can't see the data that has been updated by another language? 
> 
> So my question is, how can I alter the data in shared memory in a non-Python language to pass back to Python?
> 
> Maybe you need to use a memory barrier to force the data to be seen by another cpu?
> Maybe use shm lock operation to sync both sides?
> Googling I see people talking about using stdatomic.h for this.
> 
> But I am far from clear what you would need to do.
> 
> Barry
> 
> Thanks,
> 
> Jen
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From Marco.Sulla.Python at gmail.com  Wed Feb  2 07:54:10 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Wed, 2 Feb 2022 13:54:10 +0100
Subject: Why dict.setdefault() has value as optional?
Message-ID: <CABbU2U9TQceFoC16hezeAAQwGZLtyzPN-sR=HMKsSBWpfcJq4g@mail.gmail.com>

Just out of curiosity: why dict.setdefault() has the default parameter
that.... well, has a default value (None)? I used setdefault in the past,
but I always specified a value. What's the use case of setting None by
default?

From liedtke at punkt.de  Wed Feb  2 08:24:45 2022
From: liedtke at punkt.de (Lars Liedtke)
Date: Wed, 2 Feb 2022 14:24:45 +0100
Subject: Why dict.setdefault() has value as optional?
In-Reply-To: <CABbU2U9TQceFoC16hezeAAQwGZLtyzPN-sR=HMKsSBWpfcJq4g@mail.gmail.com>
References: <CABbU2U9TQceFoC16hezeAAQwGZLtyzPN-sR=HMKsSBWpfcJq4g@mail.gmail.com>
Message-ID: <0535618f-80fd-31d7-aae2-02dda24de009@punkt.de>

This is a quite philosophical queston if you look at it in general:
"What value do you give a variable, that is not set?"

You are right, at first it seems strange to have a default of None. But 
how do you want to signal that no default is set yet? Especially if you 
think of a dict that can have multiple keys with each different values 
of different types?

Have fun in the rabbithole ;-)

Cheers

Lars

Am 02.02.22 um 13:54 schrieb Marco Sulla:
> Just out of curiosity: why dict.setdefault() has the default parameter
> that.... well, has a default value (None)? I used setdefault in the past,
> but I always specified a value. What's the use case of setting None by
> default?

-- 
punkt.de GmbH
Lars Liedtke
.infrastructure

Kaiserallee 13a	
76133 Karlsruhe

Tel. +49 721 9109 500
https://infrastructure.punkt.de
info at punkt.de

AG Mannheim 108285
Gesch?ftsf?hrer: J?rgen Egeling, Daniel Lienert, Fabian Stein


From wlfraed at ix.netcom.com  Wed Feb  2 00:30:40 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Wed, 02 Feb 2022 00:30:40 -0500
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
 <MureHAB----2@tutanota.com>
 <mailman.11.1643758824.2976.python-list@python.org>
Message-ID: <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com>

On Wed, 2 Feb 2022 00:40:22 +0100 (CET), Jen Kris <jenkris at tutanota.com>
declaimed the following:

>
> breakup = int.from_bytes(byte_val, "big")
>print("this is breakup " + str(breakup))
>
>Python prints:? this is breakup 32894
>
>Note that I had to switch from little endian to big endian.? Python is little endian by default, but in this case it's big endian.? 
>
	Look at the struct module. I'm pretty certain it has flags for big or
little end, or system native (that, or run your integers through the
various "network byte order" functions that I think C and Python both
support.

https://www.gta.ufrj.br/ensino/eel878/sockets/htonsman.html


>However, if anyone on this list knows how to pass data from a non-Python language to Python in multiprocessing.shared_memory please let me (and the list) know.? 

	<pondering>MMU cache lines not writing through to RAM? Can't find
anything on Google to force a cache flush</pondering> Can you test on a
different OS? (Windows vs Linux)



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From Marco.Sulla.Python at gmail.com  Wed Feb  2 10:21:27 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Wed, 2 Feb 2022 16:21:27 +0100
Subject: Why dict.setdefault() has value as optional?
In-Reply-To: <0535618f-80fd-31d7-aae2-02dda24de009@punkt.de>
References: <CABbU2U9TQceFoC16hezeAAQwGZLtyzPN-sR=HMKsSBWpfcJq4g@mail.gmail.com>
 <0535618f-80fd-31d7-aae2-02dda24de009@punkt.de>
Message-ID: <CABbU2U8HNG_VKvzxBoMh_ZdQvuGJREm5xYt-Or5itEKTRFXSfQ@mail.gmail.com>

On Wed, 2 Feb 2022 at 14:34, Lars Liedtke <liedtke at punkt.de> wrote:
>
> This is a quite philosophical queston if you look at it in general:
> "What value do you give a variable, that is not set?"

Maybe I expressed my question badly. My existential doubt is why
setdefault has an optional parameter for the value and not a required
parameter. I'm not asking why the default is None.

From dieter at handshake.de  Wed Feb  2 12:58:31 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Wed, 2 Feb 2022 18:58:31 +0100
Subject: http.client and dns lookups
In-Reply-To: <vmaocix7f2.ln2@news.c0t0d0s0.de>
References: <25080.12886.84182.252875@ixdm.fritz.box>
 <D9D4F283-BBCC-480A-922C-ABE0CF3740B6@barrys-emacs.org>
 <mailman.7.1643735165.2976.python-list@python.org>
 <vmaocix7f2.ln2@news.c0t0d0s0.de>
Message-ID: <25082.50759.381474.118187@ixdm.fritz.box>

Michael Welle wrote at 2022-2-1 19:28 +0100:
> ...
>That doesn't happen when the 'real' issue occurs. Attaching strace to
>the Python process I can see that resolv.conf is stat'ed and open'ed. I
>guess now I'm more confused than before ;). There must be an additional
>condition that I'm missing.

The DNS service routinely uses caches. Outdated cache values
can cause (apparently non deterministic) failures.

From avigross at verizon.net  Wed Feb  2 13:16:07 2022
From: avigross at verizon.net (Avi Gross)
Date: Wed, 2 Feb 2022 18:16:07 +0000 (UTC)
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com>
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
 <MureHAB----2@tutanota.com>
 <mailman.11.1643758824.2976.python-list@python.org>
 <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com>
Message-ID: <706734682.2805722.1643825767629@mail.yahoo.com>

I applaud trying to find the right solution but wonder if a more trivial solution is even being considered. It ignores big and little endians and just converts your data into another form and back.

If all you want to do is send an integer that fit in 32 bits or 64 bits, why not convert it to a character string in a form that both machines will see the same way and when read back, convert it back to an integer? 

As long as both side see the same string, this can be done in reasonable time and portably.

Or am I missing something? Is "1234" not necessarily seen in the same order, or "1.234e3" or whatever?

Obviously, if the mechanism is heavily used and multiple sides keep reading and even writing the same memory location, this is not ideal. But having different incompatible processors looking at the same memory is also not.

-----Original Message-----
From: Dennis Lee Bieber <wlfraed at ix.netcom.com>
To: python-list at python.org
Sent: Wed, Feb 2, 2022 12:30 am
Subject: Re: Data unchanged when passing data to Python in multiprocessing shared memory


On Wed, 2 Feb 2022 00:40:22 +0100 (CET), Jen Kris <jenkris at tutanota.com>

declaimed the following:



>

> breakup = int.from_bytes(byte_val, "big")

>print("this is breakup " + str(breakup))

>

>Python prints:? this is breakup 32894

>

>Note that I had to switch from little endian to big endian.? Python is little endian by default, but in this case it's big endian.? 

>

??? Look at the struct module. I'm pretty certain it has flags for big or

little end, or system native (that, or run your integers through the

various "network byte order" functions that I think C and Python both

support.



https://www.gta.ufrj.br/ensino/eel878/sockets/htonsman.html





>However, if anyone on this list knows how to pass data from a non-Python language to Python in multiprocessing.shared_memory please let me (and the list) know.? 



??? <pondering>MMU cache lines not writing through to RAM? Can't find

anything on Google to force a cache flush</pondering> Can you test on a

different OS? (Windows vs Linux)







-- 

??? Wulfraed? ? ? ? ? ? ? ?  Dennis Lee Bieber? ? ? ?  AF6VN

??? wlfraed at ix.netcom.com? ? http://wlfraed.microdiversity.freeddns.org/

-- 

https://mail.python.org/mailman/listinfo/python-list


From jenkris at tutanota.com  Wed Feb  2 13:16:19 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Wed, 2 Feb 2022 19:16:19 +0100 (CET)
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com>
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
 <MureHAB----2@tutanota.com>
 <mailman.11.1643758824.2976.python-list@python.org>
 <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com>
Message-ID: <MuvdgZg--3-2@tutanota.com>

It's not clear to me from the struct module whether it can actually auto-detect endianness.? I think it must be specified, just as I had to do with int.from_bytes().? In my case endianness was dictated by how the four bytes were populated, starting with the zero bytes on the left.? 


Feb 1, 2022, 21:30 by wlfraed at ix.netcom.com:

> On Wed, 2 Feb 2022 00:40:22 +0100 (CET), Jen Kris <jenkris at tutanota.com>
> declaimed the following:
>
>>
>> breakup = int.from_bytes(byte_val, "big")
>>
> >print("this is breakup " + str(breakup))
>
>>
>>
> >Python prints:? this is breakup 32894
>
>>
>>
> >Note that I had to switch from little endian to big endian.? Python is little endian by default, but in this case it's big endian.? 
>
>>
>>
> Look at the struct module. I'm pretty certain it has flags for big or
> little end, or system native (that, or run your integers through the
> various "network byte order" functions that I think C and Python both
> support.
>
> https://www.gta.ufrj.br/ensino/eel878/sockets/htonsman.html
>
>
> >However, if anyone on this list knows how to pass data from a non-Python language to Python in multiprocessing.shared_memory please let me (and the list) know.? 
>
>  <pondering>MMU cache lines not writing through to RAM? Can't find
> anything on Google to force a cache flush</pondering> Can you test on a
> different OS? (Windows vs Linux)
>
>
>
> -- 
>  Wulfraed                 Dennis Lee Bieber         AF6VN
>  wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From jenkris at tutanota.com  Wed Feb  2 13:27:09 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Wed, 2 Feb 2022 19:27:09 +0100 (CET)
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <706734682.2805722.1643825767629@mail.yahoo.com>
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
 <MureHAB----2@tutanota.com>
 <mailman.11.1643758824.2976.python-list@python.org>
 <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com>
 <706734682.2805722.1643825767629@mail.yahoo.com>
Message-ID: <MuvgAhH--3-2@tutanota.com>

An ASCII string will not work.? If you convert 32894 to an ascii string you will have five bytes, but you need four.? In my original post I showed the C program I used to convert any 32-bit number to 4 bytes.? 


Feb 2, 2022, 10:16 by python-list at python.org:

> I applaud trying to find the right solution but wonder if a more trivial solution is even being considered. It ignores big and little endians and just converts your data into another form and back.
>
> If all you want to do is send an integer that fit in 32 bits or 64 bits, why not convert it to a character string in a form that both machines will see the same way and when read back, convert it back to an integer? 
>
> As long as both side see the same string, this can be done in reasonable time and portably.
>
> Or am I missing something? Is "1234" not necessarily seen in the same order, or "1.234e3" or whatever?
>
> Obviously, if the mechanism is heavily used and multiple sides keep reading and even writing the same memory location, this is not ideal. But having different incompatible processors looking at the same memory is also not.
>
> -----Original Message-----
> From: Dennis Lee Bieber <wlfraed at ix.netcom.com>
> To: python-list at python.org
> Sent: Wed, Feb 2, 2022 12:30 am
> Subject: Re: Data unchanged when passing data to Python in multiprocessing shared memory
>
>
> On Wed, 2 Feb 2022 00:40:22 +0100 (CET), Jen Kris <jenkris at tutanota.com>
>
> declaimed the following:
>
>
>
>>
>>
>> breakup = int.from_bytes(byte_val, "big")
>>
>
> >print("this is breakup " + str(breakup))
>
>>
>>
>
> >Python prints:? this is breakup 32894
>
>>
>>
>
> >Note that I had to switch from little endian to big endian.? Python is little endian by default, but in this case it's big endian.? 
>
>>
>>
>
> ??? Look at the struct module. I'm pretty certain it has flags for big or
>
> little end, or system native (that, or run your integers through the
>
> various "network byte order" functions that I think C and Python both
>
> support.
>
>
>
> https://www.gta.ufrj.br/ensino/eel878/sockets/htonsman.html
>
>
>
>
>
> >However, if anyone on this list knows how to pass data from a non-Python language to Python in multiprocessing.shared_memory please let me (and the list) know.? 
>
>
>
> ??? <pondering>MMU cache lines not writing through to RAM? Can't find
>
> anything on Google to force a cache flush</pondering> Can you test on a
>
> different OS? (Windows vs Linux)
>
>
>
>
>
>
>
> -- 
>
> ??? Wulfraed? ? ? ? ? ? ? ?  Dennis Lee Bieber? ? ? ?  AF6VN
>
> ??? wlfraed at ix.netcom.com? ? http://wlfraed.microdiversity.freeddns.org/
>
> -- 
>
> https://mail.python.org/mailman/listinfo/python-list
>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From wlfraed at ix.netcom.com  Wed Feb  2 13:41:57 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Wed, 02 Feb 2022 13:41:57 -0500
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
 <MureHAB----2@tutanota.com>
 <mailman.11.1643758824.2976.python-list@python.org>
 <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com> <MuvdgZg--3-2@tutanota.com>
 <mailman.18.1643825781.2976.python-list@python.org>
Message-ID: <67jlvg55ku567ouehf5o19imaqbbfipsbe@4ax.com>

On Wed, 2 Feb 2022 19:16:19 +0100 (CET), Jen Kris <jenkris at tutanota.com>
declaimed the following:

>It's not clear to me from the struct module whether it can actually auto-detect endianness.? I think it must be specified, just as I had to do with int.from_bytes().? In my case endianness was dictated by how the four bytes were populated, starting with the zero bytes on the left.?

	Which is why I also suggested maybe looking at the various network/host
translation calls. They are in the socket module of Python, and should also
be available in most C standard libraries...

https://docs.python.org/3/library/socket.html#other-functions

"""

socket.ntohl(x)

    Convert 32-bit positive integers from network to host byte order. On
machines where the host byte order is the same as network byte order, this
is a no-op; otherwise, it performs a 4-byte swap operation.

socket.ntohs(x)

    Convert 16-bit positive integers from network to host byte order. On
machines where the host byte order is the same as network byte order, this
is a no-op; otherwise, it performs a 2-byte swap operation.

    Changed in version 3.10: Raises OverflowError if x does not fit in a
16-bit unsigned integer.

socket.htonl(x)

    Convert 32-bit positive integers from host to network byte order. On
machines where the host byte order is the same as network byte order, this
is a no-op; otherwise, it performs a 4-byte swap operation.

socket.htons(x)

    Convert 16-bit positive integers from host to network byte order. On
machines where the host byte order is the same as network byte order, this
is a no-op; otherwise, it performs a 2-byte swap operation.

    Changed in version 3.10: Raises OverflowError if x does not fit in a
16-bit unsigned integer.
"""

https://docs.python.org/3/library/struct.html
"""
Byte Order, Size, and Alignment

By default, C types are represented in the machine?s native format and byte
order, and properly aligned by skipping pad bytes if necessary (according
to the rules used by the C compiler).

Alternatively, the first character of the format string can be used to
indicate the byte order, size and alignment of the packed data, according
to the following table:

Character	Byte order	Size	Alignment
	@		native		native	native
	=		native		standard	none
	<		little-endian	standard	none
	>		big-endian	standard	none
	!		network (= big-endian)			<<<<<<
						standard	none	<<<<<<

If the first character is not one of these, '@' is assumed.
"""

	Since all the programs in your situation are running on the same
machine, it would appear that at least one of them is NOT formatting
integers in native host mode -- and I don't think it is Python.


https://www.tutorialspoint.com/unix_sockets/network_byte_orders.htm
"""
These functions are macros and result in the insertion of conversion source
code into the calling program. On little-endian machines, the code will
change the values around to network byte order. On big-endian machines, no
code is inserted since none is needed; the functions are defined as null.
"""



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From Cecil at decebal.nl  Wed Feb  2 14:06:44 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Wed, 02 Feb 2022 20:06:44 +0100
Subject: Waht do you think about my repeated_timer class
Message-ID: <87zgn9jdff.fsf@munus.decebal.nl>

I need (sometimes) to repeatedly execute a function. For this I wrote
the below class. What do you think about it?
    from threading  import Timer



    class repeated_timer(object):
        def __init__(self, fn, interval, start = False):
            if not callable(fn):
                raise TypeError('{} is not a function'.format(fn))
            self._fn         = fn
            self._check_interval(interval)
            self._interval   = interval
            self._timer      = None
            self._is_running = False
            if start:
                self.start()

        def _check_interval(self, interval):
            if not type(interval) in [int, float]:
                raise TypeError('{} is not numeric'.format(interval))
            if interval <= 0:
                raise ValueError('{} is not greater as 0'.format(interval))

        def _next(self):
            self._timer = Timer(self._interval, self._run)
            self._timer.start()

        def _run(self):
            self._next()
            self._fn()

        def set_interval(self, interval):
            self._check_interval(interval)
            self._interval = interval

        def start(self):
            if not self._is_running:
                self._next()
                self._is_running = True

        def stop(self):
            if self._is_running:
                self._timer.cancel()
                self._timer      = None
                self._is_running = False

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From Marco.Sulla.Python at gmail.com  Wed Feb  2 16:09:02 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Wed, 2 Feb 2022 22:09:02 +0100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <87zgn9jdff.fsf@munus.decebal.nl>
References: <87zgn9jdff.fsf@munus.decebal.nl>
Message-ID: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>

You could add a __del__ that calls stop :)

On Wed, 2 Feb 2022 at 21:23, Cecil Westerhof via Python-list
<python-list at python.org> wrote:
>
> I need (sometimes) to repeatedly execute a function. For this I wrote
> the below class. What do you think about it?
>     from threading  import Timer
>
>
>
>     class repeated_timer(object):
>         def __init__(self, fn, interval, start = False):
>             if not callable(fn):
>                 raise TypeError('{} is not a function'.format(fn))
>             self._fn         = fn
>             self._check_interval(interval)
>             self._interval   = interval
>             self._timer      = None
>             self._is_running = False
>             if start:
>                 self.start()
>
>         def _check_interval(self, interval):
>             if not type(interval) in [int, float]:
>                 raise TypeError('{} is not numeric'.format(interval))
>             if interval <= 0:
>                 raise ValueError('{} is not greater as 0'.format(interval))
>
>         def _next(self):
>             self._timer = Timer(self._interval, self._run)
>             self._timer.start()
>
>         def _run(self):
>             self._next()
>             self._fn()
>
>         def set_interval(self, interval):
>             self._check_interval(interval)
>             self._interval = interval
>
>         def start(self):
>             if not self._is_running:
>                 self._next()
>                 self._is_running = True
>
>         def stop(self):
>             if self._is_running:
>                 self._timer.cancel()
>                 self._timer      = None
>                 self._is_running = False
>
> --
> Cecil Westerhof
> Senior Software Engineer
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
> --
> https://mail.python.org/mailman/listinfo/python-list

From barry at barrys-emacs.org  Wed Feb  2 17:26:33 2022
From: barry at barrys-emacs.org (Barry)
Date: Wed, 2 Feb 2022 22:26:33 +0000
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <MuvdgZg--3-2@tutanota.com>
References: <MuvdgZg--3-2@tutanota.com>
Message-ID: <29DFF330-168B-4114-84C6-966CEBFBA0FD@barrys-emacs.org>



> On 2 Feb 2022, at 18:19, Jen Kris via Python-list <python-list at python.org> wrote:
> 
> ?It's not clear to me from the struct module whether it can actually auto-detect endianness. 

It is impossible to auto detect endian in the general case.

> I think it must be specified, just as I had to do with int.from_bytes().  In my case endianness was dictated by how the four bytes were populated, starting with the zero bytes on the left.  

You can specify the endian explicitly in the format strings. It?s all in the docs.

Barry
> 
> 
> Feb 1, 2022, 21:30 by wlfraed at ix.netcom.com:
> 
>> On Wed, 2 Feb 2022 00:40:22 +0100 (CET), Jen Kris <jenkris at tutanota.com>
>> declaimed the following:
>> 
>>> 
>>> breakup = int.from_bytes(byte_val, "big")
>>> 
>>> print("this is breakup " + str(breakup))
>> 
>>> 
>>> 
>>> Python prints:  this is breakup 32894
>> 
>>> 
>>> 
>>> Note that I had to switch from little endian to big endian.  Python is little endian by default, but in this case it's big endian.  
>> 
>>> 
>>> 
>> Look at the struct module. I'm pretty certain it has flags for big or
>> little end, or system native (that, or run your integers through the
>> various "network byte order" functions that I think C and Python both
>> support.
>> 
>> https://www.gta.ufrj.br/ensino/eel878/sockets/htonsman.html
>> 
>> 
>>> However, if anyone on this list knows how to pass data from a non-Python language to Python in multiprocessing.shared_memory please let me (and the list) know.  
>> 
>> <pondering>MMU cache lines not writing through to RAM? Can't find
>> anything on Google to force a cache flush</pondering> Can you test on a
>> different OS? (Windows vs Linux)
>> 
>> 
>> 
>> -- 
>> Wulfraed                 Dennis Lee Bieber         AF6VN
>> wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From barry at barrys-emacs.org  Wed Feb  2 17:31:52 2022
From: barry at barrys-emacs.org (Barry)
Date: Wed, 2 Feb 2022 22:31:52 +0000
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
References: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
Message-ID: <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>



> On 2 Feb 2022, at 21:12, Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
> 
> ?You could add a __del__ that calls stop :)

Didn?t python3 make this non deterministic when del is called?

I thought the recommendation is to not rely on __del__ in python3 code.

Barry
> 
>> On Wed, 2 Feb 2022 at 21:23, Cecil Westerhof via Python-list
>> <python-list at python.org> wrote:
>> 
>> I need (sometimes) to repeatedly execute a function. For this I wrote
>> the below class. What do you think about it?
>>    from threading  import Timer
>> 
>> 
>> 
>>    class repeated_timer(object):
>>        def __init__(self, fn, interval, start = False):
>>            if not callable(fn):
>>                raise TypeError('{} is not a function'.format(fn))
>>            self._fn         = fn
>>            self._check_interval(interval)
>>            self._interval   = interval
>>            self._timer      = None
>>            self._is_running = False
>>            if start:
>>                self.start()
>> 
>>        def _check_interval(self, interval):
>>            if not type(interval) in [int, float]:
>>                raise TypeError('{} is not numeric'.format(interval))
>>            if interval <= 0:
>>                raise ValueError('{} is not greater as 0'.format(interval))
>> 
>>        def _next(self):
>>            self._timer = Timer(self._interval, self._run)
>>            self._timer.start()
>> 
>>        def _run(self):
>>            self._next()
>>            self._fn()
>> 
>>        def set_interval(self, interval):
>>            self._check_interval(interval)
>>            self._interval = interval
>> 
>>        def start(self):
>>            if not self._is_running:
>>                self._next()
>>                self._is_running = True
>> 
>>        def stop(self):
>>            if self._is_running:
>>                self._timer.cancel()
>>                self._timer      = None
>>                self._is_running = False
>> 
>> --
>> Cecil Westerhof
>> Senior Software Engineer
>> LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>> --
>> https://mail.python.org/mailman/listinfo/python-list
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From rosuav at gmail.com  Wed Feb  2 17:58:46 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 3 Feb 2022 09:58:46 +1100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
References: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
 <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
Message-ID: <CAPTjJmpqQW-T3OHAJi_n1BkzJCYE9VuRCv+n-v=T6A1_3o+Axw@mail.gmail.com>

On Thu, 3 Feb 2022 at 09:33, Barry <barry at barrys-emacs.org> wrote:
>
>
>
> > On 2 Feb 2022, at 21:12, Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
> >
> > ?You could add a __del__ that calls stop :)
>
> Didn?t python3 make this non deterministic when del is called?
>
> I thought the recommendation is to not rely on __del__ in python3 code.
>

The __del__ method is called when the object is being disposed of.
Aside from some considerations about interpreter shutdown, it's
perfectly reliable... as long as you understand that objects don't get
disposed of when they "go out of scope", but when they run out of
references and get garbage collected. So for instance, you can't
depend on "x = Spam(); x = Ham()" to dispose of the Spam object
instantly, because it might not get garbage collected instantly; but
you can depend on __del__ getting called when the Spam object gets
garbage collected.

With that said, I can guarantee you that a __del__ method is NOT the
right way to call stop, for one simple reason: the object will keep
its own references (via the timer) so long as it is running. So it
won't get garbage collected, and in fact, this is very important to it
being reliable. Consider this simpler example:

def call_soon(func):
    t = threading.Timer(10, func)
    t.start()

When this function returns, you can no longer refer to the object 't'.
Will the function be called at the appropriate time? Yes, it
absolutely will, and it would be highly surprising if it didn't! So
the thread itself keeps a reference to the important objects.

(Side point: The OP's code is quite inefficient, as it creates a new
thread for each reiteration, but there's nothing wrong with that if
you're looking for something simple.)

The main reason to "not rely on __del__" (and, by the way, that's
nothing to do with whether it's Python 2 or Python 3, this has been
true since early Py2 and possibly earlier) is that you don't know when
the object will be disposed of. So if you want to guarantee that
something is cleaned up, what you need is a way for the object itself
to still exist, but the corresponding resource to be cleaned up. A
classic example is a file object:

def read_then_write(fn):
    with open(fn) as read_file:
        data = read_file.read()
    print(read_file)
    # ... transform the data as required ...
    with open(fn, "w") as write_file:
        write_file.write(data)
    print(write_file)

After each 'with' block, the file object is still there. You can refer
to it. Nothing has destroyed the object. But the file has been closed,
guaranteeing that you can safely reopen it in a different mode
(regardless of your OS). The same could be done with this timer; an
__exit__ method would make a lot of sense here, and would allow the
timer to be used in a with block to govern its execution. (It also
isn't really necessary, but if you want a good Pythonic way to show
the beginning and end of its use area, a 'with' block is the way to
go.)

ChrisA

From Cecil at decebal.nl  Wed Feb  2 19:27:07 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 03 Feb 2022 01:27:07 +0100
Subject: Waht do you think about my repeated_timer class
References: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
 <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
 <CAPTjJmpqQW-T3OHAJi_n1BkzJCYE9VuRCv+n-v=T6A1_3o+Axw@mail.gmail.com>
 <mailman.23.1643842739.2976.python-list@python.org>
Message-ID: <87k0eckd5w.fsf@munus.decebal.nl>

Chris Angelico <rosuav at gmail.com> writes:

> On Thu, 3 Feb 2022 at 09:33, Barry <barry at barrys-emacs.org> wrote:
> (Side point: The OP's code is quite inefficient, as it creates a new
> thread for each reiteration, but there's nothing wrong with that if
> you're looking for something simple.)

It is just something I wrote fast. How could I do this in a better way?


> (regardless of your OS). The same could be done with this timer; an
> __exit__ method would make a lot of sense here, and would allow the
> timer to be used in a with block to govern its execution. (It also
> isn't really necessary, but if you want a good Pythonic way to show
> the beginning and end of its use area, a 'with' block is the way to
> go.)

I will look into that.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From Cecil at decebal.nl  Wed Feb  2 19:54:54 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 03 Feb 2022 01:54:54 +0100
Subject: Waht do you think about my repeated_timer class
References: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
 <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
 <CAPTjJmpqQW-T3OHAJi_n1BkzJCYE9VuRCv+n-v=T6A1_3o+Axw@mail.gmail.com>
 <mailman.23.1643842739.2976.python-list@python.org>
 <87k0eckd5w.fsf@munus.decebal.nl>
Message-ID: <87fsp0kbvl.fsf@munus.decebal.nl>

Cecil Westerhof <Cecil at decebal.nl> writes:

>> (regardless of your OS). The same could be done with this timer; an
>> __exit__ method would make a lot of sense here, and would allow the
>> timer to be used in a with block to govern its execution. (It also
>> isn't really necessary, but if you want a good Pythonic way to show
>> the beginning and end of its use area, a 'with' block is the way to
>> go.)
>
> I will look into that.

Implemented:
    from threading  import Timer



    class repeated_timer(object):
        def __init__(self, fn, interval, start = False):
            if not callable(fn):
                raise TypeError('{} is not a function'.format(fn))
            self._fn         = fn
            self._check_interval(interval)
            self._interval   = interval
            self._timer      = None
            self._is_running = False
            if start:
                self.start()

        def __enter__(self):
            return self

        def __exit__(self, exc_type, exc_val, exc_tb):
            self.stop()

        def _check_interval(self, interval):
            if not type(interval) in [int, float]:
                raise TypeError('{} is not numeric'.format(interval))
            if interval <= 0:
                raise ValueError('{} is not greater as 0'.format(interval))

        def _next(self):
            self._timer = Timer(self._interval, self._run)
            self._timer.start()

        def _run(self):
            self._next()
            self._fn()

        def set_interval(self, interval):
            self._check_interval(interval)
            self._interval = interval

        def start(self):
            if not self._is_running:
                self._is_running = True
                self._next()

        def stop(self):
            if self._is_running:
                self._timer.cancel()
                self._timer      = None
                self._is_running = False

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From cs at cskk.id.au  Wed Feb  2 20:39:43 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Thu, 3 Feb 2022 12:39:43 +1100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <87fsp0kbvl.fsf@munus.decebal.nl>
References: <87fsp0kbvl.fsf@munus.decebal.nl>
Message-ID: <YfsyX0swyT9ebSs2@cskk.homeip.net>

You have:

    def _check_interval(self, interval):
        if not type(interval) in [int, float]:
            raise TypeError('{} is not numeric'.format(interval))

This check is better written:

    if not isinstance(interval, (int,float)):

which handles subclasses of these types (but note that bool subclasses 
int :-) normally we don't care), or behaviourally:

    try:
        interval = float(interval)
    except ValueError as e:
        raise TypeError(
            "cannot convert %s:%r to float: %s"
            % (type(interval).__name__, interval, e)) from e

which tries to convert to float and fails if that does not work, which 
supports classes with a __float__ method (these classes are rare, but 
decimal.Decimal is one example).

I'd probably use the isinstance() form myself.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From rosuav at gmail.com  Wed Feb  2 21:23:22 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 3 Feb 2022 13:23:22 +1100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <87k0eckd5w.fsf@munus.decebal.nl>
References: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
 <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
 <CAPTjJmpqQW-T3OHAJi_n1BkzJCYE9VuRCv+n-v=T6A1_3o+Axw@mail.gmail.com>
 <mailman.23.1643842739.2976.python-list@python.org>
 <87k0eckd5w.fsf@munus.decebal.nl>
Message-ID: <CAPTjJmp52PZc-toTuBYDdJUr=2Jku6uwtVeUFAiYi7fv4+WnJQ@mail.gmail.com>

On Thu, 3 Feb 2022 at 12:24, Cecil Westerhof via Python-list
<python-list at python.org> wrote:
>
> Chris Angelico <rosuav at gmail.com> writes:
>
> > On Thu, 3 Feb 2022 at 09:33, Barry <barry at barrys-emacs.org> wrote:
> > (Side point: The OP's code is quite inefficient, as it creates a new
> > thread for each reiteration, but there's nothing wrong with that if
> > you're looking for something simple.)
>
> It is just something I wrote fast. How could I do this in a better way?

I'll answer your question, but first and foremost: Your code was fine,
and if something does what it's supposed to, that is the most
important. Everything else is minor.

But for other ways to do things, I would recommend creating a single
thread function and spawning a single thread to run it, and then
having that function call the target every N seconds. Also, consider
subclassing Thread rather than subclassing object (which, btw, is the
default; you don't need to say "class X(object)"), which will
automatically give your object all the methods of a timer.

If you feel like it, you could even look into ways to do things
without threads, but that would be a much bigger change :)

But remember: when your code does what it's supposed to, it is *fine*,
and doesn't need changing. I'm not saying that your code is bad :)

ChrisA

From avigross at verizon.net  Wed Feb  2 21:31:16 2022
From: avigross at verizon.net (Avi Gross)
Date: Thu, 3 Feb 2022 02:31:16 +0000 (UTC)
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <MuvgAhH--3-2@tutanota.com>
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
 <MureHAB----2@tutanota.com>
 <mailman.11.1643758824.2976.python-list@python.org>
 <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com>
 <706734682.2805722.1643825767629@mail.yahoo.com> <MuvgAhH--3-2@tutanota.com>
Message-ID: <796338694.2916887.1643855476163@mail.yahoo.com>

Jen,

I would not be shocked at incompatibilities in the system described making it hard to exchange anything, including text, but am not clear if there is a limitation of four bytes in what can be shared. For me, a character string can use any number of contiguous bytes in memory that some kind of pointer or table lookup provides access to. 

Clearly you could squeeze larger number in by not writing decimals but using hexadecimal notation that adds a to f as valid entries, and of course you can make your own customized base 32 or if you use more symbols (such as upper and lower case as different) you could  add a few symbols and stretch it out to base 60 or 64.

Like I said, I may have missed something. If you KNOW that system A will interact with systems B and C and so on, and each machine knows what kind it is, and especially if it knows which kind of machine left it something in shared memory, there has to be a way to coordinate between them. If character strings in ASCII or UTF8 or EBCDIC or some kind of raw are allowed, and used with just a few characters that can spell out numbers, I would think much is possible. If needed, perhaps a fixed size could be set aside and the string use a null terminator if shorter. 

Of course if this big-endian issue also scrambles bytes used in strings, forget it.

Or, maybe shared memory is not the easy way to go, even it it might be faster.



-----Original Message-----
From: Jen Kris <jenkris at tutanota.com>
To: Avi Gross <avigross at verizon.net>
Cc: python-list at python.org <python-list at python.org>
Sent: Wed, Feb 2, 2022 1:27 pm
Subject: Re: Data unchanged when passing data to Python in multiprocessing shared memory



An ASCII string will not work.? If you convert 32894 to an ascii string you will have five bytes, but you need four.? In my original post I showed the C program I used to convert any 32-bit number to 4 bytes.? 






Feb 2, 2022, 10:16 by python-list at python.org:

I applaud trying to find the right solution but wonder if a more trivial solution is even being considered. It ignores big and little endians and just converts your data into another form and back.



If all you want to do is send an integer that fit in 32 bits or 64 bits, why not convert it to a character string in a form that both machines will see the same way and when read back, convert it back to an integer? 



As long as both side see the same string, this can be done in reasonable time and portably.



Or am I missing something? Is "1234" not necessarily seen in the same order, or "1.234e3" or whatever?



Obviously, if the mechanism is heavily used and multiple sides keep reading and even writing the same memory location, this is not ideal. But having different incompatible processors looking at the same memory is also not.



-----Original Message-----

From: Dennis Lee Bieber <wlfraed at ix.netcom.com>

To: python-list at python.org

Sent: Wed, Feb 2, 2022 12:30 am

Subject: Re: Data unchanged when passing data to Python in multiprocessing shared memory





On Wed, 2 Feb 2022 00:40:22 +0100 (CET), Jen Kris <jenkris at tutanota.com>



declaimed the following:






breakup = int.from_bytes(byte_val, "big")



>print("this is breakup " + str(breakup))





>Python prints:? this is breakup 32894





>Note that I had to switch from little endian to big endian.? Python is little endian by default, but in this case it's big endian.? 





??? Look at the struct module. I'm pretty certain it has flags for big or



little end, or system native (that, or run your integers through the



various "network byte order" functions that I think C and Python both



support.







https://www.gta.ufrj.br/ensino/eel878/sockets/htonsman.html











>However, if anyone on this list knows how to pass data from a non-Python language to Python in multiprocessing.shared_memory please let me (and the list) know.? 







??? <pondering>MMU cache lines not writing through to RAM? Can't find



anything on Google to force a cache flush</pondering> Can you test on a



different OS? (Windows vs Linux)















-- 



??? Wulfraed? ? ? ? ? ? ? ?  Dennis Lee Bieber? ? ? ?  AF6VN



??? wlfraed at ix.netcom.com? ? http://wlfraed.microdiversity.freeddns.org/



-- 



https://mail.python.org/mailman/listinfo/python-list



-- 

https://mail.python.org/mailman/listinfo/python-list



  


From rosuav at gmail.com  Wed Feb  2 21:35:15 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 3 Feb 2022 13:35:15 +1100
Subject: Data unchanged when passing data to Python in multiprocessing
 shared memory
In-Reply-To: <796338694.2916887.1643855476163@mail.yahoo.com>
References: <Muqv9er--3-2@tutanota.com>
 <35BA674C-1AAF-43B3-9203-BBF7472E6AE4@barrys-emacs.org>
 <MureHAB----2@tutanota.com>
 <mailman.11.1643758824.2976.python-list@python.org>
 <s44kvg91u4dbvq1nh2uoaj9gd7o20d5fka@4ax.com>
 <706734682.2805722.1643825767629@mail.yahoo.com>
 <MuvgAhH--3-2@tutanota.com> <796338694.2916887.1643855476163@mail.yahoo.com>
Message-ID: <CAPTjJmrvC=R0KnJG_MiWe9i2KOB1twyvyuebcK_tRvLuJspbbw@mail.gmail.com>

On Thu, 3 Feb 2022 at 13:32, Avi Gross via Python-list
<python-list at python.org> wrote:
>
> Jen,
>
> I would not be shocked at incompatibilities in the system described making it hard to exchange anything, including text, but am not clear if there is a limitation of four bytes in what can be shared. For me, a character string can use any number of contiguous bytes in memory that some kind of pointer or table lookup provides access to.
>
> Clearly you could squeeze larger number in by not writing decimals but using hexadecimal notation that adds a to f as valid entries, and of course you can make your own customized base 32 or if you use more symbols (such as upper and lower case as different) you could  add a few symbols and stretch it out to base 60 or 64.
>
> Like I said, I may have missed something. If you KNOW that system A will interact with systems B and C and so on, and each machine knows what kind it is, and especially if it knows which kind of machine left it something in shared memory, there has to be a way to coordinate between them. If character strings in ASCII or UTF8 or EBCDIC or some kind of raw are allowed, and used with just a few characters that can spell out numbers, I would think much is possible. If needed, perhaps a fixed size could be set aside and the string use a null terminator if shorter.
>
> Of course if this big-endian issue also scrambles bytes used in strings, forget it.
>
> Or, maybe shared memory is not the easy way to go, even it it might be faster.
>

Structs are fine, and are highly unlikely to be the OP's problem.
Switching to a different system won't help.

Unfortunately I can't offer any serious help about the actual problem,
as I don't know what's going on with the shared memory issue.

ChrisA

From Cecil at decebal.nl  Wed Feb  2 21:17:07 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 03 Feb 2022 03:17:07 +0100
Subject: Waht do you think about my repeated_timer class
References: <87fsp0kbvl.fsf@munus.decebal.nl>
 <YfsyX0swyT9ebSs2@cskk.homeip.net>
 <mailman.24.1643853485.2976.python-list@python.org>
Message-ID: <87bkzok82k.fsf@munus.decebal.nl>

Cameron Simpson <cs at cskk.id.au> writes:

> You have:
>
>     def _check_interval(self, interval):
>         if not type(interval) in [int, float]:
>             raise TypeError('{} is not numeric'.format(interval))
>
> This check is better written:
>
>     if not isinstance(interval, (int,float)):
>
> which handles subclasses of these types (but note that bool subclasses 
> int :-)

Done, thanks.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From 2QdxY4RzWzUUiLuE at potatochowder.com  Wed Feb  2 22:31:42 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Wed, 2 Feb 2022 21:31:42 -0600
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <YfsyX0swyT9ebSs2@cskk.homeip.net>
References: <87fsp0kbvl.fsf@munus.decebal.nl>
 <YfsyX0swyT9ebSs2@cskk.homeip.net>
Message-ID: <YftMntVfWblJKlvB@scrozzle>

On 2022-02-03 at 12:39:43 +1100,
Cameron Simpson <cs at cskk.id.au> wrote:

> You have:
> 
>     def _check_interval(self, interval):
>         if not type(interval) in [int, float]:
>             raise TypeError('{} is not numeric'.format(interval))
> 
> This check is better written:
> 
>     if not isinstance(interval, (int,float)):
> 
> which handles subclasses of these types (but note that bool subclasses 
> int :-) normally we don't care), or behaviourally:
> 
>     try:
>         interval = float(interval)
>     except ValueError as e:
>         raise TypeError(
>             "cannot convert %s:%r to float: %s"
>             % (type(interval).__name__, interval, e)) from e
> 
> which tries to convert to float and fails if that does not work, which 
> supports classes with a __float__ method (these classes are rare, but 
> decimal.Decimal is one example).

I think this can be simplified for time intervals to the following:

    if interval <= 0:
        raise ValueError(...)

which accepts non-negative real values; throws ValueError for negative
real values; and TypeError for other stuff, including complex numbers
(pathological types notwithstanding).  One thing that doesn't work right
is NaNs, but I'm sure it's not the only code that acts weirdly when
faced with a NaM (curiously, Timer accepts a NaN, but the text I get
from help(Timer) in Python 3.10.2 is, well, broken).

FWIW, I'd find some way to tell users the units (seconds, milliseconds,
fortnights, etc.) instead of making them wade through your code to find
the call to (and possibly the [broken] help text of) Timer.

From rosuav at gmail.com  Wed Feb  2 23:07:22 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 3 Feb 2022 15:07:22 +1100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <YftMntVfWblJKlvB@scrozzle>
References: <87fsp0kbvl.fsf@munus.decebal.nl>
 <YfsyX0swyT9ebSs2@cskk.homeip.net> <YftMntVfWblJKlvB@scrozzle>
Message-ID: <CAPTjJmqcohrT3wiTYqhJVscHXi5Dn5AnUjCd=_uch=iiwVdPRA@mail.gmail.com>

On Thu, 3 Feb 2022 at 14:52, <2QdxY4RzWzUUiLuE at potatochowder.com> wrote:
>
> On 2022-02-03 at 12:39:43 +1100,
> Cameron Simpson <cs at cskk.id.au> wrote:
>
> > You have:
> >
> >     def _check_interval(self, interval):
> >         if not type(interval) in [int, float]:
> >             raise TypeError('{} is not numeric'.format(interval))
> >
> > This check is better written:
> >
> >     if not isinstance(interval, (int,float)):
> >
> > which handles subclasses of these types (but note that bool subclasses
> > int :-) normally we don't care), or behaviourally:
> >
> >     try:
> >         interval = float(interval)
> >     except ValueError as e:
> >         raise TypeError(
> >             "cannot convert %s:%r to float: %s"
> >             % (type(interval).__name__, interval, e)) from e
> >
> > which tries to convert to float and fails if that does not work, which
> > supports classes with a __float__ method (these classes are rare, but
> > decimal.Decimal is one example).
>
> I think this can be simplified for time intervals to the following:
>
>     if interval <= 0:
>         raise ValueError(...)

That's checking something quite different, though. Casting to float
will accept anything that can be, well, cast to float, but checking
for less than or equal to zero demands that it already be some sort of
number. It's debatable which check is more correct, but certainly this
is not a simplification of the other code, it's a distinctly different
validation.

> which accepts non-negative real values; throws ValueError for negative
> real values; and TypeError for other stuff, including complex numbers
> (pathological types notwithstanding).  One thing that doesn't work right
> is NaNs, but I'm sure it's not the only code that acts weirdly when
> faced with a NaM (curiously, Timer accepts a NaN, but the text I get
> from help(Timer) in Python 3.10.2 is, well, broken).

Strange. The text I get in 3.11.0a1 is fine. But in any case, there's
always the docs on the web.

https://docs.python.org/3/library/threading.html#timer-objects

> FWIW, I'd find some way to tell users the units (seconds, milliseconds,
> fortnights, etc.) instead of making them wade through your code to find
> the call to (and possibly the [broken] help text of) Timer.

In anything in Python, assume that the unit is seconds. With anything
that accepts floats (where these can be distinguished from integers),
assume the unit is seconds. If it accepts micro or nanoseconds, it'll
almost certainly be called "high resolution timer" (unless it accepts
two args, sec and us/ns, but that's pretty obvious), so you can
generally exclude those too. The only real question is whether
sleep(int) takes seconds or milliseconds, which isn't a problem here.

Citation: I've slept in many many programming languages and
frameworks. Which sounds seriously weird, but you're all programmers,
you know what I mean :)

ChrisA

From 2QdxY4RzWzUUiLuE at potatochowder.com  Wed Feb  2 23:27:24 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Wed, 2 Feb 2022 22:27:24 -0600
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <CAPTjJmqcohrT3wiTYqhJVscHXi5Dn5AnUjCd=_uch=iiwVdPRA@mail.gmail.com>
References: <87fsp0kbvl.fsf@munus.decebal.nl>
 <YfsyX0swyT9ebSs2@cskk.homeip.net> <YftMntVfWblJKlvB@scrozzle>
 <CAPTjJmqcohrT3wiTYqhJVscHXi5Dn5AnUjCd=_uch=iiwVdPRA@mail.gmail.com>
Message-ID: <YftZrHF20orBipmY@scrozzle>

On 2022-02-03 at 15:07:22 +1100,
Chris Angelico <rosuav at gmail.com> wrote:

> On Thu, 3 Feb 2022 at 14:52, <2QdxY4RzWzUUiLuE at potatochowder.com> wrote:
> >
> > On 2022-02-03 at 12:39:43 +1100,
> > Cameron Simpson <cs at cskk.id.au> wrote:
> >
> > > You have:
> > >
> > >     def _check_interval(self, interval):
> > >         if not type(interval) in [int, float]:
> > >             raise TypeError('{} is not numeric'.format(interval))
> > >
> > > This check is better written:
> > >
> > >     if not isinstance(interval, (int,float)):
> > >
> > > which handles subclasses of these types (but note that bool subclasses
> > > int :-) normally we don't care), or behaviourally:
> > >
> > >     try:
> > >         interval = float(interval)
> > >     except ValueError as e:
> > >         raise TypeError(
> > >             "cannot convert %s:%r to float: %s"
> > >             % (type(interval).__name__, interval, e)) from e
> > >
> > > which tries to convert to float and fails if that does not work, which
> > > supports classes with a __float__ method (these classes are rare, but
> > > decimal.Decimal is one example).
> >
> > I think this can be simplified for time intervals to the following:
> >
> >     if interval <= 0:
> >         raise ValueError(...)
> 
> That's checking something quite different, though. Casting to float
> will accept anything that can be, well, cast to float, but checking
> for less than or equal to zero demands that it already be some sort of
> number. It's debatable which check is more correct, but certainly this
> is not a simplification of the other code, it's a distinctly different
> validation.

Okay, "simplified" isn't quite the right word.  Given two examples (with
known deficiencies) and no actual use cases or specifications, I added a
third example, which I believed was simpler (and arguably better in one
or more ways, which I explained), than the others.

> > which accepts non-negative real values; throws ValueError for negative
> > real values; and TypeError for other stuff, including complex numbers
> > (pathological types notwithstanding).  One thing that doesn't work right
> > is NaNs, but I'm sure it's not the only code that acts weirdly when
> > faced with a NaM (curiously, Timer accepts a NaN, but the text I get
> > from help(Timer) in Python 3.10.2 is, well, broken).
> 
> Strange. The text I get in 3.11.0a1 is fine. But in any case, there's
> always the docs on the web.
> 
> https://docs.python.org/3/library/threading.html#timer-objects

help(Timer) is built into my REPL (and likely consumed by development
systems and IDEs everywhere).  No web necessary.

> > FWIW, I'd find some way to tell users the units (seconds, milliseconds,
> > fortnights, etc.) instead of making them wade through your code to find
> > the call to (and possibly the [broken] help text of) Timer.
> 
> In anything in Python, assume that the unit is seconds. With anything
> that accepts floats (where these can be distinguished from integers),
> assume the unit is seconds. If it accepts micro or nanoseconds, it'll
> almost certainly be called "high resolution timer" (unless it accepts
> two args, sec and us/ns, but that's pretty obvious), so you can
> generally exclude those too. The only real question is whether
> sleep(int) takes seconds or milliseconds, which isn't a problem here.
> 
> Citation: I've slept in many many programming languages and
> frameworks. Which sounds seriously weird, but you're all programmers,
> you know what I mean :)

I've slept in enough programming environments to know better than to
assume anything.  ;-)

From rosuav at gmail.com  Wed Feb  2 23:37:01 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 3 Feb 2022 15:37:01 +1100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <YftZrHF20orBipmY@scrozzle>
References: <87fsp0kbvl.fsf@munus.decebal.nl>
 <YfsyX0swyT9ebSs2@cskk.homeip.net> <YftMntVfWblJKlvB@scrozzle>
 <CAPTjJmqcohrT3wiTYqhJVscHXi5Dn5AnUjCd=_uch=iiwVdPRA@mail.gmail.com>
 <YftZrHF20orBipmY@scrozzle>
Message-ID: <CAPTjJmpvNO66V0i9Q9-k0uBvBQzZKi3aC1LxqJ2m4CkvxrZtpw@mail.gmail.com>

On Thu, 3 Feb 2022 at 15:28, <2QdxY4RzWzUUiLuE at potatochowder.com> wrote:
>
> On 2022-02-03 at 15:07:22 +1100,
> Chris Angelico <rosuav at gmail.com> wrote:
>
> > On Thu, 3 Feb 2022 at 14:52, <2QdxY4RzWzUUiLuE at potatochowder.com> wrote:
> > >
> > > On 2022-02-03 at 12:39:43 +1100,
> > > Cameron Simpson <cs at cskk.id.au> wrote:
> > >
> > > > You have:
> > > >
> > > >     def _check_interval(self, interval):
> > > >         if not type(interval) in [int, float]:
> > > >             raise TypeError('{} is not numeric'.format(interval))
> > > >
> > > > This check is better written:
> > > >
> > > >     if not isinstance(interval, (int,float)):
> > > >
> > > > which handles subclasses of these types (but note that bool subclasses
> > > > int :-) normally we don't care), or behaviourally:
> > > >
> > > >     try:
> > > >         interval = float(interval)
> > > >     except ValueError as e:
> > > >         raise TypeError(
> > > >             "cannot convert %s:%r to float: %s"
> > > >             % (type(interval).__name__, interval, e)) from e
> > > >
> > > > which tries to convert to float and fails if that does not work, which
> > > > supports classes with a __float__ method (these classes are rare, but
> > > > decimal.Decimal is one example).
> > >
> > > I think this can be simplified for time intervals to the following:
> > >
> > >     if interval <= 0:
> > >         raise ValueError(...)
> >
> > That's checking something quite different, though. Casting to float
> > will accept anything that can be, well, cast to float, but checking
> > for less than or equal to zero demands that it already be some sort of
> > number. It's debatable which check is more correct, but certainly this
> > is not a simplification of the other code, it's a distinctly different
> > validation.
>
> Okay, "simplified" isn't quite the right word.  Given two examples (with
> known deficiencies) and no actual use cases or specifications, I added a
> third example, which I believed was simpler (and arguably better in one
> or more ways, which I explained), than the others.

Fair :) I'm unsure which is the best check here, or whether it's worth
having any check at all (you could just let the Timer - or
time.sleep() - do the checks).

> > Strange. The text I get in 3.11.0a1 is fine. But in any case, there's
> > always the docs on the web.
> >
> > https://docs.python.org/3/library/threading.html#timer-objects
>
> help(Timer) is built into my REPL (and likely consumed by development
> systems and IDEs everywhere).  No web necessary.

Agreed, and my primary response is that it's fine in my 3.11, so it's
probably something fixable. The web docs are a good fallback though.

> I've slept in enough programming environments to know better than to
> assume anything.  ;-)

Excellent :)

ChrisA

From Cecil at decebal.nl  Wed Feb  2 22:58:01 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 03 Feb 2022 04:58:01 +0100
Subject: Waht do you think about my repeated_timer class
References: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
 <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
 <CAPTjJmpqQW-T3OHAJi_n1BkzJCYE9VuRCv+n-v=T6A1_3o+Axw@mail.gmail.com>
 <mailman.23.1643842739.2976.python-list@python.org>
 <87k0eckd5w.fsf@munus.decebal.nl>
 <CAPTjJmp52PZc-toTuBYDdJUr=2Jku6uwtVeUFAiYi7fv4+WnJQ@mail.gmail.com>
 <mailman.25.1643855014.2976.python-list@python.org>
Message-ID: <877dack3ee.fsf@munus.decebal.nl>

Chris Angelico <rosuav at gmail.com> writes:

>> > (Side point: The OP's code is quite inefficient, as it creates a new
>> > thread for each reiteration, but there's nothing wrong with that if
>> > you're looking for something simple.)
>>
>> It is just something I wrote fast. How could I do this in a better way?
>
> I'll answer your question, but first and foremost: Your code was fine,
> and if something does what it's supposed to, that is the most
> important. Everything else is minor.

I like to write efficient code and it never hurts to write better code
as just doing what it is supposed to do. ;-)

And in my case interval is .5 seconds and when someone is going to use
it with an even smaller interval ?


> But for other ways to do things, I would recommend creating a single
> thread function and spawning a single thread to run it, and then
> having that function call the target every N seconds. Also, consider

Have to be careful that timing keeps correct when target takes a 'lot'
of time.
Something to ponder about, but can wait.


> subclassing Thread rather than subclassing object (which, btw, is the
> default; you don't need to say "class X(object)"), which will
> automatically give your object all the methods of a timer.

Of-course. I should have thought about that. :'-(

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From rosuav at gmail.com  Wed Feb  2 23:49:57 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 3 Feb 2022 15:49:57 +1100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <877dack3ee.fsf@munus.decebal.nl>
References: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
 <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
 <CAPTjJmpqQW-T3OHAJi_n1BkzJCYE9VuRCv+n-v=T6A1_3o+Axw@mail.gmail.com>
 <mailman.23.1643842739.2976.python-list@python.org>
 <87k0eckd5w.fsf@munus.decebal.nl>
 <CAPTjJmp52PZc-toTuBYDdJUr=2Jku6uwtVeUFAiYi7fv4+WnJQ@mail.gmail.com>
 <mailman.25.1643855014.2976.python-list@python.org>
 <877dack3ee.fsf@munus.decebal.nl>
Message-ID: <CAPTjJmqL+8bDM6n=KBPGwa0xua9Up9jgR06MsDadX2jS6tDv=g@mail.gmail.com>

On Thu, 3 Feb 2022 at 15:43, Cecil Westerhof via Python-list
<python-list at python.org> wrote:
>
> Chris Angelico <rosuav at gmail.com> writes:
>
> >> > (Side point: The OP's code is quite inefficient, as it creates a new
> >> > thread for each reiteration, but there's nothing wrong with that if
> >> > you're looking for something simple.)
> >>
> >> It is just something I wrote fast. How could I do this in a better way?
> >
> > I'll answer your question, but first and foremost: Your code was fine,
> > and if something does what it's supposed to, that is the most
> > important. Everything else is minor.
>
> I like to write efficient code and it never hurts to write better code
> as just doing what it is supposed to do. ;-)
>
> And in my case interval is .5 seconds and when someone is going to use
> it with an even smaller interval ?
>
>
> > But for other ways to do things, I would recommend creating a single
> > thread function and spawning a single thread to run it, and then
> > having that function call the target every N seconds. Also, consider
>
> Have to be careful that timing keeps correct when target takes a 'lot'
> of time.
> Something to ponder about, but can wait.

Ah. In that case, I would recommend a different look at things.
Instead of waiting X time, then firing the event, then waiting X time,
consider instead an interval timer based on monotonic time:

https://docs.python.org/3/library/time.html#time.monotonic

When the timer starts, record the current monotonic time, and sleep
one interval. After the function returns, sleep the remainder of one
interval. It's up to you what happens if you ever find that the next
time point has already passed - do you call the function immediately,
or skip and wait for the next moment?

Interval timers have some complexity to them, but it's worth putting
in the time (pun intended) to figure out how these things work :)

ChrisA

From Cecil at decebal.nl  Wed Feb  2 23:52:19 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 03 Feb 2022 05:52:19 +0100
Subject: Waht do you think about my repeated_timer class
References: <87fsp0kbvl.fsf@munus.decebal.nl>
 <YfsyX0swyT9ebSs2@cskk.homeip.net> <YftMntVfWblJKlvB@scrozzle>
 <mailman.28.1643860267.2976.python-list@python.org>
Message-ID: <87zgn8imbg.fsf@munus.decebal.nl>

2QdxY4RzWzUUiLuE at potatochowder.com writes:

> FWIW, I'd find some way to tell users the units (seconds, milliseconds,
> fortnights, etc.) instead of making them wade through your code to find
> the call to (and possibly the [broken] help text of) Timer.

You mean with docstring?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From Cecil at decebal.nl  Thu Feb  3 00:00:18 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 03 Feb 2022 06:00:18 +0100
Subject: Waht do you think about my repeated_timer class
References: <87zgn9jdff.fsf@munus.decebal.nl>
Message-ID: <87v8xwily5.fsf@munus.decebal.nl>

Cecil Westerhof <Cecil at decebal.nl> writes:

> I need (sometimes) to repeatedly execute a function. For this I wrote
> the below class. What do you think about it?

I wrote some unit test for the class. Is this the correct way to do
this?
For example in test_correct_params_no_start I check four things. Some
people say you should not check more as one.

By the way it was good that I defined __enter__ and __exit__. :-D


    #!/usr/bin/env python3



    import unittest

    from threading  import Timer



    from repeated_timer import repeated_timer


    class test_repeated_timer(unittest.TestCase):
       def test_interval_zero(self):
          with self.assertRaises(ValueError):
              rt = repeated_timer(dummy_fn, 0)

       def test_interval_negative(self):
          with self.assertRaises(ValueError):
              rt = repeated_timer(dummy_fn, -1)

       def test_interval_string(self):
          with self.assertRaises(TypeError):
              rt = repeated_timer(dummy_fn, '.4')

       def test_non_function(self):
          with self.assertRaises(TypeError):
              rt = repeated_timer('dummy_fn', .5)

       def test_correct_params_no_start(self):
           rt = repeated_timer(dummy_fn, .5)
           self.assertEqual(rt._fn,       dummy_fn)
           self.assertEqual(rt._interval, .5)
           self.assertEqual(rt._timer,    None)
           self.assertFalse(rt._is_running)

       def test_correct_params_do_start(self):
           with repeated_timer(dummy_fn, .375, True) as rt:
               self.assertEqual(rt._fn,       dummy_fn)
               self.assertEqual(rt._interval, .375)
               self.assertTrue (isinstance(rt._timer, Timer), 'There should be a timer')
               self.assertTrue (rt._is_running,               'Should be running')

       def test__start_later(self):
           with repeated_timer(dummy_fn, .5) as rt:
               self.assertEqual(rt._fn,       dummy_fn)
               self.assertEqual(rt._interval, .5)
               self.assertEqual(rt._timer,    None)
               self.assertFalse(rt._is_running)
               rt.start()
               self.assertEqual(rt._fn,       dummy_fn)
               self.assertEqual(rt._interval, .5)
               self.assertTrue (isinstance(rt._timer, Timer), 'There should be a timer')
               self.assertTrue (rt._is_running,               'Should be running')



    def dummy_fn():
        pass



    if __name__ == '__main__':
        unittest.main()


>     from threading  import Timer
>
>
>
>     class repeated_timer(object):
>         def __init__(self, fn, interval, start = False):
>             if not callable(fn):
>                 raise TypeError('{} is not a function'.format(fn))
>             self._fn         = fn
>             self._check_interval(interval)
>             self._interval   = interval
>             self._timer      = None
>             self._is_running = False
>             if start:
>                 self.start()
>
>         def _check_interval(self, interval):
>             if not type(interval) in [int, float]:
>                 raise TypeError('{} is not numeric'.format(interval))
>             if interval <= 0:
>                 raise ValueError('{} is not greater as 0'.format(interval))
>
>         def _next(self):
>             self._timer = Timer(self._interval, self._run)
>             self._timer.start()
>
>         def _run(self):
>             self._next()
>             self._fn()
>
>         def set_interval(self, interval):
>             self._check_interval(interval)
>             self._interval = interval
>
>         def start(self):
>             if not self._is_running:
>                 self._next()
>                 self._is_running = True
>
>         def stop(self):
>             if self._is_running:
>                 self._timer.cancel()
>                 self._timer      = None
>                 self._is_running = False

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From Cecil at decebal.nl  Thu Feb  3 00:44:00 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 03 Feb 2022 06:44:00 +0100
Subject: Waht do you think about my repeated_timer class
References: <CABbU2U8fCh-qR5QdyFwgBAmQEi+6nzsbNK=2Jjk=QZxHf=h3CQ@mail.gmail.com>
 <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
 <CAPTjJmpqQW-T3OHAJi_n1BkzJCYE9VuRCv+n-v=T6A1_3o+Axw@mail.gmail.com>
 <mailman.23.1643842739.2976.python-list@python.org>
 <87k0eckd5w.fsf@munus.decebal.nl>
 <CAPTjJmp52PZc-toTuBYDdJUr=2Jku6uwtVeUFAiYi7fv4+WnJQ@mail.gmail.com>
 <mailman.25.1643855014.2976.python-list@python.org>
 <877dack3ee.fsf@munus.decebal.nl>
 <CAPTjJmqL+8bDM6n=KBPGwa0xua9Up9jgR06MsDadX2jS6tDv=g@mail.gmail.com>
 <mailman.32.1643863810.2976.python-list@python.org>
Message-ID: <87r18kijxb.fsf@munus.decebal.nl>

Chris Angelico <rosuav at gmail.com> writes:

> On Thu, 3 Feb 2022 at 15:43, Cecil Westerhof via Python-list
> <python-list at python.org> wrote:
>>
>> Chris Angelico <rosuav at gmail.com> writes:
>>
>> >> > (Side point: The OP's code is quite inefficient, as it creates a new
>> >> > thread for each reiteration, but there's nothing wrong with that if
>> >> > you're looking for something simple.)
>> >>
>> >> It is just something I wrote fast. How could I do this in a better way?
>> >
>> > I'll answer your question, but first and foremost: Your code was fine,
>> > and if something does what it's supposed to, that is the most
>> > important. Everything else is minor.
>>
>> I like to write efficient code and it never hurts to write better code
>> as just doing what it is supposed to do. ;-)
>>
>> And in my case interval is .5 seconds and when someone is going to use
>> it with an even smaller interval ?
>>
>>
>> > But for other ways to do things, I would recommend creating a single
>> > thread function and spawning a single thread to run it, and then
>> > having that function call the target every N seconds. Also, consider
>>
>> Have to be careful that timing keeps correct when target takes a 'lot'
>> of time.
>> Something to ponder about, but can wait.
>
> Ah. In that case, I would recommend a different look at things.
> Instead of waiting X time, then firing the event, then waiting X time,
> consider instead an interval timer based on monotonic time:
>
> https://docs.python.org/3/library/time.html#time.monotonic
>
> When the timer starts, record the current monotonic time, and sleep
> one interval. After the function returns, sleep the remainder of one
> interval. It's up to you what happens if you ever find that the next
> time point has already passed - do you call the function immediately,
> or skip and wait for the next moment?
>
> Interval timers have some complexity to them, but it's worth putting
> in the time (pun intended) to figure out how these things work :)

I will look into it.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From 2QdxY4RzWzUUiLuE at potatochowder.com  Thu Feb  3 06:37:25 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Thu, 3 Feb 2022 05:37:25 -0600
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <87zgn8imbg.fsf@munus.decebal.nl>
References: <87fsp0kbvl.fsf@munus.decebal.nl>
 <YfsyX0swyT9ebSs2@cskk.homeip.net> <YftMntVfWblJKlvB@scrozzle>
 <mailman.28.1643860267.2976.python-list@python.org>
 <87zgn8imbg.fsf@munus.decebal.nl>
Message-ID: <Yfu+dQVpBVZRtVbF@scrozzle>

On 2022-02-03 at 05:52:19 +0100,
Cecil Westerhof via Python-list <python-list at python.org> wrote:

> 2QdxY4RzWzUUiLuE at potatochowder.com writes:
> 
> > FWIW, I'd find some way to tell users the units (seconds, milliseconds,
> > fortnights, etc.) instead of making them wade through your code to find
> > the call to (and possibly the [broken] help text of) Timer.
> 
> You mean with docstring?

Docstring, comments, error/exception text, external documentation, URLs
or other references in the source code (docstring, comments), the name
of the object in question (e.g., instead of "interval," call it
"interval_seconds," or "seconds_between_runs").  *Something*.  Or more
than one of the above.  A reference to the Timer class's documentation.
There are a lot of options.

Pick a place (or more than one!) that will be fairly conspicuous in as
many situations to as many users (including future you) or potential
users (including your future code) as possible.

In a perfect world, you will receive feedback and/or use your own code,
and improve your choice(s) and method(s) over time.

From sjeik_appie at hotmail.com  Thu Feb  3 09:12:06 2022
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 03 Feb 2022 15:12:06 +0100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <177517EB-2AA2-47AA-B974-100188449961@barrys-emacs.org>
Message-ID: <DB6PR01MB38955B70C5FFC45A8A50A2BD83289@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Feb 2, 2022 23:31, Barry <barry at barrys-emacs.org> wrote:

     > On 2 Feb 2022, at 21:12, Marco Sulla <Marco.Sulla.Python at gmail.com>
     wrote:
     >
     > You could add a __del__ that calls stop :)

     Didn't python3 make this non deterministic when del is called?

     I thought the recommendation is to not rely on __del__ in python3 code.

   ==>
   Adding __del__ also poses chalenges is you would like to support pypy:
   "There are a few extra implications from the difference in the GC. Most
   notably, if an object has a __del__, the __del__ is never called more than
   once in PyPy; but CPython will call the same __del__ several times if the
   object is resurrected and dies again (at least it is reliably so in older
   CPythons; newer CPythons try to call destructors not more than once, but
   there are counter-examples). The __del__ methods are called in "the right"
   order if they are on objects pointing to each other, as in CPython, but
   unlike CPython, if there is a dead cycle of objects referencing each
   other, their __del__ methods are called anyway; CPython would instead put
   them into the list garbage of the gc module."
   https://doc.pypy.org/en/latest/cpython_differences.html

From etemcetin.toptani at std.yeditepe.edu.tr  Thu Feb  3 02:26:11 2022
From: etemcetin.toptani at std.yeditepe.edu.tr (=?UTF-8?B?RVRFTSDDh0VUxLBOIFRPUFRBTsSw?=)
Date: Thu, 3 Feb 2022 10:26:11 +0300
Subject: A Short Survey To Understand Practitioner' Perspectives Towards The
 Requirements Engineering
Message-ID: <CAJ48VLhyXkuHOCakX0Ei3SdiRM=+6DqUF6qBOuqhXCryWFt5Vg@mail.gmail.com>

Dear Sir or Madam,


We prepared a short survey to understand practitioners? perspectives
towards the requirements engineering. Our survey basically aims to clarify
on many aspects of the requirements engineering applied in industry, including
(i) requirements gathering and specifications, (ii) requirements
modifications, (iii) requirements analysis, and (iv) requirements
transformation. The survey results will be submitted to a reputable journal
on software engineering.


The survey takes about 2-5 minutes to participate, we would be so grateful
if you could separate your time. Also, please circulate the email to anyone
who may be interested.


The survey link: https://forms.gle/DhLqr15GXVhJhzzy6


All the best,

 Etem ?etin Toptani

-- 
*Bu mesaj? yazd?rmadan ?nce ?evreye verebilece?iniz zararlar? bir kez daha 
d???n?n?z.?*
*Think of the environment once more before printing out this 
message.*

-- 
*Bu mesaj? yazd?rmadan ?nce ?evreye verebilece?iniz zararlar? bir kez daha 
d???n?n?z.?*
*Think of the environment once more before printing out this 
message.*

From sjeik_appie at hotmail.com  Thu Feb  3 10:32:58 2022
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 03 Feb 2022 16:32:58 +0100
Subject: Pypy with Cython
Message-ID: <DB6PR01MB38954E7FA1F2820F99BD451A83289@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   Hi,
   I inherited a fairly large codebase that I need to port to Python 3. Since
   the program was running quite slow I am also running the unittests against
   pypy3.8. It's a long running program that does lots of pairwise
   comparisons of string values in two files. Some parts of the program (e.g
   a modulo 11 digit check) are implemented in Cython. Should I use pure
   Python instead when using Pypy? I compiled the Cython modules for pypy and
   they work, but I'm afraid they might just slow things down.
   Thanks!
   Albert-Jan

From drsalists at gmail.com  Thu Feb  3 11:01:48 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Thu, 3 Feb 2022 08:01:48 -0800
Subject: Pypy with Cython
In-Reply-To: <DB6PR01MB38954E7FA1F2820F99BD451A83289@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <DB6PR01MB38954E7FA1F2820F99BD451A83289@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <CAGGBd_qUSFQb+4yRx_AGb1g7bxdpCrcZ1HMgrLyUMc-mXVW98g@mail.gmail.com>

The best answer to "is this slower on Pypy" is probably to measure.

Sometimes it makes sense to rewrite C extension modules in pure python for
pypy.

On Thu, Feb 3, 2022 at 7:33 AM Albert-Jan Roskam <sjeik_appie at hotmail.com>
wrote:

>    Hi,
>    I inherited a fairly large codebase that I need to port to Python 3.
> Since
>    the program was running quite slow I am also running the unittests
> against
>    pypy3.8. It's a long running program that does lots of pairwise
>    comparisons of string values in two files. Some parts of the program
> (e.g
>    a modulo 11 digit check) are implemented in Cython. Should I use pure
>    Python instead when using Pypy? I compiled the Cython modules for pypy
> and
>    they work, but I'm afraid they might just slow things down.
>    Thanks!
>    Albert-Jan
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From sjeik_appie at hotmail.com  Thu Feb  3 11:07:45 2022
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Thu, 03 Feb 2022 17:07:45 +0100
Subject: Pypy with Cython
In-Reply-To: <CAGGBd_qUSFQb+4yRx_AGb1g7bxdpCrcZ1HMgrLyUMc-mXVW98g@mail.gmail.com>
Message-ID: <DB6PR01MB38959EE997444EC2B3536B4383289@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Feb 3, 2022 17:01, Dan Stromberg <drsalists at gmail.com> wrote:

     > The best answer to "is this slower on

     > Pypy" is probably to measure.
     > Sometimes it makes sense to rewrite C

     > extension modules in pure python for pypy.

   ====
   Hi Dan, thanks. What profiler do you recommend I normally use cProfile,
   but I was thinking about this
   one: https://pyinstrument.readthedocs.io/en/latest/index.html

From grant.b.edwards at gmail.com  Thu Feb  3 13:57:56 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Thu, 03 Feb 2022 10:57:56 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
Message-ID: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>

I've got a small ssl server app. I want to require a certificate from
the client, so I'm using a context with

context.verify_mode = ssl.CERT_REQUIRED

But, I want all certificates accepted. How do I disable client
certificate verification?

--
Grant




From grant.b.edwards at gmail.com  Thu Feb  3 14:17:17 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Thu, 03 Feb 2022 11:17:17 -0800 (PST)
Subject: ssl: why wrap newly accept()ed connections?
Message-ID: <61fc2a3d.1c69fb81.c508c.1f92@mx.google.com>

According to the docs, when you accept() an ssl connection,
you need to wrap the new connection:

 https://docs.python.org/3/library/ssl.html?highlight=ssl#ssl-sockets

   When a client connects, you?ll call accept() on the socket to get
   the new socket from the other end, and use the context?s
   SSLContext.wrap_socket() method to create a server-side SSL socket
   for the connection:

    while True:
        newsocket, fromaddr = bindsocket.accept()
        connstream = context.wrap_socket(newsocket, server_side=True)
        try:
            deal_with_client(connstream)
        finally:
            connstream.shutdown(socket.SHUT_RDWR)
            connstream.close()
           
However, example server code I've found does not wrap the newly
accepted connection. I've checked, and newsocket is already an
<ssl:SSLSocket> object.  The examples I've seen/tried simply call
..recv() and .send() methods of newsocket, and that seems to work fine.

What is the purpose of wrapping newsocket?



From kushal at locationd.net  Thu Feb  3 14:37:35 2022
From: kushal at locationd.net (Kushal Kumaran)
Date: Thu, 03 Feb 2022 11:37:35 -0800
Subject: ssl: why wrap newly accept()ed connections?
In-Reply-To: <61fc2a3d.1c69fb81.c508c.1f92@mx.google.com> (Grant Edwards's
 message of "Thu, 03 Feb 2022 11:17:17 -0800 (PST)")
References: <61fc2a3d.1c69fb81.c508c.1f92@mx.google.com>
Message-ID: <87k0ebkagw.fsf@locationd.net>

On Thu, Feb 03 2022 at 11:17:17 AM, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> According to the docs, when you accept() an ssl connection,
> you need to wrap the new connection:
>
>  https://docs.python.org/3/library/ssl.html?highlight=ssl#ssl-sockets
>
>    When a client connects, you?ll call accept() on the socket to get
>    the new socket from the other end, and use the context?s
>    SSLContext.wrap_socket() method to create a server-side SSL socket
>    for the connection:
>
>     while True:
>         newsocket, fromaddr = bindsocket.accept()
>         connstream = context.wrap_socket(newsocket, server_side=True)
>         try:
>             deal_with_client(connstream)
>         finally:
>             connstream.shutdown(socket.SHUT_RDWR)
>             connstream.close()
>            
> However, example server code I've found does not wrap the newly
> accepted connection. I've checked, and newsocket is already an
> <ssl:SSLSocket> object.  The examples I've seen/tried simply call
> ..recv() and .send() methods of newsocket, and that seems to work fine.
>
> What is the purpose of wrapping newsocket?

That section is talking about using an "ordinary" socket for the server.
bindsocket is a socket.socket.  If bindsocket was already a
ssl.SSLSocket, the wrapping would be already done by accept.

I suppose this kind of functionality is useful for protocols that start
off as cleartext and then switch to TLS (such as the mail-related
protocols that use STARTTLS).

-- 
regards,
kushal

From kushal at locationd.net  Thu Feb  3 14:32:45 2022
From: kushal at locationd.net (Kushal Kumaran)
Date: Thu, 03 Feb 2022 11:32:45 -0800
Subject: ssl server: how to disable client cert verfication?
In-Reply-To: <61fc25b4.1c69fb81.ea933.f956@mx.google.com> (Grant Edwards's
 message of "Thu, 03 Feb 2022 10:57:56 -0800 (PST)")
References: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
Message-ID: <87o83nkaoy.fsf@locationd.net>

On Thu, Feb 03 2022 at 10:57:56 AM, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> I've got a small ssl server app. I want to require a certificate from
> the client, so I'm using a context with
>
> context.verify_mode = ssl.CERT_REQUIRED
>
> But, I want all certificates accepted. How do I disable client
> certificate verification?
>

Perhaps you can explain what your goal is.  Which kinds of client
certificates do you want to permit (to the best of my knowledge, none of
these can be actually allowed):

- expired certificates

- self-signed certificates

- certificates signed by untrusted CA

- completely garbage certificates (bad signature, etc.)

I don't see what benefit you expect from requiring client certificates
if you don't care what the certificate says.  Why not simply set
verify_mode to SSL_NONE and use other authentication mechanisms?

-- 
regards,
kushal

From grant.b.edwards at gmail.com  Thu Feb  3 16:24:11 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Thu, 03 Feb 2022 13:24:11 -0800 (PST)
Subject: ssl: why wrap newly accept()ed connections?
References: <61fc2a3d.1c69fb81.c508c.1f92@mx.google.com>
 <87k0ebkagw.fsf@locationd.net>
Message-ID: <61fc47fb.1c69fb81.f7ff7.0203@mx.google.com>

On 2022-02-03, Kushal Kumaran <kushal at locationd.net> wrote:
>
>> [...]
>> However, example server code I've found does not wrap the newly
>> accepted connection. I've checked, and newsocket is already an
>> <ssl:SSLSocket> object. [...]
>>
>> What is the purpose of wrapping newsocket?
>
> That section is talking about using an "ordinary" socket for the server.

Yep, I missed that.

> bindsocket is a socket.socket.  If bindsocket was already a
> ssl.SSLSocket, the wrapping would be already done by accept.

Thanks!

--
Grant


From grant.b.edwards at gmail.com  Thu Feb  3 16:32:04 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Thu, 03 Feb 2022 13:32:04 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
References: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
 <87o83nkaoy.fsf@locationd.net>
Message-ID: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>

On 2022-02-03, Kushal Kumaran <kushal at locationd.net> wrote:

> On Thu, Feb 03 2022 at 10:57:56 AM, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>> I've got a small ssl server app. I want to require a certificate from
>> the client, so I'm using a context with
>>
>> context.verify_mode = ssl.CERT_REQUIRED
>>
>> But, I want all certificates accepted. How do I disable client
>> certificate verification?
>>
>
> Perhaps you can explain what your goal is.

It's a troubleshooting utility for displaying a client's certificate.

> Which kinds of client certificates do you want to permit

All of them. Anything that's parsable as an X509 certificate no matter
how "invalid" it is.

> (to the best of my knowledge, none of these can be actually allowed):
>
> - expired certificates
> - self-signed certificates
> - certificates signed by untrusted CA
> - completely garbage certificates (bad signature, etc.)
>
> I don't see what benefit you expect from requiring client
> certificates if you don't care what the certificate says.

I do care what it says. The whole point is to find out what it says.

I just don't want it validated by the SSL layer: I want to print it
out. That seems to be trivial to do for server certificates using
"openssl s_client", but I can't find any way to do it for client
certficates.

> Why not simply set verify_mode to SSL_NONE and use other
> authentication mechanisms?

I'm not interested in doing any authentication.

I just want to require that the client provide a certificate and then
print it out using print(connection.getpeercert())

--
Grant



From barry at barrys-emacs.org  Thu Feb  3 16:42:32 2022
From: barry at barrys-emacs.org (Barry)
Date: Thu, 3 Feb 2022 21:42:32 +0000
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <877dack3ee.fsf@munus.decebal.nl>
References: <877dack3ee.fsf@munus.decebal.nl>
Message-ID: <43D1E67E-B1C9-4CC7-AC72-351B7005EC33@barrys-emacs.org>



> On 3 Feb 2022, at 04:45, Cecil Westerhof via Python-list <python-list at python.org> wrote:
> 
> Have to be careful that timing keeps correct when target takes a 'lot'
> of time.
> Something to ponder about, but can wait.

You have noticed that your class does call the function at the repeat interval but
rather at the repeat interval plus processing time.

The way to fix this is to subtract the last processing elapsed time for the next interval.
Sort of a software phase locked loop.

Just before you call the run function record the time.time() as start_time.
Then you can calculate next_interval = max( .001, interval - time.time() - start_time)
I use 1ms as the min interval.

Barry



From barry at barrys-emacs.org  Thu Feb  3 16:52:41 2022
From: barry at barrys-emacs.org (Barry)
Date: Thu, 3 Feb 2022 21:52:41 +0000
Subject: ssl server: how to disable client cert verfication?
In-Reply-To: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
References: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
Message-ID: <15D2E951-9767-4A40-8EAC-DDA63D611ACF@barrys-emacs.org>



> On 3 Feb 2022, at 21:34, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> 
> ?On 2022-02-03, Kushal Kumaran <kushal at locationd.net> wrote:
> 
>>> On Thu, Feb 03 2022 at 10:57:56 AM, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>>> I've got a small ssl server app. I want to require a certificate from
>>> the client, so I'm using a context with
>>> 
>>> context.verify_mode = ssl.CERT_REQUIRED
>>> 
>>> But, I want all certificates accepted. How do I disable client
>>> certificate verification?
>>> 
>> 
>> Perhaps you can explain what your goal is.
> 
> It's a troubleshooting utility for displaying a client's certificate.
> 
>> Which kinds of client certificates do you want to permit
> 
> All of them. Anything that's parsable as an X509 certificate no matter
> how "invalid" it is.
> 
>> (to the best of my knowledge, none of these can be actually allowed):
>> 
>> - expired certificates
>> - self-signed certificates
>> - certificates signed by untrusted CA
>> - completely garbage certificates (bad signature, etc.)
>> 
>> I don't see what benefit you expect from requiring client
>> certificates if you don't care what the certificate says.
> 
> I do care what it says. The whole point is to find out what it says.
> 
> I just don't want it validated by the SSL layer: I want to print it
> out. That seems to be trivial to do for server certificates using
> "openssl s_client", but I can't find any way to do it for client
> certficates.
> 
>> Why not simply set verify_mode to SSL_NONE and use other
>> authentication mechanisms?
> 
> I'm not interested in doing any authentication.
> 
> I just want to require that the client provide a certificate and then
> print it out using print(connection.getpeercert())

I am not near the pc with the code on. But in outline you provide a ssl context that
returns  true for the validation of the cert always. You also get to have x509 cert
in your hands. I use pyopenssl to play with x.509 certs.

Let me know if this is not enough info and I will dig out the code I have that
does this custom cert stuff.

Barry

> 
> --
> Grant
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From grant.b.edwards at gmail.com  Thu Feb  3 17:36:25 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Thu, 03 Feb 2022 14:36:25 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
References: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
 <15D2E951-9767-4A40-8EAC-DDA63D611ACF@barrys-emacs.org>
Message-ID: <61fc58e9.1c69fb81.f1e67.01bd@mx.google.com>

On 2022-02-03, Barry <barry at barrys-emacs.org> wrote:
>
>> [...] I just want to require that the client provide a certificate
>> and then print it out using print(connection.getpeercert())
>
> I am not near the pc with the code on. But in outline you provide a
> ssl context that returns true for the validation of the cert always.

I thought that was what I was asking.

How do you create an ssl context that requests a client certificate
but then treats any received client certificate as valid?

I've looked through the ssl.Context documentation multiple times, and
haven't been able to spot any option or flag that disables client
certificate validation or allows the user to override the actual
client certificate validation process.

> You also get to have x509 cert in your hands. I use pyopenssl to
> play with x.509 certs.

I don't have any problem getting and printing the certificate once the
connection is established. The problem is preventing the handshake
from failing when the client certificate isn't valid and signed by a
CA provided to the context with .load_verify_locations().

> Let me know if this is not enough info and I will dig out the code I
> have that does this custom cert stuff.

--
Grant

From Cecil at decebal.nl  Thu Feb  3 17:41:42 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Thu, 03 Feb 2022 23:41:42 +0100
Subject: Waht do you think about my repeated_timer class
References: <877dack3ee.fsf@munus.decebal.nl>
 <43D1E67E-B1C9-4CC7-AC72-351B7005EC33@barrys-emacs.org>
 <mailman.12.1643924563.27178.python-list@python.org>
Message-ID: <87czk3indl.fsf@munus.decebal.nl>

Barry <barry at barrys-emacs.org> writes:

>> On 3 Feb 2022, at 04:45, Cecil Westerhof via Python-list <python-list at python.org> wrote:
>> 
>> Have to be careful that timing keeps correct when target takes a 'lot'
>> of time.
>> Something to ponder about, but can wait.
>
> You have noticed that your class does call the function at the repeat interval but
> rather at the repeat interval plus processing time.

Nope:
        def _next(self):
            self._timer = Timer(self._interval, self._run)
            self._timer.start()

        def _run(self):
            self._next()
            self._fn()

In _run I first set the new timer and then I execute the function. So
that will go mostly OK.


> The way to fix this is to subtract the last processing elapsed time for the next interval.
> Sort of a software phase locked loop.
>
> Just before you call the run function record the time.time() as start_time.
> Then you can calculate next_interval = max( .001, interval - time.time() - start_time)
> I use 1ms as the min interval.

But I am working on a complete rewrite to create a more efficient
class. (This means I have to change also the code that uses it.) There
I have to do something like you suggest. (I am already working on it.)


Personally I am also of the opinion that the function should finish in
less as 10% from the interval. (That was one of my rewrites.)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From greg.ewing at canterbury.ac.nz  Thu Feb  3 18:04:02 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 4 Feb 2022 12:04:02 +1300
Subject: Pypy with Cython
In-Reply-To: <mailman.5.1643904475.27178.python-list@python.org>
References: <CAGGBd_qUSFQb+4yRx_AGb1g7bxdpCrcZ1HMgrLyUMc-mXVW98g@mail.gmail.com>
 <DB6PR01MB38959EE997444EC2B3536B4383289@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
 <mailman.5.1643904475.27178.python-list@python.org>
Message-ID: <j635b5Fl6vdU1@mid.individual.net>

On 4/02/22 5:07 am, Albert-Jan Roskam wrote:
>     On Feb 3, 2022 17:01, Dan Stromberg <drsalists at gmail.com> wrote:
> 
>     What profiler do you recommend

If it runs for that long, just measuring execution time should
be enough. Python comes with a "timeit" module to help with
that, or you can use whatever your OS provides (e.g. the
"time" shell command in unices).

-- 
Greg

From oscar.j.benjamin at gmail.com  Thu Feb  3 19:35:05 2022
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 4 Feb 2022 00:35:05 +0000
Subject: Pypy with Cython
In-Reply-To: <j635b5Fl6vdU1@mid.individual.net>
References: <CAGGBd_qUSFQb+4yRx_AGb1g7bxdpCrcZ1HMgrLyUMc-mXVW98g@mail.gmail.com>
 <DB6PR01MB38959EE997444EC2B3536B4383289@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
 <mailman.5.1643904475.27178.python-list@python.org>
 <j635b5Fl6vdU1@mid.individual.net>
Message-ID: <CAHVvXxRDAwXDv_G_aRqAuYKmaDi_SQZ2_YWvY2e8BN4iKTfvBw@mail.gmail.com>

On Thu, 3 Feb 2022 at 23:16, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> On 4/02/22 5:07 am, Albert-Jan Roskam wrote:
> >     On Feb 3, 2022 17:01, Dan Stromberg <drsalists at gmail.com> wrote:
> >
> >     What profiler do you recommend
>
> If it runs for that long, just measuring execution time should
> be enough. Python comes with a "timeit" module to help with
> that, or you can use whatever your OS provides (e.g. the
> "time" shell command in unices).

Yes, exactly. It's important to distinguish between timing or
benchmarking as compared to profiling.

When you use a profiler it does not usually give an accurate
representation of the time taken by the same code when the profiler is
not running because of the overhead added by the profiler itself. The
purpose of the profiler is instead to give lots of information that
can help you to *think* about how to make the code faster (e.g. what
proportion of time is spent in different functions or how many times
is a function called etc). This information is useful for considering
at a *high level* what parts of the code could be improved to make it
faster by e.g. calling a particular function less or making that
function faster.

The timeit module can be used to time micro-operations i.e. things
that take nanoseconds or maybe milliseconds. It repeats an operation
in a loop which is often needed to get reliable timings for things
that are otherwise too fast to reliably time from a single run. It can
give information that helps to identify good approaches to try at a
*low level* e.g. when optimising a single line of code.

If you want to *evaluate* whether or not a change actually makes your
*whole* program faster you should just run your code normally but time
how long it takes (which is what the unix "time" command does). You
can also use time.time() from Python for this. Profilers and timeit
help to identify ways to speed up your code but should not be used as
the way to assess the final impact of the changes you make to a long
running program.

--
Oscar

From rosuav at gmail.com  Thu Feb  3 19:55:55 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 4 Feb 2022 11:55:55 +1100
Subject: ssl server: how to disable client cert verfication?
In-Reply-To: <61fc58e9.1c69fb81.f1e67.01bd@mx.google.com>
References: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
 <15D2E951-9767-4A40-8EAC-DDA63D611ACF@barrys-emacs.org>
 <61fc58e9.1c69fb81.f1e67.01bd@mx.google.com>
Message-ID: <CAPTjJmo=3SO9xDC8S_TgPQZpCYQ8-91MuY9xo37evERfrm2u_g@mail.gmail.com>

On Fri, 4 Feb 2022 at 09:37, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> I've looked through the ssl.Context documentation multiple times, and
> haven't been able to spot any option or flag that disables client
> certificate validation or allows the user to override the actual
> client certificate validation process.

What you're doing is a little unusual, so my first thought would be to
subclass Context and override whatever method does the checks.

ChrisA

From kushal at locationd.net  Thu Feb  3 21:25:05 2022
From: kushal at locationd.net (Kushal Kumaran)
Date: Thu, 03 Feb 2022 18:25:05 -0800
Subject: ssl server: how to disable client cert verfication?
In-Reply-To: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com> (Grant Edwards's
 message of "Thu, 03 Feb 2022 13:32:04 -0800 (PST)")
References: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
 <87o83nkaoy.fsf@locationd.net>
 <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
Message-ID: <87bkznqsfy.fsf@locationd.net>

On Thu, Feb 03 2022 at 01:32:04 PM, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> On 2022-02-03, Kushal Kumaran <kushal at locationd.net> wrote:
>
>> On Thu, Feb 03 2022 at 10:57:56 AM, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>>> I've got a small ssl server app. I want to require a certificate from
>>> the client, so I'm using a context with
>>>
>>> context.verify_mode = ssl.CERT_REQUIRED
>>>
>>> But, I want all certificates accepted. How do I disable client
>>> certificate verification?
>>>
>>
>> Perhaps you can explain what your goal is.
>
> It's a troubleshooting utility for displaying a client's certificate.
>
>> Which kinds of client certificates do you want to permit
>
> All of them. Anything that's parsable as an X509 certificate no matter
> how "invalid" it is.
>

Does `openssl x509 -in <filename> -text -noout` do what you want?

>> (to the best of my knowledge, none of these can be actually allowed):
>>
>> - expired certificates
>> - self-signed certificates
>> - certificates signed by untrusted CA
>> - completely garbage certificates (bad signature, etc.)
>>
>> I don't see what benefit you expect from requiring client
>> certificates if you don't care what the certificate says.
>
> I do care what it says. The whole point is to find out what it says.
>
> I just don't want it validated by the SSL layer: I want to print it
> out. That seems to be trivial to do for server certificates using
> "openssl s_client", but I can't find any way to do it for client
> certficates.
>

In your place, I would simply use the openssl x509 command.  If I wanted
more/different info, I would write a script to load the certificate and
printed out the relevant info.  If this functionality must be provided
by a server, I would write it so that a certificate could be POSTed to
the server (without using client certificates), and it would in turn do
the parsing equivalent to what the standalone script would do and
respond with the relevant info.  (But I hear X.509 parsing is an
esoteric mess, and it's unclear to me what demons lie in the area of
parsing untrusted X.509 content).

I don't know how to use the stdlib's ssl module to do this kind of
parsing.  The cryptography package makes this simple though:

https://cryptography.io/en/latest/x509/reference/#loading-certificates

>> Why not simply set verify_mode to SSL_NONE and use other
>> authentication mechanisms?
>
> I'm not interested in doing any authentication.
>
> I just want to require that the client provide a certificate and then
> print it out using print(connection.getpeercert())
>

-- 
regards,
kushal

From Cecil at decebal.nl  Fri Feb  4 09:28:00 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 04 Feb 2022 15:28:00 +0100
Subject: Waht do you think about my repeated_timer class
References: <87zgn9jdff.fsf@munus.decebal.nl>
Message-ID: <875ypuiu4v.fsf@munus.decebal.nl>

Cecil Westerhof <Cecil at decebal.nl> writes:

It was already not a good name, but I am rewriting the class
completely, so now the name is a complete bumper. (No more timer.) I
am thinking about naming the class repeating_thread, but I cannot say
that I find it a very good name. So if someone has a good idea for a
name ?

The class starts a thread where every by the user defined interval a
by the user define function is called.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From ethan at stoneleaf.us  Fri Feb  4 10:28:01 2022
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 4 Feb 2022 07:28:01 -0800
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <875ypuiu4v.fsf@munus.decebal.nl>
References: <87zgn9jdff.fsf@munus.decebal.nl> <875ypuiu4v.fsf@munus.decebal.nl>
Message-ID: <35a16d4b-efdd-9243-4c01-3882042e8dbe@stoneleaf.us>

On 2/4/22 6:28 AM, Cecil Westerhof via Python-list wrote:

 > It was already not a good name, but I am rewriting the class
 > completely, so now the name is a complete bumper. (No more timer.) I
 > am thinking about naming the class repeating_thread, but I cannot say
 > that I find it a very good name. So if someone has a good idea for a
 > name ?
 >
 > The class starts a thread where every by the user defined interval a
 > by the user define function is called.

How about `timed_repeat` or `repeat_timer`?

--
~Ethan~

From sinamobasheri at outlook.com  Fri Feb  4 10:55:50 2022
From: sinamobasheri at outlook.com (Sina Mobasheri)
Date: Fri, 4 Feb 2022 15:55:50 +0000
Subject: PYTHONPATH vs Python Virtual Environment
Message-ID: <AS8PR06MB7592E5A5D8ECE41C599E2963AD299@AS8PR06MB7592.eurprd06.prod.outlook.com>

it's not good title defiantly and I don't mean to compare apples and oranges

when I start using python virtual environment it was because isolation proposes and everyone say about its benefits in isolation and working with different versions of the same package in different projects

but recently I start using pip install --target <package> for zipapp<https://docs.python.org/3.10/library/zipapp.html#creating-standalone-applications-with-zipapp> things, and then I use this pip's option (--target) and add its target folder to PYTHONPATH and target folder's bin directory to PATH, so it's like virtual environment to me

I'm curious what is a problem with this approach (edges), what are other benefits of the virtual environment that this way can't offer? most tutorials talk about isolation and managing of packages versions but I think maybe it's more than that maybe?

From Cecil at decebal.nl  Fri Feb  4 12:17:12 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 04 Feb 2022 18:17:12 +0100
Subject: Waht do you think about my repeated_timer class
References: <87zgn9jdff.fsf@munus.decebal.nl> <875ypuiu4v.fsf@munus.decebal.nl>
 <35a16d4b-efdd-9243-4c01-3882042e8dbe@stoneleaf.us>
 <mailman.3.1643988490.7010.python-list@python.org>
Message-ID: <871r0iimav.fsf@munus.decebal.nl>

Ethan Furman <ethan at stoneleaf.us> writes:

> On 2/4/22 6:28 AM, Cecil Westerhof via Python-list wrote:
>
>> It was already not a good name, but I am rewriting the class
>> completely, so now the name is a complete bumper. (No more timer.) I
>> am thinking about naming the class repeating_thread, but I cannot say
>> that I find it a very good name. So if someone has a good idea for a
>> name ?
>>
>> The class starts a thread where every by the user defined interval a
>> by the user define function is called.
>
> How about `timed_repeat` or `repeat_timer`?

timed_repeat does have something.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From grant.b.edwards at gmail.com  Fri Feb  4 13:15:09 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 04 Feb 2022 10:15:09 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
References: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
 <15D2E951-9767-4A40-8EAC-DDA63D611ACF@barrys-emacs.org>
 <61fc58e9.1c69fb81.f1e67.01bd@mx.google.com>
 <CAPTjJmo=3SO9xDC8S_TgPQZpCYQ8-91MuY9xo37evERfrm2u_g@mail.gmail.com>
Message-ID: <61fd6d2d.1c69fb81.a8d6d.71e9@mx.google.com>

On 2022-02-04, Chris Angelico <rosuav at gmail.com> wrote:
> On Fri, 4 Feb 2022 at 09:37, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>> I've looked through the ssl.Context documentation multiple times, and
>> haven't been able to spot any option or flag that disables client
>> certificate validation or allows the user to override the actual
>> client certificate validation process.
>
> What you're doing is a little unusual, so my first thought would be to
> subclass Context and override whatever method does the checks.

I've done a dir() on the Context object, and I don't see anything that
looks like a method to do the checks. I suspect that the Context
object doesn't actually _do_ anything, it just hold a reference to an
underlying openssl context object and allow to to change its
configuration values.

--
Grant



From grant.b.edwards at gmail.com  Fri Feb  4 13:24:39 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 04 Feb 2022 10:24:39 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
References: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
 <87o83nkaoy.fsf@locationd.net> <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
 <87bkznqsfy.fsf@locationd.net>
Message-ID: <61fd6f67.1c69fb81.5db12.7425@mx.google.com>

On 2022-02-04, Kushal Kumaran <kushal at locationd.net> wrote:

>> It's a troubleshooting utility for displaying a client's certificate.
>>
>>> Which kinds of client certificates do you want to permit
>>
>> All of them. Anything that's parsable as an X509 certificate no matter
>> how "invalid" it is.
>>
>
> Does `openssl x509 -in <filename> -text -noout` do what you want?

Where does <filename> come from?

>> I just don't want it validated by the SSL layer: I want to print it
>> out. That seems to be trivial to do for server certificates using
>> "openssl s_client", but I can't find any way to do it for client
>> certficates.
>
> In your place, I would simply use the openssl x509 command.

How does the x509 command obtain the certificate from the
client/server handshake?

> If I wanted more/different info, I would write a script to load the
> certificate and printed out the relevant info.

How does one "load the certificate" from the client?

> If this functionality must be provided by a server,

> I would write it so that a certificate could be POSTed to
> the server (without using client certificates),

The problem is in getting the certificate is provided by the client
during the handshake with the server. Don't worry about how to
parse/print it -- I can deal with that.

> I don't know how to use the stdlib's ssl module to do this kind of
> parsing.

I'm not asking about parsing x509 certificates. That's not the
problem.

The problem is _getting_ the client certificate that was provided
during the client/server handshake. That's trivial if the handshake
was successful. The problem is obtaining the client certificate when
the handshake fails. I was hoping there was a way to disable client
certificate validation so that the handshake will succeed and then
allow me to get the client certificate from the connection object.

--
Grant




From barry at barrys-emacs.org  Fri Feb  4 13:19:27 2022
From: barry at barrys-emacs.org (Barry)
Date: Fri, 4 Feb 2022 18:19:27 +0000
Subject: ssl server: how to disable client cert verfication?
In-Reply-To: <61fd6d2d.1c69fb81.a8d6d.71e9@mx.google.com>
References: <61fd6d2d.1c69fb81.a8d6d.71e9@mx.google.com>
Message-ID: <C0261B0D-F4EB-4F16-AEFA-0994EA474F51@barrys-emacs.org>



> On 4 Feb 2022, at 18:17, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> 
> ?On 2022-02-04, Chris Angelico <rosuav at gmail.com> wrote:
>>> On Fri, 4 Feb 2022 at 09:37, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>>> I've looked through the ssl.Context documentation multiple times, and
>>> haven't been able to spot any option or flag that disables client
>>> certificate validation or allows the user to override the actual
>>> client certificate validation process.
>> 
>> What you're doing is a little unusual, so my first thought would be to
>> subclass Context and override whatever method does the checks.
> 
> I've done a dir() on the Context object, and I don't see anything that
> looks like a method to do the checks. I suspect that the Context
> object doesn't actually _do_ anything, it just hold a reference to an
> underlying openssl context object and allow to to change its
> configuration values.

We started with the OpenSSL api and looked see what it provided.
Then looked for how to access that from python.

Barry

> --
> Grant
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From christian at python.org  Fri Feb  4 13:59:00 2022
From: christian at python.org (Christian Heimes)
Date: Fri, 4 Feb 2022 19:59:00 +0100
Subject: ssl server: how to disable client cert verfication?
In-Reply-To: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
References: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
Message-ID: <dc15a63b-cd2c-fc97-70e4-c7ca5e65bf80@python.org>

On 03/02/2022 19.57, Grant Edwards wrote:
> I've got a small ssl server app. I want to require a certificate from
> the client, so I'm using a context with
> 
> context.verify_mode = ssl.CERT_REQUIRED
> 
> But, I want all certificates accepted. How do I disable client
> certificate verification?

You can't. Python's ssl module does not expose the necessary feature to 
override the verification callback SSL_CTX_set_verify(). PyOpenSSL lets 
you set a callback and ignore any and all errors.

Christian

From christian at python.org  Fri Feb  4 14:01:53 2022
From: christian at python.org (Christian Heimes)
Date: Fri, 4 Feb 2022 20:01:53 +0100
Subject: ssl server: how to disable client cert verfication?
In-Reply-To: <61fd6f67.1c69fb81.5db12.7425@mx.google.com>
References: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
 <87o83nkaoy.fsf@locationd.net> <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
 <87bkznqsfy.fsf@locationd.net> <61fd6f67.1c69fb81.5db12.7425@mx.google.com>
Message-ID: <7dc6a1d4-7776-247d-355f-5246a555af6d@python.org>

On 04/02/2022 19.24, Grant Edwards wrote:
> The problem is _getting_ the client certificate that was provided
> during the client/server handshake. That's trivial if the handshake
> was successful. The problem is obtaining the client certificate when
> the handshake fails. I was hoping there was a way to disable client
> certificate validation so that the handshake will succeed and then
> allow me to get the client certificate from the connection object.

FYI, it's more complicated in TLS 1.3. Post-handshake authentication 
(PHA) can happen out-of-bounce. Only TLS 1.2 performs client cert auth 
during handshake or renegotiation.

Christian

From codewizard at gmail.com  Fri Feb  4 12:40:51 2022
From: codewizard at gmail.com (Igor Berger)
Date: Fri, 4 Feb 2022 09:40:51 -0800 (PST)
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <871r0iimav.fsf@munus.decebal.nl>
References: <87zgn9jdff.fsf@munus.decebal.nl> <875ypuiu4v.fsf@munus.decebal.nl>
 <35a16d4b-efdd-9243-4c01-3882042e8dbe@stoneleaf.us>
 <mailman.3.1643988490.7010.python-list@python.org>
 <871r0iimav.fsf@munus.decebal.nl>
Message-ID: <0fe85127-cfa8-49b8-910e-4c60fa859f49n@googlegroups.com>

On Friday, February 4, 2022 at 12:28:53 PM UTC-5, Cecil Westerhof wrote:
> Ethan Furman <et... at stoneleaf.us> writes: 
> 
> > On 2/4/22 6:28 AM, Cecil Westerhof via Python-list wrote: 
> > 
> >> It was already not a good name, but I am rewriting the class 
> >> completely, so now the name is a complete bumper. (No more timer.) I 
> >> am thinking about naming the class repeating_thread, but I cannot say 
> >> that I find it a very good name. So if someone has a good idea for a 
> >> name ? 
> >> 
> >> The class starts a thread where every by the user defined interval a 
> >> by the user define function is called. 
> > 
> > How about `timed_repeat` or `repeat_timer`?
> timed_repeat does have something.
> -- 
> Cecil Westerhof 
> Senior Software Engineer 
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof

run_periodically() ?

From Cecil at decebal.nl  Fri Feb  4 13:22:24 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 04 Feb 2022 19:22:24 +0100
Subject: Waht do you think about my repeated_timer class
References: <87zgn9jdff.fsf@munus.decebal.nl> <875ypuiu4v.fsf@munus.decebal.nl>
 <35a16d4b-efdd-9243-4c01-3882042e8dbe@stoneleaf.us>
 <mailman.3.1643988490.7010.python-list@python.org>
 <871r0iimav.fsf@munus.decebal.nl>
 <0fe85127-cfa8-49b8-910e-4c60fa859f49n@googlegroups.com>
Message-ID: <87wniah4pr.fsf@munus.decebal.nl>

Igor Berger <codewizard at gmail.com> writes:

> On Friday, February 4, 2022 at 12:28:53 PM UTC-5, Cecil Westerhof wrote:
>> Ethan Furman <et... at stoneleaf.us> writes: 
>> 
>> > On 2/4/22 6:28 AM, Cecil Westerhof via Python-list wrote: 
>> > 
>> >> It was already not a good name, but I am rewriting the class 
>> >> completely, so now the name is a complete bumper. (No more timer.) I 
>> >> am thinking about naming the class repeating_thread, but I cannot say 
>> >> that I find it a very good name. So if someone has a good idea for a 
>> >> name ? 
>> >> 
>> >> The class starts a thread where every by the user defined interval a 
>> >> by the user define function is called. 
>> > 
>> > How about `timed_repeat` or `repeat_timer`?
>> timed_repeat does have something.
>
> run_periodically() ?

Also better as mine.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From dieter at handshake.de  Fri Feb  4 14:25:33 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Fri, 4 Feb 2022 20:25:33 +0100
Subject: PYTHONPATH vs Python Virtual Environment
In-Reply-To: <AS8PR06MB7592E5A5D8ECE41C599E2963AD299@AS8PR06MB7592.eurprd06.prod.outlook.com>
References: <AS8PR06MB7592E5A5D8ECE41C599E2963AD299@AS8PR06MB7592.eurprd06.prod.outlook.com>
Message-ID: <25085.32173.663142.706239@ixdm.fritz.box>

Sina Mobasheri wrote at 2022-2-4 15:55 +0000:
>it's not good title defiantly and I don't mean to compare apples and oranges
>
>when I start using python virtual environment it was because isolation proposes and everyone say about its benefits in isolation and working with different versions of the same package in different projects
>
>but recently I start using pip install --target <package> for zipapp<https://docs.python.org/3.10/library/zipapp.html#creating-standalone-applications-with-zipapp> things, and then I use this pip's option (--target) and add its target folder to PYTHONPATH and target folder's bin directory to PATH, so it's like virtual environment to me
>
>I'm curious what is a problem with this approach (edges), what are other benefits of the virtual environment that this way can't offer? most tutorials talk about isolation and managing of packages versions but I think maybe it's more than that maybe?

Usually, PYTHONPATH is global.
This can make a problem when you have different applications
which require different package versions.

Of course, you can use a wrapper for your application which locally
sets PYTHONPATH appropriately.

From dieter at handshake.de  Fri Feb  4 14:28:03 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Fri, 4 Feb 2022 20:28:03 +0100
Subject: ssl server: how to disable client cert verfication?
In-Reply-To: <61fc58e9.1c69fb81.f1e67.01bd@mx.google.com>
References: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
 <15D2E951-9767-4A40-8EAC-DDA63D611ACF@barrys-emacs.org>
 <61fc58e9.1c69fb81.f1e67.01bd@mx.google.com>
Message-ID: <25085.32323.499265.960572@ixdm.fritz.box>

Grant Edwards wrote at 2022-2-3 14:36 -0800:
>On 2022-02-03, Barry <barry at barrys-emacs.org> wrote:
> ...
>I've looked through the ssl.Context documentation multiple times, and
>haven't been able to spot any option or flag that disables client
>certificate validation or allows the user to override the actual
>client certificate validation process.

Note that Python does not do the certificate validation itself
but delegates this to the underlying SSL library.
Thus, this library would need to support your use case.
It may not as your scenario is quite special.

From grant.b.edwards at gmail.com  Fri Feb  4 14:40:30 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 04 Feb 2022 11:40:30 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
References: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
 <87o83nkaoy.fsf@locationd.net> <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
 <87bkznqsfy.fsf@locationd.net> <61fd6f67.1c69fb81.5db12.7425@mx.google.com>
 <7dc6a1d4-7776-247d-355f-5246a555af6d@python.org>
Message-ID: <61fd812e.1c69fb81.5988a.7a1b@mx.google.com>

On 2022-02-04, Christian Heimes <christian at python.org> wrote:
> On 04/02/2022 19.24, Grant Edwards wrote:
>> The problem is _getting_ the client certificate that was provided
>> during the client/server handshake. That's trivial if the handshake
>> was successful. The problem is obtaining the client certificate when
>> the handshake fails. I was hoping there was a way to disable client
>> certificate validation so that the handshake will succeed and then
>> allow me to get the client certificate from the connection object.
>
> FYI, it's more complicated in TLS 1.3. Post-handshake authentication 
> (PHA) can happen out-of-bounce. Only TLS 1.2 performs client cert auth 
> during handshake or renegotiation.

That's fine. I can force TLS 1.2 to be used. I don't think there are
going to be situations where the choice of 1.2 vs 1.3 will affect what
certificate is supplied by the client.

The 1.3 PHA would also be OK as long as

 1. I can disable verification of the client certificate that's
    obtained via PHA.

 2. I can obtain the client certificated that was sent during PHA.

What's odd is that it's trivial to do what I want from the client side
using "openssl s_client", but there doesn't seem to be any way to do
the corresponding using "openssl s_server".

I'm beginning to suspect this is a deficiency in the openssl library
itself.

--
Grant



From grant.b.edwards at gmail.com  Fri Feb  4 14:42:16 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 04 Feb 2022 11:42:16 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
References: <61fd6d2d.1c69fb81.a8d6d.71e9@mx.google.com>
 <C0261B0D-F4EB-4F16-AEFA-0994EA474F51@barrys-emacs.org>
Message-ID: <61fd8198.1c69fb81.e7e36.7b58@mx.google.com>

On 2022-02-04, Barry <barry at barrys-emacs.org> wrote:
>>
>>> What you're doing is a little unusual, so my first thought would be to
>>> subclass Context and override whatever method does the checks.
>> 
>> I've done a dir() on the Context object, and I don't see anything that
>> looks like a method to do the checks. I suspect that the Context
>> object doesn't actually _do_ anything, it just hold a reference to an
>> underlying openssl context object and allow to to change its
>> configuration values.
>
> We started with the OpenSSL api and looked see what it provided.
> Then looked for how to access that from python.

Right. I now suspect this is something missing from the oponssl server
side library code. It's trivial to do the same thing from the client
side (ignore the validity of the server certificate).

--
Grant




From grant.b.edwards at gmail.com  Fri Feb  4 14:43:28 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 04 Feb 2022 11:43:28 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
References: <61fc25b4.1c69fb81.ea933.f956@mx.google.com>
 <dc15a63b-cd2c-fc97-70e4-c7ca5e65bf80@python.org>
Message-ID: <61fd81e0.1c69fb81.869b.7952@mx.google.com>

On 2022-02-04, Christian Heimes <christian at python.org> wrote:
> On 03/02/2022 19.57, Grant Edwards wrote:
>> I've got a small ssl server app. I want to require a certificate from
>> the client, so I'm using a context with
>> 
>> context.verify_mode = ssl.CERT_REQUIRED
>> 
>> But, I want all certificates accepted. How do I disable client
>> certificate verification?
>
> You can't. Python's ssl module does not expose the necessary feature to 
> override the verification callback SSL_CTX_set_verify(). PyOpenSSL lets 
> you set a callback and ignore any and all errors.

Thanks! I'll look into that.

Since "openssl s_client" didn't seem to have any option to ignore
client cert validity, I was starting to wonder if ignoring it was
simply impossible with openssl.

--
Grant




From grant.b.edwards at gmail.com  Fri Feb  4 14:46:15 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 04 Feb 2022 11:46:15 -0800 (PST)
Subject: ssl server: how to disable client cert verfication?
References: <61fc49d4.1c69fb81.a405c.5b87@mx.google.com>
 <15D2E951-9767-4A40-8EAC-DDA63D611ACF@barrys-emacs.org>
 <61fc58e9.1c69fb81.f1e67.01bd@mx.google.com>
 <25085.32323.499265.960572@ixdm.fritz.box>
Message-ID: <61fd8287.1c69fb81.df9f1.7c3c@mx.google.com>

On 2022-02-04, Dieter Maurer <dieter at handshake.de> wrote:
> Grant Edwards wrote at 2022-2-3 14:36 -0800:
>>On 2022-02-03, Barry <barry at barrys-emacs.org> wrote:
>> ...
>>I've looked through the ssl.Context documentation multiple times, and
>>haven't been able to spot any option or flag that disables client
>>certificate validation or allows the user to override the actual
>>client certificate validation process.
>
> Note that Python does not do the certificate validation itself
> but delegates this to the underlying SSL library.
> Thus, this library would need to support your use case.
> It may not as your scenario is quite special.

The corresponding scenario is easily supported for the client
side. Even "openssl s_client" offers the option to ignore cert
validation failures and print the cert anyway. It seems odd that
s_server can't do the same.

--
Grant


From rosuav at gmail.com  Fri Feb  4 16:00:17 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 5 Feb 2022 08:00:17 +1100
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <871r0iimav.fsf@munus.decebal.nl>
References: <87zgn9jdff.fsf@munus.decebal.nl> <875ypuiu4v.fsf@munus.decebal.nl>
 <35a16d4b-efdd-9243-4c01-3882042e8dbe@stoneleaf.us>
 <mailman.3.1643988490.7010.python-list@python.org>
 <871r0iimav.fsf@munus.decebal.nl>
Message-ID: <CAPTjJmr=mWXWU5EQeyU5Ca-atmabWGdthxJp2UH2oAmQF+2U-g@mail.gmail.com>

On Sat, 5 Feb 2022 at 04:33, Cecil Westerhof via Python-list
<python-list at python.org> wrote:
>
> Ethan Furman <ethan at stoneleaf.us> writes:
>
> > On 2/4/22 6:28 AM, Cecil Westerhof via Python-list wrote:
> >
> >> It was already not a good name, but I am rewriting the class
> >> completely, so now the name is a complete bumper. (No more timer.) I
> >> am thinking about naming the class repeating_thread, but I cannot say
> >> that I find it a very good name. So if someone has a good idea for a
> >> name ?
> >>
> >> The class starts a thread where every by the user defined interval a
> >> by the user define function is called.
> >
> > How about `timed_repeat` or `repeat_timer`?
>
> timed_repeat does have something.
>

What about "interval timer"?

ChrisA

From martinp.dipaola at gmail.com  Fri Feb  4 18:07:54 2022
From: martinp.dipaola at gmail.com (Martin Di Paola)
Date: Fri, 4 Feb 2022 23:07:54 +0000
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <87czk3indl.fsf@munus.decebal.nl>
References: <877dack3ee.fsf@munus.decebal.nl>
 <43D1E67E-B1C9-4CC7-AC72-351B7005EC33@barrys-emacs.org>
 <mailman.12.1643924563.27178.python-list@python.org>
 <87czk3indl.fsf@munus.decebal.nl>
Message-ID: <20220204230754.dfimkkaackqdzjrq@gmail.com>


>In _run I first set the new timer and then I execute the function. So
>that will go mostly OK.

Yes, that's correct however you are not taking into consideration the 
imprecision of the timers.

Timer will call the next _run() after self._interval *plus* some unknown 
arbitrary time (and extra delay).

Let's assume that when you setup an 1 sec Timer but the Timer calls 
_run() after 1.01 secs due this unknown extra delay.

The first time fn() should be called after 1 sec since the begin but it 
is called after 1.01 secs so the extra delay was of 0.01 sec.

The second time fn() should be called after 2 secs since the begin but 
it is called after 2.02 secs. The second fn() was delayed not by 0.01 
but by 0.02 secs.

The third fn() will be delayed by 0.03 secs and so on.

This arbitrary delay is very small however it will sum up on each 
iteration and depending of your application can be a serious problem.

I wrote a post about this and how to create "constant rate loops" which 
fixes this problem:
https://book-of-gehn.github.io/articles/2019/10/23/Constant-Rate-Loop.html

In the post I also describe two solutions (with their trade-offs) for 
when the target function fn() takes longer than the self._interval time.

See if it helps.

Thanks,
Martin.


On Thu, Feb 03, 2022 at 11:41:42PM +0100, Cecil Westerhof via 
Python-list wrote:
>Barry <barry at barrys-emacs.org> writes:
>
>>> On 3 Feb 2022, at 04:45, Cecil Westerhof via Python-list <python-list at python.org> wrote:
>>>
>>> Have to be careful that timing keeps correct when target takes a 'lot'
>>> of time.
>>> Something to ponder about, but can wait.
>>
>> You have noticed that your class does call the function at the repeat interval but
>> rather at the repeat interval plus processing time.
>
>Nope:
>        def _next(self):
>            self._timer = Timer(self._interval, self._run)
>            self._timer.start()
>
>        def _run(self):
>            self._next()
>            self._fn()
>
>In _run I first set the new timer and then I execute the function. So
>that will go mostly OK.
>
>
>> The way to fix this is to subtract the last processing elapsed time for the next interval.
>> Sort of a software phase locked loop.
>>
>> Just before you call the run function record the time.time() as start_time.
>> Then you can calculate next_interval = max( .001, interval - time.time() - start_time)
>> I use 1ms as the min interval.
>
>But I am working on a complete rewrite to create a more efficient
>class. (This means I have to change also the code that uses it.) There
>I have to do something like you suggest. (I am already working on it.)
>
>
>Personally I am also of the opinion that the function should finish in
>less as 10% from the interval. (That was one of my rewrites.)
>
>-- 
>Cecil Westerhof
>Senior Software Engineer
>LinkedIn: http://www.linkedin.com/in/cecilwesterhof
>-- 
>https://mail.python.org/mailman/listinfo/python-list

From Cecil at decebal.nl  Fri Feb  4 15:39:33 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 04 Feb 2022 21:39:33 +0100
Subject: RuntimeError, or user defined exception
Message-ID: <87sfsygyd6.fsf@munus.decebal.nl>

I am creating a class that will call a user defined function on user
defined intervals. In my opinion it is an error when the function
takes more as 10% of interval, or more as half a second. What is
better: just using RuntimeError, or creating two exception classes for
this?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From Cecil at decebal.nl  Fri Feb  4 17:08:59 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Fri, 04 Feb 2022 23:08:59 +0100
Subject: Waht do you think about my repeated_timer class
References: <87zgn9jdff.fsf@munus.decebal.nl> <875ypuiu4v.fsf@munus.decebal.nl>
 <35a16d4b-efdd-9243-4c01-3882042e8dbe@stoneleaf.us>
 <mailman.3.1643988490.7010.python-list@python.org>
 <871r0iimav.fsf@munus.decebal.nl>
 <CAPTjJmr=mWXWU5EQeyU5Ca-atmabWGdthxJp2UH2oAmQF+2U-g@mail.gmail.com>
 <mailman.16.1644008429.7010.python-list@python.org>
Message-ID: <87o83mgu84.fsf@munus.decebal.nl>

Chris Angelico <rosuav at gmail.com> writes:

> On Sat, 5 Feb 2022 at 04:33, Cecil Westerhof via Python-list
> <python-list at python.org> wrote:
>>
>> Ethan Furman <ethan at stoneleaf.us> writes:
>>
>> > On 2/4/22 6:28 AM, Cecil Westerhof via Python-list wrote:
>> >
>> >> It was already not a good name, but I am rewriting the class
>> >> completely, so now the name is a complete bumper. (No more timer.) I
>> >> am thinking about naming the class repeating_thread, but I cannot say
>> >> that I find it a very good name. So if someone has a good idea for a
>> >> name ?
>> >>
>> >> The class starts a thread where every by the user defined interval a
>> >> by the user define function is called.
>> >
>> > How about `timed_repeat` or `repeat_timer`?
>>
>> timed_repeat does have something.
>>
>
> What about "interval timer"?

It is not really a timer, but on the other hand it seems to be a
timer. (White box, black box.)

I will add it. I do not have to decide before I publish the class, so
I can let it simmer for a while. ;-)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt  Fri Feb  4 19:37:24 2022
From: p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt (Paulo da Silva)
Date: Sat, 5 Feb 2022 00:37:24 +0000
Subject: Writing a package
Message-ID: <stkgs4$ki2$1@gioia.aioe.org>

Hello!

Let's say I have a dir src containing another dir named foo and a script 
test.py.

So, I have
src/foo (dir)
src/test.py (script)

test.py has the folloing code:

import foo as f
c=f.C()

I am inside src and want to run python test.py.

How can I create the class C inside src/foo dir if it is possible at all?

Thanks.

From cs at cskk.id.au  Fri Feb  4 21:01:45 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 5 Feb 2022 13:01:45 +1100
Subject: Writing a package
In-Reply-To: <stkgs4$ki2$1@gioia.aioe.org>
References: <stkgs4$ki2$1@gioia.aioe.org>
Message-ID: <Yf3aiXfssha7ctcA@cskk.homeip.net>

On 05Feb2022 00:37, Paulo da Silva <p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt> wrote:
>Let's say I have a dir src containing another dir named foo and a 
>script test.py.
>
>So, I have
>src/foo (dir)
>src/test.py (script)
>
>test.py has the folloing code:
>
>import foo as f
>c=f.C()
>
>I am inside src and want to run python test.py.
>
>How can I create the class C inside src/foo dir if it is possible at 
>all?

Define it in the file "src/foo/__init__.py".

When you go:

    import blah

Python reaches for the file "blah.py" or "blah/__init__.py" (this second 
path is for "packages").

Cheers,
Cameron Simpson <cs at cskk.id.au>

From avigross at verizon.net  Fri Feb  4 23:18:39 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 5 Feb 2022 04:18:39 +0000 (UTC)
Subject: Waht do you think about my repeated_timer class
In-Reply-To: <0fe85127-cfa8-49b8-910e-4c60fa859f49n@googlegroups.com>
References: <87zgn9jdff.fsf@munus.decebal.nl> <875ypuiu4v.fsf@munus.decebal.nl>
 <35a16d4b-efdd-9243-4c01-3882042e8dbe@stoneleaf.us>
 <mailman.3.1643988490.7010.python-list@python.org>
 <871r0iimav.fsf@munus.decebal.nl>
 <0fe85127-cfa8-49b8-910e-4c60fa859f49n@googlegroups.com>
Message-ID: <344758048.67586.1644034719302@mail.yahoo.com>

time_after_time
But to be more pythonic, throw some double underscores before and after.

-----Original Message-----
From: Igor Berger <codewizard at gmail.com>
To: python-list at python.org
Sent: Fri, Feb 4, 2022 12:40 pm
Subject: Re: Waht do you think about my repeated_timer class

On Friday, February 4, 2022 at 12:28:53 PM UTC-5, Cecil Westerhof wrote:
> Ethan Furman <et... at stoneleaf.us> writes: 
> 
> > On 2/4/22 6:28 AM, Cecil Westerhof via Python-list wrote: 
> > 
> >> It was already not a good name, but I am rewriting the class 
> >> completely, so now the name is a complete bumper. (No more timer.) I 
> >> am thinking about naming the class repeating_thread, but I cannot say 
> >> that I find it a very good name. So if someone has a good idea for a 
> >> name ? 
> >> 
> >> The class starts a thread where every by the user defined interval a 
> >> by the user define function is called. 
> > 
> > How about `timed_repeat` or `repeat_timer`?
> timed_repeat does have something.
> -- 
> Cecil Westerhof 
> Senior Software Engineer 
> LinkedIn: http://www.linkedin.com/in/cecilwesterhof

run_periodically() ?
-- 
https://mail.python.org/mailman/listinfo/python-list

From p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt  Fri Feb  4 23:20:41 2022
From: p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt (Paulo da Silva)
Date: Sat, 5 Feb 2022 04:20:41 +0000
Subject: Writing a package
References: <stkgs4$ki2$1@gioia.aioe.org> <Yf3aiXfssha7ctcA@cskk.homeip.net>
 <mailman.18.1644026866.7010.python-list@python.org>
Message-ID: <stktup$ei1$3@gioia.aioe.org>

?s 02:01 de 05/02/22, Cameron Simpson escreveu:
> On 05Feb2022 00:37, Paulo da Silva <p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt> wrote:
>> Let's say I have a dir src containing another dir named foo and a
>> script test.py.
>>
>> So, I have
>> src/foo (dir)
>> src/test.py (script)
>>
>> test.py has the folloing code:
>>
>> import foo as f
>> c=f.C()
>>
>> I am inside src and want to run python test.py.
>>
>> How can I create the class C inside src/foo dir if it is possible at
>> all?
> 
> Define it in the file "src/foo/__init__.py".
> 
> When you go:
> 
>      import blah
> 
> Python reaches for the file "blah.py" or "blah/__init__.py" (this second
> path is for "packages").
> 

Yes, thank you.

From cs at cskk.id.au  Sat Feb  5 00:04:24 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Sat, 5 Feb 2022 16:04:24 +1100
Subject: RuntimeError, or user defined exception
In-Reply-To: <87sfsygyd6.fsf@munus.decebal.nl>
References: <87sfsygyd6.fsf@munus.decebal.nl>
Message-ID: <Yf4FWD0bFGvhNm50@cskk.homeip.net>

On 04Feb2022 21:39, Cecil Westerhof <Cecil at decebal.nl> wrote:
>I am creating a class that will call a user defined function on user
>defined intervals. In my opinion it is an error when the function
>takes more as 10% of interval, or more as half a second. What is
>better: just using RuntimeError, or creating two exception classes for
>this?

You could reuse TimeoutError, a builtin exception raised by OS calls 
which time out: https://docs.python.org/3/library/exceptions.html#TimeoutError

I wouldn't use RuntimeError. Its core purpose is interpreter failures 
IIRC, and I use it myself for situations which should never occur - 
things indicating an actual bug in my code.

Specificly, I do not expect to try to catch RuntimeError, and broadly 
don't expect anything else too either (with some narrow exceptions).  
Whereas you would very reasonably catch TimeoutError as a situation 
which can readily occur. _Particularly_ with a tool which runs code 
supplied from outside - it _might_ do anything.

IMO your situation is a misbehaviour, not an internal bug. I'd reach for 
TimeoutError.

An example from my own code where I use a RuntimeError:

    class FSTagsCommand(BaseCommand, TagsCommandMixin):
      ''' `fstags` main command line utility.
      '''
      GETOPT_SPEC = 'o:'
      [.......]
      def apply_opt(self, opt, val):
        ''' Apply command line option.
        '''
        options = self.options
        if opt == '-o':
          options.ontology_path = val
        else:
          raise RuntimeError("unhandled option")

Here, the BaseCommand superclass parses command line options according 
to GETOPT_SPEC. The apply_opt() method gets called with any valid 
options. GETOPT_SPEC='o:', so seeing anything except '-o' in apply_opt() 
indicates a mismatch between GETOPT_SPEC and my apply_opt implementation 
(other options get rejected by BaseCommand). So _if_ my code gets there, 
that is an internal bug of my own.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From createkmontalbion at gmail.com  Sun Feb  6 03:14:07 2022
From: createkmontalbion at gmail.com (createkmontalbion at gmail.com)
Date: Sun, 6 Feb 2022 13:44:07 +0530
Subject: Openning Python program
Message-ID: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>

   I cannot open python after downloading it keeps going to modify/uninstall
   ?? please help

   ?

   Sent from [1]Mail for Windows

   ?

   --------------------------------------------------------------------------

                 This email has been checked for viruses by Avast antivirus
   [2]Avast logo software.
                 [3]www.avast.com

References

   Visible links
   1. https://go.microsoft.com/fwlink/?LinkId=550986
   2. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient
   3. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient

From python at mrabarnett.plus.com  Sun Feb  6 10:41:18 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 6 Feb 2022 15:41:18 +0000
Subject: Openning Python program
In-Reply-To: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
Message-ID: <3e9e5488-3271-9937-74eb-352e02abd3f3@mrabarnett.plus.com>

On 2022-02-06 08:14, createkmontalbion at gmail.com wrote:
>     I cannot open python after downloading it keeps going to modify/uninstall
>     ?? please help
> 
It sounds like you're just re-running the installer.

The installer should've installed IDLE. Try that for editing programs.

From wlfraed at ix.netcom.com  Sun Feb  6 10:48:42 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sun, 06 Feb 2022 10:48:42 -0500
Subject: Openning Python program
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
Message-ID: <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>

On Sun, 6 Feb 2022 13:44:07 +0530, "createkmontalbion at gmail.com"
<createkmontalbion at gmail.com> declaimed the following:

>   I cannot open python after downloading it keeps going to modify/uninstall
>   ?? please help
>

	Stop clicking on the INSTALLER. What you downloaded is just the program
that installs and configures Python on your system. Stuff it away someplace
safe should you need to modify the current installation, but otherwise just
forget that it exists.

	Python is a command line interpreter -- not an IDE. Open a command
shell and see if it finds Python:

Microsoft Windows [Version 10.0.19041.1415]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Wulfraed>python
Python ActivePython 3.8.2 (ActiveState Software Inc.) based on
 on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

NOTE: there may be a "launcher" installed that is supposed to find Python
without requiring one to edit the system PATH environment variable -- but I
tend to avoid it: M$ and some other applications seem to keep hijacking
which Python gets priority, and it often invokes anything but the version I
want. [If on installs Visual Studio and doesn't read carefully, it will
install its version of Python and the R statistics system] (Though it
appears that today is a good day...)

C:\Users\Wulfraed>py
Python ActivePython 3.8.2 (ActiveState Software Inc.) based on
 on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

	Pick your preferred programming text editor, write some script, and run
it from the command line

C:\Users\Wulfraed>type junk.py

print("Let me out of here!")

C:\Users\Wulfraed>python junk.py
Let me out of here!

C:\Users\Wulfraed>junk.py
Let me out of here!

C:\Users\Wulfraed>junk
Let me out of here!

C:\Users\Wulfraed>python junk
python: can't open file 'junk': [Errno 2] No such file or directory

C:\Users\Wulfraed>

NOTE: I have Windows configured to accept .py as an executable file, with
the Python interpreter as the "runner", which is how the lines without
"python" function -- but if you explicitly invoke python with a file name
you must provide the full name.

C:\Users\Wulfraed>assoc .py
.py=Python.File

C:\Users\Wulfraed>ftype python.file
python.file="C:\Python38\python.exe" "%1" %*


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From eryksun at gmail.com  Sun Feb  6 20:29:59 2022
From: eryksun at gmail.com (Eryk Sun)
Date: Sun, 6 Feb 2022 19:29:59 -0600
Subject: Openning Python program
In-Reply-To: <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
Message-ID: <CACL+1as+hisHTST_99Vmg_DVaVMh7UjEs4Btr8_v-qhogr-Zxg@mail.gmail.com>

On 2/6/22, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>
> NOTE: there may be a "launcher" installed that is supposed to find Python
> without requiring one to edit the system PATH environment variable -- but I
> tend to avoid it: M$ and some other applications seem to keep hijacking
> which Python gets priority, and it often invokes anything but the version I
> want. [If on installs Visual Studio and doesn't read carefully, it will
> install its version of Python and the R statistics system] (Though it
> appears that today is a good day...)

No other applications have anything to do with how the py launcher
works. It depends on the Python installations that are registered in
the "Software\Python\PythonCore" registry key in HKCU and HKLM. If the
launcher is installed, the command template of the "Python.File"
ProgID is configured to use the launcher, e.g. `C:\Windows\py.exe "%L"
%*`. Whether or not other applications take control of the ".py" file
association, and whether the user allows this to happen, is a separate
issue.

Unless the launcher is configured otherwise, or the version to use is
set by a shebang line in the script, the launcher defaults to running
the highest version number of Python that's installed. If there are
multiple installations for the same version, it prefers the system's
native architecture (e.g. x64 in 64-bit Windows). You can list the
available versions and paths via `py -0p`.

The launcher allows setting the default version to use, and also
separate defaults for 2.x and 3.x (i.e. for the `-2` and `-3`
command-line options). This is simplest to set via the PY_PYTHON,
PY_PYTHON2, and PY_PYTHON3 environment variables, but it can also be
configured in "%LocalAppData%\py.ini".

 > NOTE: I have Windows configured to accept .py as an executable file, with
> the Python interpreter as the "runner", which is how the lines without
> "python" function -- but if you explicitly invoke python with a file name
> you must provide the full name.
>
> C:\Users\Wulfraed>assoc .py
> .py=Python.File
>
> C:\Users\Wulfraed>ftype python.file
> python.file="C:\Python38\python.exe" "%1" %*

The CMD shell's ASSOC and FTYPE commands are out of date and should
not be relied on. The returned or set values from those commands are
often irrelevant to what the shell API actually does.

ASSOC and FTYPE operate on only the "HKLM\Software\Classes" setting
(i.e. file associations set for the machine), which was sufficient
when these commands were added in Windows NT 4.0 (1996). But since
Windows 2000, HKCR is a merged view that prefers values set in
"HKCU\Software\Classes" (i.e. file associations set for the current
user; e.g. Python is installed for the current user).

Also, Explorer caches file associations per the user's last "open
with" choice in
"HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts".
When the shell API computes the file association for a file, this user
choice overrides the HKCR settings. Moreover, the user choice for each
association in the shell's "FileExts" key can be locked down in its
"UserChoice" subkey (i.e. the user opted to always open with the app).

From arj.python at gmail.com  Mon Feb  7 04:17:27 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Mon, 7 Feb 2022 13:17:27 +0400
Subject: Best way to check if there is internet?
Message-ID: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>

Greetings,

Using the standard library or 3rd party libraries, what's the
best way to check if there is internet? Checking if google.com
is reachable is good but I wonder if there is a more native,
protocol-oriented
way of knowing?

Even this URL recommends checking if a domain is up as a way to check for
internet connectivity:
https://www.ibm.com/support/pages/checking-network-connectivity-when-using-python-and-ibm-resilient-circuits

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From rosuav at gmail.com  Mon Feb  7 04:27:51 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 7 Feb 2022 20:27:51 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
Message-ID: <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>

On Mon, 7 Feb 2022 at 20:18, Abdur-Rahmaan Janhangeer
<arj.python at gmail.com> wrote:
>
> Greetings,
>
> Using the standard library or 3rd party libraries, what's the
> best way to check if there is internet? Checking if google.com
> is reachable is good but I wonder if there is a more native,
> protocol-oriented
> way of knowing?
>

What do you mean by "if there is internet"? How low a level of
connection do you want to test? You could ping an IP address that you
know and can guarantee will respond. You could attempt a DNS lookup.
You could try an HTTP request. Each one can fail in different ways,
for different reasons. It's best to test what you actually care about.

> Even this URL recommends checking if a domain is up as a way to check for
> internet connectivity:
> https://www.ibm.com/support/pages/checking-network-connectivity-when-using-python-and-ibm-resilient-circuits

That talks about a misconfigured proxy as being the most likely cause.
Is that something you're trying to test for?

There is no single concept of "there is internet". (Other than, in a
trivial sense, that the internet does exist.) What you need to know is
"can I use the internet?", and ultimately, that depends on what you
need it to do.

ChrisA

From arj.python at gmail.com  Mon Feb  7 04:33:54 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Mon, 7 Feb 2022 13:33:54 +0400
Subject: Best way to check if there is internet?
In-Reply-To: <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
Message-ID: <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>

Popular browsers tell: No internet connection detected. A function that
goes in the same sense. Unless they too are pinging Google.com to check ...

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius


On Mon, Feb 7, 2022 at 1:28 PM Chris Angelico <rosuav at gmail.com> wrote:

> On Mon, 7 Feb 2022 at 20:18, Abdur-Rahmaan Janhangeer
> <arj.python at gmail.com> wrote:
> >
> > Greetings,
> >
> > Using the standard library or 3rd party libraries, what's the
> > best way to check if there is internet? Checking if google.com
> > is reachable is good but I wonder if there is a more native,
> > protocol-oriented
> > way of knowing?
> >
>
> What do you mean by "if there is internet"? How low a level of
> connection do you want to test? You could ping an IP address that you
> know and can guarantee will respond. You could attempt a DNS lookup.
> You could try an HTTP request. Each one can fail in different ways,
> for different reasons. It's best to test what you actually care about.
>
> > Even this URL recommends checking if a domain is up as a way to check for
> > internet connectivity:
> >
> https://www.ibm.com/support/pages/checking-network-connectivity-when-using-python-and-ibm-resilient-circuits
>
> That talks about a misconfigured proxy as being the most likely cause.
> Is that something you're trying to test for?
>
> There is no single concept of "there is internet". (Other than, in a
> trivial sense, that the internet does exist.) What you need to know is
> "can I use the internet?", and ultimately, that depends on what you
> need it to do.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From liedtke at punkt.de  Mon Feb  7 07:47:14 2022
From: liedtke at punkt.de (Lars Liedtke)
Date: Mon, 7 Feb 2022 13:47:14 +0100
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
Message-ID: <4b995eff-0945-b434-5ea9-bf0304965ada@punkt.de>

Each Browser is doing it differently and even Windows or 
Linux-Desktopmanagers (NetworkManager).

I have had cases, where one tool told me I had Internet and another one 
I hadn't.

So What Chris Angelico wrote is propably the best way, ping e.g. Google, 
do a DNS lookup and try http for Status 200. Each with its own 
Errorhandling and if you have got all three, then it is propable that 
you have "Internet".

Cheers

Lars

Am 07.02.22 um 10:33 schrieb Abdur-Rahmaan Janhangeer:
> Popular browsers tell: No internet connection detected. A function that
> goes in the same sense. Unless they too are pinging Google.com to check ...
>
> Kind Regards,
>
> Abdur-Rahmaan Janhangeer
> about <https://compileralchemy.github.io/> | blog
> <https://www.pythonkitchen.com>
> github <https://github.com/Abdur-RahmaanJ>
> Mauritius
>
>
> On Mon, Feb 7, 2022 at 1:28 PM Chris Angelico <rosuav at gmail.com> wrote:
>
>> On Mon, 7 Feb 2022 at 20:18, Abdur-Rahmaan Janhangeer
>> <arj.python at gmail.com> wrote:
>>> Greetings,
>>>
>>> Using the standard library or 3rd party libraries, what's the
>>> best way to check if there is internet? Checking if google.com
>>> is reachable is good but I wonder if there is a more native,
>>> protocol-oriented
>>> way of knowing?
>>>
>> What do you mean by "if there is internet"? How low a level of
>> connection do you want to test? You could ping an IP address that you
>> know and can guarantee will respond. You could attempt a DNS lookup.
>> You could try an HTTP request. Each one can fail in different ways,
>> for different reasons. It's best to test what you actually care about.
>>
>>> Even this URL recommends checking if a domain is up as a way to check for
>>> internet connectivity:
>>>
>> https://www.ibm.com/support/pages/checking-network-connectivity-when-using-python-and-ibm-resilient-circuits
>>
>> That talks about a misconfigured proxy as being the most likely cause.
>> Is that something you're trying to test for?
>>
>> There is no single concept of "there is internet". (Other than, in a
>> trivial sense, that the internet does exist.) What you need to know is
>> "can I use the internet?", and ultimately, that depends on what you
>> need it to do.
>>
>> ChrisA
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
-- 
punkt.de GmbH
Lars Liedtke
.infrastructure

Kaiserallee 13a	
76133 Karlsruhe

Tel. +49 721 9109 500
https://infrastructure.punkt.de
info at punkt.de

AG Mannheim 108285
Gesch?ftsf?hrer: J?rgen Egeling, Daniel Lienert, Fabian Stein


From liedtke at punkt.de  Mon Feb  7 07:54:37 2022
From: liedtke at punkt.de (Lars Liedtke)
Date: Mon, 7 Feb 2022 13:54:37 +0100
Subject: Best way to check if there is internet?
In-Reply-To: <4b995eff-0945-b434-5ea9-bf0304965ada@punkt.de>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <4b995eff-0945-b434-5ea9-bf0304965ada@punkt.de>
Message-ID: <5a1bde4f-3ad8-dfc1-0eaf-ff866921248d@punkt.de>

P.S.: for ping a startingpoint can be icmplib, for dns dnspython and for 
http requests. But consider as well if you want to check for IPv4 and/or 
IPv6 Connectivity.

Am 07.02.22 um 13:47 schrieb Lars Liedtke:
> Each Browser is doing it differently and even Windows or 
> Linux-Desktopmanagers (NetworkManager).
>
> I have had cases, where one tool told me I had Internet and another 
> one I hadn't.
>
> So What Chris Angelico wrote is propably the best way, ping e.g. 
> Google, do a DNS lookup and try http for Status 200. Each with its own 
> Errorhandling and if you have got all three, then it is propable that 
> you have "Internet".
>
> Cheers
>
> Lars
>
> Am 07.02.22 um 10:33 schrieb Abdur-Rahmaan Janhangeer:
>> Popular browsers tell: No internet connection detected. A function that
>> goes in the same sense. Unless they too are pinging Google.com to 
>> check ...
>>
>> Kind Regards,
>>
>> Abdur-Rahmaan Janhangeer
>> about <https://compileralchemy.github.io/> | blog
>> <https://www.pythonkitchen.com>
>> github <https://github.com/Abdur-RahmaanJ>
>> Mauritius
>>
>>
>> On Mon, Feb 7, 2022 at 1:28 PM Chris Angelico <rosuav at gmail.com> wrote:
>>
>>> On Mon, 7 Feb 2022 at 20:18, Abdur-Rahmaan Janhangeer
>>> <arj.python at gmail.com> wrote:
>>>> Greetings,
>>>>
>>>> Using the standard library or 3rd party libraries, what's the
>>>> best way to check if there is internet? Checking if google.com
>>>> is reachable is good but I wonder if there is a more native,
>>>> protocol-oriented
>>>> way of knowing?
>>>>
>>> What do you mean by "if there is internet"? How low a level of
>>> connection do you want to test? You could ping an IP address that you
>>> know and can guarantee will respond. You could attempt a DNS lookup.
>>> You could try an HTTP request. Each one can fail in different ways,
>>> for different reasons. It's best to test what you actually care about.
>>>
>>>> Even this URL recommends checking if a domain is up as a way to 
>>>> check for
>>>> internet connectivity:
>>>>
>>> https://www.ibm.com/support/pages/checking-network-connectivity-when-using-python-and-ibm-resilient-circuits 
>>>
>>>
>>> That talks about a misconfigured proxy as being the most likely cause.
>>> Is that something you're trying to test for?
>>>
>>> There is no single concept of "there is internet". (Other than, in a
>>> trivial sense, that the internet does exist.) What you need to know is
>>> "can I use the internet?", and ultimately, that depends on what you
>>> need it to do.
>>>
>>> ChrisA
>>> -- 
>>> https://mail.python.org/mailman/listinfo/python-list
>>>
-- 
punkt.de GmbH
Lars Liedtke
.infrastructure

Kaiserallee 13a	
76133 Karlsruhe

Tel. +49 721 9109 500
https://infrastructure.punkt.de
info at punkt.de

AG Mannheim 108285
Gesch?ftsf?hrer: J?rgen Egeling, Daniel Lienert, Fabian Stein


From rosuav at gmail.com  Mon Feb  7 07:59:04 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 7 Feb 2022 23:59:04 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <4b995eff-0945-b434-5ea9-bf0304965ada@punkt.de>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <4b995eff-0945-b434-5ea9-bf0304965ada@punkt.de>
Message-ID: <CAPTjJmrHywvB1-Fah5HdrP-yHRL3KpUSrKWQ6d4zrAvaeNWK2g@mail.gmail.com>

On Mon, 7 Feb 2022 at 23:48, Lars Liedtke <liedtke at punkt.de> wrote:
>
> Each Browser is doing it differently and even Windows or
> Linux-Desktopmanagers (NetworkManager).
>
> I have had cases, where one tool told me I had Internet and another one
> I hadn't.

Not at all surprised :) Though that situation would probably be best
described as "degraded service". (Unless one tool was straight-up
misconfigured, of course.)

> So What Chris Angelico wrote is propably the best way, ping e.g. Google,
> do a DNS lookup and try http for Status 200. Each with its own
> Errorhandling and if you have got all three, then it is propable that
> you have "Internet".
>

More specifically, what I'm saying is that the best way to determine
whether you "have internet" is to do the exact thing that you care
about. What do you need the internet for? Downloading things from web
sites? Then attempt an HTTP request. Do you need UDP transmission?
Then send a UDP packet somewhere and wait for a response. Etc.

ChrisA

From grant.b.edwards at gmail.com  Mon Feb  7 10:52:19 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 07 Feb 2022 07:52:19 -0800 (PST)
Subject: Openning Python program
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
Message-ID: <62014033.1c69fb81.1c113.fab9@mx.google.com>

On 2022-02-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> On Sun, 6 Feb 2022 13:44:07 +0530, "createkmontalbion at gmail.com"
><createkmontalbion at gmail.com> declaimed the following:
>
>>   I cannot open python after downloading it keeps going to modify/uninstall
>>   ?? please help
>
> Stop clicking on the INSTALLER. What you downloaded is just the program
> that installs and configures Python on your system. Stuff it away someplace
> safe should you need to modify the current installation, but otherwise just
> forget that it exists.

This is _still_ a problem after all these years and countless
identical complaints?

How difficult would it be to add a few lines of text to the installer
welcome screen explaining that you've just started the Python
INSTALLER, and if you've already done the installation and want to
"run Python" try <whatever>?

--
Grant



From rosuav at gmail.com  Mon Feb  7 10:56:18 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Feb 2022 02:56:18 +1100
Subject: Openning Python program
In-Reply-To: <62014033.1c69fb81.1c113.fab9@mx.google.com>
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
 <62014033.1c69fb81.1c113.fab9@mx.google.com>
Message-ID: <CAPTjJmrmdvDWxSOXsD47Bk8745miHwsDF8dYw103NReYmY73jw@mail.gmail.com>

On Tue, 8 Feb 2022 at 02:53, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>
> On 2022-02-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> > On Sun, 6 Feb 2022 13:44:07 +0530, "createkmontalbion at gmail.com"
> ><createkmontalbion at gmail.com> declaimed the following:
> >
> >>   I cannot open python after downloading it keeps going to modify/uninstall
> >>   ?? please help
> >
> > Stop clicking on the INSTALLER. What you downloaded is just the program
> > that installs and configures Python on your system. Stuff it away someplace
> > safe should you need to modify the current installation, but otherwise just
> > forget that it exists.
>
> This is _still_ a problem after all these years and countless
> identical complaints?
>
> How difficult would it be to add a few lines of text to the installer
> welcome screen explaining that you've just started the Python
> INSTALLER, and if you've already done the installation and want to
> "run Python" try <whatever>?
>

How difficult would it be to get people to read those lines, though?

ChrisA

From barry at barrys-emacs.org  Mon Feb  7 11:30:33 2022
From: barry at barrys-emacs.org (Barry)
Date: Mon, 7 Feb 2022 16:30:33 +0000
Subject: Openning Python program
In-Reply-To: <62014033.1c69fb81.1c113.fab9@mx.google.com>
References: <62014033.1c69fb81.1c113.fab9@mx.google.com>
Message-ID: <C6DA9D39-3881-4011-AA71-B40BB52AA9BF@barrys-emacs.org>



> On 7 Feb 2022, at 15:55, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> 
> ?On 2022-02-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>> On Sun, 6 Feb 2022 13:44:07 +0530, "createkmontalbion at gmail.com"
>> <createkmontalbion at gmail.com> declaimed the following:
>> 
>>>  I cannot open python after downloading it keeps going to modify/uninstall
>>>  ?? please help
>> 
>> Stop clicking on the INSTALLER. What you downloaded is just the program
>> that installs and configures Python on your system. Stuff it away someplace
>> safe should you need to modify the current installation, but otherwise just
>> forget that it exists.
> 
> This is _still_ a problem after all these years and countless
> identical complaints?
> 
> How difficult would it be to add a few lines of text to the installer
> welcome screen explaining that you've just started the Python
> INSTALLER, and if you've already done the installation and want to
> "run Python" try <whatever>?

Better yet include the word setup in the installer .exe name.
Just like almost every other installer does.

Barry

> 
> --
> Grant
> 
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From grant.b.edwards at gmail.com  Mon Feb  7 11:45:27 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 07 Feb 2022 08:45:27 -0800 (PST)
Subject: Openning Python program
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
 <62014033.1c69fb81.1c113.fab9@mx.google.com>
 <CAPTjJmrmdvDWxSOXsD47Bk8745miHwsDF8dYw103NReYmY73jw@mail.gmail.com>
Message-ID: <62014ca7.1c69fb81.9fa7a.aa4e@mx.google.com>

On 2022-02-07, Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, 8 Feb 2022 at 02:53, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>>
>>On 2022-02-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>>> On Sun, 6 Feb 2022 13:44:07 +0530, "createkmontalbion at gmail.com"
>>><createkmontalbion at gmail.com> declaimed the following:
>>>
>>>>   I cannot open python after downloading it keeps going to
>>>>   modify/uninstall ?? please help
>
>> This is _still_ a problem after all these years and countless
>> identical complaints?
>
>> How difficult would it be to add a few lines of text to the
>> installer welcome screen explaining that you've just started the
>> Python INSTALLER, and if you've already done the installation and
>> want to "run Python" try <whatever>?
>
> How difficult would it be to get people to read those lines, though?

There is that...

It's been ages since I installed Python on Windows, so maybe that info
has already been added and I've put my foot in it?

--
Grant



From grant.b.edwards at gmail.com  Mon Feb  7 11:46:48 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 07 Feb 2022 08:46:48 -0800 (PST)
Subject: Openning Python program
References: <62014033.1c69fb81.1c113.fab9@mx.google.com>
 <C6DA9D39-3881-4011-AA71-B40BB52AA9BF@barrys-emacs.org>
Message-ID: <62014cf8.1c69fb81.44262.6695@mx.google.com>

On 2022-02-07, Barry <barry at barrys-emacs.org> wrote:
>> On 7 Feb 2022, at 15:55, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>> ?On 2022-02-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>>> On Sun, 6 Feb 2022 13:44:07 +0530, "createkmontalbion at gmail.com"
>>> <createkmontalbion at gmail.com> declaimed the following:
>>> 
>>>>  I cannot open python after downloading it keeps going to modify/uninstall
>>>>  ?? please help
[...]
>> This is _still_ a problem after all these years and countless
>> identical complaints?
>> 
>> How difficult would it be to add a few lines of text to the installer
>> welcome screen explaining that you've just started the Python
>> INSTALLER, and if you've already done the installation and want to
>> "run Python" try <whatever>?
>
> Better yet include the word setup in the installer .exe name.
> Just like almost every other installer does.

That's also been suggested repeatedly over the years also.

--
Grant



From carolblessy15 at gmail.com  Mon Feb  7 02:30:41 2022
From: carolblessy15 at gmail.com (blessy carol)
Date: Sun, 6 Feb 2022 23:30:41 -0800 (PST)
Subject: Logging user activity
Message-ID: <98256dd6-c4ef-413e-aa50-caa5e1841d4fn@googlegroups.com>

Hi,

I have this task where I have to create log files to record user activity whenever they make an entry or view something. Also, I have to create Database log file whenever someone accessed or manipulated the data in the database. The code is written python and used django framework. I've connected django with oracle cloud database. So now I want to if the basic logging details can be used to store the record of these activities in the log file in the server. It  would be of great help. 

Thanks & Regards,
Blessy.

From wlfraed at ix.netcom.com  Mon Feb  7 11:55:39 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Mon, 07 Feb 2022 11:55:39 -0500
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
Message-ID: <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>

On Mon, 7 Feb 2022 13:33:54 +0400, Abdur-Rahmaan Janhangeer
<arj.python at gmail.com> declaimed the following:

>Popular browsers tell: No internet connection detected. A function that
>goes in the same sense. Unless they too are pinging Google.com to check ...
>

	Ah, but WHEN do those browsers report that? When attempting to connect
to whatever the default "home" page has been set to? (Mine is configured to
use https://www.google.com as the default page -- if my router is down,
obviously the browser will time-out waiting for a response from Google, and
report "no network").

	Pretty much any discovery of "no network" occurs when the application
attempts to make a normal connection to some target -- using whatever
protocol is normal for that application -- and fails to get a response.
ping is not a solution -- it is possible for firewalls to be configured to
drop with no response specific packets. A firewall configured to DROP
rather than REJECT results in a machine that just "isn't there" to outside
poking. That doesn't mean that the network is down -- only that the machine
you tried to poke is ignoring you.

	Also, for a machine freshly booted, with no cache, even pinging Google
first requires making contact with a DNS server to ask for Google's IP
address. With no network, the DNS look-up will fail before ping even tries
to hit Google.

	Consider that UDP is often used in a "fire and forget" mode -- packets
get sent to the network interface for forwarding, but there is no
expectation that the transport system will return success/failure packets.
For UDP, any such has to be built into the application level protocol(s).
TCP, OTOH, /is/ a "connected" protocol expecting to receive ACK/NAK packets
for each one it sends out. If it doesn't receive either it will, after some
time-out period, declare a broken connection.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From Cecil at decebal.nl  Mon Feb  7 11:44:49 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 07 Feb 2022 17:44:49 +0100
Subject: Openning Python program
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
 <62014033.1c69fb81.1c113.fab9@mx.google.com>
 <CAPTjJmrmdvDWxSOXsD47Bk8745miHwsDF8dYw103NReYmY73jw@mail.gmail.com>
 <mailman.31.1644249391.7010.python-list@python.org>
Message-ID: <875ypq1v9a.fsf@munus.decebal.nl>

Chris Angelico <rosuav at gmail.com> writes:

> On Tue, 8 Feb 2022 at 02:53, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>>
>> On 2022-02-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>> > On Sun, 6 Feb 2022 13:44:07 +0530, "createkmontalbion at gmail.com"
>> ><createkmontalbion at gmail.com> declaimed the following:
>> >
>> >>   I cannot open python after downloading it keeps going to modify/uninstall
>> >>   ?? please help
>> >
>> > Stop clicking on the INSTALLER. What you downloaded is just the program
>> > that installs and configures Python on your system. Stuff it away someplace
>> > safe should you need to modify the current installation, but otherwise just
>> > forget that it exists.
>>
>> This is _still_ a problem after all these years and countless
>> identical complaints?
>>
>> How difficult would it be to add a few lines of text to the installer
>> welcome screen explaining that you've just started the Python
>> INSTALLER, and if you've already done the installation and want to
>> "run Python" try <whatever>?
>>
>
> How difficult would it be to get people to read those lines, though?

That does remind me about a system administrator who wanted to make a
point. He changed something on the server so all the Windows computers
started up and gave a message:
        If you want to continue: click Cancel

The help-desk became flooded with calls. I think he did a great job of
showing a vulnerability. But it was not appreciated and he was fired.
:'-(

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From rosuav at gmail.com  Mon Feb  7 12:39:13 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Feb 2022 04:39:13 +1100
Subject: Openning Python program
In-Reply-To: <875ypq1v9a.fsf@munus.decebal.nl>
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
 <62014033.1c69fb81.1c113.fab9@mx.google.com>
 <CAPTjJmrmdvDWxSOXsD47Bk8745miHwsDF8dYw103NReYmY73jw@mail.gmail.com>
 <mailman.31.1644249391.7010.python-list@python.org>
 <875ypq1v9a.fsf@munus.decebal.nl>
Message-ID: <CAPTjJmpzMQPHbrWHxZxMkMXmWTxWdN6T2UJVqa4SjC0gn7Ef4w@mail.gmail.com>

On Tue, 8 Feb 2022 at 04:30, Cecil Westerhof via Python-list
<python-list at python.org> wrote:
>
> Chris Angelico <rosuav at gmail.com> writes:
>
> > On Tue, 8 Feb 2022 at 02:53, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> >>
> >> On 2022-02-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> >> > On Sun, 6 Feb 2022 13:44:07 +0530, "createkmontalbion at gmail.com"
> >> ><createkmontalbion at gmail.com> declaimed the following:
> >> >
> >> >>   I cannot open python after downloading it keeps going to modify/uninstall
> >> >>   ?? please help
> >> >
> >> > Stop clicking on the INSTALLER. What you downloaded is just the program
> >> > that installs and configures Python on your system. Stuff it away someplace
> >> > safe should you need to modify the current installation, but otherwise just
> >> > forget that it exists.
> >>
> >> This is _still_ a problem after all these years and countless
> >> identical complaints?
> >>
> >> How difficult would it be to add a few lines of text to the installer
> >> welcome screen explaining that you've just started the Python
> >> INSTALLER, and if you've already done the installation and want to
> >> "run Python" try <whatever>?
> >>
> >
> > How difficult would it be to get people to read those lines, though?
>
> That does remind me about a system administrator who wanted to make a
> point. He changed something on the server so all the Windows computers
> started up and gave a message:
>         If you want to continue: click Cancel
>
> The help-desk became flooded with calls. I think he did a great job of
> showing a vulnerability. But it was not appreciated and he was fired.
> :'-(
>

First image in this collection:

https://thedailywtf.com/articles/How-Do-I-Use-This

For those who can't click on links, it's a screenshot of a
confirmation dialogue. The user asked to cancel all the current
transfers, and the system wanted to check that the user really wanted
to do that; if you do indeed want to cancel those transfers, click
"Cancel", but if you actually don't want to, click "Cancel" instead.

ChrisA

From auriocus at gmx.de  Mon Feb  7 14:33:31 2022
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Mon, 7 Feb 2022 20:33:31 +0100
Subject: Correct way to setup a package with both compiled C code and Python
 code?
Message-ID: <strs6d$nqq$1@dont-email.me>

Hi all,

we've developed a Python pacakge which consists of both a compiled 
extension module and some helper functions in Python. Is there a 
tutorial on how to package such an extension?

Most resources I found for distutils describe either building an 
extension or pure python modules. Currently I have a structure like this:

ABCD/__init__.py
ABCD/main.py
ccode/some.c
ccode/some.h
ccode/Makefile

The Makefile compiles the C code and creates ABCD/some.so, which 
"main.py" then imports. This works, but not e.g. on Windows and it's not 
integrated into pip, of course.

We're soon going to publish the package as open source code and it would 
be great to do "pip install ABCD" ultimately. Is there a simple example 
out there how to achieve this?

Additionally, we use OpenMP in the C code for parallelism. This is easy 
in the Makefile, one has to pass "-fopenmp" to gcc and "/openmp" to 
msvc. Is there a way to set this flag automatically depending on the 
compiler?

Best regards,

          Christian

From grant.b.edwards at gmail.com  Mon Feb  7 14:40:24 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 07 Feb 2022 11:40:24 -0800 (PST)
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
Message-ID: <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>

On 2022-02-07, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:

> Also, for a machine freshly booted, with no cache, even pinging
> Google first requires making contact with a DNS server to ask for
> Google's IP address. With no network, the DNS look-up will fail
> before ping even tries to hit Google.

Ah, c'mon... Every geek worth his salt knows a few real world IP
addresses without relying on DNS. If you want to "ping Google", it's

 $ ping 8.8.8.8
or
 $ ping 8.8.4.4

:)

If that doesn't work, then you ask 'route -n' for the IP address of
the default gateway, and try pinging that. It's possible your default
gateway is alive but configured to ignore ICMP ping requests, but I've
never run into one like that.

But, as has been pointed out previously "if there is internet" is too
vague a question to have an answer.

If all you have is proxied access to outside HTTPS servers, then I
would consider the answer to be "no", but most people would say "yes"
they have internet.

If all you have is NAT'ed outbound TCP connections, even more people
would say "yes they have internet", but I would still answer "partially".

--
Grant

From Cecil at decebal.nl  Mon Feb  7 14:36:12 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Mon, 07 Feb 2022 20:36:12 +0100
Subject: Openning Python program
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
 <62014033.1c69fb81.1c113.fab9@mx.google.com>
 <CAPTjJmrmdvDWxSOXsD47Bk8745miHwsDF8dYw103NReYmY73jw@mail.gmail.com>
 <mailman.31.1644249391.7010.python-list@python.org>
 <875ypq1v9a.fsf@munus.decebal.nl>
 <CAPTjJmpzMQPHbrWHxZxMkMXmWTxWdN6T2UJVqa4SjC0gn7Ef4w@mail.gmail.com>
 <mailman.35.1644255566.7010.python-list@python.org>
Message-ID: <871r0e1nbn.fsf@munus.decebal.nl>

Chris Angelico <rosuav at gmail.com> writes:

>> > How difficult would it be to get people to read those lines, though?
>>
>> That does remind me about a system administrator who wanted to make a
>> point. He changed something on the server so all the Windows computers
>> started up and gave a message:
>>         If you want to continue: click Cancel
>>
>> The help-desk became flooded with calls. I think he did a great job of
>> showing a vulnerability. But it was not appreciated and he was fired.
>> :'-(
>>
>
> First image in this collection:
>
> https://thedailywtf.com/articles/How-Do-I-Use-This
>
> For those who can't click on links, it's a screenshot of a
> confirmation dialogue. The user asked to cancel all the current
> transfers, and the system wanted to check that the user really wanted
> to do that; if you do indeed want to cancel those transfers, click
> "Cancel", but if you actually don't want to, click "Cancel" instead.

His dialog was crystal clear. The problem was that most users just
click OK without reading the message. And that was what his little
experiment showed.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From rosuav at gmail.com  Mon Feb  7 14:51:20 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Feb 2022 06:51:20 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
Message-ID: <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>

On Tue, 8 Feb 2022 at 06:41, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> But, as has been pointed out previously "if there is internet" is too
> vague a question to have an answer.
>
> If all you have is proxied access to outside HTTPS servers, then I
> would consider the answer to be "no", but most people would say "yes"
> they have internet.

Some day, we'll have people on Mars. They won't have TCP connections -
at least, not unless servers start supporting connection timeouts
measured in minutes or hours - but it wouldn't surprise me if some
sort of caching proxy system is deployed.

On the other hand, it also wouldn't surprise me if we do everything at
a high level instead - have a Martian PyPI mirror, Debian package
mirror, etc, etc, etc - and then build a mirror synchronization
protocol that uses UDP.

Either way, though: would a person on Mars "have the internet"? Yes,
but not the internet as we know it...

ChrisA

From rosuav at gmail.com  Mon Feb  7 14:53:24 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Feb 2022 06:53:24 +1100
Subject: Openning Python program
In-Reply-To: <871r0e1nbn.fsf@munus.decebal.nl>
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
 <62014033.1c69fb81.1c113.fab9@mx.google.com>
 <CAPTjJmrmdvDWxSOXsD47Bk8745miHwsDF8dYw103NReYmY73jw@mail.gmail.com>
 <mailman.31.1644249391.7010.python-list@python.org>
 <875ypq1v9a.fsf@munus.decebal.nl>
 <CAPTjJmpzMQPHbrWHxZxMkMXmWTxWdN6T2UJVqa4SjC0gn7Ef4w@mail.gmail.com>
 <mailman.35.1644255566.7010.python-list@python.org>
 <871r0e1nbn.fsf@munus.decebal.nl>
Message-ID: <CAPTjJmrmrrysy2FUVDD-2JB8z6QuSXcKccvHo+ODSbguaHwiUQ@mail.gmail.com>

On Tue, 8 Feb 2022 at 06:51, Cecil Westerhof via Python-list
<python-list at python.org> wrote:
>
> Chris Angelico <rosuav at gmail.com> writes:
>
> >> > How difficult would it be to get people to read those lines, though?
> >>
> >> That does remind me about a system administrator who wanted to make a
> >> point. He changed something on the server so all the Windows computers
> >> started up and gave a message:
> >>         If you want to continue: click Cancel
> >>
> >> The help-desk became flooded with calls. I think he did a great job of
> >> showing a vulnerability. But it was not appreciated and he was fired.
> >> :'-(
> >>
> >
> > First image in this collection:
> >
> > https://thedailywtf.com/articles/How-Do-I-Use-This
> >
> > For those who can't click on links, it's a screenshot of a
> > confirmation dialogue. The user asked to cancel all the current
> > transfers, and the system wanted to check that the user really wanted
> > to do that; if you do indeed want to cancel those transfers, click
> > "Cancel", but if you actually don't want to, click "Cancel" instead.
>
> His dialog was crystal clear. The problem was that most users just
> click OK without reading the message. And that was what his little
> experiment showed.
>

Ah. Yes, that... that sounds like a very familiar and serious vulnerability.

ChrisA

From wlfraed at ix.netcom.com  Mon Feb  7 15:41:33 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Mon, 07 Feb 2022 15:41:33 -0500
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <mailman.36.1644262827.7010.python-list@python.org>
Message-ID: <eo030hlfr3duum9dcm101k6mh480ea901o@4ax.com>

On Mon, 07 Feb 2022 11:40:24 -0800 (PST), Grant Edwards
<grant.b.edwards at gmail.com> declaimed the following:

>On 2022-02-07, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>
>> Also, for a machine freshly booted, with no cache, even pinging
>> Google first requires making contact with a DNS server to ask for
>> Google's IP address. With no network, the DNS look-up will fail
>> before ping even tries to hit Google.
>
>Ah, c'mon... Every geek worth his salt knows a few real world IP
>addresses without relying on DNS. If you want to "ping Google", it's
>
> $ ping 8.8.8.8
>or
> $ ping 8.8.4.4
>

	Which happen to be Google's DNS servers -- not what most think of as
"Google"

C:\Users\Wulfraed>ping www.google.com -4

Pinging www.google.com [142.251.45.36] with 32 bytes of data:
...


>:)
>
>If that doesn't work, then you ask 'route -n' for the IP address of
>the default gateway, and try pinging that. It's possible your default
>gateway is alive but configured to ignore ICMP ping requests, but I've
>never run into one like that.
>

	No "route -n" here...

C:\Users\Wulfraed>route -n

Manipulates network routing tables.

ROUTE [-f] [-p] [-4|-6] command [destination]
                  [MASK netmask]  [gateway] [METRIC metric]  [IF interface]

  -f           Clears the routing tables of all gateway entries.  If this
is
               used in conjunction with one of the commands, the tables are
               cleared prior to running the command.

  -p           When used with the ADD command, makes a route persistent
across
               boots of the system. By default, routes are not preserved
               when the system is restarted. Ignored for all other
commands,
               which always affect the appropriate persistent routes.

  -4           Force using IPv4.

  -6           Force using IPv6.

  command      One of these:
                 PRINT     Prints  a route
                 ADD       Adds    a route
                 DELETE    Deletes a route
                 CHANGE    Modifies an existing route
  destination  Specifies the host.
  MASK         Specifies that the next parameter is the 'netmask' value.
  netmask      Specifies a subnet mask value for this route entry.
               If not specified, it defaults to 255.255.255.255.
  gateway      Specifies gateway.
  interface    the interface number for the specified route.
  METRIC       specifies the metric, ie. cost for the destination.
...


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From grant.b.edwards at gmail.com  Mon Feb  7 16:02:42 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 07 Feb 2022 13:02:42 -0800 (PST)
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <mailman.36.1644262827.7010.python-list@python.org>
 <eo030hlfr3duum9dcm101k6mh480ea901o@4ax.com>
Message-ID: <620188f2.1c69fb81.b86cd.913e@mx.google.com>

On 2022-02-07, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> On Mon, 07 Feb 2022 11:40:24 -0800 (PST), Grant Edwards
><grant.b.edwards at gmail.com> declaimed the following:
>
>>On 2022-02-07, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>>
>>> Also, for a machine freshly booted, with no cache, even pinging
>>> Google first requires making contact with a DNS server to ask for
>>> Google's IP address. With no network, the DNS look-up will fail
>>> before ping even tries to hit Google.
>>
>>Ah, c'mon... Every geek worth his salt knows a few real world IP
>>addresses without relying on DNS. If you want to "ping Google", it's
>>
>> $ ping 8.8.8.8
>>or
>> $ ping 8.8.4.4
>>
>
> Which happen to be Google's DNS servers -- not what most think of as
> "Google"

Right. So even asking "do you have Google" is too vague. :)

> Manipulates network routing tables.

Sorry, I didn't know that the Windows "route" command didn't recognize the
standard -n option.

--
Grant


From cs at cskk.id.au  Mon Feb  7 16:25:33 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 8 Feb 2022 08:25:33 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
References: <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
Message-ID: <YgGOTQqcg5l7cuC9@cskk.homeip.net>

On 07Feb2022 11:40, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>Ah, c'mon... Every geek worth his salt knows a few real world IP
>addresses without relying on DNS. If you want to "ping Google", it's
> $ ping 8.8.8.8
> $ ping 8.8.4.4

And Cloudflare is 1.1.1.1. Speaking as someone who's seen his upstream 
provider have good G connectivity and degraded C connectiviy :-(

>If that doesn't work, then you ask 'route -n' for the IP address of
>the default gateway,

I tend to just check for the presence of the default route for services 
which should be "up" when there's internet. (Persistent ssh tunnels, in 
the main.) Not even a ping.

Some of those tunnels are further conditioned on a specific ping.

"ping -c 5 -q ip-addr-here" can be a simple Boolean test.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From cs at cskk.id.au  Mon Feb  7 16:28:42 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 8 Feb 2022 08:28:42 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
References: <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
Message-ID: <YgGPCnP/Tup/u3a+@cskk.homeip.net>

On 08Feb2022 06:51, Chris Angelico <rosuav at gmail.com> wrote:
>Some day, we'll have people on Mars. They won't have TCP connections -
>at least, not unless servers start supporting connection timeouts
>measured in minutes or hours - but it wouldn't surprise me if some
>sort of caching proxy system is deployed.

The TCP ESTABLISHED state has no timeouts at all (though intemediate 
stateful things can get bored).  The setup/teardown do though :-) But 
they can be proxied. Our previous satellite modem proxied TCP locally 
and ran a more-suitable-satellite protocol from the modem to the 
downstation, where it became TCP again and went on its way.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From rosuav at gmail.com  Mon Feb  7 16:59:28 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Feb 2022 08:59:28 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <YgGPCnP/Tup/u3a+@cskk.homeip.net>
References: <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
 <YgGPCnP/Tup/u3a+@cskk.homeip.net>
Message-ID: <CAPTjJmqtrysddUQC5vXObqEpA3s5WbF0riVPFrXsfPiei2S8yQ@mail.gmail.com>

On Tue, 8 Feb 2022 at 08:30, Cameron Simpson <cs at cskk.id.au> wrote:
>
> On 08Feb2022 06:51, Chris Angelico <rosuav at gmail.com> wrote:
> >Some day, we'll have people on Mars. They won't have TCP connections -
> >at least, not unless servers start supporting connection timeouts
> >measured in minutes or hours - but it wouldn't surprise me if some
> >sort of caching proxy system is deployed.
>
> The TCP ESTABLISHED state has no timeouts at all (though intemediate
> stateful things can get bored).  The setup/teardown do though :-)

That's what I mean - not a lot of point trying to establish a TCP
socket if the other end will give up on you after a mere ninety
seconds. It's extremely convenient that an established connection
lasts forever until touched in some way (even across reconfigurations
of the network - which is how you can redo a server's network over
SSH), but first you have to get to that!

> But
> they can be proxied. Our previous satellite modem proxied TCP locally
> and ran a more-suitable-satellite protocol from the modem to the
> downstation, where it became TCP again and went on its way.

Yup. I'm not sure whether we'll proxy TCP, proxy HTTP, or have high
level mirrors, but one way or another, people on Mars will want their
internets plsthx.

ChrisA

From hjp-python at hjp.at  Mon Feb  7 17:09:56 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Mon, 7 Feb 2022 23:09:56 +0100
Subject: Logging user activity
In-Reply-To: <98256dd6-c4ef-413e-aa50-caa5e1841d4fn@googlegroups.com>
References: <98256dd6-c4ef-413e-aa50-caa5e1841d4fn@googlegroups.com>
Message-ID: <20220207220956.jaddrujdxuydk32k@hjp.at>

On 2022-02-06 23:30:41 -0800, blessy carol wrote:
> I have this task where I have to create log files to record user
> activity whenever they make an entry or view something. Also, I have
> to create Database log file whenever someone accessed or manipulated
> the data in the database. The code is written python and used django
> framework. I've connected django with oracle cloud database. So now I
> want to if the basic logging details can be used to store the record
> of these activities in the log file in the server.

There are three places where you can do that in a centralized manner:

1. In the database itself. AFAIK Oracle has an audit system, but I've
   never used it.
2. At the Django ORM layer. Django has the ability to log all database
   queries it makes
3. At the web request level. Your web server (probably) already logs
   every request but not necessarily the information you are interested
   in. But you could write a piece of middleware for your Django which
   extracts log-worthy information and logs that.

The first two options are probably too low-level, and especially the
second is really hard to interpret in an automated manner (which is what
you probably want to do - otherwise why log in the first place?)

So I'd try the third option. But it really depends a lot on the
structure of your application on whether it's feasible to extract all
the data you need at that point. It's possible that you will have to go
through all the views in your app, see what data they are requesting and
altering and craft appropriate log messages for each.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220207/c29eabec/attachment.sig>

From 2QdxY4RzWzUUiLuE at potatochowder.com  Mon Feb  7 17:30:05 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Mon, 7 Feb 2022 16:30:05 -0600
Subject: Best way to check if there is internet?
In-Reply-To: <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
Message-ID: <YgGdbapOzgD2fppj@scrozzle>

On 2022-02-08 at 06:51:20 +1100,
Chris Angelico <rosuav at gmail.com> wrote:

> Either way, though: would a person on Mars "have the internet"? Yes,
> but not the internet as we know it...

By current definition, they *can't* have the internet as we know it.

Wikipedia,? Mirrian-Webster,? and TechTerms.com? (the first three that
came up in my search engine, which is admittedly Earthbound for now) all
use words like "global" and phrases like "across the world" or "around
the world," all of which arguably exclude Mars, or at least a network
that covers both Earth and Mars.

IMO, Mars and its current and future living beings are better off
without the World Wide Web, too.

? https://en.wikipedia.org/wiki/Internet
? https://www.merriam-webster.com/dictionary/Internet
? https://techterms.com/definition/internet

From rosuav at gmail.com  Mon Feb  7 17:38:22 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Feb 2022 09:38:22 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <YgGdbapOzgD2fppj@scrozzle>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
 <YgGdbapOzgD2fppj@scrozzle>
Message-ID: <CAPTjJmrHgHFzob30GvwRJ2X6+UFMiUn4uCYazPMX=n88Lzi_hA@mail.gmail.com>

On Tue, 8 Feb 2022 at 09:31, <2QdxY4RzWzUUiLuE at potatochowder.com> wrote:
>
> On 2022-02-08 at 06:51:20 +1100,
> Chris Angelico <rosuav at gmail.com> wrote:
>
> > Either way, though: would a person on Mars "have the internet"? Yes,
> > but not the internet as we know it...
>
> By current definition, they *can't* have the internet as we know it.
>
> Wikipedia,? Mirrian-Webster,? and TechTerms.com? (the first three that
> came up in my search engine, which is admittedly Earthbound for now) all
> use words like "global" and phrases like "across the world" or "around
> the world," all of which arguably exclude Mars, or at least a network
> that covers both Earth and Mars.
>

Yes, well, globally-unique IDs are probably going to be
universally-unique IDs too, but we're not hugely bothered by that.
Remember, you can get targeted advertising on the ISS too...

https://xkcd.com/713/

ChrisA

From cs at cskk.id.au  Mon Feb  7 17:40:40 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 8 Feb 2022 09:40:40 +1100
Subject: Logging user activity
In-Reply-To: <98256dd6-c4ef-413e-aa50-caa5e1841d4fn@googlegroups.com>
References: <98256dd6-c4ef-413e-aa50-caa5e1841d4fn@googlegroups.com>
Message-ID: <YgGf6HRfYBcC3Ou2@cskk.homeip.net>

On 06Feb2022 23:30, blessy carol <carolblessy15 at gmail.com> wrote:
>I have this task where I have to create log files to record user 
>activity whenever they make an entry or view something. Also, I have to 
>create Database log file whenever someone accessed or manipulated the 
>data in the database. The code is written python and used django 
>framework. I've connected django with oracle cloud database. So now I 
>want to if the basic logging details can be used to store the record of 
>these activities in the log file in the server. It  would be of great 
>help.

For the file, you can do that directly in the logging configuration - 
there's a FileHandler with a bunch of configuration options (like 
rolling over to a new file etc). Hook that to your logging setup.

For the db side, make a Djanog Model for your log table and write your 
own logging handler which makes new instance of the model containing log 
information. There might even by a prebuilt Django logging thing 
available for that - I haven't looked.

We had a "log everything!" requests for a project here.

I made a generic AuditLog Django model for this. It has a timestamp, a 
description, a log_type PositiveIntegerField, a user field (because we 
were tracking things by User, can be NULL for things not driven by a 
User eg app internals), a parent (ForeignKey to the AuditLog model so 
you can make hierarchical related log entries), and an:

    entity = GenericForeignKey("entity_type", "entity_uuid")

which was a reference to some "primary" entity (Model instance) 
elsewhere where that was relevant, and a couple of JSON fields for state 
information.

This is massive overkill for your needs, but we only wanted to do this 
once, so needs like yours also use this model here.

Then you can write simple log calls which make model instances with what 
you need to log in them.

The only only real catch for us was transactions - if something fails 
and rolls back the transaction it also throws away the log entries 
because they're in the transaction. For your needs that is probably 
fine.

For us, we want to use it for error tracking/diagnosis too, so I've got 
a ticket to hand the logging to a distinct Thread (which magicly makes a 
distinct db connection, outside the transaction). That way the logging 
can survive a transaction rollback.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From greg.ewing at canterbury.ac.nz  Mon Feb  7 19:27:35 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 8 Feb 2022 13:27:35 +1300
Subject: Best way to check if there is internet?
In-Reply-To: <mailman.37.1644263492.7010.python-list@python.org>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
 <mailman.37.1644263492.7010.python-list@python.org>
Message-ID: <j6drnpFmpnvU1@mid.individual.net>

On 8/02/22 8:51 am, Chris Angelico wrote:
> Some day, we'll have people on Mars. They won't have TCP connections -
> at least, not unless servers start supporting connection timeouts
> measured in minutes or hours - but it wouldn't surprise me if some
> sort of caching proxy system is deployed.

Or the internet acquires a new protocol that's designed
for very-long-latency connections.

-- 
Greg

From ethan at stoneleaf.us  Mon Feb  7 20:18:54 2022
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 7 Feb 2022 17:18:54 -0800
Subject: Best way to check if there is internet?
In-Reply-To: <j6drnpFmpnvU1@mid.individual.net>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
 <mailman.37.1644263492.7010.python-list@python.org>
 <j6drnpFmpnvU1@mid.individual.net>
Message-ID: <afa55b81-59dd-2276-9bf8-6621eaff5b8d@stoneleaf.us>

On 2/7/22 4:27 PM, Greg Ewing wrote:
 > On 8/02/22 8:51 am, Chris Angelico wrote:

 >> Some day, we'll have people on Mars. They won't have TCP connections -
 >> at least, not unless servers start supporting connection timeouts
 >> measured in minutes or hours - but it wouldn't surprise me if some
 >> sort of caching proxy system is deployed.
 >
 > Or the internet acquires a new protocol that's designed
 > for very-long-latency connections.

RocketNet -- a massive store-and-forward protocol.  ;-)

--
~Ethan~

From torriem at gmail.com  Mon Feb  7 20:22:16 2022
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 7 Feb 2022 18:22:16 -0700
Subject: Best way to check if there is internet?
In-Reply-To: <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
Message-ID: <6a28f405-aadb-0edc-d49a-da4e193272c4@gmail.com>

On 2/7/22 12:51, Chris Angelico wrote:
> Some day, we'll have people on Mars. They won't have TCP connections -
> at least, not unless servers start supporting connection timeouts
> measured in minutes or hours - but it wouldn't surprise me if some
> sort of caching proxy system is deployed.
> 
> On the other hand, it also wouldn't surprise me if we do everything at
> a high level instead - have a Martian PyPI mirror, Debian package
> mirror, etc, etc, etc - and then build a mirror synchronization
> protocol that uses UDP.
> 
> Either way, though: would a person on Mars "have the internet"? Yes,
> but not the internet as we know it...

Fun fact.  The team running the Ingenuity helicopter on mars has shell
access to Linux running on the copter.  Obviously not interactive in the
normal sense of course, but they can batch shell commands and pass them
through the communication network to the rover, which relays them to the
copter.  Standard out is relayed back to earth at the next opportunity.
 Currently they use this remote shell access to compress all the images
after each flight and use ffmpeg to create video sequences from stills
on the copter computer itself.  They also used it to do some hacks to
temporarily fix the watchdog timing issue they had initially.  One of
the Linux gurus on the project has given several interviews to the Linux
Unplugged podcast. Fastinating stuff!

It's likely they have a python interpreter onboard as well.

From rosuav at gmail.com  Mon Feb  7 20:23:52 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 8 Feb 2022 12:23:52 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <afa55b81-59dd-2276-9bf8-6621eaff5b8d@stoneleaf.us>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
 <mailman.37.1644263492.7010.python-list@python.org>
 <j6drnpFmpnvU1@mid.individual.net>
 <afa55b81-59dd-2276-9bf8-6621eaff5b8d@stoneleaf.us>
Message-ID: <CAPTjJmre7NCjaXbof+kYCvVeUoVrVFaAXOYMZ+LK+u1ULAn3aw@mail.gmail.com>

On Tue, 8 Feb 2022 at 12:20, Ethan Furman <ethan at stoneleaf.us> wrote:
>
> On 2/7/22 4:27 PM, Greg Ewing wrote:
>  > On 8/02/22 8:51 am, Chris Angelico wrote:
>
>  >> Some day, we'll have people on Mars. They won't have TCP connections -
>  >> at least, not unless servers start supporting connection timeouts
>  >> measured in minutes or hours - but it wouldn't surprise me if some
>  >> sort of caching proxy system is deployed.
>  >
>  > Or the internet acquires a new protocol that's designed
>  > for very-long-latency connections.
>
> RocketNet -- a massive store-and-forward protocol.  ;-)
>

Definitely possible. Though wouldn't a rocket scientist call it
"store-and-prograde"? :)

ChrisA

From Karsten.Hilbert at gmx.net  Tue Feb  8 02:17:08 2022
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Tue, 8 Feb 2022 08:17:08 +0100
Subject: Aw: Re: Best way to check if there is internet?
In-Reply-To: <j6drnpFmpnvU1@mid.individual.net>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
 <mailman.37.1644263492.7010.python-list@python.org>
 <j6drnpFmpnvU1@mid.individual.net>
Message-ID: <trinity-5317a0bb-01ac-4ff7-a964-8105d99c251e-1644304628711@3c-app-gmx-bs63>

> Or the internet acquires a new protocol that's designed
> for very-long-latency connections.

That's being worked on already

https://en.wikipedia.org/wiki/NASA_Deep_Space_Network

Karsten

From tdldev at gmail.com  Tue Feb  8 07:04:04 2022
From: tdldev at gmail.com (Jack Dangler)
Date: Tue, 8 Feb 2022 07:04:04 -0500
Subject: Logging user activity
In-Reply-To: <20220207220956.jaddrujdxuydk32k@hjp.at>
References: <98256dd6-c4ef-413e-aa50-caa5e1841d4fn@googlegroups.com>
 <20220207220956.jaddrujdxuydk32k@hjp.at>
Message-ID: <ae8ca18c-3b5c-f444-54c0-7f1dfd967842@gmail.com>


On 2/7/22 5:09 PM, Peter J. Holzer wrote:
> On 2022-02-06 23:30:41 -0800, blessy carol wrote:
>> I have this task where I have to create log files to record user
>> activity whenever they make an entry or view something. Also, I have
>> to create Database log file whenever someone accessed or manipulated
>> the data in the database. The code is written python and used django
>> framework. I've connected django with oracle cloud database. So now I
>> want to if the basic logging details can be used to store the record
>> of these activities in the log file in the server.
> There are three places where you can do that in a centralized manner:
>
> 1. In the database itself. AFAIK Oracle has an audit system, but I've
>     never used it.
> 2. At the Django ORM layer. Django has the ability to log all database
>     queries it makes
> 3. At the web request level. Your web server (probably) already logs
>     every request but not necessarily the information you are interested
>     in. But you could write a piece of middleware for your Django which
>     extracts log-worthy information and logs that.
>
> The first two options are probably too low-level, and especially the
> second is really hard to interpret in an automated manner (which is what
> you probably want to do - otherwise why log in the first place?)
>
> So I'd try the third option. But it really depends a lot on the
> structure of your application on whether it's feasible to extract all
> the data you need at that point. It's possible that you will have to go
> through all the views in your app, see what data they are requesting and
> altering and craft appropriate log messages for each.
>
>          hp
>
>
Short of a long lesson on why software developers should always think of 
logging/monitoring components to everything they write, I'll just offer 
this. It's worth the read and should put you in a good position to 
complete your task.

https://www.askpython.com/django/django-logging

If you're after something that you're being paid to do, look into 
Splunk! It's a robust package that offers logging and correlation of 
data on nearly any level of complication/sophistication your customers need.

Hope this helps.

Regards

Jack


From alan at csail.mit.edu  Mon Feb  7 23:29:17 2022
From: alan at csail.mit.edu (Alan Bawden)
Date: Mon, 07 Feb 2022 23:29:17 -0500
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
 <mailman.37.1644263492.7010.python-list@python.org>
 <j6drnpFmpnvU1@mid.individual.net>
Message-ID: <864k5arnfm.fsf@williamsburg.bawden.org>

Greg Ewing <greg.ewing at canterbury.ac.nz> writes:

   On 8/02/22 8:51 am, Chris Angelico wrote:
   > Some day, we'll have people on Mars. They won't have TCP connections -
   > at least, not unless servers start supporting connection timeouts
   > measured in minutes or hours - but it wouldn't surprise me if some
   > sort of caching proxy system is deployed.

   Or the internet acquires a new protocol that's designed
   for very-long-latency connections.

Very much a hot topic:

https://www.nasa.gov/directorates/heo/scan/engineering/technology/disruption_tolerant_networking_overview
https://en.wikipedia.org/wiki/Delay-tolerant_networking
https://datatracker.ietf.org/doc/html/rfc4838
https://datatracker.ietf.org/doc/html/rfc5050

-- 
Alan Bawden

From alan at csail.mit.edu  Mon Feb  7 23:46:55 2022
From: alan at csail.mit.edu (Alan Bawden)
Date: Mon, 07 Feb 2022 23:46:55 -0500
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
 <620175a8.1c69fb81.9e8e3.b1ec@mx.google.com>
 <CAPTjJmp24wExC3+W9pOv0xUUih2qqryWzSbHfSOMpLwG2KVoMw@mail.gmail.com>
 <mailman.37.1644263492.7010.python-list@python.org>
 <j6drnpFmpnvU1@mid.individual.net>
 <864k5arnfm.fsf@williamsburg.bawden.org>
Message-ID: <86zgn2q81s.fsf@williamsburg.bawden.org>

And I missed one that was just published last month:

https://datatracker.ietf.org/doc/html/rfc9171

Unlike RFC 5050, this version of the protocol actually claims to be a
"Proposed Standard".

-- 
Alan Bawden

From Cecil at decebal.nl  Tue Feb  8 03:05:46 2022
From: Cecil at decebal.nl (Cecil Westerhof)
Date: Tue, 08 Feb 2022 09:05:46 +0100
Subject: Openning Python program
References: <78B09374-BD35-4184-AAB0-3B2A3479BCF2@hxcore.ol>
 <mailman.21.1644157604.7010.python-list@python.org>
 <0eqvvglenoq7b2ud0l0lf3q4ph4dneli4j@4ax.com>
 <62014033.1c69fb81.1c113.fab9@mx.google.com>
 <CAPTjJmrmdvDWxSOXsD47Bk8745miHwsDF8dYw103NReYmY73jw@mail.gmail.com>
 <mailman.31.1644249391.7010.python-list@python.org>
 <875ypq1v9a.fsf@munus.decebal.nl>
 <CAPTjJmpzMQPHbrWHxZxMkMXmWTxWdN6T2UJVqa4SjC0gn7Ef4w@mail.gmail.com>
 <mailman.35.1644255566.7010.python-list@python.org>
 <871r0e1nbn.fsf@munus.decebal.nl>
 <CAPTjJmrmrrysy2FUVDD-2JB8z6QuSXcKccvHo+ODSbguaHwiUQ@mail.gmail.com>
 <mailman.38.1644263616.7010.python-list@python.org>
Message-ID: <87wni5zsth.fsf@munus.decebal.nl>

Chris Angelico <rosuav at gmail.com> writes:

> On Tue, 8 Feb 2022 at 06:51, Cecil Westerhof via Python-list
> <python-list at python.org> wrote:
>>
>> Chris Angelico <rosuav at gmail.com> writes:
>>
>> >> > How difficult would it be to get people to read those lines, though?
>> >>
>> >> That does remind me about a system administrator who wanted to make a
>> >> point. He changed something on the server so all the Windows computers
>> >> started up and gave a message:
>> >>         If you want to continue: click Cancel
>> >>
>> >> The help-desk became flooded with calls. I think he did a great job of
>> >> showing a vulnerability. But it was not appreciated and he was fired.
>> >> :'-(
>> >>
>> >
>> > First image in this collection:
>> >
>> > https://thedailywtf.com/articles/How-Do-I-Use-This
>> >
>> > For those who can't click on links, it's a screenshot of a
>> > confirmation dialogue. The user asked to cancel all the current
>> > transfers, and the system wanted to check that the user really wanted
>> > to do that; if you do indeed want to cancel those transfers, click
>> > "Cancel", but if you actually don't want to, click "Cancel" instead.
>>
>> His dialog was crystal clear. The problem was that most users just
>> click OK without reading the message. And that was what his little
>> experiment showed.
>>
>
> Ah. Yes, that... that sounds like a very familiar and serious vulnerability.

I really think that always clicking on OK without ever checking what
is going to happen is a problem. It at least shows that 'user
friendly' software can be counter productive.

>From long ago:
     Are you sure that you want to take this action?

     Yes.


     Are you really sure that you want to take this action?

     Yes.


     Are you really, really sure that you want to take this action.

     Yes. (With a curse under the breath.)

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

From etemcetin.toptani at std.yeditepe.edu.tr  Tue Feb  8 03:52:11 2022
From: etemcetin.toptani at std.yeditepe.edu.tr (=?UTF-8?B?RVRFTSDDh0VUxLBOIFRPUFRBTsSw?=)
Date: Tue, 8 Feb 2022 11:52:11 +0300
Subject: Kind Remainder: How do Practitioners Approach towards Requirements
 Engineering?
Message-ID: <CAJ48VLhzw2EWrhiNjPiWPsP42PZYhDmYdt6ViYKXnOzoiqdPSQ@mail.gmail.com>

Dear Sir or Madam,


We prepared a short survey to understand practitioners? perspectives
towards the requirements engineering. Our survey basically aims to clarify
on many aspects of the requirements engineering applied in industry, including
(i) requirements gathering and specifications, (ii) requirements
modifications, (iii) requirements analysis, and (iv) requirements
transformation. We will use the results to publish a journal paper on the
practitioners' perspectives towards requirements engineering


The survey takes about 2-7 minutes to participate. We will be so grateful
if you can separate just a few minutes of you. Also, please circulate the
email to anyone who may be interested.


The survey link: https://forms.gle/DhLqr15GXVhJhzzy6


All the best,

Etem ?etin Toptani

-- 
*Bu mesaj? yazd?rmadan ?nce ?evreye verebilece?iniz zararlar? bir kez daha 
d???n?n?z.?*
*Think of the environment once more before printing out this 
message.*

-- 
*Bu mesaj? yazd?rmadan ?nce ?evreye verebilece?iniz zararlar? bir kez daha 
d???n?n?z.?*
*Think of the environment once more before printing out this 
message.*

From liedtke at punkt.de  Tue Feb  8 09:13:07 2022
From: liedtke at punkt.de (Lars Liedtke)
Date: Tue, 8 Feb 2022 15:13:07 +0100
Subject: How do you log in your projects?
Message-ID: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>

Hello,

inspired by this thread 
https://mail.python.org/pipermail/python-list/2022-February/905167.html 
I finally came around to send in this question.

I know how to use python logging generally. Being on the DevOps-Side of 
things at the moment, I naturally began liking to run tools, which not 
only have logging, but also have different levels of logging 
configurable, because often when you run a couple of different services 
and software projects, logging is often the first help you turn to, and 
sometimes it is the only way to know what is going on. Not that the code 
of most (FLOSS) software is not available, but you simply do not have 
the time to read into the code of every piece of software you run.

So of course I want to write software on my own, if I have to or wpuld 
like to, that has got a decent amount of logging. But how often to 
actually log things? Of course the obvious answer is "it depends".

So how do you do your logging?
- On a line per line basis? on a function/method basis?
- Do you use decorators to mark beginnings and ends of methods/functions 
in log files?
- Which kind of variable contents do you write into your logfiles? Of 
course you shouldn't leak secrets...
- How do you decide, which kind of log message goes into which level?
- How do you prevent logging cluttering your actual code?

Maybe there are some questions I forgot, so please feel free to add 
whatever you think might help finding the best way of logging (tm)

Cheers

Lars

-- 
punkt.de GmbH
Lars Liedtke
.infrastructure

Kaiserallee 13a	
76133 Karlsruhe

Tel. +49 721 9109 500
https://infrastructure.punkt.de
info at punkt.de

AG Mannheim 108285
Gesch?ftsf?hrer: J?rgen Egeling, Daniel Lienert, Fabian Stein


From loris.bennett at fu-berlin.de  Tue Feb  8 08:55:51 2022
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Tue, 08 Feb 2022 14:55:51 +0100
Subject: SQLAlchemy: When to initialise a session
Message-ID: <87h799fons.fsf@hornfels.zedat.fu-berlin.de>

Hi,

I am writing a fairly simple command-line application which will just
add or delete an entry in a database and then generate a corresponding
email.

I am using SQLAlchemy to wrap a class around a database and have

  class DatebaseWrapper():
      """Encapsulation of the database"""

      def __init__(self, url):
          self.engine = create_engine(url)
 
Should I extend the initialisation to

      def __init__(self, url):
          self.engine = create_engine(url)
          self.session = sessionmaker(self.engine)

since each there will be only one session per call of the program?

Or, since I am writing the database wrapper as its own module for
possible reuse, should the program using the wrapper class
initialise the session itself?

Cheers,

Loris
 
-- 
This signature is currently under construction.

From loris.bennett at fu-berlin.de  Tue Feb  8 09:12:45 2022
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Tue, 08 Feb 2022 15:12:45 +0100
Subject: SQLAlchemy: When to initialise a session
References: <87h799fons.fsf@hornfels.zedat.fu-berlin.de>
Message-ID: <87czjxfnvm.fsf@hornfels.zedat.fu-berlin.de>

"Loris Bennett" <loris.bennett at fu-berlin.de> writes:

> Hi,
>
> I am writing a fairly simple command-line application which will just
> add or delete an entry in a database and then generate a corresponding
> email.
>
> I am using SQLAlchemy to wrap a class around a database and have
>
>   class DatebaseWrapper():
>       """Encapsulation of the database"""
>
>       def __init__(self, url):
>           self.engine = create_engine(url)
>  
> Should I extend the initialisation to
>
>       def __init__(self, url):
>           self.engine = create_engine(url)
>           self.session = sessionmaker(self.engine)
>
> since each there will be only one session per call of the program?
>
> Or, since I am writing the database wrapper as its own module for
> possible reuse, should the program using the wrapper class
> initialise the session itself?

Turns out this is all explained here:

  https://docs.sqlalchemy.org/en/14/orm/session_basics.html#session-frequently-asked-questions

Sorry for the noise.

-- 
This signature is currently under construction.

From dieter at handshake.de  Tue Feb  8 12:57:57 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Tue, 8 Feb 2022 18:57:57 +0100
Subject: Correct way to setup a package with both compiled C code and
 Python code?
In-Reply-To: <strs6d$nqq$1@dont-email.me>
References: <strs6d$nqq$1@dont-email.me>
Message-ID: <25090.44837.132493.228601@ixdm.fritz.box>

Christian Gollwitzer wrote at 2022-2-7 20:33 +0100:
>we've developed a Python pacakge which consists of both a compiled
>extension module and some helper functions in Python. Is there a
>tutorial on how to package such an extension?

Look at "https://package.python.org",
especially "https://packaging.python.org/en/latest/guides/packaging-binary-extensions/".

From Marco.Sulla.Python at gmail.com  Tue Feb  8 15:40:07 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Tue, 8 Feb 2022 21:40:07 +0100
Subject: How do you log in your projects?
In-Reply-To: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>
References: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>
Message-ID: <CABbU2U-jTZXzNkG89q0PbXJuYLVpiOwJMR43bMMOZDnaRxu=kA@mail.gmail.com>

These are a lot of questions. I hope we're not off topic.
I don't know if mine are best practices. I can tell what I try to do.

On Tue, 8 Feb 2022 at 15:15, Lars Liedtke <liedtke at punkt.de> wrote:
> - On a line per line basis? on a function/method basis?

I usually log the start and end of functions. I could also log inside
a branch or in other parts of the function/method.

> - Do you use decorators to mark beginnings and ends of methods/functions
> in log files?

No, since I put the function parameters in the first log. But I think
that such a decorator it's not bad.

> - Which kind of variable contents do you write into your logfiles? Of
> course you shouldn't leak secrets...

Well, all the data that is useful to understand what the code is
doing. It's better to repeat the essential data to identify a specific
call in all the logs of the function, so if it is called
simultaneously by more clients you can distinguish them

> - How do you decide, which kind of log message goes into which level?

It depends on the importance, the verbosity and the occurrences of the logs.

> - How do you prevent logging cluttering your actual code?

I have the opposite problem, I should log more. So I can't answer your question.

From barry at barrys-emacs.org  Tue Feb  8 17:09:12 2022
From: barry at barrys-emacs.org (Barry)
Date: Tue, 8 Feb 2022 22:09:12 +0000
Subject: How do you log in your projects?
In-Reply-To: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>
References: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>
Message-ID: <84B940A1-5624-42D3-A21A-983C3B43CD16@barrys-emacs.org>



> On 8 Feb 2022, at 14:15, Lars Liedtke <liedtke at punkt.de> wrote:
> 
> ?Hello,
> 
> inspired by this thread https://mail.python.org/pipermail/python-list/2022-February/905167.html I finally came around to send in this question.
> 
> I know how to use python logging generally. Being on the DevOps-Side of things at the moment, I naturally began liking to run tools, which not only have logging, but also have different levels of logging configurable, because often when you run a couple of different services and software projects, logging is often the first help you turn to, and sometimes it is the only way to know what is going on. Not that the code of most (FLOSS) software is not available, but you simply do not have the time to read into the code of every piece of software you run.
> 
> So of course I want to write software on my own, if I have to or wpuld like to, that has got a decent amount of logging. But how often to actually log things? Of course the obvious answer is "it depends".
> 
> So how do you do your logging?
> - On a line per line basis? on a function/method basis?
> - Do you use decorators to mark beginnings and ends of methods/functions in log files?
> - Which kind of variable contents do you write into your logfiles? Of course you shouldn't leak secrets...
> - How do you decide, which kind of log message goes into which level?
> - How do you prevent logging cluttering your actual code?

You seem to be thinking about how to log the flow of the code.
That is useful to log when debugging your code. But depending
on your performance goals you may have to remove from the
production version.

But it is not what you do to be able to maintain a production service.
In this case you log events that the service is processing.

Barry

> 
> Maybe there are some questions I forgot, so please feel free to add whatever you think might help finding the best way of logging (tm)
> 
> Cheers
> 
> Lars
> 
> -- 
> punkt.de GmbH
> Lars Liedtke
> .infrastructure
> 
> Kaiserallee 13a    
> 76133 Karlsruhe
> 
> Tel. +49 721 9109 500
> https://infrastructure.punkt.de
> info at punkt.de
> 
> AG Mannheim 108285
> Gesch?ftsf?hrer: J?rgen Egeling, Daniel Lienert, Fabian Stein
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From jenkris at tutanota.com  Tue Feb  8 20:12:02 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Wed, 9 Feb 2022 02:12:02 +0100 (CET)
Subject: Can't get iterator in the C API
Message-ID: <MvR1O5n--3-2@tutanota.com>

I am using the Python C API to load the Gutenberg corpus from the nltk library and iterate through the sentences.? The Python code I am trying to replicate is:

from nltk.corpus import gutenberg
for i, fileid in enumerate(gutenberg.fileids()):
??????? sentences = gutenberg.sents(fileid)
??????? etc

where gutenberg.fileids is, of course, iterable.?

I use the following C API code to import the module and get pointers:

int64_t Call_PyModule()
{
??? PyObject *pModule, *pName, *pSubMod, *pFidMod, *pFidSeqIter,*pSentMod;

??? pName = PyUnicode_FromString("nltk.corpus");
??? pModule = PyImport_Import(pName);

??? if (pModule == 0x0){
??????? PyErr_Print();
??????? return 1; }

??? pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
??? pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
??? pSentMod = PyObject_GetAttrString(pSubMod, "sents");

??? pFidIter = PyObject_GetIter(pFidMod);
??? int ckseq_ok = PySeqIter_Check(pFidMod);
??? pFidSeqIter? = PySeqIter_New(pFidMod);

??? return 0;
}

pSubMod, pFidMod and pSentMod all return valid pointers, but the iterator lines return zero:?

pFidIter = PyObject_GetIter(pFidMod);
int ckseq_ok = PySeqIter_Check(pFidMod);
pFidSeqIter? = PySeqIter_New(pFidMod);

So the C API thinks gutenberg.fileids is not iterable, but it is.? What am I doing wrong?



From avigross at verizon.net  Tue Feb  8 12:23:53 2022
From: avigross at verizon.net (Avi Gross)
Date: Tue, 8 Feb 2022 17:23:53 +0000 (UTC)
Subject: Kind Remainder: How do Practitioners Approach towards
 Requirements Engineering?
In-Reply-To: <CAJ48VLhzw2EWrhiNjPiWPsP42PZYhDmYdt6ViYKXnOzoiqdPSQ@mail.gmail.com>
References: <CAJ48VLhzw2EWrhiNjPiWPsP42PZYhDmYdt6ViYKXnOzoiqdPSQ@mail.gmail.com>
Message-ID: <224151487.658801.1644341033349@mail.yahoo.com>

Is there any way to get this mesage to stop showing up in my mailbox in what seems to be a shotgun approach asking everybody everywhere they can think of to participate in something very amorphous and at the same time having NOTHING particular to do with Python?
I consider things like this SPAM and will not click on anything so have no idea if it is legitimate, but I suggest multiple requests with no real details become an imposition. Anyone who wanted to do the survey has had a chance. Why bully the rest of us?
Is there some policy that might apply here about not being relevant?


-----Original Message-----
From: ETEM ?ET?N TOPTAN? <etemcetin.toptani at std.yeditepe.edu.tr>
To: python-list at python.org
Sent: Tue, Feb 8, 2022 3:52 am
Subject: Kind Remainder: How do Practitioners Approach towards Requirements Engineering?

Dear Sir or Madam,


We prepared a short survey to understand practitioners? perspectives
towards the requirements engineering. Our survey basically aims to clarify
on many aspects of the requirements engineering applied in industry, including
(i) requirements gathering and specifications, (ii) requirements
modifications, (iii) requirements analysis, and (iv) requirements
transformation. We will use the results to publish a journal paper on the
practitioners' perspectives towards requirements engineering


The survey takes about 2-7 minutes to participate. We will be so grateful
if you can separate just a few minutes of you. Also, please circulate the
email to anyone who may be interested.


The survey link: https://forms.gle/DhLqr15GXVhJhzzy6


All the best,

Etem ?etin Toptani

-- 
*Bu mesaj? yazd?rmadan ?nce ?evreye verebilece?iniz zararlar? bir kez daha 
d???n?n?z.?*
*Think of the environment once more before printing out this 
message.*

-- 
*Bu mesaj? yazd?rmadan ?nce ?evreye verebilece?iniz zararlar? bir kez daha 
d???n?n?z.?*
*Think of the environment once more before printing out this 
message.*
-- 
https://mail.python.org/mailman/listinfo/python-list

From auriocus at gmx.de  Tue Feb  8 16:43:53 2022
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Tue, 8 Feb 2022 22:43:53 +0100
Subject: Correct way to setup a package with both compiled C code and
 Python code?
In-Reply-To: <mailman.54.1644343084.7010.python-list@python.org>
References: <strs6d$nqq$1@dont-email.me>
 <25090.44837.132493.228601@ixdm.fritz.box>
 <mailman.54.1644343084.7010.python-list@python.org>
Message-ID: <stuo6r$10o$1@dont-email.me>

Am 08.02.22 um 18:57 schrieb Dieter Maurer:
> Christian Gollwitzer wrote at 2022-2-7 20:33 +0100:
>> we've developed a Python pacakge which consists of both a compiled
>> extension module and some helper functions in Python. Is there a
>> tutorial on how to package such an extension?
> 
> Look at "https://package.python.org",
> especially "https://packaging.python.org/en/latest/guides/packaging-binary-extensions/".

Thank you, but that page is more like a high-level description, it talks 
a lot about the differences between C code and Python and how it can be 
combined using SWIG etc, but I already have a working extension.

My question was more targeted at how to write the setup.py file in this 
case such that both the compiled code and the Python code gets loaded.

In the meantime, I found a similar example for C++ code with CMake and 
pybind11 here:

	https://github.com/benjaminjack/python_cpp_example

That gave me enough to create a similar thing for a pure CPython extension:

	https://github.com/auriocus/python_setuptools_sandbox

Now, I still have to figure out how to get this on PyPI and how to 
enable OpenMP during compilation.

	Christian

From python at mrabarnett.plus.com  Tue Feb  8 21:10:51 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 9 Feb 2022 02:10:51 +0000
Subject: Can't get iterator in the C API
In-Reply-To: <MvR1O5n--3-2@tutanota.com>
References: <MvR1O5n--3-2@tutanota.com>
Message-ID: <58f81cfe-1843-32bc-3b2a-b6919b2c308a@mrabarnett.plus.com>

On 2022-02-09 01:12, Jen Kris via Python-list wrote:
> I am using the Python C API to load the Gutenberg corpus from the nltk library and iterate through the sentences.? The Python code I am trying to replicate is:
> 
> from nltk.corpus import gutenberg
> for i, fileid in enumerate(gutenberg.fileids()):
>  ??????? sentences = gutenberg.sents(fileid)
>  ??????? etc
> 
> where gutenberg.fileids is, of course, iterable.
> 
> I use the following C API code to import the module and get pointers:
> 
> int64_t Call_PyModule()
> {
>  ??? PyObject *pModule, *pName, *pSubMod, *pFidMod, *pFidSeqIter,*pSentMod;
> 
>  ??? pName = PyUnicode_FromString("nltk.corpus");
>  ??? pModule = PyImport_Import(pName);
> 
>  ??? if (pModule == 0x0){
>  ??????? PyErr_Print();
>  ??????? return 1; }
> 
>  ??? pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>  ??? pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>  ??? pSentMod = PyObject_GetAttrString(pSubMod, "sents");
> 
>  ??? pFidIter = PyObject_GetIter(pFidMod);
>  ??? int ckseq_ok = PySeqIter_Check(pFidMod);
>  ??? pFidSeqIter? = PySeqIter_New(pFidMod);
> 
>  ??? return 0;
> }
> 
> pSubMod, pFidMod and pSentMod all return valid pointers, but the iterator lines return zero:
> 
> pFidIter = PyObject_GetIter(pFidMod);
> int ckseq_ok = PySeqIter_Check(pFidMod);
> pFidSeqIter? = PySeqIter_New(pFidMod);
> 
> So the C API thinks gutenberg.fileids is not iterable, but it is.? What am I doing wrong?
> 
Look at your Python code. You have "gutenberg.fileids()", so the 
'fileids' attribute is not an iterable itself, but a method that you 
need to call to get the iterable.

From arj.python at gmail.com  Wed Feb  9 02:15:34 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Wed, 9 Feb 2022 11:15:34 +0400
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
Message-ID: <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>

Thanks everybody for the answers. It was very enlightening. Here's my
solution:

# using rich console
def ensure_internet():
console = Console()
domains = [
'https://google.com',
'https://yahoo.com',
'https://bing.com',
'https://www.ecosia.org',
'https://www.wikipedia.org'
]
results = []
with console.status("Checking internet ...", spinner="dots"):
for domain in domains:
try:
requests.get(domain)
results.append(1)
except Exception as e:
results.append(0)
if not any(results):
print('No internet connection')
sys.exit()

gist link:
https://gist.github.com/Abdur-rahmaanJ/7917dc5ab7f5d2aa37b2723909be08f7

I think for me having the internet means ability to request urls

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From gisle.vanem at gmail.com  Wed Feb  9 03:21:41 2022
From: gisle.vanem at gmail.com (Gisle Vanem)
Date: Wed, 9 Feb 2022 09:21:41 +0100
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
Message-ID: <15ece024-270c-f616-e90e-a0e8f90fd1bc@gmail.com>

Abdur-Rahmaan Janhangeer wrote:

> Thanks everybody for the answers. It was very enlightening. Here's my
> solution:
> 
> # using rich console
> def ensure_internet():
> console = Console()
> domains = [
> 'https://google.com',
> 'https://yahoo.com',
> 'https://bing.com',
> 'https://www.ecosia.org',
> 'https://www.wikipedia.org'
> ]
> results = []
> with console.status("Checking internet ...", spinner="dots"):
> for domain in domains:
> try:
> requests.get(domain)
> results.append(1)
> except Exception as e:
> results.append(0)
> if not any(results):
> print('No internet connection')
> sys.exit()

Was this supposed to be a self-contained working
program? It doesn't work here.

Were/what is 'Console()' for example?

-- 
--gv

From arj.python at gmail.com  Wed Feb  9 03:33:40 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Wed, 9 Feb 2022 12:33:40 +0400
Subject: Best way to check if there is internet?
In-Reply-To: <15ece024-270c-f616-e90e-a0e8f90fd1bc@gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <15ece024-270c-f616-e90e-a0e8f90fd1bc@gmail.com>
Message-ID: <CADrxXXnd_-4U2M=vdbFZk++b3FBkvF6e+3RPMmJnOGg9scQudw@mail.gmail.com>

It's a demo to get the idea behind, uses requests and the rich library.

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius


On Wed, Feb 9, 2022 at 12:24 PM Gisle Vanem <gisle.vanem at gmail.com> wrote:

> Abdur-Rahmaan Janhangeer wrote:
>
> > Thanks everybody for the answers. It was very enlightening. Here's my
> > solution:
> >
> > # using rich console
> > def ensure_internet():
> > console = Console()
> > domains = [
> > 'https://google.com',
> > 'https://yahoo.com',
> > 'https://bing.com',
> > 'https://www.ecosia.org',
> > 'https://www.wikipedia.org'
> > ]
> > results = []
> > with console.status("Checking internet ...", spinner="dots"):
> > for domain in domains:
> > try:
> > requests.get(domain)
> > results.append(1)
> > except Exception as e:
> > results.append(0)
> > if not any(results):
> > print('No internet connection')
> > sys.exit()
>
> Was this supposed to be a self-contained working
> program? It doesn't work here.
>
> Were/what is 'Console()' for example?
>
> --
> --gv
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From gisle.vanem at gmail.com  Wed Feb  9 05:29:03 2022
From: gisle.vanem at gmail.com (Gisle Vanem)
Date: Wed, 9 Feb 2022 11:29:03 +0100
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXnd_-4U2M=vdbFZk++b3FBkvF6e+3RPMmJnOGg9scQudw@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <15ece024-270c-f616-e90e-a0e8f90fd1bc@gmail.com>
 <CADrxXXnd_-4U2M=vdbFZk++b3FBkvF6e+3RPMmJnOGg9scQudw@mail.gmail.com>
Message-ID: <0adcd933-0e0c-3cf7-c0de-c723bfe356a6@gmail.com>

Abdur-Rahmaan Janhangeer wrote:

> It's a demo to get the idea behind, uses requests and the rich library.

I'd never heard of 'Rich'. But d/l it and playing
with it, I came up with this version with fat green
spinner-bar:

import requests
from rich.console import Console

def ensure_internet():
     console = Console()
     domains = [ "https://google.com",
                 "https://yahoo.com",
                 "https://bing.com",
                 "https://www.ecosia.org",
                 "https://www.wikipedia.org" ]
     results = []
     with console.status ("Checking internet ...", spinner="material") as c:
         for domain in domains:
             c._spinner.text = "Checking %-40s" % domain
             try:
                 requests.get(domain)
                 results.append(1)
             except Exception as e:
                 results.append(0)

     if not any(results):
        print("No internet connection")

ensure_internet()

-------------------

Rich' is IMO a pretty amazing package.

--gv

From p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt  Tue Feb  8 21:17:43 2022
From: p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt (Paulo da Silva)
Date: Wed, 9 Feb 2022 02:17:43 +0000
Subject: Unpacking lists in a f string
Message-ID: <stv887$ics$1@gioia.aioe.org>

Hi!

Let's say I have two lists of equal length but with a variable number of 
elements. For ex.:

l1=['a','b','c']
l2=['j','k','l']

I want to build a string like this
"foo a j, b k, c l bar"

Is it possible to achieve this with f strings or any other 
simple/efficient way?

Thanks for any help/comments.

From python at example.invalid  Tue Feb  8 22:43:07 2022
From: python at example.invalid (Python)
Date: Wed, 9 Feb 2022 04:43:07 +0100
Subject: Unpacking lists in a f string
References: <stv887$ics$1@gioia.aioe.org>
Message-ID: <stvd84$45r$1@gioia.aioe.org>

Paulo da Silva wrote:
> Hi!
> 
> Let's say I have two lists of equal length but with a variable number of 
> elements. For ex.:
> 
> l1=['a','b','c']
> l2=['j','k','l']
> 
> I want to build a string like this
> "foo a j, b k, c l bar"
> 
> Is it possible to achieve this with f strings or any other 
> simple/efficient way?

'foo ' + ', '.join(f"{one} {two}" for one,two in zip(l1,l2)) + ' bar'


From narshad.380 at gmail.com  Wed Feb  9 02:46:15 2022
From: narshad.380 at gmail.com (NArshad)
Date: Tue, 8 Feb 2022 23:46:15 -0800 (PST)
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
Message-ID: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>

When I enter data using Tkinter form in an Excel file when the excel file is closed there is no error but when I enter data using Tkinter form when the excel is already open following error comes:



Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Users\Dani Brothers\Anaconda3\lib\tkinter\__init__.py", line 1705, in __call__
    return self.func(*args)
  File "D:/Python/Book Bank/New folder/PyCharm/Final/Excel.py", line 61, in SaveBook
    workbook.save(filename="BookBank.xlsx")
  File "C:\Users\Dani Brothers\Anaconda3\lib\site-packages\openpyxl\workbook\workbook.py", line 392, in save
    save_workbook(self, filename)
  File "C:\Users\Dani Brothers\Anaconda3\lib\site-packages\openpyxl\writer\excel.py", line 291, in save_workbook
    archive = ZipFile(filename, 'w', ZIP_DEFLATED, allowZip64=True)
  File "C:\Users\Dani Brothers\Anaconda3\lib\zipfile.py", line 1207, in __init__
    self.fp = io.open(file, filemode)
PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'



What to do to correct this error? I have already searched on google search many times but no solution was found.

From auriocus at gmx.de  Wed Feb  9 07:45:08 2022
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Wed, 9 Feb 2022 13:45:08 +0100
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
In-Reply-To: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
Message-ID: <su0d1v$ju2$1@dont-email.me>

Am 09.02.22 um 08:46 schrieb NArshad:
> When I enter data using Tkinter form in an Excel file when the excel file is closed there is no error but when I enter data using Tkinter form when the excel is already open following error comes:

> PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
> 
> 
> 
> What to do to correct this error? I have already searched on google search many times but no solution was found.

It's impossible. Excel locks the file deliberately when it is open, so 
that you can't overwrite it from a different program. Otherwise, the 
file could become inconsistent.

The correct way to handle it in the GUI is to tell the user via a 
message box that the file is open and can't be written.

An alternative to writing the file directly would be that you remote 
control Excel; I think it provides a DDE API: 
https://support.microsoft.com/en-us/office/dde-function-79e8b21c-2054-4b48-9ceb-d2cf38dc17f9

	Christian

From akkana at shallowsky.com  Wed Feb  9 10:32:54 2022
From: akkana at shallowsky.com (Akkana Peck)
Date: Wed, 9 Feb 2022 08:32:54 -0700
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
Message-ID: <YgPepryEwuGFG09O@shallowsky.com>

Abdur-Rahmaan Janhangeer writes:
> results = []
> with console.status("Checking internet ...", spinner="dots"):
> for domain in domains:
> try:
> requests.get(domain)
> results.append(1)
> except Exception as e:
> results.append(0)
> if not any(results):
> print('No internet connection')
> sys.exit()
> 
> gist link:
> https://gist.github.com/Abdur-rahmaanJ/7917dc5ab7f5d2aa37b2723909be08f7
> 
> I think for me having the internet means ability to request urls

Request urls and get the right answer?

This won't work if you're behind a captive portal: every URL you
try to get will return successfully, but the content will be
the captive portal page.

You need to compare the output of at least one of those pages to
known output to be sure you're really connected.

        ...Akkana

From jenkris at tutanota.com  Wed Feb  9 10:44:31 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Wed, 9 Feb 2022 16:44:31 +0100 (CET)
Subject: Can't get iterator in the C API
In-Reply-To: <58f81cfe-1843-32bc-3b2a-b6919b2c308a@mrabarnett.plus.com>
References: <MvR1O5n--3-2@tutanota.com>
 <58f81cfe-1843-32bc-3b2a-b6919b2c308a@mrabarnett.plus.com>
Message-ID: <MvU94eb--7-2@tutanota.com>

Thank you for clarifying that.? Now on to getting the iterator from the method.? 

Jen


Feb 8, 2022, 18:10 by python at mrabarnett.plus.com:

> On 2022-02-09 01:12, Jen Kris via Python-list wrote:
>
>> I am using the Python C API to load the Gutenberg corpus from the nltk library and iterate through the sentences.? The Python code I am trying to replicate is:
>>
>> from nltk.corpus import gutenberg
>> for i, fileid in enumerate(gutenberg.fileids()):
>>  ??????? sentences = gutenberg.sents(fileid)
>>  ??????? etc
>>
>> where gutenberg.fileids is, of course, iterable.
>>
>> I use the following C API code to import the module and get pointers:
>>
>> int64_t Call_PyModule()
>> {
>>  ??? PyObject *pModule, *pName, *pSubMod, *pFidMod, *pFidSeqIter,*pSentMod;
>>
>>  ??? pName = PyUnicode_FromString("nltk.corpus");
>>  ??? pModule = PyImport_Import(pName);
>>
>>  ??? if (pModule == 0x0){
>>  ??????? PyErr_Print();
>>  ??????? return 1; }
>>
>>  ??? pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>>  ??? pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>>  ??? pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>>
>>  ??? pFidIter = PyObject_GetIter(pFidMod);
>>  ??? int ckseq_ok = PySeqIter_Check(pFidMod);
>>  ??? pFidSeqIter? = PySeqIter_New(pFidMod);
>>
>>  ??? return 0;
>> }
>>
>> pSubMod, pFidMod and pSentMod all return valid pointers, but the iterator lines return zero:
>>
>> pFidIter = PyObject_GetIter(pFidMod);
>> int ckseq_ok = PySeqIter_Check(pFidMod);
>> pFidSeqIter? = PySeqIter_New(pFidMod);
>>
>> So the C API thinks gutenberg.fileids is not iterable, but it is.? What am I doing wrong?
>>
> Look at your Python code. You have "gutenberg.fileids()", so the 'fileids' attribute is not an iterable itself, but a method that you need to call to get the iterable.
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From jkn_gg at nicorp.f9.co.uk  Wed Feb  9 10:24:30 2022
From: jkn_gg at nicorp.f9.co.uk (jkn)
Date: Wed, 9 Feb 2022 07:24:30 -0800 (PST)
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
In-Reply-To: <su0d1v$ju2$1@dont-email.me>
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
 <su0d1v$ju2$1@dont-email.me>
Message-ID: <61cde595-c553-44b4-8392-2cae9ed1d78en@googlegroups.com>

On Wednesday, February 9, 2022 at 12:46:49 PM UTC, Christian Gollwitzer wrote:
> Am 09.02.22 um 08:46 schrieb NArshad:
> > When I enter data using Tkinter form in an Excel file when the excel file is closed there is no error but when I enter data using Tkinter form when the excel is already open following error comes:
> > PermissionError: [Errno 13] Permission denied: 'Abc.xlsx' 
> > 
> > 
> > 
> > What to do to correct this error? I have already searched on google search many times but no solution was found.
> It's impossible. Excel locks the file deliberately when it is open, so 
> that you can't overwrite it from a different program. Otherwise, the 
> file could become inconsistent. 
> 
> The correct way to handle it in the GUI is to tell the user via a 
> message box that the file is open and can't be written. 
> 
> An alternative to writing the file directly would be that you remote 
> control Excel; I think it provides a DDE API: 
> https://support.microsoft.com/en-us/office/dde-function-79e8b21c-2054-4b48-9ceb-d2cf38dc17f9 
> 
DDE? Wow, that takes me back...

From david at lowryduda.com  Wed Feb  9 11:00:02 2022
From: david at lowryduda.com (David Lowry-Duda)
Date: Wed, 9 Feb 2022 11:00:02 -0500
Subject: Unpacking lists in a f string
In-Reply-To: <stv887$ics$1@gioia.aioe.org>
References: <stv887$ics$1@gioia.aioe.org>
Message-ID: <YgPlAhLmC5qqtHhS@icerm-dld>

> l1=['a','b','c']
> l2=['j','k','l']
> 
> I want to build a string like this
> "foo a j, b k, c l bar"
> Is it possible to achieve this with f strings or any other 
> simple/efficient way?

Here is a small list of things that want to be done (and natural ways to 
perform them)

1. pair items in the lists (using list comprehension),
2. format the pairs (using an f-string),
3. separate the pairs with commas (using join), and
4. incorporate surrounding "foo" "bar" text. (using an f-string)

It would be possible to have

    print(f"foo {', '.join(f'{first} {second}' for first, second in zip(l1, l2))} bar")

But I find nested f-strings to be confusing, and avoid them. This is a 
case where I actually prefer format.

    print("foo {} bar".format(
        ', '.join(f'{first} {second}' for first, second in zip(l1, l2))
    ))

It's still a one-liner, but it's a one liner that I find easier to 
parse. Or perhaps I would explicitly construct the string first and then 
use it?

    innerstr = ', '.join(f'{first} {second} for first, second in zip(l1, l2))
    print(f"foo {innerstr} bar")

These ideas are what feel natural to me.

- DLD

From arj.python at gmail.com  Wed Feb  9 12:21:11 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Wed, 9 Feb 2022 21:21:11 +0400
Subject: Best way to check if there is internet?
In-Reply-To: <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CAPTjJmog2LaVSqLX=6C2azDw_ngTMKkVaiVPPWBHq9g4DXpcNg@mail.gmail.com>
 <CADrxXXkeZmCKGCvCAH4Vcjj2F3-BkkMjUtF=1bo-W0KpGvd2AA@mail.gmail.com>
 <mailman.26.1644226448.7010.python-list@python.org>
 <tsi20hdfug18l5uvp7nfdgun4hq9k8n1ig@4ax.com>
Message-ID: <CADrxXXmH96L+HDHwf-tvX36b4z+DTiKikQvYLg1aBBbFERO04Q@mail.gmail.com>

>  Ah, but WHEN do those browsers report that? When attempting to connect
to whatever the default "home" page has been set to? (Mine is configured to
use https://www.google.com as the default page -- if my router is down,
obviously the browser will time-out waiting for a response from Google, and
report "no network").

Yes you are right ... lol i misread the error i guess

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From arj.python at gmail.com  Wed Feb  9 12:24:12 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Wed, 9 Feb 2022 21:24:12 +0400
Subject: Best way to check if there is internet?
In-Reply-To: <YgPepryEwuGFG09O@shallowsky.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <YgPepryEwuGFG09O@shallowsky.com>
Message-ID: <CADrxXXmcD5NXG2B6S8m0JhLrXCPiwth1cCjpY1Xmu=MUGOZ9hg@mail.gmail.com>

> This won't work if you're behind a captive portal: every URL you
try to get will return successfully, but the content will be
the captive portal page.

Yes agree a pretty common scenario if you are moving around ...
The solution is to test against a webpage you know the content
will always be the same. Thanks for pointing out!

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From dieter at handshake.de  Wed Feb  9 12:46:06 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Wed, 9 Feb 2022 18:46:06 +0100
Subject: Correct way to setup a package with both compiled C code and
 Python code?
In-Reply-To: <stuo6r$10o$1@dont-email.me>
References: <strs6d$nqq$1@dont-email.me>
 <25090.44837.132493.228601@ixdm.fritz.box>
 <mailman.54.1644343084.7010.python-list@python.org>
 <stuo6r$10o$1@dont-email.me>
Message-ID: <25091.64990.74396.410926@ixdm.fritz.box>

Christian Gollwitzer wrote at 2022-2-8 22:43 +0100:
>Am 08.02.22 um 18:57 schrieb Dieter Maurer:
>> Christian Gollwitzer wrote at 2022-2-7 20:33 +0100:
>>> we've developed a Python pacakge which consists of both a compiled
>>> extension module and some helper functions in Python. Is there a
>>> tutorial on how to package such an extension?
>>
>> Look at "https://package.python.org",
>> especially "https://packaging.python.org/en/latest/guides/packaging-binary-extensions/".
>
>Thank you, but that page is more like a high-level description, it talks
>a lot about the differences between C code and Python and how it can be
>combined using SWIG etc, but I already have a working extension.

Packaging and distribution was originally handled by `distutils`;
other tools, e.g. `setuptools`, extended `distutils` and have
gained wide spread acceptance. As a consequence, `distutils`
has been deprecated in favor of those other tools.
I assume the use of `setuptools` below.

Extensions are described via the `ext_modules` parameter of
the `setup` function. Its value is a sequence of
`setuptools.Extension` instances.

As of Python 3.11, `setuptools.Extension` is still a minor enhancement
of `distutils.extension.Extension`. This is documented via
docstrings in its source.
Read this documentation and come back if questions remain.

From python at mrabarnett.plus.com  Wed Feb  9 13:50:12 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 9 Feb 2022 18:50:12 +0000
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
In-Reply-To: <su0d1v$ju2$1@dont-email.me>
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
 <su0d1v$ju2$1@dont-email.me>
Message-ID: <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>

On 2022-02-09 12:45, Christian Gollwitzer wrote:
> Am 09.02.22 um 08:46 schrieb NArshad:
>> When I enter data using Tkinter form in an Excel file when the excel file is closed there is no error but when I enter data using Tkinter form when the excel is already open following error comes:
> 
>> PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
>> 
>> 
>> 
>> What to do to correct this error? I have already searched on google search many times but no solution was found.
> 
> It's impossible. Excel locks the file deliberately when it is open, so
> that you can't overwrite it from a different program. Otherwise, the
> file could become inconsistent.
> 
It's the same the other way too; you can't open the file in Excel while 
Python has it open.

> The correct way to handle it in the GUI is to tell the user via a
> message box that the file is open and can't be written.
> 
> An alternative to writing the file directly would be that you remote
> control Excel; I think it provides a DDE API:
> https://support.microsoft.com/en-us/office/dde-function-79e8b21c-2054-4b48-9ceb-d2cf38dc17f9
> 

From p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt  Wed Feb  9 13:13:33 2022
From: p_d_a_s_i_l_v_a_ns at nonetnoaddress.pt (Paulo da Silva)
Date: Wed, 9 Feb 2022 18:13:33 +0000
Subject: Unpacking lists in a f string
References: <stv887$ics$1@gioia.aioe.org>
Message-ID: <su108d$1l9r$1@gioia.aioe.org>

?s 02:17 de 09/02/22, Paulo da Silva escreveu:
> Hi!
> 
> Let's say I have two lists of equal length but with a variable number of 
> elements. For ex.:
> 
> l1=['a','b','c']
> l2=['j','k','l']
> 
> I want to build a string like this
> "foo a j, b k, c l bar"
> 
> Is it possible to achieve this with f strings or any other 
> simple/efficient way?
> 
> Thanks for any help/comments.

Thank you for your responses.

From wlfraed at ix.netcom.com  Wed Feb  9 14:18:46 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Wed, 09 Feb 2022 14:18:46 -0500
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
Message-ID: <na280hhqstkv8vkvtqf3ksm9gu3d14sh9v@4ax.com>

On Tue, 8 Feb 2022 23:46:15 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>When I enter data using Tkinter form in an Excel file when the excel file is closed there is no error but when I enter data using Tkinter form when the excel is already open following error comes:
>
<SNIP>

>PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
>

>What to do to correct this error? I have already searched on google search many times but no solution was found.

	Well -- at the basic level... Make sure that only ONE user/application
has access to the file at any given moment.

	Excel (and spreadsheets opened by it) are single-user applications for
a reason -- to prevent the data in the file from being changed without the
application knowing about it. They are not databases designed to allow
multiple users to make changes.

	YOU will have to implement transaction control over the file. At the
rudimentary level (since you can't change how Excel itself operates)
whenever you intend to save changes to the data you have to notify any user
that has the file open that they need to close/save the file; when that has
been done you have to reread the file (because they may have made changes
to the contents that you haven't seen yet), reapply any of your changes (if
still valid conditions), then save the file. You can then notify the other
users that the file is updated and can be opened by them. If you don't
reread/validate the data before applying your changes, you would wipe out
any changes made by others.

	Now, if you have total control over the applications that will access
the file (which essentially means: NOBODY will use Excel itself) you could
write a client/server scheme. In this scheme you would have one process
(the server) as the only program that does anything with the spreadsheet
file and its contents. Your client programs would connect (TCP sockets most
likely) to the server process and send it "commands" (something to be
defined/documented is an interface control document); the server would
parse the commands and implement changes to the file data, and/or return
any requested data to the client. That, at least avoids the file level
conflicts. You still have to figure out how to handle the case where two
clients try to update the same record (record level locking). 

	One possibility is to have every record contain a time-stamp of the
last change -- and I'm going to assume the "commands" are record based, not
cell based -- and the command protocol for update sends back the time-stamp
originally read; the server code would compare the time-stamp with what
that record has in the file -- if they are the same, update the entire
record including a new time-stamp; if different, return a "conflict" status
with the current state of the record (with time-stamp) -- the client can
then compare the new record with the stale one, make any changes, and retry
the update. NOTE: this takes care of single record conflicts, but will help
if a "transaction" has to update multiple records since there is no history
to allow "all or none succeed" logic to be implemented.

"Commands"
	READREC <recno>
				returns status (no such record) and (if valid) specified
record including last time-stamp
	WRITEREC <recno> <record including last time-stamp>
				returns status (update conflict and current record contents
with time-stamp); if recno is unused, can pass 0 for timestamp to write new
record; upon write the time-stamp is updated
	UPDATE
				basically same as WRITEREC
	FIND <column list> <find-string>
				returns list of recno for records matching the search (not
the records themselves)
	

	A proper client/server database handles most of the problem of keeping
the data (file) uncorrupted... Though clients attempting to update the same
record will have one succeed, the other get an exception -- which would be
handled by: rereading the record(s) one attempted to update, confirming the
conditions for the update are still valid, and reapplying the update.
Multiple record updates within a transaction are possible.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From martinp.dipaola at gmail.com  Wed Feb  9 14:38:23 2022
From: martinp.dipaola at gmail.com (Martin Di Paola)
Date: Wed, 9 Feb 2022 19:38:23 +0000
Subject: How do you log in your projects?
In-Reply-To: <CABbU2U-jTZXzNkG89q0PbXJuYLVpiOwJMR43bMMOZDnaRxu=kA@mail.gmail.com>
References: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>
 <CABbU2U-jTZXzNkG89q0PbXJuYLVpiOwJMR43bMMOZDnaRxu=kA@mail.gmail.com>
Message-ID: <20220209193823.ez7ewt4sr7joplix@gmail.com>

>> - On a line per line basis? on a function/method basis?

In general I prefer logging line by line instead per function.

It is easy to add a bunch of decorators to the functions and get the 
logs of all the program but I most of the time I end up with very 
confusing logs.

There are exceptions, yes, but I prefer the line by line where the log 
should explain what is doing the code.

>> - Which kind of variable contents do you write into your logfiles?
>> - How do you decide, which kind of log message goes into which level?
>> - How do you prevent logging cluttering your actual code?

These three comes to the same answer: I think on whom is going to read 
the logs.

If the logs are meant to be read by my users I log high level messages,
specially before parts that can take a while (like the classic 
"Loading...").

If I log variables, those must be the ones set by the users so he/she 
can understand how he/she is controlling the behaviour of the program.

For exceptions I print the message but not the traceback. Across the 
code tag some important functions to put an extra message that will 
enhance the final message printed to the user.

https://github.com/byexamples/byexample/blob/master/byexample/common.py#L192-L238

For example:

     for example in examples:
         with enhance_exceptions(example, ...):
             foo()

So if an exception is raised by foo(), enhance_exceptions() will attach 
to it useful information for the user from the example variable.

In the main, then I do the pretty print
https://github.com/byexamples/byexample/blob/master/byexample/byexample.py#L17-L22

If the user of the logs is me or any other developer I write more debugging stuff.

My approach is to not log anything and when I have to debug something 
I use a debugger + some prints. When the issue is fixed I review which 
prints would be super useful and I turn them into logs and the rest is 
deleted.


On Tue, Feb 08, 2022 at 09:40:07PM +0100, Marco Sulla wrote:
>These are a lot of questions. I hope we're not off topic.
>I don't know if mine are best practices. I can tell what I try to do.
>
>On Tue, 8 Feb 2022 at 15:15, Lars Liedtke <liedtke at punkt.de> wrote:
>> - On a line per line basis? on a function/method basis?
>
>I usually log the start and end of functions. I could also log inside
>a branch or in other parts of the function/method.
>
>> - Do you use decorators to mark beginnings and ends of methods/functions
>> in log files?
>
>No, since I put the function parameters in the first log. But I think
>that such a decorator it's not bad.
>
>> - Which kind of variable contents do you write into your logfiles? Of
>> course you shouldn't leak secrets...
>
>Well, all the data that is useful to understand what the code is
>doing. It's better to repeat the essential data to identify a specific
>call in all the logs of the function, so if it is called
>simultaneously by more clients you can distinguish them
>
>> - How do you decide, which kind of log message goes into which level?
>
>It depends on the importance, the verbosity and the occurrences of the logs.
>
>> - How do you prevent logging cluttering your actual code?
>
>I have the opposite problem, I should log more. So I can't answer your question.
>-- 
>https://mail.python.org/mailman/listinfo/python-list

From kvratkin at yandex.ru  Wed Feb  9 14:38:48 2022
From: kvratkin at yandex.ru (Kirill Ratkin)
Date: Wed, 9 Feb 2022 22:38:48 +0300
Subject: Unpacking lists in a f string
In-Reply-To: <su108d$1l9r$1@gioia.aioe.org>
References: <stv887$ics$1@gioia.aioe.org> <su108d$1l9r$1@gioia.aioe.org>
Message-ID: <78619ba8-bb4e-5eca-a26e-95a14ce70a12@yandex.ru>

Hi.

Try this:

f"foo {','.join([f'{a} {b}' for a,b in list(zip(l1,l2))])} bar"

09.02.2022 21:13, Paulo da Silva ?????:
> ?s 02:17 de 09/02/22, Paulo da Silva escreveu:
>> Hi!
>>
>> Let's say I have two lists of equal length but with a variable number 
>> of elements. For ex.:
>>
>> l1=['a','b','c']
>> l2=['j','k','l']
>>
>> I want to build a string like this
>> "foo a j, b k, c l bar"
>>
>> Is it possible to achieve this with f strings or any other 
>> simple/efficient way?
>>
>> Thanks for any help/comments.
>
> Thank you for your responses.

From jenkris at tutanota.com  Wed Feb  9 19:40:59 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Thu, 10 Feb 2022 01:40:59 +0100 (CET)
Subject: C API  PyObject_Call segfaults with string
Message-ID: <MvW3s4j--3-2@tutanota.com>

This is a follow-on to a question I asked yesterday, which was answered by MRAB.?? I'm using the Python C API to load the Gutenberg corpus from the nltk library and iterate through the sentences.? The Python code I am trying to replicate is:

from nltk.corpus import gutenberg
for i, fileid in enumerate(gutenberg.fileids()):
??????? sentences = gutenberg.sents(fileid)
??????? etc

I have everything finished down to the last line (sentences = gutenberg.sents(fileid)) where I use? PyObject_Call to call gutenberg.sents, but it segfaults.? The fileid is a string -- the first fileid in this corpus is "austen-emma.txt."??

pName = PyUnicode_FromString("nltk.corpus");
pModule = PyImport_Import(pName);

pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
pSentMod = PyObject_GetAttrString(pSubMod, "sents");

pFileIds = PyObject_CallObject(pFidMod, 0);
pListItem = PyList_GetItem(pFileIds, listIndex);
pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
pListStr = PyBytes_AS_STRING(pListStrE);
Py_DECREF(pListStrE);

// sentences = gutenberg.sents(fileid)
PyObject *c_args = Py_BuildValue("s", pListStr);??
PyObject *NullPtr = 0;
pSents = PyObject_Call(pSentMod, c_args, NullPtr);

The final line segfaults:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6e4e8d5 in _PyEval_EvalCodeWithName ()
?? from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0

My guess is the problem is in Py_BuildValue, which returns a pointer but it may not be constructed correctly.? I also tried it with "O" and it doesn't segfault but it returns 0x0.?

I'm new to using the C API.? Thanks for any help.?

Jen



From songofacandy at gmail.com  Wed Feb  9 19:52:34 2022
From: songofacandy at gmail.com (Inada Naoki)
Date: Thu, 10 Feb 2022 09:52:34 +0900
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <MvW3s4j--3-2@tutanota.com>
References: <MvW3s4j--3-2@tutanota.com>
Message-ID: <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>

On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
<python-list at python.org> wrote:
>
> I have everything finished down to the last line (sentences = gutenberg.sents(fileid)) where I use  PyObject_Call to call gutenberg.sents, but it segfaults.  The fileid is a string -- the first fileid in this corpus is "austen-emma.txt."
>
> pName = PyUnicode_FromString("nltk.corpus");
> pModule = PyImport_Import(pName);
>
> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>
> pFileIds = PyObject_CallObject(pFidMod, 0);
> pListItem = PyList_GetItem(pFileIds, listIndex);
> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
> pListStr = PyBytes_AS_STRING(pListStrE);
> Py_DECREF(pListStrE);

HERE.
PyBytes_AS_STRING() returns pointer in the pListStrE Object.
So Py_DECREF(pListStrE) makes pListStr a dangling pointer.

>
> // sentences = gutenberg.sents(fileid)
> PyObject *c_args = Py_BuildValue("s", pListStr);

Why do you encode&decode pListStrE?
Why don't you use just pListStrE?

> PyObject *NullPtr = 0;
> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>

c_args must tuple, but you passed a unicode object here.
Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue


> The final line segfaults:
> Program received signal SIGSEGV, Segmentation fault.
> 0x00007ffff6e4e8d5 in _PyEval_EvalCodeWithName ()
>    from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>
> My guess is the problem is in Py_BuildValue, which returns a pointer but it may not be constructed correctly.  I also tried it with "O" and it doesn't segfault but it returns 0x0.
>
> I'm new to using the C API.  Thanks for any help.
>
> Jen
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list

Bests,

-- 
Inada Naoki  <songofacandy at gmail.com>

From jenkris at tutanota.com  Wed Feb  9 20:05:19 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Thu, 10 Feb 2022 02:05:19 +0100 (CET)
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
Message-ID: <MvW9RVE--3-2@tutanota.com>

Thanks for your reply.? 

I eliminated the DECREF and now it doesn't segfault but it returns 0x0.? Same when I substitute pListStrE for pListStr.? pListStr contains the string representation of the fileid, so it seemed like the one to use.? According to? http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, PyBuildValue "builds a tuple only if its format string contains two or more format units" and that doc contains examples.?


Feb 9, 2022, 16:52 by songofacandy at gmail.com:

> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
> <python-list at python.org> wrote:
>
>>
>> I have everything finished down to the last line (sentences = gutenberg.sents(fileid)) where I use  PyObject_Call to call gutenberg.sents, but it segfaults.  The fileid is a string -- the first fileid in this corpus is "austen-emma.txt."
>>
>> pName = PyUnicode_FromString("nltk.corpus");
>> pModule = PyImport_Import(pName);
>>
>> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>>
>> pFileIds = PyObject_CallObject(pFidMod, 0);
>> pListItem = PyList_GetItem(pFileIds, listIndex);
>> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>> pListStr = PyBytes_AS_STRING(pListStrE);
>> Py_DECREF(pListStrE);
>>
>
> HERE.
> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>
>>
>> // sentences = gutenberg.sents(fileid)
>> PyObject *c_args = Py_BuildValue("s", pListStr);
>>
>
> Why do you encode&decode pListStrE?
> Why don't you use just pListStrE?
>
>> PyObject *NullPtr = 0;
>> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>>
>
> c_args must tuple, but you passed a unicode object here.
> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>
>
>> The final line segfaults:
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00007ffff6e4e8d5 in _PyEval_EvalCodeWithName ()
>>  from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>>
>> My guess is the problem is in Py_BuildValue, which returns a pointer but it may not be constructed correctly.  I also tried it with "O" and it doesn't segfault but it returns 0x0.
>>
>> I'm new to using the C API.  Thanks for any help.
>>
>> Jen
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>
> Bests,
>
> -- 
> Inada Naoki  <songofacandy at gmail.com>
>


From songofacandy at gmail.com  Wed Feb  9 20:08:21 2022
From: songofacandy at gmail.com (Inada Naoki)
Date: Thu, 10 Feb 2022 10:08:21 +0900
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <MvW9RVE--3-2@tutanota.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
Message-ID: <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>

On Thu, Feb 10, 2022 at 10:05 AM Jen Kris <jenkris at tutanota.com> wrote:
>
> Thanks for your reply.
>
> I eliminated the DECREF and now it doesn't segfault but it returns 0x0.  Same when I substitute pListStrE for pListStr.  pListStr contains the string representation of the fileid, so it seemed like the one to use.  According to  http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, PyBuildValue "builds a tuple only if its format string contains two or more format units" and that doc contains examples.
>

Yes, and PyObject_Call accept tuple, not str.


https://docs.python.org/3/c-api/call.html#c.PyObject_Call

>
> Feb 9, 2022, 16:52 by songofacandy at gmail.com:
>
> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
> <python-list at python.org> wrote:
>
>
> I have everything finished down to the last line (sentences = gutenberg.sents(fileid)) where I use PyObject_Call to call gutenberg.sents, but it segfaults. The fileid is a string -- the first fileid in this corpus is "austen-emma.txt."
>
> pName = PyUnicode_FromString("nltk.corpus");
> pModule = PyImport_Import(pName);
>
> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>
> pFileIds = PyObject_CallObject(pFidMod, 0);
> pListItem = PyList_GetItem(pFileIds, listIndex);
> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
> pListStr = PyBytes_AS_STRING(pListStrE);
> Py_DECREF(pListStrE);
>
>
> HERE.
> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>
>
> // sentences = gutenberg.sents(fileid)
> PyObject *c_args = Py_BuildValue("s", pListStr);
>
>
> Why do you encode&decode pListStrE?
> Why don't you use just pListStrE?
>
> PyObject *NullPtr = 0;
> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>
>
> c_args must tuple, but you passed a unicode object here.
> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>
> The final line segfaults:
> Program received signal SIGSEGV, Segmentation fault.
> 0x00007ffff6e4e8d5 in _PyEval_EvalCodeWithName ()
> from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>
> My guess is the problem is in Py_BuildValue, which returns a pointer but it may not be constructed correctly. I also tried it with "O" and it doesn't segfault but it returns 0x0.
>
> I'm new to using the C API. Thanks for any help.
>
> Jen
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
> Bests,
>
> --
> Inada Naoki <songofacandy at gmail.com>
>
>


-- 
Inada Naoki  <songofacandy at gmail.com>

From jenkris at tutanota.com  Wed Feb  9 20:15:20 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Thu, 10 Feb 2022 02:15:20 +0100 (CET)
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
Message-ID: <MvWBjNR--3-2@tutanota.com>

Right you are.? In that case should I use Py_BuildValue and convert to tuple (because it won't return a tuple for a one-arg), or should I just convert pListStr to tuple?? Thanks for your help.? 


Feb 9, 2022, 17:08 by songofacandy at gmail.com:

> On Thu, Feb 10, 2022 at 10:05 AM Jen Kris <jenkris at tutanota.com> wrote:
>
>>
>> Thanks for your reply.
>>
>> I eliminated the DECREF and now it doesn't segfault but it returns 0x0.  Same when I substitute pListStrE for pListStr.  pListStr contains the string representation of the fileid, so it seemed like the one to use.  According to  http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, PyBuildValue "builds a tuple only if its format string contains two or more format units" and that doc contains examples.
>>
>
> Yes, and PyObject_Call accept tuple, not str.
>
>
> https://docs.python.org/3/c-api/call.html#c.PyObject_Call
>
>>
>> Feb 9, 2022, 16:52 by songofacandy at gmail.com:
>>
>> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
>> <python-list at python.org> wrote:
>>
>>
>> I have everything finished down to the last line (sentences = gutenberg.sents(fileid)) where I use PyObject_Call to call gutenberg.sents, but it segfaults. The fileid is a string -- the first fileid in this corpus is "austen-emma.txt."
>>
>> pName = PyUnicode_FromString("nltk.corpus");
>> pModule = PyImport_Import(pName);
>>
>> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>>
>> pFileIds = PyObject_CallObject(pFidMod, 0);
>> pListItem = PyList_GetItem(pFileIds, listIndex);
>> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>> pListStr = PyBytes_AS_STRING(pListStrE);
>> Py_DECREF(pListStrE);
>>
>>
>> HERE.
>> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
>> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>>
>>
>> // sentences = gutenberg.sents(fileid)
>> PyObject *c_args = Py_BuildValue("s", pListStr);
>>
>>
>> Why do you encode&decode pListStrE?
>> Why don't you use just pListStrE?
>>
>> PyObject *NullPtr = 0;
>> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>>
>>
>> c_args must tuple, but you passed a unicode object here.
>> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>>
>> The final line segfaults:
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00007ffff6e4e8d5 in _PyEval_EvalCodeWithName ()
>> from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>>
>> My guess is the problem is in Py_BuildValue, which returns a pointer but it may not be constructed correctly. I also tried it with "O" and it doesn't segfault but it returns 0x0.
>>
>> I'm new to using the C API. Thanks for any help.
>>
>> Jen
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>>
>> Bests,
>>
>> --
>> Inada Naoki <songofacandy at gmail.com>
>>
>
>
> -- 
> Inada Naoki  <songofacandy at gmail.com>
>


From songofacandy at gmail.com  Wed Feb  9 20:23:47 2022
From: songofacandy at gmail.com (Inada Naoki)
Date: Thu, 10 Feb 2022 10:23:47 +0900
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <MvWBjNR--3-2@tutanota.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
Message-ID: <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>

// https://docs.python.org/3/c-api/call.html#c.PyObject_CallNoArgs
// This function is only for one arg. Python >= 3.9 is required.
pSents = PyObject_CallOneArg(pSentMod, pListItem);

Or

// https://docs.python.org/3/c-api/call.html#c.PyObject_CallFunctionObjArgs
// This function can call function with multiple arguments. Can be
used with Python <3.9 too.
pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);

On Thu, Feb 10, 2022 at 10:15 AM Jen Kris <jenkris at tutanota.com> wrote:
>
> Right you are.  In that case should I use Py_BuildValue and convert to tuple (because it won't return a tuple for a one-arg), or should I just convert pListStr to tuple?  Thanks for your help.
>
>
> Feb 9, 2022, 17:08 by songofacandy at gmail.com:
>
> On Thu, Feb 10, 2022 at 10:05 AM Jen Kris <jenkris at tutanota.com> wrote:
>
>
> Thanks for your reply.
>
> I eliminated the DECREF and now it doesn't segfault but it returns 0x0. Same when I substitute pListStrE for pListStr. pListStr contains the string representation of the fileid, so it seemed like the one to use. According to http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, PyBuildValue "builds a tuple only if its format string contains two or more format units" and that doc contains examples.
>
>
> Yes, and PyObject_Call accept tuple, not str.
>
>
> https://docs.python.org/3/c-api/call.html#c.PyObject_Call
>
>
> Feb 9, 2022, 16:52 by songofacandy at gmail.com:
>
> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
> <python-list at python.org> wrote:
>
>
> I have everything finished down to the last line (sentences = gutenberg.sents(fileid)) where I use PyObject_Call to call gutenberg.sents, but it segfaults. The fileid is a string -- the first fileid in this corpus is "austen-emma.txt."
>
> pName = PyUnicode_FromString("nltk.corpus");
> pModule = PyImport_Import(pName);
>
> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>
> pFileIds = PyObject_CallObject(pFidMod, 0);
> pListItem = PyList_GetItem(pFileIds, listIndex);
> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
> pListStr = PyBytes_AS_STRING(pListStrE);
> Py_DECREF(pListStrE);
>
>
> HERE.
> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>
>
> // sentences = gutenberg.sents(fileid)
> PyObject *c_args = Py_BuildValue("s", pListStr);
>
>
> Why do you encode&decode pListStrE?
> Why don't you use just pListStrE?
>
> PyObject *NullPtr = 0;
> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>
>
> c_args must tuple, but you passed a unicode object here.
> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>
> The final line segfaults:
> Program received signal SIGSEGV, Segmentation fault.
> 0x00007ffff6e4e8d5 in _PyEval_EvalCodeWithName ()
> from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>
> My guess is the problem is in Py_BuildValue, which returns a pointer but it may not be constructed correctly. I also tried it with "O" and it doesn't segfault but it returns 0x0.
>
> I'm new to using the C API. Thanks for any help.
>
> Jen
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
> Bests,
>
> --
> Inada Naoki <songofacandy at gmail.com>
>
>
>
> --
> Inada Naoki <songofacandy at gmail.com>
>
>


-- 
Inada Naoki  <songofacandy at gmail.com>

From jenkris at tutanota.com  Wed Feb  9 20:37:00 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Thu, 10 Feb 2022 02:37:00 +0100 (CET)
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
 <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
Message-ID: <MvWGgaJ--B-2@tutanota.com>

I'm using Python 3.8 so I tried your second choice:

pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);

but pSents is 0x0.? pSentMod and pListItem are valid pointers.? 


Feb 9, 2022, 17:23 by songofacandy at gmail.com:

> // https://docs.python.org/3/c-api/call.html#c.PyObject_CallNoArgs
> // This function is only for one arg. Python >= 3.9 is required.
> pSents = PyObject_CallOneArg(pSentMod, pListItem);
>
> Or
>
> // https://docs.python.org/3/c-api/call.html#c.PyObject_CallFunctionObjArgs
> // This function can call function with multiple arguments. Can be
> used with Python <3.9 too.
> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>
> On Thu, Feb 10, 2022 at 10:15 AM Jen Kris <jenkris at tutanota.com> wrote:
>
>>
>> Right you are.  In that case should I use Py_BuildValue and convert to tuple (because it won't return a tuple for a one-arg), or should I just convert pListStr to tuple?  Thanks for your help.
>>
>>
>> Feb 9, 2022, 17:08 by songofacandy at gmail.com:
>>
>> On Thu, Feb 10, 2022 at 10:05 AM Jen Kris <jenkris at tutanota.com> wrote:
>>
>>
>> Thanks for your reply.
>>
>> I eliminated the DECREF and now it doesn't segfault but it returns 0x0. Same when I substitute pListStrE for pListStr. pListStr contains the string representation of the fileid, so it seemed like the one to use. According to http://web.mit.edu/people/amliu/vrut/python/ext/buildValue.html, PyBuildValue "builds a tuple only if its format string contains two or more format units" and that doc contains examples.
>>
>>
>> Yes, and PyObject_Call accept tuple, not str.
>>
>>
>> https://docs.python.org/3/c-api/call.html#c.PyObject_Call
>>
>>
>> Feb 9, 2022, 16:52 by songofacandy at gmail.com:
>>
>> On Thu, Feb 10, 2022 at 9:42 AM Jen Kris via Python-list
>> <python-list at python.org> wrote:
>>
>>
>> I have everything finished down to the last line (sentences = gutenberg.sents(fileid)) where I use PyObject_Call to call gutenberg.sents, but it segfaults. The fileid is a string -- the first fileid in this corpus is "austen-emma.txt."
>>
>> pName = PyUnicode_FromString("nltk.corpus");
>> pModule = PyImport_Import(pName);
>>
>> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>>
>> pFileIds = PyObject_CallObject(pFidMod, 0);
>> pListItem = PyList_GetItem(pFileIds, listIndex);
>> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>> pListStr = PyBytes_AS_STRING(pListStrE);
>> Py_DECREF(pListStrE);
>>
>>
>> HERE.
>> PyBytes_AS_STRING() returns pointer in the pListStrE Object.
>> So Py_DECREF(pListStrE) makes pListStr a dangling pointer.
>>
>>
>> // sentences = gutenberg.sents(fileid)
>> PyObject *c_args = Py_BuildValue("s", pListStr);
>>
>>
>> Why do you encode&decode pListStrE?
>> Why don't you use just pListStrE?
>>
>> PyObject *NullPtr = 0;
>> pSents = PyObject_Call(pSentMod, c_args, NullPtr);
>>
>>
>> c_args must tuple, but you passed a unicode object here.
>> Read https://docs.python.org/3/c-api/arg.html#c.Py_BuildValue
>>
>> The final line segfaults:
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00007ffff6e4e8d5 in _PyEval_EvalCodeWithName ()
>> from /usr/lib/x86_64-linux-gnu/libpython3.8.so.1.0
>>
>> My guess is the problem is in Py_BuildValue, which returns a pointer but it may not be constructed correctly. I also tried it with "O" and it doesn't segfault but it returns 0x0.
>>
>> I'm new to using the C API. Thanks for any help.
>>
>> Jen
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>>
>> Bests,
>>
>> --
>> Inada Naoki <songofacandy at gmail.com>
>>
>>
>>
>> --
>> Inada Naoki <songofacandy at gmail.com>
>>
>
>
> -- 
> Inada Naoki  <songofacandy at gmail.com>
>


From songofacandy at gmail.com  Wed Feb  9 20:40:35 2022
From: songofacandy at gmail.com (Inada Naoki)
Date: Thu, 10 Feb 2022 10:40:35 +0900
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <MvWGgaJ--B-2@tutanota.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
 <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
 <MvWGgaJ--B-2@tutanota.com>
Message-ID: <CAEfz+Tx4rpN6+fqrPMd-oeuNxitGPbGf5NYCqZCdAJa6htW1gQ@mail.gmail.com>

On Thu, Feb 10, 2022 at 10:37 AM Jen Kris <jenkris at tutanota.com> wrote:
>
> I'm using Python 3.8 so I tried your second choice:
>
> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>
> but pSents is 0x0.  pSentMod and pListItem are valid pointers.
>

It means exception happened.
If you are writing Python/C function, return NULL (e.g. `if (pSents ==
NULL) return NULL`)
Then Python show the exception and traceback for you.

-- 
Inada Naoki  <songofacandy at gmail.com>

From avigross at verizon.net  Wed Feb  9 20:11:40 2022
From: avigross at verizon.net (Avi Gross)
Date: Thu, 10 Feb 2022 01:11:40 +0000 (UTC)
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXmcD5NXG2B6S8m0JhLrXCPiwth1cCjpY1Xmu=MUGOZ9hg@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <YgPepryEwuGFG09O@shallowsky.com>
 <CADrxXXmcD5NXG2B6S8m0JhLrXCPiwth1cCjpY1Xmu=MUGOZ9hg@mail.gmail.com>
Message-ID: <1236667679.90129.1644455500101@mail.yahoo.com>

Actually, you may want a dynamic page. Pages may sometimes be delivered from some cache along the way perhaps within your own company firewall if someone else recently accessed them.
Consider a page that returns the current time or like the following asks for an arithmetical calculation to add 5 and 6
https://search.yahoo.com/search?fr=mcafee&type=E211US1249G0&p=%3D5%2B6
Not a great example, and it may not work for you, but there likely is a site that can exist that accepts a request for some formatted calculation that you can specify dynamically. You can compare a date/time returned to being accurate within some seconds and asking for n*m can obviously be validated.
Of course, once such a ploy is well known, it can be intercepted and sent proper results and still fool you into assuming you had full internet access.?
So a site that you can send to and receive back using a cryptographic method might be better.



-----Original Message-----
From: Abdur-Rahmaan Janhangeer <arj.python at gmail.com>
To: Python <python-list at python.org>
Sent: Wed, Feb 9, 2022 12:24 pm
Subject: Re: Best way to check if there is internet?

> This won't work if you're behind a captive portal: every URL you
try to get will return successfully, but the content will be
the captive portal page.

Yes agree a pretty common scenario if you are moving around ...
The solution is to test against a webpage you know the content
will always be the same. Thanks for pointing out!

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius
-- 
https://mail.python.org/mailman/listinfo/python-list

From jenkris at tutanota.com  Wed Feb  9 20:47:51 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Thu, 10 Feb 2022 02:47:51 +0100 (CET)
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <CAEfz+Tx4rpN6+fqrPMd-oeuNxitGPbGf5NYCqZCdAJa6htW1gQ@mail.gmail.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
 <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
 <MvWGgaJ--B-2@tutanota.com>
 <CAEfz+Tx4rpN6+fqrPMd-oeuNxitGPbGf5NYCqZCdAJa6htW1gQ@mail.gmail.com>
Message-ID: <MvWJAZS--3-2@tutanota.com>

I'll do that and post back tomorrow.? The office is closing and I have to leave now (I'm in Seattle).? Thanks again for your help.? 


Feb 9, 2022, 17:40 by songofacandy at gmail.com:

> On Thu, Feb 10, 2022 at 10:37 AM Jen Kris <jenkris at tutanota.com> wrote:
>
>>
>> I'm using Python 3.8 so I tried your second choice:
>>
>> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>>
>> but pSents is 0x0.  pSentMod and pListItem are valid pointers.
>>
>
> It means exception happened.
> If you are writing Python/C function, return NULL (e.g. `if (pSents ==
> NULL) return NULL`)
> Then Python show the exception and traceback for you.
>
> -- 
> Inada Naoki  <songofacandy at gmail.com>
>


From python at mrabarnett.plus.com  Wed Feb  9 21:43:45 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 10 Feb 2022 02:43:45 +0000
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <MvWGgaJ--B-2@tutanota.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
 <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
 <MvWGgaJ--B-2@tutanota.com>
Message-ID: <36767883-dafa-84f0-ee65-478965a6ae8f@mrabarnett.plus.com>

On 2022-02-10 01:37, Jen Kris via Python-list wrote:
> I'm using Python 3.8 so I tried your second choice:
> 
> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
> 
> but pSents is 0x0.? pSentMod and pListItem are valid pointers.
> 
'PyObject_CallFunction' looks like a good one to use:

"""PyObject* PyObject_CallFunction(PyObject *callable, const char 
*format, ...)

Call a callable Python object callable, with a variable number of C 
arguments. The C arguments are described using a Py_BuildValue() style 
format string. The format can be NULL, indicating that no arguments are 
provided.
"""

[snip]

What I do is add comments to keep track of what objects I have 
references to at each point and whether they are new references or could 
be NULL.

For example:

     pName = PyUnicode_FromString("nltk.corpus");
     //> pName+?

This means that 'pName' contains a reference, '+' means that it's a new 
reference, and '?' means that it could be NULL (usually due to an 
exception, but not always) so I need to check it.

Continuing in this vein:

     pModule = PyImport_Import(pName);
     //> pName+? pModule+?

     pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
     //> pName+? pModule+? pSubMod+?
     pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
     //> pName+? pModule+? pSubMod+? pFidMod+?
     pSentMod = PyObject_GetAttrString(pSubMod, "sents");
     //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+?

     pFileIds = PyObject_CallObject(pFidMod, 0);
     //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+?
     pListItem = PyList_GetItem(pFileIds, listIndex);
     //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+? pListItem?
     pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
     //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? 
PyObject_CallObject+? pListItem? pListStrE+?

As you can see, there's a lot of leaked references building up.

Note how after:

     pListItem = PyList_GetItem(pFileIds, listIndex);

the addition is:

     //> pListItem?

This means that 'pListItem' contains a borrowed (not new) reference, but 
could be NULL.

I find it easiest to DECREF as soon as I no longer need the reference 
and remove a name from the list as soon I no longer need it (and 
DECREFed where).

For example:

     pName = PyUnicode_FromString("nltk.corpus");
     //> pName+?
     if (!pName)
         goto error;
     //> pName+
     pModule = PyImport_Import(pName);
     //> pName+ pModule+?
     Py_DECREF(pName);
     //> pModule+?
     if (!pModule)
         goto error;
     //> pModule+

I find that doing this greatly reduces the chances of getting the 
reference counting wrong, and I can remove the comments once I've 
finished the function I'm writing.

From narshad.380 at gmail.com  Thu Feb 10 01:40:48 2022
From: narshad.380 at gmail.com (NArshad)
Date: Wed, 9 Feb 2022 22:40:48 -0800 (PST)
Subject: How to solve the given problem?
Message-ID: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>


Assume that there is a pattern of feeding for a special fish in a day (10 hours a day) as below:
                               150    100    30    30    30    20    20    10    5    5
Today, the fish is fed in the second hour 60 unit instead of 100 unit Accidently. Implement some methods to distribute the remaining 40 unit in the rest of the day and propose the new patterns. Try to keep the distribution similar to the current feeding pattern. 
Note: pay attention that the total feeding amounts should be fix in a day.

From auriocus at gmx.de  Thu Feb 10 02:30:15 2022
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Thu, 10 Feb 2022 08:30:15 +0100
Subject: How to solve the given problem?
In-Reply-To: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
References: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
Message-ID: <su2eu8$58b$1@dont-email.me>

Am 10.02.22 um 07:40 schrieb NArshad:
> 
> Assume that there is a pattern of feeding for a special fish in a day (10 hours a day) as below:
>                                 150    100    30    30    30    20    20    10    5    5
> Today, the fish is fed in the second hour 60 unit instead of 100 unit Accidently. Implement some methods to distribute the remaining 40 unit in the rest of the day and propose the new patterns. Try to keep the distribution similar to the current feeding pattern.
> Note: pay attention that the total feeding amounts should be fix in a day.

This is not a Python problem, it's a math problem and most probably a 
homework problem. Actually the question already tells you how to solve 
it. There are 40 units of fish-food left and you should distribute them 
proportionally to the rest of the day. Sum up the numbers from 3rd to 
last, add 40, and then distribute this to proportionally to each day.

You'll end up with fractinoal numbers in the general case, so you'll 
have to find a method to fairly distribute the units, if you wan to 
stick with integers.

You can also check out the various algorithms for distributing seats in 
a parliament, it is almost the same problem.

	Christian

From rosuav at gmail.com  Thu Feb 10 03:40:55 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 10 Feb 2022 19:40:55 +1100
Subject: How to solve the given problem?
In-Reply-To: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
References: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
Message-ID: <CAPTjJmoAXy5wuHEbgbbL3KadjSx+3YJ-a3OUtAqpY2wSSRVefQ@mail.gmail.com>

On Thu, 10 Feb 2022 at 18:41, NArshad <narshad.380 at gmail.com> wrote:
>
>
> Assume that there is a pattern of feeding for a special fish in a day (10 hours a day) as below:
>                                150    100    30    30    30    20    20    10    5    5
> Today, the fish is fed in the second hour 60 unit instead of 100 unit Accidently. Implement some methods to distribute the remaining 40 unit in the rest of the day and propose the new patterns. Try to keep the distribution similar to the current feeding pattern.
> Note: pay attention that the total feeding amounts should be fix in a day.

Once again, you're bringing your homework to a mailing list and
expecting us to do it for you.

Start by putting in some actual work yourself, and stop trying to
cheat your way to a good grade.

Please, can people not assist this person until there's some
demonstration of actual work being done?

ChrisA

From igorbasko at gmail.com  Thu Feb 10 03:20:40 2022
From: igorbasko at gmail.com (Igor Basko)
Date: Thu, 10 Feb 2022 10:20:40 +0200
Subject: Multiple inheritance using super() in parent classes
Message-ID: <CACXVpJBDN3_ed7T-4XEYR-JWR7xYXMqL6LFpj2L1AXcZYX26KQ@mail.gmail.com>

Hi everyone,
This is my first question here. Hope to get some clarification.
Basically this question is about multiple inheritance and the usage of
super().__init__ in parent
classes.

So I have two classes that inherit from the same base class.
For example class B and class C inherit from A:
class A:
  def __init__(self, arg1):
    pass

class B(A):
  def __init__(self, arg2):
    super().__init__(arg2)

class C(A):
  def __init__(self, arg1, arg2):
    super().__init__(arg2)

Now I would like to create a new class D that inherits from B and C.
One note, D is the only class that I am "allowed" to change. A, B and C are
provided to me as is from an external package.
class D(B, C):
  def __init__(self):
    B.__init__(self, 'arg1')
    C.__init__(self, 'arg1', 'arg2')

When I initialize D I get a TypeError.
TypeError: __init__() missing 1 required positional argument: 'arg2'
I get it from the invocation of super().__init__(arg2) inside the B class.

As I understand it, the super() inside B tries to call the __init__ of
class C,
because of the multiple inheritance and the MRO that is constructed.
But when B was implemented it wasn't aware of C and I assume,
B shouldn't be aware of C in any case.

It gives me the feeling that I'm trying to implement some bad practice
here, but I'm not sure why.

I would also like to hear your suggestions if there is a way to circumvent
it. Maybe by the approach described here:
https://rhettinger.wordpress.com/2011/05/26/super-considered-super/
wrapping B and C in some Adapter class.

Thanks for reading and any help.

From narshad.380 at gmail.com  Thu Feb 10 05:26:57 2022
From: narshad.380 at gmail.com (NArshad)
Date: Thu, 10 Feb 2022 02:26:57 -0800 (PST)
Subject: How to solve the given problem?
In-Reply-To: <mailman.84.1644482467.7010.python-list@python.org>
References: <CAPTjJmoAXy5wuHEbgbbL3KadjSx+3YJ-a3OUtAqpY2wSSRVefQ@mail.gmail.com>
 <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
 <mailman.84.1644482467.7010.python-list@python.org>
Message-ID: <ef4ede55-3e23-483f-887c-b6a45112de1bn@googlegroups.com>

-ChrisA:
You don't reply if you have problems.
When I don't find any solution elsewhere then only I place in this group 


-Christian:
One problem of different type requires the same elaboration.

Q. What technique of statistics or numerical computation or general mathematics to use to solve this problem. Can you tell one example.

Find the values of ?subscript(?) for i from 1 to n (n =100).

?subscript(1) + ?subscript(2) = 3,

?subscript(?+1) ? ?subscript(?+2) = 1, ??? ? = 1, ? , ? ? 2

?subscript(n-1) + ?subscript(n) = 3





From blindanagram at nowhere.org  Thu Feb 10 07:13:44 2022
From: blindanagram at nowhere.org (BlindAnagram)
Date: Thu, 10 Feb 2022 12:13:44 +0000
Subject: Global VS Local Subroutines
Message-ID: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>

Is there any difference in performance between these two program layouts:

    def a():
      ...
    def(b):
      c = a(b)

or

    def(b):
      def a():
        ...
      c = a(b)

I would appreciate any insights on which layout to choose in which 
circumstances.


From loris.bennett at fu-berlin.de  Thu Feb 10 08:14:30 2022
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Thu, 10 Feb 2022 14:14:30 +0100
Subject: Abstraction level at which to create SQLAlchemy ORM object
Message-ID: <87k0e2j22x.fsf@hornfels.zedat.fu-berlin.de>

Hi,

I am writing a command line program which will modify entries in a
database and am trying out SQLAlchemy.

A typical command might look like

  um --operation add --uid ada --gid coders --lang en

Parsing the arguments I get, ignoring the operation, a dict

  {uid: "ada", gid: "coders", lang: "en"}

At some point this needs to be converted into an object of the class User:

  class User(Base):
      __tablename__ = "users"

      uid = Column('uid', String, primary_key=True)
      gid = Column('gid', String)
      lang = Column('lang', String)

In a way it seems it would be economical to do the conversion as early
as possible, so I can just pass around User objects.  However, this
would mean that the entry point for the program would already be tightly
coupled to the specifics of the database interaction.  

On the other hand, delaying the conversion would mean probably having to
define my own User class, which seems like unnecessary overhead.  It
would have the advantage that I could ditch SQLAlchemy more easily if I
find it too mind-bending.

WDYT?

Cheers,

Loris

-- 
This signature is currently under construction.

From rosuav at gmail.com  Thu Feb 10 10:20:33 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 11 Feb 2022 02:20:33 +1100
Subject: Global VS Local Subroutines
In-Reply-To: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
Message-ID: <CAPTjJmrAR=kcW1_n28ww4f55T2rzxh5OFS1tOKmm8qMYN5iwWQ@mail.gmail.com>

On Fri, 11 Feb 2022 at 02:13, BlindAnagram <blindanagram at nowhere.org> wrote:
>
> Is there any difference in performance between these two program layouts:
>
>     def a():
>       ...
>     def(b):
>       c = a(b)
>
> or
>
>     def(b):
>       def a():
>         ...
>       c = a(b)
>
> I would appreciate any insights on which layout to choose in which
> circumstances.
>

Great question! The difference is that, in the second case, a() isn't
available anywhere else. So the real question is: Is that good or bad?

Does it make sense to call a() from outside of your second function,
even for testing? If so, maybe name it _a() so it's private, but keep
it global. Does a() need a lot of information from the context of your
second function? If so, it's easier to have a closure, with one
function inside another.

Both styles make sense, and your question is well put: it's a design
decision with no "right" or "wrong", just different choices with
different implications.

ChrisA

From rosuav at gmail.com  Thu Feb 10 10:23:03 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 11 Feb 2022 02:23:03 +1100
Subject: How to solve the given problem?
In-Reply-To: <ef4ede55-3e23-483f-887c-b6a45112de1bn@googlegroups.com>
References: <CAPTjJmoAXy5wuHEbgbbL3KadjSx+3YJ-a3OUtAqpY2wSSRVefQ@mail.gmail.com>
 <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
 <mailman.84.1644482467.7010.python-list@python.org>
 <ef4ede55-3e23-483f-887c-b6a45112de1bn@googlegroups.com>
Message-ID: <CAPTjJmpSECwJT5wU69yd390ezpPGPqtMakrB+NStTyiDsCNxAw@mail.gmail.com>

On Fri, 11 Feb 2022 at 02:15, NArshad <narshad.380 at gmail.com> wrote:
>
> -ChrisA:
> You don't reply if you have problems.
> When I don't find any solution elsewhere then only I place in this group
>

You're a help vampire. Stop it.

https://slash7.com/2006/12/22/vampires/

Go do some actual research instead of asking people to do your homework.

ChrisA

From __peter__ at web.de  Thu Feb 10 11:39:23 2022
From: __peter__ at web.de (Peter Otten)
Date: Thu, 10 Feb 2022 17:39:23 +0100
Subject: Multiple inheritance using super() in parent classes
In-Reply-To: <CACXVpJBDN3_ed7T-4XEYR-JWR7xYXMqL6LFpj2L1AXcZYX26KQ@mail.gmail.com>
References: <CACXVpJBDN3_ed7T-4XEYR-JWR7xYXMqL6LFpj2L1AXcZYX26KQ@mail.gmail.com>
Message-ID: <465cb192-3479-9104-7d76-df64af053c55@web.de>

On 10/02/2022 09:20, Igor Basko wrote:
> Hi everyone,
> This is my first question here. Hope to get some clarification.
> Basically this question is about multiple inheritance and the usage of
> super().__init__ in parent
> classes.
>
> So I have two classes that inherit from the same base class.
> For example class B and class C inherit from A:
> class A:
>    def __init__(self, arg1):
>      pass
>
> class B(A):
>    def __init__(self, arg2):
>      super().__init__(arg2)
>
> class C(A):
>    def __init__(self, arg1, arg2):
>      super().__init__(arg2)
>
> Now I would like to create a new class D that inherits from B and C.
> One note, D is the only class that I am "allowed" to change. A, B and C are
> provided to me as is from an external package.
> class D(B, C):
>    def __init__(self):
>      B.__init__(self, 'arg1')
>      C.__init__(self, 'arg1', 'arg2')
>
> When I initialize D I get a TypeError.
> TypeError: __init__() missing 1 required positional argument: 'arg2'
> I get it from the invocation of super().__init__(arg2) inside the B class.
>
> As I understand it, the super() inside B tries to call the __init__ of
> class C,
> because of the multiple inheritance and the MRO that is constructed.
> But when B was implemented it wasn't aware of C and I assume,
> B shouldn't be aware of C in any case.

Even when you call the initializers explicitly you pass self, and that's
probably where the MRO is taken from. You can tweak that MRO by changing
the order of the parent classes:

class D(C, B): ...

However, this will call B.__init__() twice; therefore I'd stick to super():

class D(C, B):
     def __init__(self):
         super().__init__("arg1", "arg2")

> It gives me the feeling that I'm trying to implement some bad practice
> here, but I'm not sure why.

I have the feeling that your feeling is right ;)

> I would also like to hear your suggestions if there is a way to circumvent
> it. Maybe by the approach described here:
> https://rhettinger.wordpress.com/2011/05/26/super-considered-super/
> wrapping B and C in some Adapter class.

A quick glance at that page suggests that the adapter translates the
base class into an attribute -- which is what I would have suggested,
too. Using has-a instead of is-a relations is generally something to
consider seriously before jumping into multiple inheritance. The details
may depend on the actual use case which is hidden behind your A, B, C,
and D abstraction...


From auriocus at gmx.de  Thu Feb 10 10:29:54 2022
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Thu, 10 Feb 2022 16:29:54 +0100
Subject: How to solve the given problem?
In-Reply-To: <ef4ede55-3e23-483f-887c-b6a45112de1bn@googlegroups.com>
References: <CAPTjJmoAXy5wuHEbgbbL3KadjSx+3YJ-a3OUtAqpY2wSSRVefQ@mail.gmail.com>
 <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
 <mailman.84.1644482467.7010.python-list@python.org>
 <ef4ede55-3e23-483f-887c-b6a45112de1bn@googlegroups.com>
Message-ID: <su3b5o$u45$1@dont-email.me>

Am 10.02.22 um 11:26 schrieb NArshad:
> -ChrisA:
> You don't reply if you have problems.
> When I don't find any solution elsewhere then only I place in this group
> 
> 
> -Christian:
> One problem of different type requires the same elaboration.

No it doesn't

> Q. What technique of statistics or numerical computation or general mathematics to use to solve this problem. Can you tell one example
> 
> Find the values of ?subscript(?) for i from 1 to n (n =100).


WTF? Go study maths yourself instead of asking here (Offf-topic as well)

	Christian


From blindanagram at nowhere.org  Thu Feb 10 11:15:00 2022
From: blindanagram at nowhere.org (BlindAnagram)
Date: Thu, 10 Feb 2022 16:15:00 +0000
Subject: Global VS Local Subroutines
In-Reply-To: <mailman.86.1644506445.7010.python-list@python.org>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
 <CAPTjJmrAR=kcW1_n28ww4f55T2rzxh5OFS1tOKmm8qMYN5iwWQ@mail.gmail.com>
 <mailman.86.1644506445.7010.python-list@python.org>
Message-ID: <f229cebd-57ee-f7a3-58bb-7d01ff517489@nowhere.org>

On 10/02/2022 15:20, Chris Angelico wrote:
> On Fri, 11 Feb 2022 at 02:13, BlindAnagram <blindanagram at nowhere.org> wrote:
>>
>> Is there any difference in performance between these two program layouts:
>>
>>      def a():
>>        ...
>>      def(b):
>>        c = a(b)
>>
>> or
>>
>>      def(b):
>>        def a():
>>          ...
>>        c = a(b)
>>
>> I would appreciate any insights on which layout to choose in which
>> circumstances.
>>
> 
> Great question! The difference is that, in the second case, a() isn't
> available anywhere else. So the real question is: Is that good or bad?
> 
> Does it make sense to call a() from outside of your second function,
> even for testing? If so, maybe name it _a() so it's private, but keep
> it global. Does a() need a lot of information from the context of your
> second function? If so, it's easier to have a closure, with one
> function inside another.
> 
> Both styles make sense, and your question is well put: it's a design
> decision with no "right" or "wrong", just different choices with
> different implications.
> 
> ChrisA

Thanks Chris, I thought it  was mostly a design choice but I wasn't sure 
whether there would be any significant performance issues.

    Brian

From rob.cliffe at btinternet.com  Thu Feb 10 11:52:00 2022
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 10 Feb 2022 16:52:00 +0000
Subject: Global VS Local Subroutines
In-Reply-To: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
Message-ID: <daa31623-4eb4-e812-4ed4-14c7839a6efe@btinternet.com>



On 10/02/2022 12:13, BlindAnagram wrote:
> Is there any difference in performance between these two program layouts:
>
> ?? def a():
> ???? ...
> ?? def(b):
> ???? c = a(b)
>
> or
>
> ?? def(b):
> ???? def a():
> ?????? ...
> ???? c = a(b)
>
> I would appreciate any insights on which layout to choose in which 
> circumstances.
>
The way to answer questions about performance is not to guess (often 
difficult or impossible), but to measure.? Profiling tools are available 
to see where a program spends how much of its time.? But a simpler 
approach might be to try to run the guts of your program 1000 or 1000000 
times, and find how long it takes with each layout (not with a stopwatch 
? but getting the program to record the start time, end time and the 
difference).
That said, I will venture a guess (or at least, an observation):
 ??? def a():
 ??? ??? ...
is executable code.? It creates the function `a` which did not exist 
before (or creates the new version if it did).? This takes a certain 
amount of time.? In your 2nd layout, this overhead occurs every time b() 
is called.? So I would *guess* that this would be slower.? Of course it 
depends on how many times b() is called and probably on lots of other 
things.

But of course, performance is not the only consideration, as per Chris 
Angelico's answer.
Best wishes
Rob Cliffe

From rosuav at gmail.com  Thu Feb 10 12:01:39 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 11 Feb 2022 04:01:39 +1100
Subject: Global VS Local Subroutines
In-Reply-To: <daa31623-4eb4-e812-4ed4-14c7839a6efe@btinternet.com>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
 <daa31623-4eb4-e812-4ed4-14c7839a6efe@btinternet.com>
Message-ID: <CAPTjJmo9nPuiFz0P=2o_JV=hMKAx32NdM9S186Y5kGV_5Y0+8w@mail.gmail.com>

On Fri, 11 Feb 2022 at 03:57, Rob Cliffe via Python-list
<python-list at python.org> wrote:
> But of course, performance is not the only consideration, as per Chris
> Angelico's answer.

Yep. In fact, I'd say that performance is the least significant
consideration here; do what makes sense. The time difference will be
negligible.

ChrisA

From jorge.conforte at inpe.br  Fri Feb 11 12:30:59 2022
From: jorge.conforte at inpe.br (Jorge Conforte)
Date: Fri, 11 Feb 2022 14:30:59 -0300
Subject: Python LSTM forecast future values for time series
Message-ID: <53723684-e4f8-fb59-7d17-71ee6cc7e4b7@inpe.br>


HI,


I'm starting run the LSTM to forecast future values for time serie data.

please can someone give me some information on how i can predict future 
values ??for my time series using LSTM. Thanks, Conrado


From bhutsler at healing-partners.com  Thu Feb 10 12:20:35 2022
From: bhutsler at healing-partners.com (BmoreIT)
Date: Thu, 10 Feb 2022 09:20:35 -0800 (PST)
Subject: GCP Copy Files - Data Export
Message-ID: <c52016b9-b06c-4736-9956-98d308c1d693n@googlegroups.com>

I did a data export from Google to export all company data - Google Data Export

It shows the root folder and to download, I run this command (it automatically enters this command)

gsutil -m cp -r \ "gs://takeout-export-myUniqueID" \.
But I have no idea where it would save it being I am not a GCP customer, only Google Workspace. Workspace won't help because they say it's a GCP product but I am exporting from Workspace. 

Can someone let me know the proper command to run on my local machine with Google's SDK to download this folder? I was able to start the download yesterday but it said there was an invalid character in the file names so it killed the export.

Appreciate any help!

From blindanagram at nowhere.org  Thu Feb 10 12:33:59 2022
From: blindanagram at nowhere.org (BlindAnagram)
Date: Thu, 10 Feb 2022 17:33:59 +0000
Subject: Global VS Local Subroutines
In-Reply-To: <mailman.90.1644512015.7010.python-list@python.org>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
 <daa31623-4eb4-e812-4ed4-14c7839a6efe@btinternet.com>
 <mailman.90.1644512015.7010.python-list@python.org>
Message-ID: <c0298388-6b73-519c-7435-4ba4569c21de@nowhere.org>

On 10/02/2022 16:52, Rob Cliffe wrote:
> 
> 
> On 10/02/2022 12:13, BlindAnagram wrote:
>> Is there any difference in performance between these two program layouts:
>>
>> ?? def a():
>> ???? ...
>> ?? def(b):
>> ???? c = a(b)
>>
>> or
>>
>> ?? def(b):
>> ???? def a():
>> ?????? ...
>> ???? c = a(b)
>>
>> I would appreciate any insights on which layout to choose in which 
>> circumstances.
>>
> The way to answer questions about performance is not to guess (often 
> difficult or impossible), but to measure.? Profiling tools are available 
> to see where a program spends how much of its time.? But a simpler 
> approach might be to try to run the guts of your program 1000 or 1000000 
> times, and find how long it takes with each layout (not with a stopwatch 
> ? but getting the program to record the start time, end time and the 
> difference).
> That said, I will venture a guess (or at least, an observation):
>  ??? def a():
>  ??? ??? ...

Thanks for the suggestion, Rob. I considered doing this but it would 
take quite a bit of careful work to be sure of obtaining meaningful 
results so I felt it better to ask the question here in case there was 
an quick answer on performance before doing the hard work.

> is executable code.? It creates the function `a` which did not exist 
> before (or creates the new version if it did).? This takes a certain 
> amount of time.? In your 2nd layout, this overhead occurs every time b() 
> is called.? So I would *guess* that this would be slower.? Of course it 
> depends on how many times b() is called and probably on lots of other 
> things.

This is exactly what I felt too but I then wondered if the code was 
recreated dynamically or was static with just a reference being created 
on each invocation of the parent.  The overhead in this case would be 
negligible. But then I thought 'what about the context for the inner 
function invocation' - maybe its not as simple as this!  Oh dear, I need 
to understand Python's internals.

At which point I asked the question!

    Brian

From python at mrabarnett.plus.com  Thu Feb 10 14:02:27 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 10 Feb 2022 19:02:27 +0000
Subject: GCP Copy Files - Data Export
In-Reply-To: <c52016b9-b06c-4736-9956-98d308c1d693n@googlegroups.com>
References: <c52016b9-b06c-4736-9956-98d308c1d693n@googlegroups.com>
Message-ID: <a0ede338-2f49-614c-01e1-fd9a22403536@mrabarnett.plus.com>

On 2022-02-10 17:20, BmoreIT wrote:
> I did a data export from Google to export all company data - Google Data Export
> 
> It shows the root folder and to download, I run this command (it automatically enters this command)
> 
> gsutil -m cp -r \ "gs://takeout-export-myUniqueID" \.
> But I have no idea where it would save it being I am not a GCP customer, only Google Workspace. Workspace won't help because they say it's a GCP product but I am exporting from Workspace.
> 
> Can someone let me know the proper command to run on my local machine with Google's SDK to download this folder? I was able to start the download yesterday but it said there was an invalid character in the file names so it killed the export.
> 
> Appreciate any help!

On this page:

     https://cloud.google.com/storage/docs/gsutil/commands/cp

I think the problem might be that you have an surplus backslash.

Does this work?

     gsutil cp -r gs://takeout-export-myUniqueID .

This should copy recursively from gs://takeout-export-myUniqueID to the 
current folder.

From eryksun at gmail.com  Thu Feb 10 14:39:49 2022
From: eryksun at gmail.com (Eryk Sun)
Date: Thu, 10 Feb 2022 13:39:49 -0600
Subject: Global VS Local Subroutines
In-Reply-To: <c0298388-6b73-519c-7435-4ba4569c21de@nowhere.org>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
 <daa31623-4eb4-e812-4ed4-14c7839a6efe@btinternet.com>
 <mailman.90.1644512015.7010.python-list@python.org>
 <c0298388-6b73-519c-7435-4ba4569c21de@nowhere.org>
Message-ID: <CACL+1auO2mF0ZCSdyeVYRSFNcfH0drtHGtpwLheYjMZd37Qakg@mail.gmail.com>

On 2/10/22, BlindAnagram <blindanagram at nowhere.org> wrote:
>
> This is exactly what I felt too but I then wondered if the code was
> recreated dynamically or was static with just a reference being created
> on each invocation of the parent.  The overhead in this case would be
> negligible. But then I thought 'what about the context for the inner
> function invocation' - maybe its not as simple as this!  Oh dear, I need
> to understand Python's internals.

The bytecode for a() is a constant in b(). But the function object
itself isn't cached. Each time b() is called, a new function object
a() has to be created, with its defaults, keyword-only defaults, and
closure. The more complicated the latter are, the more work is
involved. Generally, however, the time to create a() will be an
insignificant fraction of the execution time of b(). If the latter
isn't the case, the cost only accumulates to something significant if
b() is called in a tight loop for an extended period.

From wlfraed at ix.netcom.com  Thu Feb 10 13:21:25 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 10 Feb 2022 13:21:25 -0500
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
 <su0d1v$ju2$1@dont-email.me>
 <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>
 <mailman.70.1644432802.7010.python-list@python.org>
Message-ID: <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>

On Wed, 9 Feb 2022 18:50:12 +0000, MRAB <python at mrabarnett.plus.com>
declaimed the following:

>On 2022-02-09 12:45, Christian Gollwitzer wrote:

>> It's impossible. Excel locks the file deliberately when it is open, so
>> that you can't overwrite it from a different program. Otherwise, the
>> file could become inconsistent.
>> 
>It's the same the other way too; you can't open the file in Excel while 
>Python has it open.
>
	While not tested with Excel, I /have/ encountered cases where an
application has locked the file for writing, but multiple readers are
permitted. Those would fail then if one attempts to write. {The other view
point is a library that does a complete open/read\write-all/close to memory
-- such an application might open/read/close, then Excel opens/locks, with
the application only learning of the change when it attempts the
open/write/close cycle}


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From wlfraed at ix.netcom.com  Thu Feb 10 13:37:12 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 10 Feb 2022 13:37:12 -0500
Subject: How to solve the given problem?
References: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
Message-ID: <q2ma0htuv58na9jqbt5vlk5efig8tql0d3@4ax.com>

On Wed, 9 Feb 2022 22:40:48 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>
>Assume that there is a pattern of feeding for a special fish in a day (10 hours a day) as below:
>                               150    100    30    30    30    20    20    10    5    5
>Today, the fish is fed in the second hour 60 unit instead of 100 unit Accidently. Implement some methods to distribute the remaining 40 unit in the rest of the day and propose the new patterns. Try to keep the distribution similar to the current feeding pattern. 
>Note: pay attention that the total feeding amounts should be fix in a day.

	This is very obviously phrased as a homework problem. WE DO NOT DO
HOMEWORK!

	Write some Python code, present the code, your input data, and the
output it produced along with an example of the output you expected, and we
may be able to point out what parts of Python you made a mistake with. If
we get ambitious we might even correct an algorithm, not just explain
errors in the usage of Python.

	I'm going a tad too far but...

HINT:
	You need to compute the percentage of remaining scheduled units each
takes, then apply those percentages to the left-over from the mis-feed.

	NONE of this has anything to do with Python itself -- it is purely
algorithm development that applies with any programming language -- or even
none.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From rosuav at gmail.com  Thu Feb 10 14:43:06 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 11 Feb 2022 06:43:06 +1100
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
In-Reply-To: <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
 <su0d1v$ju2$1@dont-email.me>
 <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>
 <mailman.70.1644432802.7010.python-list@python.org>
 <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>
Message-ID: <CAPTjJmq+6UNruEGAHMyBQOtCk5LpwOz2caVkfhm=U8Mn4SqDNA@mail.gmail.com>

On Fri, 11 Feb 2022 at 06:41, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>
> On Wed, 9 Feb 2022 18:50:12 +0000, MRAB <python at mrabarnett.plus.com>
> declaimed the following:
>
> >On 2022-02-09 12:45, Christian Gollwitzer wrote:
>
> >> It's impossible. Excel locks the file deliberately when it is open, so
> >> that you can't overwrite it from a different program. Otherwise, the
> >> file could become inconsistent.
> >>
> >It's the same the other way too; you can't open the file in Excel while
> >Python has it open.
> >
>         While not tested with Excel, I /have/ encountered cases where an
> application has locked the file for writing, but multiple readers are
> permitted. Those would fail then if one attempts to write. {The other view
> point is a library that does a complete open/read\write-all/close to memory
> -- such an application might open/read/close, then Excel opens/locks, with
> the application only learning of the change when it attempts the
> open/write/close cycle}
>

Yeah, I doubt Excel is that sophisticated. It's built on an assumption
of single-user operation.

ChrisA

From jenkris at tutanota.com  Thu Feb 10 15:00:51 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Thu, 10 Feb 2022 21:00:51 +0100 (CET)
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <CAEfz+Tx4rpN6+fqrPMd-oeuNxitGPbGf5NYCqZCdAJa6htW1gQ@mail.gmail.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
 <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
 <MvWGgaJ--B-2@tutanota.com>
 <CAEfz+Tx4rpN6+fqrPMd-oeuNxitGPbGf5NYCqZCdAJa6htW1gQ@mail.gmail.com>
Message-ID: <Mv_DLCJ--3-2@tutanota.com>

With the help of PyErr_Print() I have it solved.? Here is the final code (the part relevant to sents):

?? Py_ssize_t listIndex = 0;
?? pListItem = PyList_GetItem(pFileIds, listIndex);
?? pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
?? pListStr = PyBytes_AS_STRING(pListStrE); // Borrowed pointer

?? // Then:? sentences = gutenberg.sents(fileid) - this is a sequence item
?? PyObject *c_args = Py_BuildValue("s", pListStr);
?? PyObject *args_tuple = PyTuple_New(1);
?? PyTuple_SetItem(args_tuple, 0, c_args);

?? pSents = PyObject_CallObject(pSentMod, args_tuple);

?? if ( pSents == 0x0){
?????? PyErr_Print();
?????? return return_value; }

As you mentioned yesterday, CallObject needs a tuple, so that was the problem.? Now it works.? 

You also asked why I don't just use pListStrE.? I tried that and got a long error message from PyErr_Print.? I'm not far enough along in my C_API work to understand why, but it doesn't work.? 

Thanks very much for your help on this.? 

Jen


Feb 9, 2022, 17:40 by songofacandy at gmail.com:

> On Thu, Feb 10, 2022 at 10:37 AM Jen Kris <jenkris at tutanota.com> wrote:
>
>>
>> I'm using Python 3.8 so I tried your second choice:
>>
>> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>>
>> but pSents is 0x0.  pSentMod and pListItem are valid pointers.
>>
>
> It means exception happened.
> If you are writing Python/C function, return NULL (e.g. `if (pSents ==
> NULL) return NULL`)
> Then Python show the exception and traceback for you.
>
> -- 
> Inada Naoki  <songofacandy at gmail.com>
>


From jenkris at tutanota.com  Thu Feb 10 15:04:05 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Thu, 10 Feb 2022 21:04:05 +0100 (CET)
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <36767883-dafa-84f0-ee65-478965a6ae8f@mrabarnett.plus.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
 <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
 <MvWGgaJ--B-2@tutanota.com>
 <36767883-dafa-84f0-ee65-478965a6ae8f@mrabarnett.plus.com>
Message-ID: <Mv_E4Xm--3-2@tutanota.com>

Hi and thanks very much for your comments on reference counting.? Since I'm new to the C_API that will help a lot.? I know that reference counting is one of the difficult issues with the C API.? 

I just posted a reply to Inada Naoki showing how I solved the problem I posted yesterday.? 

Thanks much for your help.

Jen


Feb 9, 2022, 18:43 by python at mrabarnett.plus.com:

> On 2022-02-10 01:37, Jen Kris via Python-list wrote:
>
>> I'm using Python 3.8 so I tried your second choice:
>>
>> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>>
>> but pSents is 0x0.? pSentMod and pListItem are valid pointers.
>>
> 'PyObject_CallFunction' looks like a good one to use:
>
> """PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...)
>
> Call a callable Python object callable, with a variable number of C arguments. The C arguments are described using a Py_BuildValue() style format string. The format can be NULL, indicating that no arguments are provided.
> """
>
> [snip]
>
> What I do is add comments to keep track of what objects I have references to at each point and whether they are new references or could be NULL.
>
> For example:
>
>  pName = PyUnicode_FromString("nltk.corpus");
>  //> pName+?
>
> This means that 'pName' contains a reference, '+' means that it's a new reference, and '?' means that it could be NULL (usually due to an exception, but not always) so I need to check it.
>
> Continuing in this vein:
>
>  pModule = PyImport_Import(pName);
>  //> pName+? pModule+?
>
>  pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
>  //> pName+? pModule+? pSubMod+?
>  pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
>  //> pName+? pModule+? pSubMod+? pFidMod+?
>  pSentMod = PyObject_GetAttrString(pSubMod, "sents");
>  //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+?
>
>  pFileIds = PyObject_CallObject(pFidMod, 0);
>  //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? PyObject_CallObject+?
>  pListItem = PyList_GetItem(pFileIds, listIndex);
>  //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? PyObject_CallObject+? pListItem?
>  pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>  //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? PyObject_CallObject+? pListItem? pListStrE+?
>
> As you can see, there's a lot of leaked references building up.
>
> Note how after:
>
>  pListItem = PyList_GetItem(pFileIds, listIndex);
>
> the addition is:
>
>  //> pListItem?
>
> This means that 'pListItem' contains a borrowed (not new) reference, but could be NULL.
>
> I find it easiest to DECREF as soon as I no longer need the reference and remove a name from the list as soon I no longer need it (and DECREFed where).
>
> For example:
>
>  pName = PyUnicode_FromString("nltk.corpus");
>  //> pName+?
>  if (!pName)
>  goto error;
>  //> pName+
>  pModule = PyImport_Import(pName);
>  //> pName+ pModule+?
>  Py_DECREF(pName);
>  //> pModule+?
>  if (!pModule)
>  goto error;
>  //> pModule+
>
> I find that doing this greatly reduces the chances of getting the reference counting wrong, and I can remove the comments once I've finished the function I'm writing.
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From python at mrabarnett.plus.com  Thu Feb 10 15:43:10 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Thu, 10 Feb 2022 20:43:10 +0000
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <Mv_DLCJ--3-2@tutanota.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
 <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
 <MvWGgaJ--B-2@tutanota.com>
 <CAEfz+Tx4rpN6+fqrPMd-oeuNxitGPbGf5NYCqZCdAJa6htW1gQ@mail.gmail.com>
 <Mv_DLCJ--3-2@tutanota.com>
Message-ID: <bf9c3200-f5b0-6ed5-5229-d4095021f6b4@mrabarnett.plus.com>

On 2022-02-10 20:00, Jen Kris via Python-list wrote:
> With the help of PyErr_Print() I have it solved.? Here is the final code (the part relevant to sents):
> 
>  ?? Py_ssize_t listIndex = 0;
>  ?? pListItem = PyList_GetItem(pFileIds, listIndex);
>  ?? pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>  ?? pListStr = PyBytes_AS_STRING(pListStrE); // Borrowed pointer
> 
>  ?? // Then:? sentences = gutenberg.sents(fileid) - this is a sequence item
>  ?? PyObject *c_args = Py_BuildValue("s", pListStr);
>  ?? PyObject *args_tuple = PyTuple_New(1);
>  ?? PyTuple_SetItem(args_tuple, 0, c_args);
> 
>  ?? pSents = PyObject_CallObject(pSentMod, args_tuple);
> 
>  ?? if ( pSents == 0x0){
>  ?????? PyErr_Print();
>  ?????? return return_value; }
> 
> As you mentioned yesterday, CallObject needs a tuple, so that was the problem.? Now it works.
> 
> You also asked why I don't just use pListStrE.? I tried that and got a long error message from PyErr_Print.? I'm not far enough along in my C_API work to understand why, but it doesn't work.
> 
> Thanks very much for your help on this.
> 
You're encoding a Unicode string to a UTF-8 bytestring:

     pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");

then pointing to the bytes of that UTF-8 bytestring:

     pListStr = PyBytes_AS_STRING(pListStrE); // Borrowed pointer

then making a Unicode string from those UTF-8 bytes:

     PyObject *c_args = Py_BuildValue("s", pListStr);

You might was well just use the original Unicode string!

Try this instead:

    Py_ssize_t listIndex = 0;
    pListItem = PyList_GetItem(pFileIds, listIndex);
    //> pListItem?

    pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem, 0);
    //> pSents+?

    if (pSents == 0x0){
        PyErr_Print();
        return return_value;
    }

> 
> 
> Feb 9, 2022, 17:40 by songofacandy at gmail.com:
> 
>> On Thu, Feb 10, 2022 at 10:37 AM Jen Kris <jenkris at tutanota.com> wrote:
>>
>>>
>>> I'm using Python 3.8 so I tried your second choice:
>>>
>>> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>>>
>>> but pSents is 0x0.  pSentMod and pListItem are valid pointers.
>>>
>>
>> It means exception happened.
>> If you are writing Python/C function, return NULL (e.g. `if (pSents ==
>> NULL) return NULL`)
>> Then Python show the exception and traceback for you.
>>

From auriocus at gmx.de  Thu Feb 10 15:39:05 2022
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Thu, 10 Feb 2022 21:39:05 +0100
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
In-Reply-To: <mailman.95.1644522198.7010.python-list@python.org>
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
 <su0d1v$ju2$1@dont-email.me>
 <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>
 <mailman.70.1644432802.7010.python-list@python.org>
 <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>
 <CAPTjJmq+6UNruEGAHMyBQOtCk5LpwOz2caVkfhm=U8Mn4SqDNA@mail.gmail.com>
 <mailman.95.1644522198.7010.python-list@python.org>
Message-ID: <su3tb4$ac5$1@dont-email.me>

Am 10.02.22 um 20:43 schrieb Chris Angelico:
> On Fri, 11 Feb 2022 at 06:41, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>>          While not tested with Excel, I /have/ encountered cases where an
>> application has locked the file for writing, but multiple readers are
>> permitted. Those would fail then if one attempts to write. {The other view
>> point is a library that does a complete open/read\write-all/close to memory
>> -- such an application might open/read/close, then Excel opens/locks, with
>> the application only learning of the change when it attempts the
>> open/write/close cycle}
>>
> 
> Yeah, I doubt Excel is that sophisticated. It's built on an assumption
> of single-user operation.
> 

It guards against multiple user opening the same file over network 
drives. All MS applications create lock files with weird names like 
~.original-name.xlsx etc. If you open a file on the network share, and a 
colleague tries to open it from a second machine, then he will get the 
message "File locked by user xy from machine z". See here for word: 
https://support.microsoft.com/en-us/topic/-the-document-is-locked-for-editing-by-another-user-error-message-when-you-try-to-open-a-document-in-word-10b92aeb-2e23-25e0-9110-370af6edb638

I believe (haven't tested) that this is cooperative locking only and it 
doesn't help if you alter the file from another program. On the same 
machine though, I think that Excel opens the file with a flag to lock it 
from other processes. At least that was my observation and also what the 
OP has described.

Hence it is impossible to concurrently write from Python into an open 
Excel file. One might ask what the real problem is the user is trying to 
solve. Is Excel a requirement, can it be swapped by a database engine?

Best regards,

	Christian

From rosuav at gmail.com  Thu Feb 10 16:11:32 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 11 Feb 2022 08:11:32 +1100
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
In-Reply-To: <su3tb4$ac5$1@dont-email.me>
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
 <su0d1v$ju2$1@dont-email.me>
 <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>
 <mailman.70.1644432802.7010.python-list@python.org>
 <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>
 <CAPTjJmq+6UNruEGAHMyBQOtCk5LpwOz2caVkfhm=U8Mn4SqDNA@mail.gmail.com>
 <mailman.95.1644522198.7010.python-list@python.org>
 <su3tb4$ac5$1@dont-email.me>
Message-ID: <CAPTjJmp4W9mMwuND85wST67vyC_KtnB6RbXPHqqNOgHJSKT3rQ@mail.gmail.com>

On Fri, 11 Feb 2022 at 07:57, Christian Gollwitzer <auriocus at gmx.de> wrote:
>
> Am 10.02.22 um 20:43 schrieb Chris Angelico:
> > On Fri, 11 Feb 2022 at 06:41, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> >>          While not tested with Excel, I /have/ encountered cases where an
> >> application has locked the file for writing, but multiple readers are
> >> permitted. Those would fail then if one attempts to write. {The other view
> >> point is a library that does a complete open/read\write-all/close to memory
> >> -- such an application might open/read/close, then Excel opens/locks, with
> >> the application only learning of the change when it attempts the
> >> open/write/close cycle}
> >>
> >
> > Yeah, I doubt Excel is that sophisticated. It's built on an assumption
> > of single-user operation.
> >
>
> It guards against multiple user opening the same file over network
> drives. All MS applications create lock files with weird names like
> ~.original-name.xlsx etc. If you open a file on the network share, and a
> colleague tries to open it from a second machine, then he will get the
> message "File locked by user xy from machine z". See here for word:
> https://support.microsoft.com/en-us/topic/-the-document-is-locked-for-editing-by-another-user-error-message-when-you-try-to-open-a-document-in-word-10b92aeb-2e23-25e0-9110-370af6edb638
>

Yeah, but that's still just hard locking, no "one writer multiple
readers" system or anything.

> I believe (haven't tested) that this is cooperative locking only and it
> doesn't help if you alter the file from another program. On the same
> machine though, I think that Excel opens the file with a flag to lock it
> from other processes. At least that was my observation and also what the
> OP has described.
>

That sounds right; and, again, it's just a simple exclusive lock.
Excel doesn't have the sophistication to need or want anything more
than simple "I have this file, nobody else touch it" exclusive
locking.

ChrisA

From anthra.norell at bluewin.ch  Thu Feb 10 16:43:32 2022
From: anthra.norell at bluewin.ch (Friedrich Rentsch)
Date: Thu, 10 Feb 2022 22:43:32 +0100
Subject: Global VS Local Subroutines
In-Reply-To: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
Message-ID: <c11cb7b0-68b9-9597-f0e8-501d32221144@bluewin.ch>

I believe to have observed a difference which also might be worth 
noting: the imbedded function a() (second example) has access to all of 
the imbedding function's variables, which might be an efficiency factor 
with lots of variables. The access is read-only, though. If the inner 
function writes to one of the readable external variables, that variable 
becomes local to the inner function.

Frederic

On 2/10/22 1:13 PM, BlindAnagram wrote:
> Is there any difference in performance between these two program layouts:
>
> ?? def a():
> ???? ...
> ?? def(b):
> ???? c = a(b)
>
> or
>
> ?? def(b):
> ???? def a():
> ?????? ...
> ???? c = a(b)
>
> I would appreciate any insights on which layout to choose in which 
> circumstances.
>


From Marco.Sulla.Python at gmail.com  Thu Feb 10 16:45:54 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Thu, 10 Feb 2022 22:45:54 +0100
Subject: How do you log in your projects?
In-Reply-To: <20220209193823.ez7ewt4sr7joplix@gmail.com>
References: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>
 <CABbU2U-jTZXzNkG89q0PbXJuYLVpiOwJMR43bMMOZDnaRxu=kA@mail.gmail.com>
 <20220209193823.ez7ewt4sr7joplix@gmail.com>
Message-ID: <CABbU2U8ON1x8t+umOc84vqS2haPuNTb+v9=_hRZD6BBq45wpJw@mail.gmail.com>

On Wed, 9 Feb 2022 at 20:40, Martin Di Paola <martinp.dipaola at gmail.com> wrote:
>
> If the logs are meant to be read by my users I log high level messages,
> specially before parts that can take a while (like the classic
> "Loading...").

? Logs are not intended to be read by end users. Logs are primarily
used to understand what the code is doing in a production environment.
They could also be used to gather metrics data.

Why should you log to give a message instead of simply using a print?

> For exceptions I print the message but not the traceback.

Why? Traceback is vital to understand what and where the problem is. I
think you're confusing logs with messages. The stack trace can be
logged (I would say must), but the end user generally sees a vague
message with some hints, unless the program is used internally only.

From Marco.Sulla.Python at gmail.com  Thu Feb 10 16:51:39 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Thu, 10 Feb 2022 22:51:39 +0100
Subject: Global VS Local Subroutines
In-Reply-To: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
Message-ID: <CABbU2U-du=AoyezBb3BiyFpqk55KwEDXnSRuxkefLS5AG49nSw@mail.gmail.com>

I agree with Chris. I don't know if it was already written: if you
want a local function for speed reasons, you can use the classic
approach of a main function.

From Marco.Sulla.Python at gmail.com  Thu Feb 10 16:54:34 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Thu, 10 Feb 2022 22:54:34 +0100
Subject: How to solve the given problem?
In-Reply-To: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
References: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
Message-ID: <CABbU2U9cEVSzH6r2z-vTiBh7ZOghVd2ApjUAv42f0pTfx=EUHQ@mail.gmail.com>

Narshad, I propose you post your questions to StackOverflow. I'm sure
they will be very happy.

From jenkris at tutanota.com  Thu Feb 10 17:13:21 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Thu, 10 Feb 2022 23:13:21 +0100 (CET)
Subject: C API PyObject_Call segfaults with string
In-Reply-To: <bf9c3200-f5b0-6ed5-5229-d4095021f6b4@mrabarnett.plus.com>
References: <MvW3s4j--3-2@tutanota.com>
 <CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
 <MvW9RVE--3-2@tutanota.com>
 <CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
 <MvWBjNR--3-2@tutanota.com>
 <CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
 <MvWGgaJ--B-2@tutanota.com>
 <CAEfz+Tx4rpN6+fqrPMd-oeuNxitGPbGf5NYCqZCdAJa6htW1gQ@mail.gmail.com>
 <Mv_DLCJ--3-2@tutanota.com>
 <bf9c3200-f5b0-6ed5-5229-d4095021f6b4@mrabarnett.plus.com>
Message-ID: <Mv_gfBu--3-2@tutanota.com>

Thank you for that suggestion.? It allowed me to replace six lines of code with one.? :)


Feb 10, 2022, 12:43 by python at mrabarnett.plus.com:

> On 2022-02-10 20:00, Jen Kris via Python-list wrote:
>
>> With the help of PyErr_Print() I have it solved.? Here is the final code (the part relevant to sents):
>>
>>  ?? Py_ssize_t listIndex = 0;
>>  ?? pListItem = PyList_GetItem(pFileIds, listIndex);
>>  ?? pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>>  ?? pListStr = PyBytes_AS_STRING(pListStrE); // Borrowed pointer
>>
>>  ?? // Then:? sentences = gutenberg.sents(fileid) - this is a sequence item
>>  ?? PyObject *c_args = Py_BuildValue("s", pListStr);
>>  ?? PyObject *args_tuple = PyTuple_New(1);
>>  ?? PyTuple_SetItem(args_tuple, 0, c_args);
>>
>>  ?? pSents = PyObject_CallObject(pSentMod, args_tuple);
>>
>>  ?? if ( pSents == 0x0){
>>  ?????? PyErr_Print();
>>  ?????? return return_value; }
>>
>> As you mentioned yesterday, CallObject needs a tuple, so that was the problem.? Now it works.
>>
>> You also asked why I don't just use pListStrE.? I tried that and got a long error message from PyErr_Print.? I'm not far enough along in my C_API work to understand why, but it doesn't work.
>>
>> Thanks very much for your help on this.
>>
> You're encoding a Unicode string to a UTF-8 bytestring:
>
>  pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
>
> then pointing to the bytes of that UTF-8 bytestring:
>
>  pListStr = PyBytes_AS_STRING(pListStrE); // Borrowed pointer
>
> then making a Unicode string from those UTF-8 bytes:
>
>  PyObject *c_args = Py_BuildValue("s", pListStr);
>
> You might was well just use the original Unicode string!
>
> Try this instead:
>
>  Py_ssize_t listIndex = 0;
>  pListItem = PyList_GetItem(pFileIds, listIndex);
>  //> pListItem?
>
>  pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem, 0);
>  //> pSents+?
>
>  if (pSents == 0x0){
>  PyErr_Print();
>  return return_value;
>  }
>
>>
>>
>> Feb 9, 2022, 17:40 by songofacandy at gmail.com:
>>
>>> On Thu, Feb 10, 2022 at 10:37 AM Jen Kris <jenkris at tutanota.com> wrote:
>>>
>>>>
>>>> I'm using Python 3.8 so I tried your second choice:
>>>>
>>>> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>>>>
>>>> but pSents is 0x0.  pSentMod and pListItem are valid pointers.
>>>>
>>>
>>> It means exception happened.
>>> If you are writing Python/C function, return NULL (e.g. `if (pSents ==
>>> NULL) return NULL`)
>>> Then Python show the exception and traceback for you.
>>>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From rob.cliffe at btinternet.com  Thu Feb 10 17:32:08 2022
From: rob.cliffe at btinternet.com (Rob Cliffe)
Date: Thu, 10 Feb 2022 22:32:08 +0000
Subject: Global VS Local Subroutines
In-Reply-To: <c11cb7b0-68b9-9597-f0e8-501d32221144@bluewin.ch>
References: <Npadnd4vFJ7lnJj_nZ2dnUU7-a3NnZ2d@brightview.co.uk>
 <c11cb7b0-68b9-9597-f0e8-501d32221144@bluewin.ch>
Message-ID: <1cdb4be9-551d-33b7-e9c2-18332a59752e@btinternet.com>



On 10/02/2022 21:43, Friedrich Rentsch wrote:
> I believe to have observed a difference which also might be worth 
> noting: the imbedded function a() (second example) has access to all 
> of the imbedding function's variables, which might be an efficiency 
> factor with lots of variables. The access is read-only, though. If the 
> inner function writes to one of the readable external variables, that 
> variable becomes local to the inner function.
You can make it continue to refer to the variables of the imbedding 
function, i.e. b(), by declaring them non-local, e.g.
 ??? nonlocal c
Rob Cliffe
>
>
> Frederic
>
> On 2/10/22 1:13 PM, BlindAnagram wrote:
>> Is there any difference in performance between these two program 
>> layouts:
>>
>> ?? def a():
>> ???? ...
>> ?? def(b):
>> ???? c = a(b)
>>
>> or
>>
>> ?? def(b):
>> ???? def a():
>> ?????? ...
>> ???? c = a(b)
>>
>> I would appreciate any insights on which layout to choose in which 
>> circumstances.
>>
>


From avigross at verizon.net  Thu Feb 10 17:57:05 2022
From: avigross at verizon.net (Avi Gross)
Date: Thu, 10 Feb 2022 22:57:05 +0000 (UTC)
Subject: How to solve the given problem?
In-Reply-To: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
References: <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
Message-ID: <1323924283.239248.1644533825969@mail.yahoo.com>

Anyone think someone keeps asking homework questions? This seems absolutely unrelated to earlier discussions from this person. Jobs often tend to remain focused.
I opt out after frustration with earlier exchanges with NArshad about library books and EXCEL ...


-----Original Message-----
From: NArshad <narshad.380 at gmail.com>
To: python-list at python.org
Sent: Thu, Feb 10, 2022 1:40 am
Subject: How to solve the given problem?


Assume that there is a pattern of feeding for a special fish in a day (10 hours a day) as below:
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 150? ? 100? ? 30? ? 30? ? 30? ? 20? ? 20? ? 10? ? 5? ? 5
Today, the fish is fed in the second hour 60 unit instead of 100 unit Accidently. Implement some methods to distribute the remaining 40 unit in the rest of the day and propose the new patterns. Try to keep the distribution similar to the current feeding pattern. 
Note: pay attention that the total feeding amounts should be fix in a day.
-- 
https://mail.python.org/mailman/listinfo/python-list

From cs at cskk.id.au  Thu Feb 10 18:59:46 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Fri, 11 Feb 2022 10:59:46 +1100
Subject: Abstraction level at which to create SQLAlchemy ORM object
In-Reply-To: <87k0e2j22x.fsf@hornfels.zedat.fu-berlin.de>
References: <87k0e2j22x.fsf@hornfels.zedat.fu-berlin.de>
Message-ID: <YgWm8hz5chR51d60@cskk.homeip.net>

On 10Feb2022 14:14, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>I am writing a command line program which will modify entries in a
>database and am trying out SQLAlchemy.
>
>A typical command might look like
>
>  um --operation add --uid ada --gid coders --lang en
>
>Parsing the arguments I get, ignoring the operation, a dict
>
>  {uid: "ada", gid: "coders", lang: "en"}
>
>At some point this needs to be converted into an object of the class User:
>
>  class User(Base):
>      __tablename__ = "users"
>
>      uid = Column('uid', String, primary_key=True)
>      gid = Column('gid', String)
>      lang = Column('lang', String)
>
>In a way it seems it would be economical to do the conversion as early
>as possible, so I can just pass around User objects.  However, this
>would mean that the entry point for the program would already be tightly
>coupled to the specifics of the database interaction.
>
>On the other hand, delaying the conversion would mean probably having to
>define my own User class, which seems like unnecessary overhead.  It
>would have the advantage that I could ditch SQLAlchemy more easily if I
>find it too mind-bending.

If the entire persistent state of the user lives in the db I'd just 
define the User ORM type and give it whatever methods you need. So 
exactly what you've got above.

It is close to the db, but if you only interact via the methods and the 
core attributes/columns that should be mostly irrelevant to you.

If you're concerned about switching backends, maybe define an 
AbstractUser abstract class with the required methods. Then you can at 
least ensure method coverage if you make another backend:

    class AbstractUser(ABC):
        @abstractmethod
        def some_user_method(self,...):


    class SQLAUser(Base, AbstractUser):
        ... your SQLA ORM User class above ...

    User = SQLAUser

    ... everything else just talks about user ...

But you can do all of that _later_, only needed if you decide to change 
backends in a controlled manner.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From wlfraed at ix.netcom.com  Thu Feb 10 18:00:05 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 10 Feb 2022 18:00:05 -0500
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
 <su0d1v$ju2$1@dont-email.me>
 <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>
 <mailman.70.1644432802.7010.python-list@python.org>
 <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>
 <CAPTjJmq+6UNruEGAHMyBQOtCk5LpwOz2caVkfhm=U8Mn4SqDNA@mail.gmail.com>
 <mailman.95.1644522198.7010.python-list@python.org>
 <su3tb4$ac5$1@dont-email.me>
Message-ID: <6i5b0hp9k1mia779vlpl9sre0qf7qgtj8p@4ax.com>

On Thu, 10 Feb 2022 21:39:05 +0100, Christian Gollwitzer <auriocus at gmx.de>
declaimed the following:


>Hence it is impossible to concurrently write from Python into an open 
>Excel file. One might ask what the real problem is the user is trying to 
>solve. Is Excel a requirement, can it be swapped by a database engine?
>

	Based upon the path names shown by the OP, this is just a continuation
of the January thread...

Message-ID: <199c23c7-de58-44ae-a216-760c8f36c506n at googlegroups.com>
Subject: What to write or search on github to get the code for what is
written below:
From: NArshad <narshad.380 at gmail.com>
Injection-Date: Thu, 06 Jan 2022 18:55:30 +0000

... in which the OP insists they are required to manipulate a (never fully
specified) spreadsheet maintained in an Excel format file. The main
take-away in the current thread is that the OP has relinquished the idea of
a web-based application, for a local Tkinter GUI (and apparently has
actually written some code finally, as they produced a traceback message
sequence <G> -- however, the code-first_then-come-here lesson isn't
sticking if you look at their second thread of the week).


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From martinp.dipaola at gmail.com  Thu Feb 10 23:21:29 2022
From: martinp.dipaola at gmail.com (Martin Di Paola)
Date: Fri, 11 Feb 2022 04:21:29 +0000
Subject: How do you log in your projects?
In-Reply-To: <CABbU2U8ON1x8t+umOc84vqS2haPuNTb+v9=_hRZD6BBq45wpJw@mail.gmail.com>
References: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>
 <CABbU2U-jTZXzNkG89q0PbXJuYLVpiOwJMR43bMMOZDnaRxu=kA@mail.gmail.com>
 <20220209193823.ez7ewt4sr7joplix@gmail.com>
 <CABbU2U8ON1x8t+umOc84vqS2haPuNTb+v9=_hRZD6BBq45wpJw@mail.gmail.com>
Message-ID: <20220211042129.jxcttb5reludf4kj@gmail.com>

>? Logs are not intended to be read by end users. Logs are primarily
>used to understand what the code is doing in a production environment.
>They could also be used to gather metrics data.
>
>Why should you log to give a message instead of simply using a print?

You are assuming that logs and prints are different but they aren't. It 
is the same technology: some string formatted in a particular way sent 
to some place (console or file generally).

But using the logging machinery instead a plain print() you win a few 
features like thread safety and log levels (think in an user that wants 
to increase the verbose level of the output)

When communicating with an user, the logs that are intended to him/her 
can be sent to the console (like a print) in addition to a file.

For user's perspective, they look just like a print.

>Why? Traceback is vital to understand what and where the problem is. I
>think you're confusing logs with messages. The stack trace can be
>logged (I would say must), but the end user generally sees a vague
>message with some hints, unless the program is used internally only.

Yes, that's exactly why the traceback is hidden by default because the 
user don't care about it. If the error is due something that the user 
did wrong, then the message should say that and, if possible, add 
a suggestion of how to do it.

For example "The file 'foo.md' was not found." is quite descriptive. If 
you add to that message a traceback, that will just clutter the console.

Tracebacks and other errors and warnings must be logged in a file.  
I totally agree with that. Specially true when we are talking with 
server-like software.

Tracebacks can be printed to the user if a more verbose output is 
enabled. In that case you could even pretty print the traceback with 
syntax highlighting.

I guess that this should be discussed case by case. May be you are 
thinking more in a case where you have a service running and logging and 
I am more thinking in a program that a human executes by hand.

What info and how is presented to the user changes quite a bit.
>-- https://mail.python.org/mailman/listinfo/python-list

From loris.bennett at fu-berlin.de  Fri Feb 11 02:05:24 2022
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Fri, 11 Feb 2022 08:05:24 +0100
Subject: Abstraction level at which to create SQLAlchemy ORM object
References: <87k0e2j22x.fsf@hornfels.zedat.fu-berlin.de>
 <YgWm8hz5chR51d60@cskk.homeip.net>
 <mailman.107.1644544140.7010.python-list@python.org>
Message-ID: <87leyhopcb.fsf@hornfels.zedat.fu-berlin.de>

Hi Cameron,

Cameron Simpson <cs at cskk.id.au> writes:

> On 10Feb2022 14:14, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>>I am writing a command line program which will modify entries in a
>>database and am trying out SQLAlchemy.
>>
>>A typical command might look like
>>
>>  um --operation add --uid ada --gid coders --lang en
>>
>>Parsing the arguments I get, ignoring the operation, a dict
>>
>>  {uid: "ada", gid: "coders", lang: "en"}
>>
>>At some point this needs to be converted into an object of the class User:
>>
>>  class User(Base):
>>      __tablename__ = "users"
>>
>>      uid = Column('uid', String, primary_key=True)
>>      gid = Column('gid', String)
>>      lang = Column('lang', String)
>>
>>In a way it seems it would be economical to do the conversion as early
>>as possible, so I can just pass around User objects.  However, this
>>would mean that the entry point for the program would already be tightly
>>coupled to the specifics of the database interaction.
>>
>>On the other hand, delaying the conversion would mean probably having to
>>define my own User class, which seems like unnecessary overhead.  It
>>would have the advantage that I could ditch SQLAlchemy more easily if I
>>find it too mind-bending.
>
> If the entire persistent state of the user lives in the db I'd just 
> define the User ORM type and give it whatever methods you need. So 
> exactly what you've got above.
>
> It is close to the db, but if you only interact via the methods and the 
> core attributes/columns that should be mostly irrelevant to you.
>
> If you're concerned about switching backends, maybe define an 
> AbstractUser abstract class with the required methods. Then you can at 
> least ensure method coverage if you make another backend:
>
>     class AbstractUser(ABC):
>         @abstractmethod
>         def some_user_method(self,...):
>
>
>     class SQLAUser(Base, AbstractUser):
>         ... your SQLA ORM User class above ...
>
>     User = SQLAUser
>
>     ... everything else just talks about user ...
>
> But you can do all of that _later_, only needed if you decide to change 
> backends in a controlled manner.

Thanks for reminding me about abstract classes, but more importantly
that I can do this kind of stuff later. 

Cheers,

Loris



From hjp-python at hjp.at  Fri Feb 11 14:37:57 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Fri, 11 Feb 2022 20:37:57 +0100
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
In-Reply-To: <CAPTjJmp4W9mMwuND85wST67vyC_KtnB6RbXPHqqNOgHJSKT3rQ@mail.gmail.com>
References: <6d76da9d-9d8d-4646-a05d-798e3e7bdcf2n@googlegroups.com>
 <su0d1v$ju2$1@dont-email.me>
 <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>
 <mailman.70.1644432802.7010.python-list@python.org>
 <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>
 <CAPTjJmq+6UNruEGAHMyBQOtCk5LpwOz2caVkfhm=U8Mn4SqDNA@mail.gmail.com>
 <mailman.95.1644522198.7010.python-list@python.org>
 <su3tb4$ac5$1@dont-email.me>
 <CAPTjJmp4W9mMwuND85wST67vyC_KtnB6RbXPHqqNOgHJSKT3rQ@mail.gmail.com>
Message-ID: <20220211193757.yf3sv3j5glknpmz6@hjp.at>

On 2022-02-11 08:11:32 +1100, Chris Angelico wrote:
> Excel doesn't have the sophistication to need or want anything more
> than simple "I have this file, nobody else touch it" exclusive
> locking.

Interestingly, Excel did have the ability for multiple users editing the
same file at some time (maybe early 2000s? Way before Google docs or
Office 365). It had to be explicitely enabled and it didn't work very
reliably (at least not with Samba as file servers), so we never really
used it but it is clear that somebody at MS thought that users needed or
at least wanted that ability.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220211/32aba9ed/attachment.sig>

From alister.ware at ntlworld.com  Fri Feb 11 16:11:27 2022
From: alister.ware at ntlworld.com (alister)
Date: Fri, 11 Feb 2022 21:11:27 -0000 (UTC)
Subject: How do you log in your projects?
References: <7feb1932-360f-03fd-5980-7f0699bba7e0@punkt.de>
 <CABbU2U-jTZXzNkG89q0PbXJuYLVpiOwJMR43bMMOZDnaRxu=kA@mail.gmail.com>
 <20220209193823.ez7ewt4sr7joplix@gmail.com>
 <mailman.71.1644435511.7010.python-list@python.org>
Message-ID: <su6jdv$1n5g$1@gioia.aioe.org>

On Wed, 9 Feb 2022 19:38:23 +0000, Martin Di Paola wrote:

>>> - On a line per line basis? on a function/method basis?
> 
> In general I prefer logging line by line instead per function.
> 
> It is easy to add a bunch of decorators to the functions and get the
> logs of all the program but I most of the time I end up with very
> confusing logs.
> 
> There are exceptions, yes, but I prefer the line by line where the log
> should explain what is doing the code.
> 
>>> - Which kind of variable contents do you write into your logfiles?
>>> - How do you decide, which kind of log message goes into which level?
>>> - How do you prevent logging cluttering your actual code?
> 
> These three comes to the same answer: I think on whom is going to read
> the logs.
> 
> If the logs are meant to be read by my users I log high level messages,
> specially before parts that can take a while (like the classic
> "Loading...").
> 
> If I log variables, those must be the ones set by the users so he/she
> can understand how he/she is controlling the behaviour of the program.
> 
> For exceptions I print the message but not the traceback. Across the
> code tag some important functions to put an extra message that will
> enhance the final message printed to the user.
> 
> https://github.com/byexamples/byexample/blob/master/byexample/
common.py#L192-L238
> 
> For example:
> 
>      for example in examples:
>          with enhance_exceptions(example, ...):
>              foo()
> 
> So if an exception is raised by foo(), enhance_exceptions() will attach
> to it useful information for the user from the example variable.
> 
> In the main, then I do the pretty print
> https://github.com/byexamples/byexample/blob/master/byexample/
byexample.py#L17-L22
> 
> If the user of the logs is me or any other developer I write more
> debugging stuff.
> 
> My approach is to not log anything and when I have to debug something I
> use a debugger + some prints. When the issue is fixed I review which
> prints would be super useful and I turn them into logs and the rest is
> deleted.
> 
> 
> On Tue, Feb 08, 2022 at 09:40:07PM +0100, Marco Sulla wrote:
>>These are a lot of questions. I hope we're not off topic.
>>I don't know if mine are best practices. I can tell what I try to do.
>>
>>On Tue, 8 Feb 2022 at 15:15, Lars Liedtke <liedtke at punkt.de> wrote:
>>> - On a line per line basis? on a function/method basis?
>>
>>I usually log the start and end of functions. I could also log inside a
>>branch or in other parts of the function/method.
>>
>>> - Do you use decorators to mark beginnings and ends of
>>> methods/functions in log files?
>>
>>No, since I put the function parameters in the first log. But I think
>>that such a decorator it's not bad.
>>
>>> - Which kind of variable contents do you write into your logfiles? Of
>>> course you shouldn't leak secrets...
>>
>>Well, all the data that is useful to understand what the code is doing.
>>It's better to repeat the essential data to identify a specific call in
>>all the logs of the function, so if it is called simultaneously by more
>>clients you can distinguish them
>>
>>> - How do you decide, which kind of log message goes into which level?
>>
>>It depends on the importance, the verbosity and the occurrences of the
>>logs.
>>
>>> - How do you prevent logging cluttering your actual code?
>>
>>I have the opposite problem, I should log more. So I can't answer your
>>question.
>>--
>>https://mail.python.org/mailman/listinfo/python-list
In my current project I use the loggin module & write to a file
I log at info level for the entry & exit of each function/method
& scatter debug level logs for arease where I need to check data is as 
expected
I have also found Ansi colour codes usefull when skiming through the 
output





-- 
If an experiment works, something has gone wrong.

From wlfraed at ix.netcom.com  Fri Feb 11 18:20:19 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Fri, 11 Feb 2022 18:20:19 -0500
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
References: <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>
 <mailman.70.1644432802.7010.python-list@python.org>
 <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>
 <CAPTjJmq+6UNruEGAHMyBQOtCk5LpwOz2caVkfhm=U8Mn4SqDNA@mail.gmail.com>
 <mailman.95.1644522198.7010.python-list@python.org>
 <su3tb4$ac5$1@dont-email.me>
 <CAPTjJmp4W9mMwuND85wST67vyC_KtnB6RbXPHqqNOgHJSKT3rQ@mail.gmail.com>
 <20220211193757.yf3sv3j5glknpmz6@hjp.at>
 <mailman.109.1644608287.7010.python-list@python.org>
Message-ID: <s8qd0hld254olr1t47ujtrrc10arb71mut@4ax.com>

On Fri, 11 Feb 2022 20:37:57 +0100, "Peter J. Holzer" <hjp-python at hjp.at>
declaimed the following:

>Interestingly, Excel did have the ability for multiple users editing the
>same file at some time (maybe early 2000s? Way before Google docs or
>Office 365). It had to be explicitely enabled and it didn't work very
>reliably (at least not with Samba as file servers), so we never really
>used it but it is clear that somebody at MS thought that users needed or
>at least wanted that ability.

	A quick Google does find mention of "shared workbooks":
https://docs.microsoft.com/en-us/office/troubleshoot/excel/use-shared-workbook
https://support.microsoft.com/en-us/office/what-happened-to-shared-workbooks-150fc205-990a-4763-82f1-6c259303fe05

	The preferred mode requires "Microsoft 365 subscription" and latest
Office version:
https://support.microsoft.com/en-us/office/collaborate-on-excel-workbooks-at-the-same-time-with-co-authoring-7152aa8b-b791-414c-a3bb-3024e46fb104

	However -- the key feature is that these are Excel-Excel(-Excel...)
operations (and the co-author mode needs M$ subscription and use of
OneDrive cloud storage). 

	Most Python libraries are working directly with the Excel format data
file, not by running Excel. 

	Caveat: there is a product that appears to use the Excel component API
to control Excel itself, not just read/write xls(*) files -- so it might
understand the older shared workbook mode, but I'd recommend a detailed
study of the API and the sharing operations first:
https://www.xlwings.org/
https://docs.xlwings.org/en/stable/
"""
xlwings (Open Source) is a BSD-licensed Python library that makes it easy
to call Python from Excel and vice versa:
    *	Scripting: Automate/interact with Excel from Python using a syntax
close to VBA.
    *	Macros: Replace VBA macros with clean and powerful Python code.
    *	UDFs: Write User Defined Functions (UDFs) in Python (Windows only).
"""

	If running under Windows, similar capability should be possible using
the win32py (or whatever the current name is) extension... Or with more
difficulty, ctypes!


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From cousinstanley at gmail.com  Sat Feb 12 11:53:39 2022
From: cousinstanley at gmail.com (Cousin Stanley)
Date: Sat, 12 Feb 2022 09:53:39 -0700
Subject: Python LSTM forecast future values for time series
References: <53723684-e4f8-fb59-7d17-71ee6cc7e4b7@inpe.br>
 <mailman.92.1644514663.7010.python-list@python.org>
Message-ID: <su8omm$b9c$1@dont-email.me>

Jorge Conforte wrote:

> ....
> I'm starting run the LSTM to forecast future values for time serie data.
> 
> please can someone give me some information 
> on how i can predict future values 
??> for my time series using LSTM. Thanks, Conrado

  I cannot personlly help but a google search
  using the following phrase seems promising .... 

    lstm predict future values
    

-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona


From shishaozhong at gmail.com  Sat Feb 12 15:15:43 2022
From: shishaozhong at gmail.com (Shaozhong SHI)
Date: Sat, 12 Feb 2022 20:15:43 +0000
Subject: URLError: <urlopen error [Errno 11001] getaddrinfo failed>
Message-ID: <CA+i5JwY6udg9TBNUKJQOnFFAG7_R-rd-U61iF7r1_w3utZMXBg@mail.gmail.com>

The following is used in a loop to get response code for each url.

print (urllib.request.urlopen(url).getcode())

However, error message says: URLError: <urlopen error [Errno 11001]
getaddrinfo failed>

Python 3.6.5 is being used to test whether url is live or not.

Can anyone shed light on this?

Regards,

David

From hjp-python at hjp.at  Sat Feb 12 15:17:01 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 12 Feb 2022 21:17:01 +0100
Subject: PermissionError: [Errno 13] Permission denied: 'Abc.xlsx'
In-Reply-To: <s8qd0hld254olr1t47ujtrrc10arb71mut@4ax.com>
References: <3026c9cf-9d36-9301-032c-2d31b61866e1@mrabarnett.plus.com>
 <mailman.70.1644432802.7010.python-list@python.org>
 <8kka0h9l1eaogdvouo3g9ng8t85c01mm3e@4ax.com>
 <CAPTjJmq+6UNruEGAHMyBQOtCk5LpwOz2caVkfhm=U8Mn4SqDNA@mail.gmail.com>
 <mailman.95.1644522198.7010.python-list@python.org>
 <su3tb4$ac5$1@dont-email.me>
 <CAPTjJmp4W9mMwuND85wST67vyC_KtnB6RbXPHqqNOgHJSKT3rQ@mail.gmail.com>
 <20220211193757.yf3sv3j5glknpmz6@hjp.at>
 <mailman.109.1644608287.7010.python-list@python.org>
 <s8qd0hld254olr1t47ujtrrc10arb71mut@4ax.com>
Message-ID: <20220212201701.cqkjttd4t3ntpuza@hjp.at>

On 2022-02-11 18:20:19 -0500, Dennis Lee Bieber wrote:
> On Fri, 11 Feb 2022 20:37:57 +0100, "Peter J. Holzer" <hjp-python at hjp.at>
> declaimed the following:
> 
> >Interestingly, Excel did have the ability for multiple users editing the
> >same file at some time (maybe early 2000s? Way before Google docs or
                                              ^^^^^^^^^^
> >Office 365). It had to be explicitely enabled and it didn't work very
   ^^^^^^^^^^
> >reliably (at least not with Samba as file servers), so we never really
> >used it but it is clear that somebody at MS thought that users needed or
> >at least wanted that ability.
> 
> 	A quick Google does find mention of "shared workbooks":
> https://docs.microsoft.com/en-us/office/troubleshoot/excel/use-shared-workbook

| To make changes to a shared workbook that was created in Microsoft Excel 97
| or a later version of Excel, you must use Excel 97 or a later version of Excel.

So apparently that ability was added in Excel 97.


> 	The preferred mode requires "Microsoft 365 subscription" and latest
> Office version:

Yes, these days you would use Office 365 for that kind of functionality,
but that didn't exist at that time (neither did cloud storage systems).

Network file systems like NFS or SMB did exist, though, and people
wanted (or at least MS thought they wanted) to collaborate on Excel
files. So Excel provided a way to do that.

That was sort of the point: That Excel was not *always* single-user.
There have been ways that multiple users could simultaneously edit the
same file for 25 years.


> 	However -- the key feature is that these are Excel-Excel(-Excel...)
> operations

I'm pretty sure that that worked entirely through the file system. So
the only thing stopping you from implementing it in a different program
was the lack of documentation (as was typical for MS in the 1990's).

But again, this wasn't my point. My point was that Excel files are
designed to be used only by a single process isn't true.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220212/e0652a19/attachment.sig>

From hjp-python at hjp.at  Sat Feb 12 15:25:34 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 12 Feb 2022 21:25:34 +0100
Subject: URLError: <urlopen error [Errno 11001] getaddrinfo failed>
In-Reply-To: <CA+i5JwY6udg9TBNUKJQOnFFAG7_R-rd-U61iF7r1_w3utZMXBg@mail.gmail.com>
References: <CA+i5JwY6udg9TBNUKJQOnFFAG7_R-rd-U61iF7r1_w3utZMXBg@mail.gmail.com>
Message-ID: <20220212202534.whgkg44pxfjxx4iy@hjp.at>

On 2022-02-12 20:15:43 +0000, Shaozhong SHI wrote:
> The following is used in a loop to get response code for each url.
> 
> print (urllib.request.urlopen(url).getcode())
> 
> However, error message says: URLError: <urlopen error [Errno 11001]
> getaddrinfo failed>
> 
> Python 3.6.5 is being used to test whether url is live or not.
> 
> Can anyone shed light on this?

getaddrinfo is the function that resolves a domain name into an IP
address. If that fails, either the domain name doesn't exist or you have
a problem with your DNS setup.

Things to test:

* Can you resolve the domain name to an address (try ?nslookup? or
  ?host? or ?dig? depending on your OS)?
* Can you access the whole URL with a different tool like ?wget? or
  ?curl??

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220212/67f0aca0/attachment.sig>

From rosuav at gmail.com  Sat Feb 12 15:25:36 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 13 Feb 2022 07:25:36 +1100
Subject: URLError: <urlopen error [Errno 11001] getaddrinfo failed>
In-Reply-To: <CA+i5JwY6udg9TBNUKJQOnFFAG7_R-rd-U61iF7r1_w3utZMXBg@mail.gmail.com>
References: <CA+i5JwY6udg9TBNUKJQOnFFAG7_R-rd-U61iF7r1_w3utZMXBg@mail.gmail.com>
Message-ID: <CAPTjJmqiA8ZYjzkkum+iNcBeNjNkyoMSEG-mWG-X+KGXXcgX+Q@mail.gmail.com>

On Sun, 13 Feb 2022 at 07:17, Shaozhong SHI <shishaozhong at gmail.com> wrote:
>
> The following is used in a loop to get response code for each url.
>
> print (urllib.request.urlopen(url).getcode())
>
> However, error message says: URLError: <urlopen error [Errno 11001]
> getaddrinfo failed>
>
> Python 3.6.5 is being used to test whether url is live or not.
>
> Can anyone shed light on this?
>

What that's saying is that it couldn't look up the domain name to get
the corresponding address. Most likely, that means you either don't
have DNS available, or the DNS server said that it couldn't find that
server. (Whether that's true or not. It's always possible for the DNS
server to be wrong.) So, if you can fairly safely assume that you have
a fully-functioning internet connection (for instance, if other URLs
can be fetched successfully), the most reasonable interpretation of
this is that the URL is, in some way, broken.

I suspect that errno 11001 is a Winsock error, in which case it would
mean "Host not found". On my Linux system, I get a chained exception
that says "Name or service not found". Most likely, you can find this
information further up in the traceback.

ChrisA

From gisle.vanem at gmail.com  Sun Feb 13 02:45:55 2022
From: gisle.vanem at gmail.com (Gisle Vanem)
Date: Sun, 13 Feb 2022 08:45:55 +0100
Subject: URLError: <urlopen error [Errno 11001] getaddrinfo failed>
In-Reply-To: <CA+i5JwY6udg9TBNUKJQOnFFAG7_R-rd-U61iF7r1_w3utZMXBg@mail.gmail.com>
References: <CA+i5JwY6udg9TBNUKJQOnFFAG7_R-rd-U61iF7r1_w3utZMXBg@mail.gmail.com>
Message-ID: <3bc439fa-f793-fe89-26d4-3a4a7ec0f4c6@gmail.com>

Shaozhong SHI wrote:

> The following is used in a loop to get response code for each url.
> 
> print (urllib.request.urlopen(url).getcode())
> 
> However, error message says: URLError: <urlopen error [Errno 11001]

11001 == WSAHOST_NOT_FOUND.

Look in any 'winsock.h' header:
   #define WSAHOST_NOT_FOUND (WSABASEERR+1001)

-- 
--gv

From martin.schoon at gmail.com  Mon Feb 14 15:54:01 2022
From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=)
Date: 14 Feb 2022 20:54:01 GMT
Subject: Saving/exporting plots from Jupyter-labs?
Message-ID: <j6vtr9F64mrU1@mid.individual.net>

I have used Jupyter notebooks for some time now. I am not a heavy
or advanced user. I find the notebook format a nice way to create 
python documents.

Now I am trying out Jupyter-labs. I like it. I have two head-
scratchers for now:

1) In notebooks I can save a plot by right-clicking on it and do
save image as. In Jupyter-lab that does not work and so far I
have not been able to figure out how to do it. Yes, I have looked
in the documentation.

2) Why is Jupyter-labs hooking up to Google-analytics?

TIA,

/Martin

From jenkris at tutanota.com  Mon Feb 14 20:05:42 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 15 Feb 2022 02:05:42 +0100 (CET)
Subject: C API - How to return a Dictionary as a Dictionary type
Message-ID: <MvuuThb--3-2@tutanota.com>

I created a dictionary with the Python C API and assigned two keys and values:

PyObject* this_dict = PyDict_New();?
const char *key = "key1";
char *val = "data_01";?
PyObject* val_p = PyUnicode_FromString(val);?
int r = PyDict_SetItemString(this_dict, key, val_p);?

// Add another k-v pair
key = "key2";
val = "data_02";?
val_p = PyUnicode_FromString(val);?
r = PyDict_SetItemString(this_dict, key, val_p);?

I need to retrieve the entire dictionary to be passed to a library function that expects a dictionary.? I used? PyDict_Items:

PyObject* pdi = PyDict_Items(this_dict);
PyObject* str_untagd = PyObject_Str(pdi);
PyObject* repr_utd = PyObject_Repr(str_untagd);
PyObject* str_utd = PyUnicode_AsEncodedString(repr_utd, "utf-8", "~E~");??
const char *bytes_d = PyBytes_AS_STRING(str_utd);
printf("REPR_UnTag: %s\n", bytes_d);

but as the docs say (https://docs.python.org/3.8/c-api/dict.html), that returns a PyListObject, not a dictionary enclosed with curly braces:?

[('key1', 'data_01'), ('key2', 'data_02')]".?

My question is, how can I get the dictionary as a dictionary type, enclosed with curly braces.? I found PyObject_GenericGetDict (https://docs.python.org/3.8/c-api/object.html) but I haven't found any documentation or explanation of how it works.?

Is PyObject_GenericGetDict what I need, or is there another way to do it?


Thanks,

Jen



From rosuav at gmail.com  Mon Feb 14 20:24:31 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 15 Feb 2022 12:24:31 +1100
Subject: C API - How to return a Dictionary as a Dictionary type
In-Reply-To: <MvuuThb--3-2@tutanota.com>
References: <MvuuThb--3-2@tutanota.com>
Message-ID: <CAPTjJmr=q7McVtrmaNW33qSbQZMQ-86bfp7oTybQ5zSSxFd6pA@mail.gmail.com>

On Tue, 15 Feb 2022 at 12:07, Jen Kris via Python-list
<python-list at python.org> wrote:
>
> I created a dictionary with the Python C API and assigned two keys and values:
>
> PyObject* this_dict = PyDict_New();
> const char *key = "key1";
> char *val = "data_01";
> PyObject* val_p = PyUnicode_FromString(val);
> int r = PyDict_SetItemString(this_dict, key, val_p);
>
> // Add another k-v pair
> key = "key2";
> val = "data_02";
> val_p = PyUnicode_FromString(val);
> r = PyDict_SetItemString(this_dict, key, val_p);
>
> I need to retrieve the entire dictionary to be passed to a library function that expects a dictionary.  I used  PyDict_Items:
>
> PyObject* pdi = PyDict_Items(this_dict);
> PyObject* str_untagd = PyObject_Str(pdi);
> PyObject* repr_utd = PyObject_Repr(str_untagd);
> PyObject* str_utd = PyUnicode_AsEncodedString(repr_utd, "utf-8", "~E~");
> const char *bytes_d = PyBytes_AS_STRING(str_utd);
> printf("REPR_UnTag: %s\n", bytes_d);
>
> but as the docs say (https://docs.python.org/3.8/c-api/dict.html), that returns a PyListObject, not a dictionary enclosed with curly braces:
>
> [('key1', 'data_01'), ('key2', 'data_02')]".
>
> My question is, how can I get the dictionary as a dictionary type, enclosed with curly braces.  I found PyObject_GenericGetDict (https://docs.python.org/3.8/c-api/object.html) but I haven't found any documentation or explanation of how it works.
>
> Is PyObject_GenericGetDict what I need, or is there another way to do it?
>

Not sure what you mean. The dict is already a dict. If you refer to
this_dict, it is a dict, right?

If you need the string representation of that, you should be able to
call PyObject_Repr just as you are, but call it on the dict, not on
the dict items.

ChrisA

From jenkris at tutanota.com  Mon Feb 14 20:35:35 2022
From: jenkris at tutanota.com (Jen Kris)
Date: Tue, 15 Feb 2022 02:35:35 +0100 (CET)
Subject: C API - How to return a Dictionary as a Dictionary type
In-Reply-To: <CAPTjJmr=q7McVtrmaNW33qSbQZMQ-86bfp7oTybQ5zSSxFd6pA@mail.gmail.com>
References: <MvuuThb--3-2@tutanota.com>
 <CAPTjJmr=q7McVtrmaNW33qSbQZMQ-86bfp7oTybQ5zSSxFd6pA@mail.gmail.com>
Message-ID: <Mvv0JcW--3-2@tutanota.com>

Yes, that works.? This is my first day with C API dictionaries.? Now that you've explained it, it makes perfect sense.? Thanks much.? 

Jen


Feb 14, 2022, 17:24 by rosuav at gmail.com:

> On Tue, 15 Feb 2022 at 12:07, Jen Kris via Python-list
> <python-list at python.org> wrote:
>
>>
>> I created a dictionary with the Python C API and assigned two keys and values:
>>
>> PyObject* this_dict = PyDict_New();
>> const char *key = "key1";
>> char *val = "data_01";
>> PyObject* val_p = PyUnicode_FromString(val);
>> int r = PyDict_SetItemString(this_dict, key, val_p);
>>
>> // Add another k-v pair
>> key = "key2";
>> val = "data_02";
>> val_p = PyUnicode_FromString(val);
>> r = PyDict_SetItemString(this_dict, key, val_p);
>>
>> I need to retrieve the entire dictionary to be passed to a library function that expects a dictionary.  I used  PyDict_Items:
>>
>> PyObject* pdi = PyDict_Items(this_dict);
>> PyObject* str_untagd = PyObject_Str(pdi);
>> PyObject* repr_utd = PyObject_Repr(str_untagd);
>> PyObject* str_utd = PyUnicode_AsEncodedString(repr_utd, "utf-8", "~E~");
>> const char *bytes_d = PyBytes_AS_STRING(str_utd);
>> printf("REPR_UnTag: %s\n", bytes_d);
>>
>> but as the docs say (https://docs.python.org/3.8/c-api/dict.html), that returns a PyListObject, not a dictionary enclosed with curly braces:
>>
>> [('key1', 'data_01'), ('key2', 'data_02')]".
>>
>> My question is, how can I get the dictionary as a dictionary type, enclosed with curly braces.  I found PyObject_GenericGetDict (https://docs.python.org/3.8/c-api/object.html) but I haven't found any documentation or explanation of how it works.
>>
>> Is PyObject_GenericGetDict what I need, or is there another way to do it?
>>
>
> Not sure what you mean. The dict is already a dict. If you refer to
> this_dict, it is a dict, right?
>
> If you need the string representation of that, you should be able to
> call PyObject_Repr just as you are, but call it on the dict, not on
> the dict items.
>
> ChrisA
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>


From mirkok.lists at googlemail.com  Tue Feb 15 00:35:18 2022
From: mirkok.lists at googlemail.com (Mirko)
Date: Tue, 15 Feb 2022 06:35:18 +0100
Subject: venv and executing other python programs
Message-ID: <620B3B96.7080309@googlemail.com>

Hi,

I have recently started using venv for my hobby-programming. There
is an annoying problem. Since venv modifies $PATH, python programs
that use the "#!/usr/bin/env python" variant of the hashbang often
fail since their additional modules aren't install inside in venv.

How to people here deal with that?

Please note: I'm not interested in discussing whether the
env-variant is good or bad. ;-) It's not that *I* use it, but
several progs in /usr/bin/.

Thanks for your time.

From reto at labrat.space  Tue Feb 15 02:22:40 2022
From: reto at labrat.space (Reto)
Date: Tue, 15 Feb 2022 08:22:40 +0100
Subject: Saving/exporting plots from Jupyter-labs?
In-Reply-To: <j6vtr9F64mrU1@mid.individual.net>
References: <j6vtr9F64mrU1@mid.individual.net>
Message-ID: <20220215072240.a24hhe3wgwepqez2@feather>

On Mon, Feb 14, 2022 at 08:54:01PM +0000, Martin Sch??n wrote:
> 1) In notebooks I can save a plot by right-clicking on it and do
> save image as. In Jupyter-lab that does not work and so far I
> have not been able to figure out how to do it. Yes, I have looked
> in the documentation.

Shift + right click brings up the usual browser menu

From reto at labrat.space  Tue Feb 15 02:21:43 2022
From: reto at labrat.space (Reto)
Date: Tue, 15 Feb 2022 08:21:43 +0100
Subject: venv and executing other python programs
In-Reply-To: <620B3B96.7080309@googlemail.com>
References: <620B3B96.7080309@googlemail.com>
Message-ID: <20220215072143.nketxewuqpniqgeu@feather>

On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
> How to people here deal with that?

Don't activate the venv for those programs then?
The point of a venv is that you only enter it when you actually want
that specific python stack.

Get yourself a terminal that can either multiplex, or add something like
tmux or screen to the mix if you frequently need other python tools
during development.
Or just install those in you venv, after all if you
do use them for dev they are part of your dependencies, so declare them
as such.

From barry at barrys-emacs.org  Tue Feb 15 02:53:50 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Tue, 15 Feb 2022 07:53:50 +0000
Subject: venv and executing other python programs
In-Reply-To: <620B3B96.7080309@googlemail.com>
References: <620B3B96.7080309@googlemail.com>
Message-ID: <CA3AFDA4-DFFF-4E18-B58E-9A6AF0CEA046@barrys-emacs.org>



> On 15 Feb 2022, at 05:35, Mirko via Python-list <python-list at python.org> wrote:
> 
> Hi,
> 
> I have recently started using venv for my hobby-programming. There
> is an annoying problem. Since venv modifies $PATH, python programs
> that use the "#!/usr/bin/env python" variant of the hashbang often
> fail since their additional modules aren't install inside in venv.

Do you mean that your python code is running another python program
via subprocess?

You would need to provide a PATH that does not include the venv so that env
works as expected.

Or are you running the program from the command line after activating the
venv?

I create the venv and use it to run the program using
 <path-to-venv>/bin/python program.py without activating the venv

> How to people here deal with that?
> 
> Please note: I'm not interested in discussing whether the
> env-variant is good or bad. ;-) It's not that *I* use it, but
> several progs in /usr/bin/.

At least for Fedora it does not allow packages to install programs
that use /usr/bin/env because of issues like you are seeing.

Barry

> 
> Thanks for your time.
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From roel at roelschroeven.net  Tue Feb 15 05:18:29 2022
From: roel at roelschroeven.net (Roel Schroeven)
Date: Tue, 15 Feb 2022 11:18:29 +0100
Subject: venv and executing other python programs
In-Reply-To: <20220215072143.nketxewuqpniqgeu@feather>
References: <620B3B96.7080309@googlemail.com>
 <20220215072143.nketxewuqpniqgeu@feather>
Message-ID: <814283fb-24b1-20ec-e44d-ec8c0317d447@roelschroeven.net>

Op 15/02/2022 om 8:21 schreef Reto:
> On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
> > How to people here deal with that?
>
> Don't activate the venv for those programs then?
> The point of a venv is that you only enter it when you actually want
> that specific python stack.
>
> Get yourself a terminal that can either multiplex, or add something like
> tmux or screen to the mix if you frequently need other python tools
> during development.
> Or just install those in you venv, after all if you
> do use them for dev they are part of your dependencies, so declare them
> as such.
Still feels like a problem to me though.

Suppose you're working on a program which, for example, prints json to 
stdout. And suppose you want to use a tool like jq (let's call it pjq as 
an example) to process that output, by piping the output of your program 
to it:

 ??? python your-program.py | pjq

That seems to me a very sensible thing to do. If pjq is written in C, 
C++, Rust, Ruby, PHP, Go, or whatever, itis not a problem at all. To the 
user of pjq, it shouldn't matter at all in what language it's written, 
and mostly it doesn't matter at all indeed.

But if pjq happens to be written in Python, it suddenly matters very 
much: it won't work in your venv (unless your venv happens to have the 
right modules).

There are ways around it of course:
- install the tool in your venv, like you say
- from outside the venv: venv/bin/python your_program.py | pjq
- from inside the venv: python your_program.py | <global python 
interpreter> <path-to-pjq>
Or something like that. At least I think those should work, I haven't 
tested it.

But it feels to wrong to have to use such workarounds simply because pjq 
happens to be written in the same language as you're writing your 
program in.

-- 
"A common mistake that people make when trying to design something completely
foolproof is to underestimate the ingenuity of complete fools."
         -- Douglas Adams


From rosuav at gmail.com  Tue Feb 15 05:34:51 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 15 Feb 2022 21:34:51 +1100
Subject: venv and executing other python programs
In-Reply-To: <814283fb-24b1-20ec-e44d-ec8c0317d447@roelschroeven.net>
References: <620B3B96.7080309@googlemail.com>
 <20220215072143.nketxewuqpniqgeu@feather>
 <814283fb-24b1-20ec-e44d-ec8c0317d447@roelschroeven.net>
Message-ID: <CAPTjJmpjFW4yLCH66kDcqSmK9=TDch=akCxnkE8wA7N6aRuSXg@mail.gmail.com>

On Tue, 15 Feb 2022 at 21:19, Roel Schroeven <roel at roelschroeven.net> wrote:
>
> Op 15/02/2022 om 8:21 schreef Reto:
> > On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
> > > How to people here deal with that?
> >
> > Don't activate the venv for those programs then?
> > The point of a venv is that you only enter it when you actually want
> > that specific python stack.
> >
> > Get yourself a terminal that can either multiplex, or add something like
> > tmux or screen to the mix if you frequently need other python tools
> > during development.
> > Or just install those in you venv, after all if you
> > do use them for dev they are part of your dependencies, so declare them
> > as such.
> Still feels like a problem to me though.
>
> Suppose you're working on a program which, for example, prints json to
> stdout. And suppose you want to use a tool like jq (let's call it pjq as
> an example) to process that output, by piping the output of your program
> to it:
>
>      python your-program.py | pjq
>
> That seems to me a very sensible thing to do. If pjq is written in C,
> C++, Rust, Ruby, PHP, Go, or whatever, itis not a problem at all. To the
> user of pjq, it shouldn't matter at all in what language it's written,
> and mostly it doesn't matter at all indeed.
>
> But if pjq happens to be written in Python, it suddenly matters very
> much: it won't work in your venv (unless your venv happens to have the
> right modules).
>
> There are ways around it of course:
> - install the tool in your venv, like you say
> - from outside the venv: venv/bin/python your_program.py | pjq
> - from inside the venv: python your_program.py | <global python
> interpreter> <path-to-pjq>
> Or something like that. At least I think those should work, I haven't
> tested it.
>
> But it feels to wrong to have to use such workarounds simply because pjq
> happens to be written in the same language as you're writing your
> program in.
>

The pjq shebang shouldn't specify /usr/bin/env unless you want it to
run in whatever Python is the current default. In other words, if it
matters, be more specific. That usually means either using
/usr/bin/python3 for your system Python, or the interpreter out of the
venv itself - /path/to/venv/bin/python3 - to specify that.

You're absolutely right, workarounds like that are wrong. But the
wrongness comes from using env when you don't want env :)

ChrisA

From martinp.dipaola at gmail.com  Tue Feb 15 07:36:53 2022
From: martinp.dipaola at gmail.com (Martin Di Paola)
Date: Tue, 15 Feb 2022 12:36:53 +0000
Subject: venv and executing other python programs
In-Reply-To: <620B3B96.7080309@googlemail.com>
References: <620B3B96.7080309@googlemail.com>
Message-ID: <20220215123653.5u4zpwrelyd2fela@gmail.com>

I did a few experiments in my machine. I created the following foo.py

   import pandas
   print("foo")

Now "pandas" is installed under Python 3 outside the venv. I can run it 
successfully calling "python3 foo.py".

If I add the shebang "#!/usr/bin/env python3" (notice the 3), I can also 
run it as "./foo.py".

Calling it as "python foo.py" or using the shebang "#!/usr/bin/env 
python" does not work and it makes sense since "pandas" is installed
only for python 3.

New I create a virtual env with "python3 -m venv xxx" and activate it.

Once inside I can run foo.py in 4 different ways:

  - python foo.py
  - python3 foo.py
  - ./foo.py (using the shebang "#!/usr/bin/env python")
  - ./foo.py (using the shebang "#!/usr/bin/env python3")

Now all of that was with "pandas" installed outside of venv but not 
inside.

I did the same experiments with another library, "cryptonita" which it 
is not installed outside but inside and I was able to executed in 
4 different ways too (inside the venv of course):

  - python foo.py
  - python3 foo.py
  - ./foo.py (using the shebang "#!/usr/bin/env python")
  - ./foo.py (using the shebang "#!/usr/bin/env python3")

Do you have a particular context where you are having troubles? May be 
there is something else going on...

Thanks,
Martin.

On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
>Hi,
>
>I have recently started using venv for my hobby-programming. There
>is an annoying problem. Since venv modifies $PATH, python programs
>that use the "#!/usr/bin/env python" variant of the hashbang often
>fail since their additional modules aren't install inside in venv.
>
>How to people here deal with that?
>
>Please note: I'm not interested in discussing whether the
>env-variant is good or bad. ;-) It's not that *I* use it, but
>several progs in /usr/bin/.
>
>Thanks for your time.
>-- 
>https://mail.python.org/mailman/listinfo/python-list

From aninditadas677 at gmail.com  Tue Feb 15 05:22:39 2022
From: aninditadas677 at gmail.com (11_Anindita Das_BCA)
Date: Tue, 15 Feb 2022 15:52:39 +0530
Subject: ERROR IN DOWNLOADING PYTHON
Message-ID: <CA+recJUWvRnymzBXej-wy-CQYS8cPua4dCnJFafLXvLU2GxDpg@mail.gmail.com>

Dear Sir/Mam
I have downloaded the latest 3.10.2 version of python for my 64?64 bit
laptop but I'm unable to work on it as my system is showing that
api-ms-win-crt-runtime-l1-1-0-dll is missing but it's not the case as i
have also downloaded this file.
Please let me know what should be my immediate step.
Looking forward to an early response from you
Yours Sincerely
Anindita Das

From martin.schoon at gmail.com  Tue Feb 15 05:47:51 2022
From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=)
Date: 15 Feb 2022 10:47:51 GMT
Subject: Saving/exporting plots from Jupyter-labs?
References: <j6vtr9F64mrU1@mid.individual.net>
 <20220215072240.a24hhe3wgwepqez2@feather>
 <mailman.119.1644909762.7010.python-list@python.org>
Message-ID: <j71emnFeuu0U1@mid.individual.net>

Den 2022-02-15 skrev Reto <reto at labrat.space>:
> On Mon, Feb 14, 2022 at 08:54:01PM +0000, Martin Sch??n wrote:
>> 1) In notebooks I can save a plot by right-clicking on it and do
>> save image as. In Jupyter-lab that does not work and so far I
>> have not been able to figure out how to do it. Yes, I have looked
>> in the documentation.
>
> Shift + right click brings up the usual browser menu

Thanks,

/Martin

From eryksun at gmail.com  Tue Feb 15 09:06:11 2022
From: eryksun at gmail.com (Eryk Sun)
Date: Tue, 15 Feb 2022 08:06:11 -0600
Subject: ERROR IN DOWNLOADING PYTHON
In-Reply-To: <CA+recJUWvRnymzBXej-wy-CQYS8cPua4dCnJFafLXvLU2GxDpg@mail.gmail.com>
References: <CA+recJUWvRnymzBXej-wy-CQYS8cPua4dCnJFafLXvLU2GxDpg@mail.gmail.com>
Message-ID: <CACL+1avtyBSCrKQKKzpuj0Fbiiob7zo73gm21QCFwzPmBPZiGw@mail.gmail.com>

On 2/15/22, 11_Anindita Das_BCA <aninditadas677 at gmail.com> wrote:
>
> I have downloaded the latest 3.10.2 version of python for my 64?64 bit
> laptop but I'm unable to work on it as my system is showing that
> api-ms-win-crt-runtime-l1-1-0-dll is missing but it's not the case as i
> have also downloaded this file.

I guess that you're using Windows 7 since Windows 10+ includes the
Universal C Runtime (ucrt) as a required system component, for which
"api-ms-win-crt-runtime-l1-1-0-dll" is a virtual API set (mapped to
ucrtbase.dll), which can never be missing, and since Windows 8.1 is
relatively uncommon (2-3% of installations).

Python 3.10 requires Windows 8.1 and above. If you're using Windows 7,
you should be able to install and use Python 3.8.10:

https://www.python.org/ftp/python/3.8.10/python-3.8.10-amd64.exe

If you need to install ucrt, here's the link for the update that was
released in 2016:

https://support.microsoft.com/kb/3118401

From mirkok.lists at googlemail.com  Tue Feb 15 09:27:16 2022
From: mirkok.lists at googlemail.com (Mirko)
Date: Tue, 15 Feb 2022 15:27:16 +0100
Subject: venv and executing other python programs
In-Reply-To: <20220215072143.nketxewuqpniqgeu@feather>
References: <620B3B96.7080309@googlemail.com>
 <20220215072143.nketxewuqpniqgeu@feather>
Message-ID: <620BB844.5030804@googlemail.com>

Am 15.02.2022 um 08:53 schrieb Barry Scott:
> Or are you running the program from the command line after activating the
> venv?

This ...

Am 15.02.2022 um 11:18 schrieb Roel Schroeven:
> Suppose you're working on a program which, for example, prints json
> to stdout. And suppose you want to use a tool like jq (let's call it
> pjq as an example) to process that output, by piping the output of
> your program to it:

... and this.


Am 15.02.2022 um 08:21 schrieb Reto:
> Don't activate the venv for those programs then?
> The point of a venv is that you only enter it when you actually want
> that specific python stack.

Well, it's not that I activate the venv *for* those troubling
programs. I activate it to work on the particular project. ;-)

> Get yourself a terminal that can either multiplex, or add something like
> tmux or screen to the mix if you frequently need other python tools
> during development.

I already do that (terminator, since I don't get tmux to behave as I
want it).


Am 15.02.2022 um 13:36 schrieb Martin Di Paola:
> I did a few experiments in my machine. I created the following foo.py
[SNIP]
> Do you have a particular context where you are having troubles? May
> be there is something else going on...

It seems to me that on your system, venv includes the site-packages
by default. For that I've found a solution: Create the venv with
--system-site-packages or in the venv's pyvenv.cfg just set/add
"include-system-site-packages = true"

Without this, I get the following (Python 3.6.9, Linux Mint 19,
Pandas installed in /usr/lib/python3/dist-packages/pandas):


[2,  0] mirko at wizbox:~$ vim /home/mirko/bin/foo.py
[2,  0] mirko at wizbox:~$ chmod +x /home/mirko/bin/foo.py
[2,  0] mirko at wizbox:~$ cat /home/mirko/bin/foo.py
#!/usr/bin/env python3

import pandas
print("Works")

[2,  0] mirko at wizbox:~$ foo.py
Works

[2,  0] mirko at wizbox:~$ python3 -m venv xxx
[2,  0] mirko at wizbox:~$ source xxx/bin/activate

(xxx) [2,  0] mirko at wizbox:~$ foo.py
Traceback (most recent call last):
  File "/home/mirko/bin/foo.py", line 3, in <module>
    import pandas
ModuleNotFoundError: No module named 'pandas'

(xxx) [2,  1] mirko at wizbox:~$ deactivate
[2,  0] mirko at wizbox:~$ foo.py
Works


So, the problem seems to be solved. Thanks all! :-)

From barry at barrys-emacs.org  Tue Feb 15 12:22:11 2022
From: barry at barrys-emacs.org (Barry)
Date: Tue, 15 Feb 2022 17:22:11 +0000
Subject: venv and executing other python programs
In-Reply-To: <620BB844.5030804@googlemail.com>
References: <620BB844.5030804@googlemail.com>
Message-ID: <DB142985-952A-46BE-B6FC-5DAFCD935F88@barrys-emacs.org>



> On 15 Feb 2022, at 14:30, Mirko via Python-list <python-list at python.org> wrote:
> 
> Well, it's not that I activate the venv *for* those troubling
> programs. I activate it to work on the particular project. ;-)

It is not necessary to activate the venv to use the python in it.
So your solution is to not do the activate.

You can run venv/bin/python without the activation.

Barry



From barry at barrys-emacs.org  Tue Feb 15 12:31:55 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Tue, 15 Feb 2022 17:31:55 +0000
Subject: venv and executing other python programs
In-Reply-To: <20220215123653.5u4zpwrelyd2fela@gmail.com>
References: <620B3B96.7080309@googlemail.com>
 <20220215123653.5u4zpwrelyd2fela@gmail.com>
Message-ID: <B8FFB248-9EFB-42E6-B4AD-B4096FB9F1F6@barrys-emacs.org>



> On 15 Feb 2022, at 12:36, Martin Di Paola <martinp.dipaola at gmail.com> wrote:
> 
> I did a few experiments in my machine. I created the following foo.py
> 
>  import pandas
>  print("foo")
> 
> Now "pandas" is installed under Python 3 outside the venv. I can run it successfully calling "python3 foo.py".
> 
> If I add the shebang "#!/usr/bin/env python3" (notice the 3), I can also run it as "./foo.py".

> 
> Calling it as "python foo.py" or using the shebang "#!/usr/bin/env python" does not work and it makes sense since "pandas" is installed
> only for python 3.
> 
> New I create a virtual env with "python3 -m venv xxx" and activate it.
> 
> Once inside I can run foo.py in 4 different ways:
> 
> - python foo.py
> - python3 foo.py
> - ./foo.py (using the shebang "#!/usr/bin/env python")
> - ./foo.py (using the shebang "#!/usr/bin/env python3")
> 
> Now all of that was with "pandas" installed outside of venv but not inside.
> 
> I did the same experiments with another library, "cryptonita" which it is not installed outside but inside and I was able to executed in 4 different ways too (inside the venv of course):
> 
> - python foo.py
> - python3 foo.py
> - ./foo.py (using the shebang "#!/usr/bin/env python")
> - ./foo.py (using the shebang "#!/usr/bin/env python3")

If you have activated the venv then any script that uses /usr/bin/env will use executables from the venv
bin folder.

I'm seeing the following in the venv I made on my mac:

 % ls venv.tmp/bin
Activate.ps1        dmgbuild*           modulegraph*        pyi-archive_viewer* pyinstaller*        python3.10@
activate            ds_store*           pip*                pyi-bindepend*      pylupdate5*         pyuic5*
activate.csh        macho_dump*         pip3*               pyi-grab_version*   pyrcc5*             wheel*
activate.fish       macho_find*         pip3.10*            pyi-makespec*       python@
colour-print*       macho_standalone*   py2applet*          pyi-set_version*    python3@

You can see that "python", "python3" and "python3.10" are present.

So it does not help to change the /usr/bin/env to use python3.
Indeed its a feature that if you use /usr/bin/env you obviously want to use the
executable from the activated venv.

I avoid all these issues by not activating the venv. Python has code to know
how to use the venv libraries that are installed in it when invoked. It does not 
depend on the activate script being run.

Barry


> 
> Do you have a particular context where you are having troubles? May be there is something else going on...
> 
> Thanks,
> Martin.
> 
> On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
>> Hi,
>> 
>> I have recently started using venv for my hobby-programming. There
>> is an annoying problem. Since venv modifies $PATH, python programs
>> that use the "#!/usr/bin/env python" variant of the hashbang often
>> fail since their additional modules aren't install inside in venv.
>> 
>> How to people here deal with that?
>> 
>> Please note: I'm not interested in discussing whether the
>> env-variant is good or bad. ;-) It's not that *I* use it, but
>> several progs in /usr/bin/.
>> 
>> Thanks for your time.
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From hjp-python at hjp.at  Tue Feb 15 17:30:18 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Tue, 15 Feb 2022 23:30:18 +0100
Subject: venv and executing other python programs
In-Reply-To: <620B3B96.7080309@googlemail.com>
References: <620B3B96.7080309@googlemail.com>
Message-ID: <20220215223018.zt2uu73ppnnbjy6k@hjp.at>

On 2022-02-15 06:35:18 +0100, Mirko via Python-list wrote:
> I have recently started using venv for my hobby-programming. There
> is an annoying problem. Since venv modifies $PATH, python programs
> that use the "#!/usr/bin/env python" variant of the hashbang often
> fail since their additional modules aren't install inside in venv.
> 
> How to people here deal with that?

Use the hashbang line to couple your scripts to their venvs.

For example, here's the start of one of my scripts:

#! /usr/local/usradm/venv/bin/python3

import json
import os
...
import psycopg2
import psycopg2.extras
import cx_Oracle
...

When this script is run, the python interpreter will inspect it's own
pathname and pull in the venv from /usr/local/usradm/venv automatically,
regardless of the environment it is started in. I could run this from
cron (which has only /usr/bin and /bin in its PATH) or from my normal
command line or from a different venv.

You could use a different venv for every script, but that's pretty
wasteful, so you probably want to organize your venvs so that several
scripts can share their environment. For example by project or product
(as in my example - "usradm" is a our in-house user administration
system) or by capabilities (for example you might want one venv with
pandas and all your favourite graph libraries for all your analytics
scripts and another with Qt5 for some gui tools).


> Please note: I'm not interested in discussing whether the
> env-variant is good or bad. ;-) It's not that *I* use it, but
> several progs in /usr/bin/.

Progs in /usr/bin should IMHO never use /usr/bin/env. They should always
use #!/usr/bin/python3. You might want to report that as a bug to your
distribution.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220215/c766589f/attachment.sig>

From martinp.dipaola at gmail.com  Tue Feb 15 21:18:58 2022
From: martinp.dipaola at gmail.com (Martin Di Paola)
Date: Wed, 16 Feb 2022 02:18:58 +0000
Subject: venv and executing other python programs
In-Reply-To: <B8FFB248-9EFB-42E6-B4AD-B4096FB9F1F6@barrys-emacs.org>
References: <620B3B96.7080309@googlemail.com>
 <20220215123653.5u4zpwrelyd2fela@gmail.com>
 <B8FFB248-9EFB-42E6-B4AD-B4096FB9F1F6@barrys-emacs.org>
Message-ID: <20220216021858.wcxg5uztdre5dysr@gmail.com>

>If you have activated the venv then any script that uses /usr/bin/env 
>will use executables from the venv
>bin folder.

That's correct. I tried to be systematic in the analysis so I tested all 
the possibilities.

>I avoid all these issues by not activating the venv. Python has code to 
>know
>how to use the venv libraries that are installed in it when invoked. It does not
>depend on the activate script being run.

I had to test this myself because I didn't believe it but you are right.  
Without having the venv activated, if the shebang explicitly points to 
the python executable of the venv, the program will have access to the 
libs installed in the environment.

The same if I do:

/home/user/venv/bin/python foo.py

Thanks for the info!
>
>Barry
>
>
>>
>> Do you have a particular context where you are having troubles? May be there is something else going on...
>>
>> Thanks,
>> Martin.
>>
>> On Tue, Feb 15, 2022 at 06:35:18AM +0100, Mirko via Python-list wrote:
>>> Hi,
>>>
>>> I have recently started using venv for my hobby-programming. There
>>> is an annoying problem. Since venv modifies $PATH, python programs
>>> that use the "#!/usr/bin/env python" variant of the hashbang often
>>> fail since their additional modules aren't install inside in venv.
>>>
>>> How to people here deal with that?
>>>
>>> Please note: I'm not interested in discussing whether the
>>> env-variant is good or bad. ;-) It's not that *I* use it, but
>>> several progs in /usr/bin/.
>>>
>>> Thanks for your time.
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>

From eryksun at gmail.com  Tue Feb 15 23:16:24 2022
From: eryksun at gmail.com (Eryk Sun)
Date: Tue, 15 Feb 2022 22:16:24 -0600
Subject: venv and executing other python programs
In-Reply-To: <20220216021858.wcxg5uztdre5dysr@gmail.com>
References: <620B3B96.7080309@googlemail.com>
 <20220215123653.5u4zpwrelyd2fela@gmail.com>
 <B8FFB248-9EFB-42E6-B4AD-B4096FB9F1F6@barrys-emacs.org>
 <20220216021858.wcxg5uztdre5dysr@gmail.com>
Message-ID: <CACL+1atPXhQJpxzmyzM=-B_xq2g4N7eJwE1i7CFU+JqHeYVorA@mail.gmail.com>

On 2/15/22, Martin Di Paola <martinp.dipaola at gmail.com> wrote:
>
> That's correct. I tried to be systematic in the analysis so I tested all
> the possibilities.

Your test results were unexpected for `python3 -m venv xxx`. By
default, virtual environments exclude the system and user site
packages. Including them should require the command-line argument
`--system-site-packages`. I'd check sys.path in the environment. Maybe
you have PYTHONPATH set.

> I had to test this myself because I didn't believe it but you are right.
> Without having the venv activated, if the shebang explicitly points to
> the python executable of the venv, the program will have access to the
> libs installed in the environment.

A virtual environment is configured by a "pyvenv.cfg" file that's
either beside the executable or one directory up. Activating an
environment is a convenience, not a requirement.

From wlfraed at ix.netcom.com  Wed Feb 16 12:15:37 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Wed, 16 Feb 2022 12:15:37 -0500
Subject: How to solve the given problem?
References: <1323924283.239248.1644533825969@mail.yahoo.com>
 <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
 <mailman.106.1644533830.7010.python-list@python.org>
 <1d8c2edf-6d36-4af5-81d5-b60d74a951a3n@googlegroups.com>
Message-ID: <vtbq0hh47arejt7m9ogncl9gig2be7hmjn@4ax.com>

On Tue, 15 Feb 2022 23:53:15 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>
>The feed remaining of the second hour will be distributed and adjusted in the next third and fourth hour not the next day because the fish must be hungry. This can be done by both taking the average or the percentage left. What I think is taking the average and then distributing in the third and fourth feed is better. If the feed is given in the next day there is a chance that the fish will not survive because the fish is special.  

	You are now defining new requirements that were not in your original
description of the  assignment (the original description had no mention of
"next day" -- only to redistribute the excess over the remainder of the
feeding schedule). It is your assignment -- code a solution based upon your
understanding of the problem, and submit it for grading (unless proven
otherwise, these have the appearance of homework assignments and, to
reiterate, we do not provide solutions to homework). OR ask the person who
gave you the assignment if your understanding is correct.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From martinp.dipaola at gmail.com  Thu Feb 17 06:44:36 2022
From: martinp.dipaola at gmail.com (Martin Di Paola)
Date: Thu, 17 Feb 2022 11:44:36 +0000
Subject: venv and executing other python programs
In-Reply-To: <CACL+1atPXhQJpxzmyzM=-B_xq2g4N7eJwE1i7CFU+JqHeYVorA@mail.gmail.com>
References: <620B3B96.7080309@googlemail.com>
 <20220215123653.5u4zpwrelyd2fela@gmail.com>
 <B8FFB248-9EFB-42E6-B4AD-B4096FB9F1F6@barrys-emacs.org>
 <20220216021858.wcxg5uztdre5dysr@gmail.com>
 <CACL+1atPXhQJpxzmyzM=-B_xq2g4N7eJwE1i7CFU+JqHeYVorA@mail.gmail.com>
Message-ID: <20220217114436.oc6vh5gg6yfes6hs@gmail.com>

>>
>> That's correct. I tried to be systematic in the analysis so I tested all
>> the possibilities.
>
>Your test results were unexpected for `python3 -m venv xxx`. By
>default, virtual environments exclude the system and user site
>packages. Including them should require the command-line argument
>`--system-site-packages`. I'd check sys.path in the environment. Maybe
>you have PYTHONPATH set.

Nope, I checked with "echo $PYTHONPATH" and nothing. I also checked 
"sys.path" within and without the environment:

Inside the environment:

['', '/usr/lib/python37.zip', '/usr/lib/python3.7', 
  '/usr/lib/python3.7/lib-dynload', 
  '/home/user/tmp/xxx/lib/python3.7/site-packages']

Outside the environment:

['', '/usr/lib/python37.zip', '/usr/lib/python3.7', 
  '/usr/lib/python3.7/lib-dynload', 
  '/home/user/.local/lib/python3.7/site-packages', 
  '/usr/local/lib/python3.7/dist-packages', 
  '/usr/lib/python3/dist-packages']

Indeed the "sys.path" inside the environment does not include system's 
site-packages.

I'll keep looking....

>A virtual environment is configured by a "pyvenv.cfg" file that's
>either beside the executable or one directory up. Activating an
>environment is a convenience, not a requirement.

Thanks, that makes a little more sense!

>-- 
>https://mail.python.org/mailman/listinfo/python-list

From wlfraed at ix.netcom.com  Thu Feb 17 13:26:17 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 17 Feb 2022 13:26:17 -0500
Subject: How to solve the given problem?
References: <1323924283.239248.1644533825969@mail.yahoo.com>
 <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
 <mailman.106.1644533830.7010.python-list@python.org>
 <1d8c2edf-6d36-4af5-81d5-b60d74a951a3n@googlegroups.com>
 <vtbq0hh47arejt7m9ogncl9gig2be7hmjn@4ax.com>
 <d089e36b-99c5-43a2-a3a1-49a599f0d90dn@googlegroups.com>
Message-ID: <1g1t0hdrj3tn1jqvnevicare8r2i26bhbb@4ax.com>

On Thu, 17 Feb 2022 02:20:53 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>I have completed the homework or what so ever it used to be its only I am telling the solution which looks to me as better as compared to what others have given like the one that Christian has given.

	This statement should probably have been how you introduced the prior
post instead of presenting something that read like a request for someone
to implement something based on your new description.

	Your immediately prior post:

>The feed remaining of the second hour will be distributed and adjusted in the next third and fourth hour not the next day because the fish must be hungry. This can be done by both taking the average or the percentage left. What I think is taking the average and then distributing in the third and fourth feed is better. If the feed is given in the next day there is a chance that the fish will not survive because the fish is special.  

	"If the feed is given in the next day..." is a non-issue, as the
original problem description (shown here)...

>Assume that there is a pattern of feeding for a special fish in a day (10 hours a day) as below:
>? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 150? ? 100? ? 30? ? 30? ? 30? ? 20? ? 20? ? 10? ? 5? ? 5
>Today, the fish is fed in the second hour 60 unit instead of 100 unit Accidently. Implement some methods to distribute the remaining 40 unit in the rest of the day and propose the new patterns. Try to keep the distribution similar to the current feeding pattern. 
>Note: pay attention that the total feeding amounts should be fix in a day.

... explicitly states that the total amount must be provided within a
single day.

	Since you didn't provide your implementation (or even just the
resultant feeding pattern), I can not determine exactly what you are doing
with regards to your "average" -- however, it seems to me that your
"average" distribution "in the third and fourth feed" fails both the "rest
of the day" and "try to keep the distribution similar..." requirements. It
is a given that you have to account for the 40 units that were not provided
in the second hour. It sounds very much like you are dumping them at 20&20
on the 3rd&4th hours, so your pattern (with the "error" feed now looks like

	150		60	50	50	30	20	20	10	5	5

A proportional solution would result in the full pattern looking (ignoring
round-off errors)

	150		60	38	38	38	25	25	13	6	6
(round-off means you have to account for 1 more unit -- I'd probably put it
on the first 38 hour, but that's only because I hand worked the proportions
on a calculator. A "proper" algorithm should probably be tracking the error
amounts [difference between integer units and real proportion] to apply the
round-off at the nearest correct location).

	Taking error term into account

units	calc	error
150
60
38		8.0		0.0
38		8.0		0.0
38		8.0		0.0
25		5.333	.333v
25		5.333	.666v
13		2.666	1.333^
				.333
6		1.333	.666v
7		1.333	.999^	(I'm only showing 3 decimal places)
				

	Of course, one could compute the data in the other direction and get

6		1.333	.333v
6		1.333	.666v
13		2.666	1.333^
				.333
25		5.333	.666v
26		5.333	.999^	(again, 3 decimal places)
				0.0
38		8.0		0.0
38		8.0		0.0
38		8.0		0.0
60
150



(v = round down, ^ round up


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From u13051995p at gmail.com  Thu Feb 17 23:58:16 2022
From: u13051995p at gmail.com (UTKARSH PANDEY)
Date: Thu, 17 Feb 2022 20:58:16 -0800 (PST)
Subject: Pickle file and send via socket
In-Reply-To: <37mdnchk-JMqHb_NnZ2dnUVZ7tidnZ2d@bt.com>
References: <f3b73b0e-1520-4768-a1d2-bb754357dfc7@googlegroups.com>
 <WtqdncCw6amw_b_NnZ2dnUVZ7tGdnZ2d@bt.com>
 <674b1cd4-5f7d-47ce-924d-76f5306746bc@googlegroups.com>
 <37mdnchk-JMqHb_NnZ2dnUVZ7tidnZ2d@bt.com>
Message-ID: <bcdfc36a-cdac-4179-af57-cd4335524997n@googlegroups.com>

On Wednesday, August 8, 2012 at 8:37:33 PM UTC+5:30, lipska the kat wrote:
> On 08/08/12 14:50, S.B wrote: 
> > On Wednesday, August 8, 2012 3:48:43 PM UTC+3, lipska the kat wrote: 
> >> On 06/08/12 14:32, S.B wrote: 
> >>
> [snip]
> > Thank you so much ! 
> > The examples are very helpful. 
> > What happens if I have a regular text file I want to send via the network. 
> > Do I need to read the file and then dump it into the "stargate" file object?
> Well according to the documentation at 
> 
> http://docs.python.org/py3k/tutorial/inputoutput.html#reading-and-writing-files 
> 
> it should be straightforward to read and write pickled files 
> Not sure why you want to pickle a text file over the network when you 
> could just stream it between ports ! 
> 
> however ... 
> 
> I'm currently getting a Unicode decode error on the first byte in the 
> stream when it gets to the other end, no idea why so I guess I have to 
> continue searching, read the documentation above and see if you can 
> figure it out, that's what I'm doing.
> lipska 
> 
> -- 
> Lipska the Kat: Troll hunter, sandbox destroyer 
> and farscape dreamer of Aeryn Sun



Directly read bytes from file and send it over the socket object from client side in while loop until all content from file is read.

Something like this.

Client side
import socket

s = socket.socket()

PORT = 9898

s.connect(("192.168.0.101",PORT))

file = open("turnover.csv","rb")
SendData = file.read(1024)

while SendData:
    s.send(SendData)
    SendData = file.read(1024)


s.close()

Server side 


import socket
s = socket.socket()
PORT =9898
print("Server is listening on port :",PORT,"\n")

s.bind(("192.168.0.101",PORT))

s.listen(10)

file = open("recv.csv","wb")
print("\n Copied file name will be recv.txt at server side\n")

while True:
    conn,addr = s.accept()
    RecvData = conn.recv(1024)
    while RecvData:
        file.write(RecvData)
        RecvData = conn.recv(1024)

    file.close()
    print("\n File has been copied successfully \n")

    conn.close()
    print("\n Server is closing the connection \n")


    break

From sarunidavid11126 at gmail.com  Fri Feb 18 02:23:02 2022
From: sarunidavid11126 at gmail.com (Saruni David)
Date: Thu, 17 Feb 2022 23:23:02 -0800 (PST)
Subject: Error installing requirements
Message-ID: <ae6c6e2d-7fc2-490a-b1dd-85a6e241165an@googlegroups.com>

PS C:\Users\Nepapa David\cpims_api> pip install -r requirements/base.txt
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support
 for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-suppor
t pip 21.0 will remove support for this functionality.
Requirement already satisfied: Django==1.8.15 in c:\users\nepapa david\cpims_api\venv1\lib\site-packages (from -r requirements/base.txt (line 1)) (1.8.15)
Requirement already satisfied: django-crispy-forms==1.6.0 in c:\users\nepapa david\cpims_api\venv1\lib\site-packages (from -r requirements/base.txt (line 2)) (1.6.0)
Requirement already satisfied: jellyfish==0.5.4 in c:\users\nepapa david\cpims_api\venv1\lib\site-packages (from -r requirements/base.txt (line 3)) (0.5.4)
Collecting pillow==3.3.2
  Using cached Pillow-3.3.2.tar.gz (10.6 MB)
Collecting psycopg2
  Using cached psycopg2-2.8.6-cp27-cp27m-win_amd64.whl (1.1 MB)
Collecting reportlab==3.3.0
  Using cached reportlab-3.3.0-cp27-none-win_amd64.whl (2.1 MB)
Collecting djangorestframework==3.3.3
  Using cached djangorestframework-3.3.3-py2.py3-none-any.whl (662 kB)
Processing c:\users\nepapa david\appdata\local\pip\cache\wheels\61\84\91\bea40ba16a87ba77cfee951da2d1d73cd8126cc3b5c1f7e327\openpyxl-2.4.0-py2.py3-none-any.whl
Collecting python-dateutil==2.4.2
  Using cached python_dateutil-2.4.2-py2.py3-none-any.whl (188 kB)
Collecting python-memcached
  Using cached python_memcached-1.59-py2.py3-none-any.whl (16 kB)
Collecting pandas==0.20.3
  Using cached pandas-0.20.3-cp27-cp27m-win_amd64.whl (8.3 MB)
Collecting xlsxwriter==0.9.8
  Using cached XlsxWriter-0.9.8-py2.py3-none-any.whl (137 kB)
Collecting django-import-export==1.2.0
  Using cached django_import_export-1.2.0-py2.py3-none-any.whl (75 kB)
Collecting sentry-sdk==0.10.0
  Using cached sentry_sdk-0.10.0-py2.py3-none-any.whl (75 kB)
Collecting schedule==0.6.0
  Using cached schedule-0.6.0-py2.py3-none-any.whl (8.7 kB)
Collecting requests==2.22.0
  Using cached requests-2.22.0-py2.py3-none-any.whl (57 kB)
Collecting django-admin-sortable
  Using cached django_admin_sortable-2.3-py2-none-any.whl (115 kB)
Processing c:\users\nepapa david\appdata\local\pip\cache\wheels\29\e8\37\8866c07502322a658fdf2b346cef1902c16494c1e0ef1b9b73\django_simple_forums-1.3.6-py2-none-any.wh
l
Collecting markdown
  Using cached Markdown-3.1.1-py2.py3-none-any.whl (87 kB)
Requirement already satisfied: setuptools>=2.2 in c:\users\nepapa david\cpims_api\venv1\lib\site-packages (from reportlab==3.3.0->-r requirements/base.txt (line 6)) (
44.1.1)
Requirement already satisfied: pip>=1.4.1 in c:\users\nepapa david\cpims_api\venv1\lib\site-packages (from reportlab==3.3.0->-r requirements/base.txt (line 6)) (20.3.
4)
Processing c:\users\nepapa david\appdata\local\pip\cache\wheels\8d\22\36\204262bf2e0e1bd954606953bc164321f6b481d4922ffb823a\et_xmlfile-1.0.1-py2-none-any.whl
Collecting jdcal
  Using cached jdcal-1.4.1-py2.py3-none-any.whl (9.5 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting numpy>=1.7.0
  Using cached numpy-1.16.6-cp27-cp27m-win_amd64.whl (11.9 MB)
Collecting pytz>=2011k
  Using cached pytz-2021.3-py2.py3-none-any.whl (503 kB)
Processing c:\users\nepapa david\appdata\local\pip\cache\wheels\2f\c3\a5\1e95754397eea31c33840174c762ea4f871cb42a75c8ea57b7\diff_match_patch-20200713-py2-none-any.whl

Processing c:\users\nepapa david\appdata\local\pip\cache\wheels\97\a8\5a\3ad933f15ee204c8c70e78bfb11a62cb578aa96320b6ee2ca8\tablib-0.14.0-py2-none-any.whl
Collecting urllib3
  Using cached urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting certifi
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting chardet<3.1.0,>=3.0.2
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting idna<2.9,>=2.5
  Using cached idna-2.8-py2.py3-none-any.whl (58 kB)
Collecting bleach
  Using cached bleach-3.3.1-py2.py3-none-any.whl (146 kB)
Collecting pymdown-extensions
  Using cached pymdown_extensions-8.0-py2.py3-none-any.whl (206 kB)
Collecting backports.csv; python_version < "3.0"
  Using cached backports.csv-1.0.7-py2.py3-none-any.whl (12 kB)
Collecting xlrd
  Using cached xlrd-2.0.1-py2.py3-none-any.whl (96 kB)
Collecting xlwt
  Using cached xlwt-1.3.0-py2.py3-none-any.whl (99 kB)
Collecting pyyaml
  Using cached PyYAML-5.4.1-cp27-cp27m-win_amd64.whl (204 kB)
Processing c:\users\nepapa david\appdata\local\pip\cache\wheels\a1\bf\e9\51b62ccd5a4772b7c00fa648dc0bd39811911c667acba2a4bb\markuppy-1.14-py2-none-any.whl
Processing c:\users\nepapa david\appdata\local\pip\cache\wheels\98\c4\95\b0df24ec6db20e710d841be844de5f7f5df8491c1e5fb91323\odfpy-1.4.1-py2.py3-none-any.whl
Collecting webencodings
  Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting packaging
  Using cached packaging-20.9-py2.py3-none-any.whl (40 kB)
Collecting defusedxml
  Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Building wheels for collected packages: pillow
  Building wheel for pillow (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'C:\Users\Nepapa David\cpims_api\venv1\Scripts\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'c:\\users\\nepapa~1\\appdata\\local
\\temp\\pip-install-fu6_er\\pillow\\setup.py'"'"'; __file__='"'"'c:\\users\\nepapa~1\\appdata\\local\\temp\\pip-install-fu6_er\\pillow\\setup.py'"'"';f=getattr(tokeni
ze, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'c:\us
ers\nepapa~1\appdata\local\temp\pip-wheel-qc83z1'
       cwd: c:\users\nepapa~1\appdata\local\temp\pip-install-fu6_er\pillow\
  Complete output (145 lines):
  Single threaded build for windows
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-2.7
  creating build\lib.win-amd64-2.7\PIL
  copying PIL\BdfFontFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\BmpImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\BufrStubImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ContainerIO.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\CurImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\DcxImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\DdsImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\EpsImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ExifTags.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\features.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\FitsStubImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\FliImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\FontFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\FpxImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\FtexImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\GbrImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\GdImageFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\GifImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\GimpGradientFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\GimpPaletteFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\GribStubImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\Hdf5StubImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\IcnsImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\IcoImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\Image.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageChops.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageCms.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageColor.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageDraw.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageDraw2.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageEnhance.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageFilter.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageFont.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageGrab.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageMath.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageMode.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageMorph.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageOps.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImagePalette.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImagePath.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageQt.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageSequence.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageShow.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageStat.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageTk.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageTransform.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImageWin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\ImtImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\IptcImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\Jpeg2KImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\JpegImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\JpegPresets.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\McIdasImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\MicImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\MpegImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\MpoImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\MspImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\OleFileIO.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PaletteFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PalmImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PcdImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PcfFontFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PcxImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PdfImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PixarImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PngImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PpmImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PsdImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PSDraw.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\PyAccess.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\SgiImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\SpiderImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\SunImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\TarIO.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\TgaImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\TiffImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\TiffTags.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\WalImageFile.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\WebPImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\WmfImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\XbmImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\XpmImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\XVThumbImagePlugin.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\_binary.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\_tkinter_finder.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\_util.py -> build\lib.win-amd64-2.7\PIL
  copying PIL\__init__.py -> build\lib.win-amd64-2.7\PIL
  running egg_info
  writing Pillow.egg-info\PKG-INFO
  writing top-level names to Pillow.egg-info\top_level.txt
  writing dependency_links to Pillow.egg-info\dependency_links.txt
  reading manifest file 'Pillow.egg-info\SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching '*.sh'
  no previously-included directories found matching 'docs\_static'
  warning: no previously-included files found matching '.coveragerc'
  warning: no previously-included files found matching '.editorconfig'
  warning: no previously-included files found matching '.landscape.yaml'
  warning: no previously-included files found matching 'appveyor.yml'
  warning: no previously-included files found matching 'build_children.sh'
  warning: no previously-included files found matching 'tox.ini'
  warning: no previously-included files matching '.git*' found anywhere in distribution
  warning: no previously-included files matching '*.pyc' found anywhere in distribution
  warning: no previously-included files matching '*.so' found anywhere in distribution
  writing manifest file 'Pillow.egg-info\SOURCES.txt'
  copying PIL\OleFileIO-README.md -> build\lib.win-amd64-2.7\PIL
  running build_ext
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "c:\users\nepapa~1\appdata\local\temp\pip-install-fu6_er\pillow\setup.py", line 753, in <module>
      zip_safe=not debug_build(), )
    File "C:\Users\Nepapa David\cpims_api\venv1\lib\site-packages\setuptools\__init__.py", line 162, in setup
      return distutils.core.setup(**attrs)
    File "C:\Python27\lib\distutils\core.py", line 151, in setup
      dist.run_commands()
    File "C:\Python27\lib\distutils\dist.py", line 953, in run_commands
      self.run_command(cmd)
    File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
      cmd_obj.run()
    File "C:\Users\Nepapa David\cpims_api\venv1\lib\site-packages\wheel\bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "C:\Python27\lib\distutils\cmd.py", line 326, in run_command
      self.distribution.run_command(command)
    File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
      cmd_obj.run()
    File "C:\Python27\lib\distutils\command\build.py", line 127, in run
      self.run_command(cmd_name)
    File "C:\Python27\lib\distutils\cmd.py", line 326, in run_command
      self.distribution.run_command(command)
    File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
      cmd_obj.run()
    File "C:\Python27\lib\distutils\command\build_ext.py", line 340, in run
      self.build_extensions()
    File "c:\users\nepapa~1\appdata\local\temp\pip-install-fu6_er\pillow\setup.py", line 521, in build_extensions
      ' using --disable-%s, aborting' % (f, f))
  ValueError: zlib is required unless explicitly disabled using --disable-zlib, aborting
  ----------------------------------------
  ERROR: Failed building wheel for pillow
  Running setup.py clean for pillow
Failed to build pillow
Installing collected packages: pillow, psycopg2, reportlab, djangorestframework, et-xmlfile, jdcal, openpyxl, six, python-dateutil, python-memcached, numpy, pytz, pan
das, xlsxwriter, diff-match-patch, backports.csv, xlrd, xlwt, pyyaml, markuppy, defusedxml, odfpy, tablib, django-import-export, urllib3, certifi, sentry-sdk, schedul
e, chardet, idna, requests, django-admin-sortable, webencodings, pyparsing, packaging, bleach, markdown, pymdown-extensions, django-simple-forums
  Attempting uninstall: pillow
    Found existing installation: pillow 6.2.2
    Uninstalling pillow-6.2.2:
      Successfully uninstalled pillow-6.2.2
    Running setup.py install for pillow ... error
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\Nepapa David\cpims_api\venv1\Scripts\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'c:\\users\\nepapa~1\\appdata\\loc
al\\temp\\pip-install-fu6_er\\pillow\\setup.py'"'"'; __file__='"'"'c:\\users\\nepapa~1\\appdata\\local\\temp\\pip-install-fu6_er\\pillow\\setup.py'"'"';f=getattr(toke
nize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'c
:\users\nepapa~1\appdata\local\temp\pip-record-gmhg6v\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\Users\Nepapa David\cpims
_api\venv1\include\site\python2.7\pillow'
         cwd: c:\users\nepapa~1\appdata\local\temp\pip-install-fu6_er\pillow\
    Complete output (147 lines):
    Single threaded build for windows
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-2.7
    creating build\lib.win-amd64-2.7\PIL
    copying PIL\BdfFontFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\BmpImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\BufrStubImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ContainerIO.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\CurImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\DcxImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\DdsImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\EpsImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ExifTags.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\features.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\FitsStubImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\FliImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\FontFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\FpxImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\FtexImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\GbrImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\GdImageFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\GifImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\GimpGradientFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\GimpPaletteFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\GribStubImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\Hdf5StubImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\IcnsImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\IcoImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\Image.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageChops.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageCms.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageColor.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageDraw.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageDraw2.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageEnhance.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageFilter.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageFont.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageGrab.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageMath.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageMode.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageMorph.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageOps.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImagePalette.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImagePath.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageQt.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageSequence.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageShow.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageStat.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageTk.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageTransform.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImageWin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\ImtImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\IptcImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\Jpeg2KImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\JpegImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\JpegPresets.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\McIdasImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\MicImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\MpegImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\MpoImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\MspImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\OleFileIO.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PaletteFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PalmImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PcdImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PcfFontFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PcxImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PdfImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PixarImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PngImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PpmImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PsdImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PSDraw.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\PyAccess.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\SgiImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\SpiderImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\SunImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\TarIO.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\TgaImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\TiffImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\TiffTags.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\WalImageFile.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\WebPImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\WmfImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\XbmImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\XpmImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\XVThumbImagePlugin.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\_binary.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\_tkinter_finder.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\_util.py -> build\lib.win-amd64-2.7\PIL
    copying PIL\__init__.py -> build\lib.win-amd64-2.7\PIL
    running egg_info
    writing Pillow.egg-info\PKG-INFO
    writing top-level names to Pillow.egg-info\top_level.txt
    writing dependency_links to Pillow.egg-info\dependency_links.txt
    reading manifest file 'Pillow.egg-info\SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    warning: no files found matching '*.sh'
    no previously-included directories found matching 'docs\_static'
    warning: no previously-included files found matching '.coveragerc'
    warning: no previously-included files found matching '.editorconfig'
    warning: no previously-included files found matching '.landscape.yaml'
    warning: no previously-included files found matching 'appveyor.yml'
    warning: no previously-included files found matching 'build_children.sh'
    warning: no previously-included files found matching 'tox.ini'
    warning: no previously-included files matching '.git*' found anywhere in distribution
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files matching '*.so' found anywhere in distribution
    writing manifest file 'Pillow.egg-info\SOURCES.txt'
    copying PIL\OleFileIO-README.md -> build\lib.win-amd64-2.7\PIL
    running build_ext
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "c:\users\nepapa~1\appdata\local\temp\pip-install-fu6_er\pillow\setup.py", line 753, in <module>
        zip_safe=not debug_build(), )
      File "C:\Users\Nepapa David\cpims_api\venv1\lib\site-packages\setuptools\__init__.py", line 162, in setup
        return distutils.core.setup(**attrs)
      File "C:\Python27\lib\distutils\core.py", line 151, in setup
        dist.run_commands()
      File "C:\Python27\lib\distutils\dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
        cmd_obj.run()
      File "C:\Users\Nepapa David\cpims_api\venv1\lib\site-packages\setuptools\command\install.py", line 61, in run
        return orig.install.run(self)
      File "C:\Python27\lib\distutils\command\install.py", line 563, in run
        self.run_command('build')
      File "C:\Python27\lib\distutils\cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
        cmd_obj.run()
      File "C:\Python27\lib\distutils\command\build.py", line 127, in run
        self.run_command(cmd_name)
      File "C:\Python27\lib\distutils\cmd.py", line 326, in run_command
        self.distribution.run_command(command)
      File "C:\Python27\lib\distutils\dist.py", line 972, in run_command
        cmd_obj.run()
      File "C:\Python27\lib\distutils\command\build_ext.py", line 340, in run
        self.build_extensions()
      File "c:\users\nepapa~1\appdata\local\temp\pip-install-fu6_er\pillow\setup.py", line 521, in build_extensions
        ' using --disable-%s, aborting' % (f, f))
    ValueError: zlib is required unless explicitly disabled using --disable-zlib, aborting
    ----------------------------------------
  Rolling back uninstall of pillow
  Moving to c:\users\nepapa david\cpims_api\venv1\lib\site-packages\pil\
   from c:\users\nepapa david\cpims_api\venv1\lib\site-packages\~il
  Moving to c:\users\nepapa david\cpims_api\venv1\lib\site-packages\pillow-6.2.2.dist-info\
   from c:\users\nepapa david\cpims_api\venv1\lib\site-packages\~illow-6.2.2.dist-info
ERROR: Command errored out with exit status 1: 'C:\Users\Nepapa David\cpims_api\venv1\Scripts\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'
c:\\users\\nepapa~1\\appdata\\local\\temp\\pip-install-fu6_er\\pillow\\setup.py'"'"'; __file__='"'"'c:\\users\\nepapa~1\\appdata\\local\\temp\\pip-install-fu6_er\\pil
low\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"
'exec'"'"'))' install --record 'c:\users\nepapa~1\appdata\local\temp\pip-record-gmhg6v\install-record.txt' --single-version-externally-managed --compile --install-hea
ders 'C:\Users\Nepapa David\cpims_api\venv1\include\site\python2.7\pillow' Check the logs for full command output.

From rosuav at gmail.com  Fri Feb 18 13:59:54 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 19 Feb 2022 05:59:54 +1100
Subject: Pickle file and send via socket
In-Reply-To: <bcdfc36a-cdac-4179-af57-cd4335524997n@googlegroups.com>
References: <f3b73b0e-1520-4768-a1d2-bb754357dfc7@googlegroups.com>
 <WtqdncCw6amw_b_NnZ2dnUVZ7tGdnZ2d@bt.com>
 <674b1cd4-5f7d-47ce-924d-76f5306746bc@googlegroups.com>
 <37mdnchk-JMqHb_NnZ2dnUVZ7tidnZ2d@bt.com>
 <bcdfc36a-cdac-4179-af57-cd4335524997n@googlegroups.com>
Message-ID: <CAPTjJmqfHoar=96naCEY40MqWZfcpP4DzD+K-PG+TwV-Uks6+A@mail.gmail.com>

On Sat, 19 Feb 2022 at 05:47, UTKARSH PANDEY <u13051995p at gmail.com> wrote:
>
> On Wednesday, August 8, 2012 at 8:37:33 PM UTC+5:30, lipska the kat wrote:
> > ...
> Directly read bytes from file and send it over the socket object from client side in while loop until all content from file is read.
>

Almost ten years. Quite an impressive ping time.

ChrisA

From mats at wichmann.us  Fri Feb 18 23:13:03 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Fri, 18 Feb 2022 21:13:03 -0700
Subject: Error installing requirements
In-Reply-To: <ae6c6e2d-7fc2-490a-b1dd-85a6e241165an@googlegroups.com>
References: <ae6c6e2d-7fc2-490a-b1dd-85a6e241165an@googlegroups.com>
Message-ID: <dc3ea0da-1a39-dc7a-61b2-ec96bb3daa5d@wichmann.us>

And the question is?

On 2/18/22 00:23, Saruni David wrote:
> PS C:\Users\Nepapa David\cpims_api> pip install -r requirements/base.txt
> DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support
>  for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-suppor
> t pip 21.0 will remove support for this functionality.

By using Python 2.7 you're constrained to very old versions of things.

> Collecting pillow==3.3.2
>   Using cached Pillow-3.3.2.tar.gz (10.6 MB)

And for this one, there was no binary wheel found, apparently, so it
fetched (or in this case used from cache) the source distribution
instead (a VERY old version, Pillow is on 9.0 these days).

So now the source distribution has to be compiled.  pip tries to arrange
that for you, but that often fails on Windows, unless you have all the
requirements set up just right.

> Building wheels for collected packages: pillow
>   Building wheel for pillow (setup.py) ... error
>   ERROR: Command errored out with exit status 1:

... and so it did.

>     File "c:\users\nepapa~1\appdata\local\temp\pip-install-fu6_er\pillow\setup.py", line 521, in build_extensions
>       ' using --disable-%s, aborting' % (f, f))
>   ValueError: zlib is required unless explicitly disabled using --disable-zlib, aborting
>   ----------------------------------------
>   ERROR: Failed building wheel for pillow

And at least one of the reasons is a missing build requirement (there
may be other lurking problems that you just haven't uncovered yet
because it stopped on that one).

Fix, and retry.

Or... maybe try using something more modern?  It's not usually necessary
to compile Pillow, usually binary wheels are available and you avoid
that problem.


From sjeik_appie at hotmail.com  Sat Feb 19 03:42:29 2022
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sat, 19 Feb 2022 09:42:29 +0100
Subject: Error installing requirements
In-Reply-To: <ae6c6e2d-7fc2-490a-b1dd-85a6e241165an@googlegroups.com>
Message-ID: <DB6PR01MB389562DBE459E2807A68720383389@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Feb 18, 2022 08:23, Saruni David <sarunidavid11126 at gmail.com> wrote:
   >> Christian Gohlke's site has a Pillow .whl for python
   2.7:?https://www.lfd.uci.edu/~gohlke/pythonlibs/#pillow

From shishaozhong at gmail.com  Sat Feb 19 06:28:31 2022
From: shishaozhong at gmail.com (Shaozhong SHI)
Date: Sat, 19 Feb 2022 11:28:31 +0000
Subject: Long running process - how to speed up?
Message-ID: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>

I have a cvs file of 932956 row and have to have time.sleep in a Python
script.  It takes a long time to process.

How can I speed up the processing?  Can I do multi-processing?

Regards,

David

From rosuav at gmail.com  Sat Feb 19 06:53:57 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 19 Feb 2022 22:53:57 +1100
Subject: Long running process - how to speed up?
In-Reply-To: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
Message-ID: <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>

On Sat, 19 Feb 2022 at 22:30, Shaozhong SHI <shishaozhong at gmail.com> wrote:
>
> I have a cvs file of 932956 row and have to have time.sleep in a Python
> script.  It takes a long time to process.
>
> How can I speed up the processing?  Can I do multi-processing?
>
Remove the time.sleep()?

ChrisA

From Karsten.Hilbert at gmx.net  Sat Feb 19 06:59:17 2022
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Sat, 19 Feb 2022 12:59:17 +0100
Subject: Aw: Re: Long running process - how to speed up?
In-Reply-To: <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
Message-ID: <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>

> > I have a cvs file of 932956 row and have to have time.sleep in a Python
> > script.  It takes a long time to process.
> >
> > How can I speed up the processing?  Can I do multi-processing?
> >
> Remove the time.sleep()?

He's attesting to only having "time.sleep" in there...

I doubt removing that will help much ;-)

Karsten

From rosuav at gmail.com  Sat Feb 19 07:03:23 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 19 Feb 2022 23:03:23 +1100
Subject: Long running process - how to speed up?
In-Reply-To: <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
 <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
Message-ID: <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>

On Sat, 19 Feb 2022 at 22:59, Karsten Hilbert <Karsten.Hilbert at gmx.net> wrote:
>
> > > I have a cvs file of 932956 row and have to have time.sleep in a Python
> > > script.  It takes a long time to process.
> > >
> > > How can I speed up the processing?  Can I do multi-processing?
> > >
> > Remove the time.sleep()?
>
> He's attesting to only having "time.sleep" in there...
>
> I doubt removing that will help much ;-)

I honestly don't understand the question, hence offering the
stupidly-obvious suggestion in the hope that it would result in a
better question. A million rows of CSV, on its own, isn't all that
much to process, so it must be the processing itself (of which we have
no information other than this reference to time.sleep) that takes all
the time.

ChrisA

From sjeik_appie at hotmail.com  Sat Feb 19 07:08:12 2022
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sat, 19 Feb 2022 13:08:12 +0100
Subject: Long running process - how to speed up?
In-Reply-To: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
Message-ID: <DB6PR01MB3895C819246A2B5A83360F3F83389@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Feb 19, 2022 12:28, Shaozhong SHI <shishaozhong at gmail.com> wrote:

     I have a cvs file of 932956 row and have to have time.sleep in a Python
     script.? It takes a long time to process.

     How can I speed up the processing?? Can I do multi-processing?

   ====
   Perhaps a dask df:?
   https://docs.dask.org/en/latest/generated/dask.dataframe.read_csv.html

From shishaozhong at gmail.com  Sat Feb 19 07:09:41 2022
From: shishaozhong at gmail.com (Shaozhong SHI)
Date: Sat, 19 Feb 2022 12:09:41 +0000
Subject: Long running process - how to speed up?
In-Reply-To: <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
 <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
 <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>
Message-ID: <CA+i5JwbJVUg7gerHARxRCnkNpjP1gOdormZkAOT6TF6s4uN8uQ@mail.gmail.com>

Can it be divided into several processes?
Regards,
David

On Saturday, 19 February 2022, Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, 19 Feb 2022 at 22:59, Karsten Hilbert <Karsten.Hilbert at gmx.net>
> wrote:
> >
> > > > I have a cvs file of 932956 row and have to have time.sleep in a
> Python
> > > > script.  It takes a long time to process.
> > > >
> > > > How can I speed up the processing?  Can I do multi-processing?
> > > >
> > > Remove the time.sleep()?
> >
> > He's attesting to only having "time.sleep" in there...
> >
> > I doubt removing that will help much ;-)
>
> I honestly don't understand the question, hence offering the
> stupidly-obvious suggestion in the hope that it would result in a
> better question. A million rows of CSV, on its own, isn't all that
> much to process, so it must be the processing itself (of which we have
> no information other than this reference to time.sleep) that takes all
> the time.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From kvratkin at yandex.ru  Sat Feb 19 07:13:57 2022
From: kvratkin at yandex.ru (Kirill Ratkin)
Date: Sat, 19 Feb 2022 15:13:57 +0300
Subject: Long running process - how to speed up?
In-Reply-To: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
Message-ID: <59b5515a-5760-9607-a67d-17e55287859c@yandex.ru>

Hi,

How I understand your script starts another script and should wait until 
second one completes its job. Right?

If so you have several options depend on how your first script is written.

If your script is async then ... there is good asyncio option

proc = await asyncio.create_subprocess_shell(
f"{execcmd}{execargs}", stdin=None, stdout=None
)
await proc.wait() In this way you can starn many workers and you don't neet to wait then in sync manner.


Anyway, just please give more info about what problem you face.

19.02.2022 14:28, Shaozhong SHI ?????:
> I have a cvs file of 932956 row and have to have time.sleep in a Python
> script.  It takes a long time to process.
>
> How can I speed up the processing?  Can I do multi-processing?
>
> Regards,
>
> David

From Marco.Sulla.Python at gmail.com  Sat Feb 19 08:46:59 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sat, 19 Feb 2022 14:46:59 +0100
Subject: Error installing requirements
In-Reply-To: <ae6c6e2d-7fc2-490a-b1dd-85a6e241165an@googlegroups.com>
References: <ae6c6e2d-7fc2-490a-b1dd-85a6e241165an@googlegroups.com>
Message-ID: <CABbU2U8kMMU+1k2B8WyDt2Oc86g+Wk7nmYAkMUED-jLT3cS_5Q@mail.gmail.com>

Maybe you compiled Python 2.7 by hand, David? It happened to me when I
tried to compile Python without zlib headers installed on my OS. Don't
know how it can be done on Windows.

From drsalists at gmail.com  Sat Feb 19 11:04:25 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Sat, 19 Feb 2022 08:04:25 -0800
Subject: Long running process - how to speed up?
In-Reply-To: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
Message-ID: <CAGGBd_ox4NxsA+St5sMkET3_nawQTB2PuNobvQKJSNQK-c=UAA@mail.gmail.com>

On Sat, Feb 19, 2022 at 3:29 AM Shaozhong SHI <shishaozhong at gmail.com>
wrote:

> I have a cvs file of 932956 row and have to have time.sleep in a Python
> script.  It takes a long time to process.
>
> How can I speed up the processing?  Can I do multi-processing?
>

How are you doing it right now?

Are you using the csv module?

You might be able to use the GNU "split" command as a prelude to using the
csv module in combination with multiprocessing.  GNU split comes with
Linuxes, but I'm sure you can get it for Windows.  MacOS comes with a
rather less powerful "split" command, but it still might work for you.

You also could try Pypy3.

HTH.

From techtasktechnologies at gmail.com  Fri Feb 18 17:46:22 2022
From: techtasktechnologies at gmail.com (Techtask Technologies)
Date: Fri, 18 Feb 2022 14:46:22 -0800 (PST)
Subject: Converting py files to .exe and .dmg
In-Reply-To: <9b206057-0593-4560-af40-442d03111770@googlegroups.com>
References: <9b206057-0593-4560-af40-442d03111770@googlegroups.com>
Message-ID: <f79321c9-9bf4-4516-8267-134299c9b4b4n@googlegroups.com>

On Monday, December 28, 2015 at 4:35:41 PM UTC+1, Brian Simms wrote:
> Hi there, 
> 
> I have done a lot of looking around online to find out how to convert Python files to .exe and .dmg files, but I am confused. Could someone provide pointers/advice as to how I can turn a Python file into a Windows .exe and Mac .dmg file. 
> 
> Thanks for any help.

You can convert Python Files to exe using Pyinstaller, this are the procedures below:
Run this commands on your terminal:
pip install pyinstaller
pyinstaller nameofyourfile.py --onefile
after that a folder should be created named Dist open it and you would see your Python file in an executable format.
I think this would help you.
~Paul Fruitful

From cosmohope14 at gmail.com  Fri Feb 18 19:22:57 2022
From: cosmohope14 at gmail.com (Cosmo Hope)
Date: Fri, 18 Feb 2022 16:22:57 -0800 (PST)
Subject: library not initialized (pygame)
In-Reply-To: <mailman.127.1619985860.3087.python-list@python.org>
References: <401c54ac-adf2-cd32-ee64-c6dfcea34755@gmail.com>
 <CAO9VG=g2-Ag6Z42K-vCYHTjCW7ddFaJdDF=1kzLfiftiViHt3A@mail.gmail.com>
 <mailman.127.1619985860.3087.python-list@python.org>
Message-ID: <a7f18df1-296d-4946-b04e-cd11db53167fn@googlegroups.com>

On Sunday, 2 May 2021 at 21:04:40 UTC+1, Michael Torrie wrote:
> On 5/2/21 1:23 PM, Quentin Bock wrote: 
> > the error apparently comes from the first instructions variable saying 
> > library not initialized not sure why, its worked before but not now :/
> I don't get that error on my Fedora 32 machine. The script ultimately 
> doesn't run because it can't find the icon png file. But the window 
> briefly appears and it seems like pygame is being initialized properly.

i have the same problem currently
have you found a solution by chance.

From learn2program at gmail.com  Sat Feb 19 07:33:59 2022
From: learn2program at gmail.com (Alan Gauld)
Date: Sat, 19 Feb 2022 12:33:59 +0000
Subject: Fwd: Re: Long running process - how to speed up?
Message-ID: <330a036b-945e-e6c6-e8fe-205172e6e8d0@yahoo.co.uk>

On 19/02/2022 11:28, Shaozhong SHI wrote:

> I have a cvs file of 932956 row 

That's not a lot in modern computing terms.

> and have to have time.sleep in a Python
> script. 

Why? Is it a requirement by your customer? Your manager?
time.sleep() is not usually helpful if you want to do
things quickly.

> It takes a long time to process.

What is a "long time"? minutes? hours? days? weeks?

It should take a million times as long as it takes to
process one row. But you have given no clue what you
are doing in each row.
- reading a database?
- reading from the network? or the internet?
- writing to a database? or the internet?
- performing highly complex math operations?

Or perhaps the processing load is in analyzing the totality
of the data after reading it all? A very different type
of problem. But we just don't know.

All of these factors will affect performance.

> How can I speed up the processing? 

It all depends on the processing.
You could try profiling your code to see where the time is spent.

> Can I do multi-processing?

Of course. But there is no guarantee that will speed things
up if there is a bottleneck on a single resource somewhere.
But it might be possible to divide and conquer and get better
speed. It all depends on what you are doing. We can't tell.

We cannot answer such a vague question with any specific
solution.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


From wlfraed at ix.netcom.com  Sat Feb 19 12:02:48 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sat, 19 Feb 2022 12:02:48 -0500
Subject: Long running process - how to speed up?
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <mailman.137.1645270124.7010.python-list@python.org>
Message-ID: <ej721hdrq1fqf8vrf62tvjrl44igg6duqc@4ax.com>

On Sat, 19 Feb 2022 11:28:31 +0000, Shaozhong SHI <shishaozhong at gmail.com>
declaimed the following:

>I have a cvs file of 932956 row and have to have time.sleep in a Python
>script.  It takes a long time to process.
>
	I'd echo the others... Unless you better explain WHY you have .sleep()
(along with how often it is called, and what duration you sleep) the first
recommendation would be to remove it.

	The most common justification for .sleep() is that one has CPU-BOUND
processing and needs to force context switches to let other operations
proceed more often than the system quantum time. Not normally a concern
given Python's GIL and the presence of multi-core chips.

	How are you processing the (near) million rows of that CSV? If you are
loading all of them into a large list you could be running Python list
reallocations, or OS page swapping (though I wouldn't expect that on most
modern systems -- maybe on a Raspberry-Pi/BeagleBone Black). Note:

>>> import sys
>>> sys.getsizeof("a")
50
>>> 

even a one-character string expands to 50 bytes. An EMPTY string takes up
51 bytes... Except for the empty string, that comes to about 49+<#chars> IF
all characters fit an 8-bit encoding -- if any non 8-bit characters are in
the string, the #chars needs to be multiplied by either 2 or 4 depending
upon the widest representation needed.

	If you are doing read-one-record, process-one-record, repeat -- and
have the .sleep() inside that loop... definitely remove the .sleep(). That
loop is already I/O bound, the fastest you can obtain is determined by how
rapidly the OS can transfer records from the file system to your program.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From martinp.dipaola at gmail.com  Sat Feb 19 14:25:48 2022
From: martinp.dipaola at gmail.com (Martin Di Paola)
Date: Sat, 19 Feb 2022 19:25:48 +0000
Subject: library not initialized (pygame)
In-Reply-To: <CAO9VG=g2-Ag6Z42K-vCYHTjCW7ddFaJdDF=1kzLfiftiViHt3A@mail.gmail.com>
References: <CAO9VG=g2-Ag6Z42K-vCYHTjCW7ddFaJdDF=1kzLfiftiViHt3A@mail.gmail.com>
Message-ID: <20220219192548.axjbqhqvpk5efybx@gmail.com>

Could you share the traceback / error that you are seeing?


On Sun, May 02, 2021 at 03:23:21PM -0400, Quentin Bock wrote:
>Code:
>#imports and variables for game
>import pygame
>from pygame import mixer
>running = True
>
>#initializes pygame
>pygame.init()
>
>#creates the pygame window
>screen = pygame.display.set_mode((1200, 800))
>
>#Title and Icon of window
>pygame.display.set_caption("3.02 Project")
>icon = pygame.image.load('3.02 icon.png')
>pygame.display.set_icon(icon)
>
>#setting up font
>pygame.font.init()
>font = pygame.font.Font('C:\Windows\Fonts\OCRAEXT.ttf', 16)
>font_x = 10
>font_y = 40
>items_picked_up = 0
>items_left = 3
>
>def main():
>    global running, event
>
>    #Game Loop
>    while running:
>        #sets screen color to black
>        screen.fill((0, 0, 0))
>
>        #checks if the user exits the window
>        for event in pygame.event.get():
>            if event.type == pygame.QUIT:
>                running = False
>                pygame.quit()
>
>        def display_instruction(x, y):
>            instructions = font.render("Each level contains 3 items you
>must pick up in each room.", True, (255, 255, 255))
>            instructions_2 = font.render("When you have picked up 3 items,
>you will advance to the next room, there are 3.", True, (255, 255, 255))
>            instructions_3 = font.render("You will be able to change the
>direction you are looking in the room, this allows you to find different
>objects.", True, (255, 255, 255))
>            clear = font.render("Click to clear the text.", True, (255,
>255, 255))
>            screen.blit(instructions, (10, 40))
>            screen.blit(instructions_2, (10, 60))
>            screen.blit(instructions_3, (10, 80))
>            screen.blit(clear, (10, 120))
>
>        if event.type == pygame.MOUSEBUTTONDOWN:
>            if event.type == pygame.MOUSEBUTTONUP:
>                screen.fill(pygame.Color('black'))  # clears the screen text
>
>        display_instruction(font_x, font_y)
>        pygame.display.update()
>
>
>main()
>
>the error apparently comes from the first instructions variable saying
>library not initialized not sure why, its worked before but not now :/
>-- 
>https://mail.python.org/mailman/listinfo/python-list

From mats at wichmann.us  Sat Feb 19 14:42:36 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Sat, 19 Feb 2022 12:42:36 -0700
Subject: Long running process - how to speed up?
In-Reply-To: <CA+i5JwbJVUg7gerHARxRCnkNpjP1gOdormZkAOT6TF6s4uN8uQ@mail.gmail.com>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
 <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
 <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>
 <CA+i5JwbJVUg7gerHARxRCnkNpjP1gOdormZkAOT6TF6s4uN8uQ@mail.gmail.com>
Message-ID: <31d366cd-2536-d528-6e6c-f22b18840dd4@wichmann.us>

On 2/19/22 05:09, Shaozhong SHI wrote:
> Can it be divided into several processes?
> Regards,
> David

The answer is: "maybe".  Multiprocessing doesn't happen for free, you
have to figure out how to divide the task up, requiring thought and
effort. We can't guess to what extent the problem you have is amenable
to multiprocessing.

Google for "dataframe" and "multiprocessing" and you should get some
hits (in my somewhat limited experience in this area, people usually
load the csv data into Pandas before they get started working with it).



From vanyp at skynet.be  Sat Feb 19 14:58:55 2022
From: vanyp at skynet.be (vanyp)
Date: Sat, 19 Feb 2022 20:58:55 +0100
Subject: PYT - How can I subscribe to a topic in the mailing list?
Message-ID: <deb1ec0b-e8a4-cee7-b718-eb0425ca8d4f@skynet.be>

And h*ow do I set the topic in my messages?*

From Karsten.Hilbert at gmx.net  Sat Feb 19 15:30:44 2022
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Sat, 19 Feb 2022 21:30:44 +0100
Subject: Aw: PYT - How can I subscribe to a topic in the mailing list?
In-Reply-To: <deb1ec0b-e8a4-cee7-b718-eb0425ca8d4f@skynet.be>
References: <deb1ec0b-e8a4-cee7-b718-eb0425ca8d4f@skynet.be>
Message-ID: <trinity-98c1266a-c3a5-4533-a4bf-a30a939f79fc-1645302643381@3c-app-gmx-bap34>

> Betreff: PYT - How can I subscribe to a topic in the mailing list?

Mailing lists don't subdivide by topic.

Your mailer may allow you to filter by Subject:.

> And h*ow do I set the topic in my messages?*

not applicable

The equivalent would be the Subject: header.

Karsten

From avigross at verizon.net  Sat Feb 19 16:44:38 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 19 Feb 2022 21:44:38 +0000 (UTC)
Subject: Long running process - how to speed up?
In-Reply-To: <330a036b-945e-e6c6-e8fe-205172e6e8d0@yahoo.co.uk>
References: <330a036b-945e-e6c6-e8fe-205172e6e8d0@yahoo.co.uk>
Message-ID: <452945610.143508.1645307078565@mail.yahoo.com>

Indeed not a clear request. Timing is everything but there are times ...
For many purposes, people may read in the entire CSV at a gulp into some data structure like a pandas DataFrame. The file is then closed and any processing done later does whatever you want.
Of course you can easily read on line at a time in Python and parse it by comma and any other processing and act on one row at a time or in small batches so you never need huge amounts of memory. But using other methods to read in the entire set of data is often better optimized and faster, and being able to do some things with the data is faster if done in vectorized fashion using add-ons like numpy and pandas. We have no idea what is being used and none of this explains a need to use some form of sleep.
Multi-processing helps only if you can make steps in the processing run in parallel without interfering with each other or making things happen out of order. Yes, you could read in the data and assign say 10,000 rows to a thread to process and then get more and assign, if done quite carefully. The results might need to be carefully combined and any shared variables might need locks and so on. Not necessarily worth it if the data is not too large and the calculations are small.

And it remains unclear where you want to sleep or why. Parallelism can be important if the sleep is to wait for the user to respond to something while processing continues in the background. Is it possible that whatever you are calling to do processing has some kind of sleep within it and you may be calling it as often as per row? In that case, ask why it does that and can you avoid that? Yes, running in parallel may let you move forward but again, it has to be done carefully and having thousands of processes sleeping at the same time may be worse!
I note badly defined questions get horrible answers. Mine included.
-----Original Message-----
From: Alan Gauld <learn2program at gmail.com>
To: python-list at python.org
Sent: Sat, Feb 19, 2022 7:33 am
Subject: Fwd: Re: Long running process - how to speed up?

On 19/02/2022 11:28, Shaozhong SHI wrote:

> I have a cvs file of 932956 row 

That's not a lot in modern computing terms.

> and have to have time.sleep in a Python
> script. 

Why? Is it a requirement by your customer? Your manager?
time.sleep() is not usually helpful if you want to do
things quickly.

> It takes a long time to process.

What is a "long time"? minutes? hours? days? weeks?

It should take a million times as long as it takes to
process one row. But you have given no clue what you
are doing in each row.
- reading a database?
- reading from the network? or the internet?
- writing to a database? or the internet?
- performing highly complex math operations?

Or perhaps the processing load is in analyzing the totality
of the data after reading it all? A very different type
of problem. But we just don't know.

All of these factors will affect performance.

> How can I speed up the processing? 

It all depends on the processing.
You could try profiling your code to see where the time is spent.

> Can I do multi-processing?

Of course. But there is no guarantee that will speed things
up if there is a bottleneck on a single resource somewhere.
But it might be possible to divide and conquer and get better
speed. It all depends on what you are doing. We can't tell.

We cannot answer such a vague question with any specific
solution.

-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos

-- 
https://mail.python.org/mailman/listinfo/python-list

From vanyp at skynet.be  Sat Feb 19 17:28:28 2022
From: vanyp at skynet.be (vanyp)
Date: Sat, 19 Feb 2022 23:28:28 +0100
Subject: PYT - The expressions described in the Python language reference
 yield only boolean values
Message-ID: <b4f0af4a-ce4e-d8d1-4ffa-d01ee6625e1a@skynet.be>

*I am trying to learn Python from the grammar given in the Python 
language reference and I am surprised.*

*Lets start here:*

*"*
*6.3.4.?Calls*

A call calls a callable object (e.g., a function 
<https://docs.python.org/3/glossary.html#term-function>) with a possibly 
empty series of arguments 
<https://docs.python.org/3/glossary.html#term-argument>:

*call *::= |primary 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-primary>|"(" [|argument_list 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-argument_list>|[","] | |comprehension 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-comprehension>|] ")"
*argument_list *::= |positional_arguments 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-positional_arguments>|["," |starred_and_keywords 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-starred_and_keywords>|]
["," |keywords_arguments 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-keywords_arguments>|]
| |starred_and_keywords 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-starred_and_keywords>|["," |keywords_arguments 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-keywords_arguments>|]
| |keywords_arguments 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-keywords_arguments>|
*positional_arguments*::= positional_item ("," positional_item)*
*positional_item *::= |assignment_expression 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-assignment_expression>|| "*" |expression 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-expression>|
*"*

*continued by:*
*"*
*6.12.?Assignment expressions*
*assignment_expression*::= [|identifier 
<https://docs.python.org/3/reference/lexical_analysis.html#grammar-token-python-grammar-identifier>|":="] |expression 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-expression>|
*"*

*Now I look at the following productions:*

*"*
*6.13.?Conditional expressions*
*conditional_expression*::= |or_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-or_test>|["if" |or_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-or_test>|"else" |expression 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-expression>|]
*expression *::= |conditional_expression 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-conditional_expression>|| |lambda_expr 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-lambda_expr>|
*"*

*The first or_test is strange, I assume it should be replaced by expression.*

*But even so I think that the only ways out of the recursion are the 
or_test or the lambda_expr.*

*And by the grammar, those evaluate to booleans as follows:*


*"*
*6.14.?Lambdas*
*lambda_expr*::= "lambda" [|parameter_list 
<https://docs.python.org/3/reference/compound_stmts.html#grammar-token-python-grammar-parameter_list>|] ":" |expression 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-expression>|
*"*

*and I conclude that the only way out of that branch is also or_test.*

*I then look at or_test:*

*"*
*6.11.?Boolean operations*
*or_test *::= |and_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-and_test>|| |or_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-or_test>|"or" |and_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-and_test>|
*and_test*::= |not_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-not_test>|| |and_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-and_test>|"and" |not_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-not_test>|
*not_test*::= |comparison 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-comparison>|| "not" |not_test 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-not_test>|
*"*

*and the ony way out is comparison, which by the semantics should return 
a boolean.*

*Looking at comparison gives:*

*"*
**6.10.?Comparisons**
*comparison *::= |or_expr 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-or_expr>|(|comp_operator 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-comp_operator>||or_expr 
<https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-or_expr>|)*
*comp_operator*::= "<" | ">" | "==" | ">=" | "<=" | "!="
| "is" ["not"] | ["not"] "in"

Comparisons yield boolean values: |True|?or |False|. Custom /rich 
comparison methods/?may return non-boolean values. In this case Python 
will call |bool() 
<https://docs.python.org/3/library/functions.html#bool>|?on such value 
in boolean contexts.

*"*

*By now I think that we are firmly stuck in boolean operations. Which 
would mean that any expression always returns a boolean value.*

*Where did I, or the language reference, go wrong?*

*Sincerely,*

*Pierre Van Nypelseer*

From rosuav at gmail.com  Sat Feb 19 20:03:08 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 20 Feb 2022 12:03:08 +1100
Subject: PYT - The expressions described in the Python language reference
 yield only boolean values
In-Reply-To: <b4f0af4a-ce4e-d8d1-4ffa-d01ee6625e1a@skynet.be>
References: <b4f0af4a-ce4e-d8d1-4ffa-d01ee6625e1a@skynet.be>
Message-ID: <CAPTjJmoTUYD3OrnRTx-LRo=_6byYTLdwiVsO-RXeaOHcYWW8dw@mail.gmail.com>

On Sun, 20 Feb 2022 at 12:00, vanyp <vanyp at skynet.be> wrote:
>
> *I am trying to learn Python from the grammar given in the Python
> language reference and I am surprised.*
>

The grammar is not the best way to learn the language. It'll show you
a lot of unnecessary details. For technical reasons, Python's grammar
defines expressions in a nested way, but the way most programmers want
to think of it is that there is operator precedence.

https://docs.python.org/3/reference/expressions.html#operator-precedence

The grammar considers that these are different types of expressions,
but they're not really different in actual usage.

ChrisA

From arj.python at gmail.com  Sun Feb 20 10:32:21 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sun, 20 Feb 2022 19:32:21 +0400
Subject: Why does not Python accept functions with no names?
Message-ID: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>

Greetings list.

Out of curiosity, why doesn't Python accept
def ():
    return '---'

()

Where the function name is ''?

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From barry at barrys-emacs.org  Sun Feb 20 11:21:33 2022
From: barry at barrys-emacs.org (Barry)
Date: Sun, 20 Feb 2022 16:21:33 +0000
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
Message-ID: <185C4014-65AA-4100-BCCA-3A66A50F8B0D@barrys-emacs.org>



> On 20 Feb 2022, at 15:35, Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:
> 
> ?Greetings list.
> 
> Out of curiosity, why doesn't Python accept
> def ():
>    return '---'
> 
> ()
> 
> Where the function name is ''?

Because there is no variable that is holding a ref to the code.
So it?s pointless.


Barry

> 
> Kind Regards,
> 
> Abdur-Rahmaan Janhangeer
> about <https://compileralchemy.github.io/> | blog
> <https://www.pythonkitchen.com>
> github <https://github.com/Abdur-RahmaanJ>
> Mauritius
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From drsalists at gmail.com  Sun Feb 20 12:06:35 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Sun, 20 Feb 2022 09:06:35 -0800
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
Message-ID: <CAGGBd_ryrOLbeefgxje7fn1nS5WVGbj3Sk8e2_1dLuz15S616w@mail.gmail.com>

On Sun, Feb 20, 2022 at 7:33 AM Abdur-Rahmaan Janhangeer <
arj.python at gmail.com> wrote:

> Greetings list.
>
> Out of curiosity, why doesn't Python accept
> def ():
>     return '---'
>
> ()
>
> Where the function name is ''?
>

() is already an empty tuple.  It would break code to change this.

From rosuav at gmail.com  Sun Feb 20 12:09:21 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 21 Feb 2022 04:09:21 +1100
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
Message-ID: <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>

On Mon, 21 Feb 2022 at 02:33, Abdur-Rahmaan Janhangeer
<arj.python at gmail.com> wrote:
>
> Greetings list.
>
> Out of curiosity, why doesn't Python

This is often the wrong question. The question is more: Why should Python?

Python doesn't do things just because there's no reason not to.

ChrisA

From arj.python at gmail.com  Sun Feb 20 12:27:14 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sun, 20 Feb 2022 21:27:14 +0400
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
Message-ID: <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>

Thanks for the answers.

@Chris Well Python deliberately throws an exception if we do not
pass in a function name. Just wanted to know why it should. As the
above case is a 100% pure anonymous function.

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From arj.python at gmail.com  Sun Feb 20 12:40:54 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sun, 20 Feb 2022 21:40:54 +0400
Subject: Why does not Python accept functions with no names?
In-Reply-To: <25106.31590.888833.723439@ixdm.fritz.box>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <25106.31590.888833.723439@ixdm.fritz.box>
Message-ID: <CADrxXX=NiuGRQidoLGGySW4u+Yy-2ZyjQeou3M=38JJZsZBTUQ@mail.gmail.com>

Yes I know about lambdas but was just an introspection about
the reasoning behind ^^

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From dieter at handshake.de  Sun Feb 20 12:33:26 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Sun, 20 Feb 2022 18:33:26 +0100
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
Message-ID: <25106.31590.888833.723439@ixdm.fritz.box>

Abdur-Rahmaan Janhangeer wrote at 2022-2-20 19:32 +0400:
>Out of curiosity, why doesn't Python accept
>def ():
>    return '---'
>
>()
>
>Where the function name is ''?

Python knows about (somewhat restricted) anonymous functions:
it calls them `lambda` expressions (the body of those functions
can only be an expression).

Your example above can be expressed as
`(lambda: '---')()`.

From shishaozhong at gmail.com  Sun Feb 20 13:05:33 2022
From: shishaozhong at gmail.com (Shaozhong SHI)
Date: Sun, 20 Feb 2022 18:05:33 +0000
Subject: Long running process - how to speed up?
In-Reply-To: <31d366cd-2536-d528-6e6c-f22b18840dd4@wichmann.us>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
 <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
 <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>
 <CA+i5JwbJVUg7gerHARxRCnkNpjP1gOdormZkAOT6TF6s4uN8uQ@mail.gmail.com>
 <31d366cd-2536-d528-6e6c-f22b18840dd4@wichmann.us>
Message-ID: <CA+i5JwaryuvPmLudYU4Scog5nZeAw9Uk0STkxStuhiEafBaD=g@mail.gmail.com>

On Sat, 19 Feb 2022 at 19:44, Mats Wichmann <mats at wichmann.us> wrote:

> On 2/19/22 05:09, Shaozhong SHI wrote:
> > Can it be divided into several processes?
> > Regards,
> > David
>
> The answer is: "maybe".  Multiprocessing doesn't happen for free, you
> have to figure out how to divide the task up, requiring thought and
> effort. We can't guess to what extent the problem you have is amenable
> to multiprocessing.
>
> Google for "dataframe" and "multiprocessing" and you should get some
> hits (in my somewhat limited experience in this area, people usually
> load the csv data into Pandas before they get started working with it).
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list


I am trying this approach,

import multiprocessing as mp

def my_func(x):
  print(x**x)

def main():
  pool = mp.Pool(mp.cpu_count())
  result = pool.map(my_func, [4,2,3])

if __name__ == "__main__":
  main()

I modified the script and set off a test run.

However, I have no idea whether this approach will be faster than
conventional approach.

Any one has idea?

Regards,

David

From shishaozhong at gmail.com  Sun Feb 20 13:09:14 2022
From: shishaozhong at gmail.com (Shaozhong SHI)
Date: Sun, 20 Feb 2022 18:09:14 +0000
Subject: Long running process - how to speed up?
In-Reply-To: <330a036b-945e-e6c6-e8fe-205172e6e8d0@yahoo.co.uk>
References: <330a036b-945e-e6c6-e8fe-205172e6e8d0@yahoo.co.uk>
Message-ID: <CA+i5Jwbi_p6QDjcwfrpvAc3jU5ZGH4_8a6eGcVT2d0mP=gPRZw@mail.gmail.com>

On Sat, 19 Feb 2022 at 18:51, Alan Gauld <learn2program at gmail.com> wrote:

> On 19/02/2022 11:28, Shaozhong SHI wrote:
>
> > I have a cvs file of 932956 row
>
> That's not a lot in modern computing terms.
>
> > and have to have time.sleep in a Python
> > script.
>
> Why? Is it a requirement by your customer? Your manager?
> time.sleep() is not usually helpful if you want to do
> things quickly.
>
> > It takes a long time to process.
>
> What is a "long time"? minutes? hours? days? weeks?
>
> It should take a million times as long as it takes to
> process one row. But you have given no clue what you
> are doing in each row.
> - reading a database?
> - reading from the network? or the internet?
> - writing to a database? or the internet?
> - performing highly complex math operations?
>
> Or perhaps the processing load is in analyzing the totality
> of the data after reading it all? A very different type
> of problem. But we just don't know.
>
> All of these factors will affect performance.
>
> > How can I speed up the processing?
>
> It all depends on the processing.
> You could try profiling your code to see where the time is spent.
>
> > Can I do multi-processing?
>
> Of course. But there is no guarantee that will speed things
> up if there is a bottleneck on a single resource somewhere.
> But it might be possible to divide and conquer and get better
> speed. It all depends on what you are doing. We can't tell.
>
> We cannot answer such a vague question with any specific
> solution.
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
> --
> https://mail.python.org/mailman/listinfo/python-list


Do not know these answers yet.  Now, it appeared to hang/stop at a point
and does not move on.

Regards,

David

From rosuav at gmail.com  Sun Feb 20 13:10:30 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 21 Feb 2022 05:10:30 +1100
Subject: Long running process - how to speed up?
In-Reply-To: <CA+i5JwaryuvPmLudYU4Scog5nZeAw9Uk0STkxStuhiEafBaD=g@mail.gmail.com>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
 <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
 <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>
 <CA+i5JwbJVUg7gerHARxRCnkNpjP1gOdormZkAOT6TF6s4uN8uQ@mail.gmail.com>
 <31d366cd-2536-d528-6e6c-f22b18840dd4@wichmann.us>
 <CA+i5JwaryuvPmLudYU4Scog5nZeAw9Uk0STkxStuhiEafBaD=g@mail.gmail.com>
Message-ID: <CAPTjJmohuvaphusTVAdgXzZ6zvdCA-5g94q5jpnuRH+mBVsBJg@mail.gmail.com>

On Mon, 21 Feb 2022 at 05:07, Shaozhong SHI <shishaozhong at gmail.com> wrote:
> However, I have no idea whether this approach will be faster than
> conventional approach.
>
> Any one has idea?

Try it. Find out. The only way to know is to measure.

I can't see the sleep call though. You may need to post your actual
code instead of trivial examples.

ChrisA

From avigross at verizon.net  Sun Feb 20 13:22:50 2022
From: avigross at verizon.net (Avi Gross)
Date: Sun, 20 Feb 2022 18:22:50 +0000 (UTC)
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
 <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>
Message-ID: <2100318715.222649.1645381370210@mail.yahoo.com>

Not really. Anonymous functions are anonymous in name only, so to speak.
When you call a function and pass it an argument that is an anonymous function definition, it is bound to a variable within the function and used mostly in a non-anonymous way. You did not bother giving it a name but it is referenced and retains an existence until it goes out of scope. Not having a name within your own scope means you do not have to do anything like deleting it further in your code. But most code that uses anonymous functions could be rewritten to use a named function.?
However, a compiler or interpreter may well be optimized to decide to ignore or even rewrite things. Consider what it might do to something like "if (5 < 4) ..." where all the code in the ellipsis can never be reached and hence it and the entire statement can be silently removed or ignored OR perhaps reported as a logical error. The code is absolutely valid in terms of syntax, but not valid as having any purpose. But what if all the above is the body of another statement like an "else" which would become a problem if you removed this code? Obviously you might have to walk up the tree and trim more.
Declaring an anonymous function in a context where it attaches to no names anywhere can be allowed to proceed, if that is important to anyone, but garbage collection will rapidly reclaim the space, I would think. So why bother allocating the space? And note if the allocation method is odd and leaves it entirely out of the tables that keep track, you then would have a memory leak.


-----Original Message-----
From: Abdur-Rahmaan Janhangeer <arj.python at gmail.com>
To: Chris Angelico <rosuav at gmail.com>
Cc: Python <python-list at python.org>
Sent: Sun, Feb 20, 2022 12:27 pm
Subject: Re: Why does not Python accept functions with no names?

Thanks for the answers.

@Chris Well Python deliberately throws an exception if we do not
pass in a function name. Just wanted to know why it should. As the
above case is a 100% pure anonymous function.

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius
-- 
https://mail.python.org/mailman/listinfo/python-list

From avigross at verizon.net  Sun Feb 20 13:40:33 2022
From: avigross at verizon.net (Avi Gross)
Date: Sun, 20 Feb 2022 18:40:33 +0000 (UTC)
Subject: Long running process - how to speed up?
In-Reply-To: <CA+i5JwaryuvPmLudYU4Scog5nZeAw9Uk0STkxStuhiEafBaD=g@mail.gmail.com>
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
 <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
 <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>
 <CA+i5JwbJVUg7gerHARxRCnkNpjP1gOdormZkAOT6TF6s4uN8uQ@mail.gmail.com>
 <31d366cd-2536-d528-6e6c-f22b18840dd4@wichmann.us>
 <CA+i5JwaryuvPmLudYU4Scog5nZeAw9Uk0STkxStuhiEafBaD=g@mail.gmail.com>
Message-ID: <1331356737.225086.1645382433160@mail.yahoo.com>

dAVId,
I would like to assume that the processing needed is quite a bit more than calculating the square of each X.
But as we are getting negligible information on anything useful, why continue interacting.?
I am trying to imagin a scenario with a million rows of sorts in a CSV (typically not used for a single column of data, as that tends not to use commas) where you read in the data and want to generate just the square of each number in what may be a disorganized order as threads are not guaranteed to do anything but interleave!
Parallelism can be used in cases where the data is segmented properly and the algorithm adjusted to fit the needs. But the overhead can be substantial. For the? trivial task mentioned, which I have to hope is not the actual task, you can get quite a decent speed by reading it into a numpy data structure and using a vectorized way to produce the squares and simply print that.?
The original question here is turning out to be mysterious as it began by asking how to speed up some slow process not yet explained and some mumbling about calling a sleep function. I note some parallel algorithms require a variant of that in that some parts must wait for other parts to complete and arrange to be dormant till signaled or schedule themselves to be woken regularly and check if things are ready for them to resume. Sleeping is a very common occurence in systems that are time-shared.


-----Original Message-----
From: Shaozhong SHI <shishaozhong at gmail.com>
To: Mats Wichmann <mats at wichmann.us>
Cc: python-list at python.org
Sent: Sun, Feb 20, 2022 1:05 pm
Subject: Re: Long running process - how to speed up?

On Sat, 19 Feb 2022 at 19:44, Mats Wichmann <mats at wichmann.us> wrote:

> On 2/19/22 05:09, Shaozhong SHI wrote:
> > Can it be divided into several processes?
> > Regards,
> > David
>
> The answer is: "maybe".? Multiprocessing doesn't happen for free, you
> have to figure out how to divide the task up, requiring thought and
> effort. We can't guess to what extent the problem you have is amenable
> to multiprocessing.
>
> Google for "dataframe" and "multiprocessing" and you should get some
> hits (in my somewhat limited experience in this area, people usually
> load the csv data into Pandas before they get started working with it).
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list


I am trying this approach,

import multiprocessing as mp

def my_func(x):
? print(x**x)

def main():
? pool = mp.Pool(mp.cpu_count())
? result = pool.map(my_func, [4,2,3])

if __name__ == "__main__":
? main()

I modified the script and set off a test run.

However, I have no idea whether this approach will be faster than
conventional approach.

Any one has idea?

Regards,

David
-- 
https://mail.python.org/mailman/listinfo/python-list

From eryksun at gmail.com  Sun Feb 20 17:36:54 2022
From: eryksun at gmail.com (Eryk Sun)
Date: Sun, 20 Feb 2022 16:36:54 -0600
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
Message-ID: <CACL+1aumxkYGUyLSNmGSCYNQn95V1cXgANtLDKzyq5Qw+gG_wA@mail.gmail.com>

On 2/20/22, Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:
>
> Out of curiosity, why doesn't Python accept
> def ():
>     return '---'

The `def` keyword is compiled as an assignment statement, not an
expression that evaluates anonymously on the stack. Using `def` as an
expression would require new syntax. For example, maybe something like
the following:

    funcs[i] = def (x, *, y=0) := (
        if x < y:
            return spam(x)
        return eggs(x)
    )

Since parsing Python depends on white space, if a `def (...) :=`
expression is multiline, the first statement on a new line would set
the indentation level, up to but not including the closing
parenthesis. (Style guides would likely restrict what's considered
good form.) The anonymous function could be called immediately. For
example:

    results[i] = def (x, *, y=0) := (
        if x < y:
            return spam(x)
        return eggs(x)
    )(z)

From rosuav at gmail.com  Sun Feb 20 17:49:23 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 21 Feb 2022 09:49:23 +1100
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CACL+1aumxkYGUyLSNmGSCYNQn95V1cXgANtLDKzyq5Qw+gG_wA@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CACL+1aumxkYGUyLSNmGSCYNQn95V1cXgANtLDKzyq5Qw+gG_wA@mail.gmail.com>
Message-ID: <CAPTjJmoOJ5qB=XvwmMv=h-nT1x_2VdevuYPKgh_LvFU4ksAjxw@mail.gmail.com>

On Mon, 21 Feb 2022 at 09:38, Eryk Sun <eryksun at gmail.com> wrote:
>
> On 2/20/22, Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:
> >
> > Out of curiosity, why doesn't Python accept
> > def ():
> >     return '---'
>
> The `def` keyword is compiled as an assignment statement, not an
> expression that evaluates anonymously on the stack. Using `def` as an
> expression would require new syntax. For example, maybe something like
> the following:
>
>     funcs[i] = def (x, *, y=0) := (
>         if x < y:
>             return spam(x)
>         return eggs(x)
>     )
>

There's always decorators.

>>> @functools.partial(operator.setitem, funcs, i)
... def _(x, *, y=0):
...     if x < y:
...             return spam(x)
...     return eggs(x)
...
>>> funcs[i]
<function _ at 0x7f6fd4b36b90>

(Has the side effect of setting _ to None.)

ChrisA

From python at example.invalid  Sun Feb 20 10:48:33 2022
From: python at example.invalid (Python)
Date: Sun, 20 Feb 2022 16:48:33 +0100
Subject: Why does not Python accept functions with no names?
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <mailman.154.1645371155.7010.python-list@python.org>
Message-ID: <sutnsc$lm1$1@gioia.aioe.org>

Abdur-Rahmaan Janhangeer wrote:
> Greetings list.
> 
> Out of curiosity, why doesn't Python accept
> def ():
>      return '---'
> 
> ()
> 
> Where the function name is ''?

For the same reason an empty sequence of characters cannot
be a variable name. Do you know any language (or formal
theory) that allows that?



From python at example.invalid  Sun Feb 20 11:33:55 2022
From: python at example.invalid (Python)
Date: Sun, 20 Feb 2022 17:33:55 +0100
Subject: Why does not Python accept functions with no names?
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <185C4014-65AA-4100-BCCA-3A66A50F8B0D@barrys-emacs.org>
 <mailman.155.1645374100.7010.python-list@python.org>
Message-ID: <sutqhf$p1$1@gioia.aioe.org>

Barry wrote:
> 
> 
>> On 20 Feb 2022, at 15:35, Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:
>>
>> ?Greetings list.
>>
>> Out of curiosity, why doesn't Python accept
>> def ():
>>     return '---'
>>
>> ()
>>
>> Where the function name is ''?
> 
> Because there is no variable that is holding a ref to the code.
> So it?s pointless.

Fun fact, it can be done, but it's (afaik) then (almost) unusable...

 >>> a
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined
 >>> locals()['a'] = 42
 >>> a
42
 >>> locals()[''] = 42
 >>> locals()['']
42
 >>> locals()[''] = (lambda x: x*42)
 >>> locals()[''](1)
42


From auriocus at gmx.de  Sun Feb 20 13:57:49 2022
From: auriocus at gmx.de (Christian Gollwitzer)
Date: Sun, 20 Feb 2022 19:57:49 +0100
Subject: Why does not Python accept functions with no names?
In-Reply-To: <sutnsc$lm1$1@gioia.aioe.org>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <mailman.154.1645371155.7010.python-list@python.org>
 <sutnsc$lm1$1@gioia.aioe.org>
Message-ID: <suu2ve$64h$1@dont-email.me>

Am 20.02.22 um 16:48 schrieb Python:
> Abdur-Rahmaan Janhangeer wrote:
>> Greetings list.
>>
>> Out of curiosity, why doesn't Python accept
>> def ():
>> ???? return '---'
>>
>> ()
>>
>> Where the function name is ''?
> 
> For the same reason an empty sequence of characters cannot
> be a variable name. Do you know any language (or formal
> theory) that allows that?

Tcl allows that:

Main console display active (Tcl8.6.9 / Tk8.6.9)
(CDEF) 49 % set "" Hallo
Hallo
(CDEF) 50 % puts ${}
Hallo
(CDEF) 51 % proc "" {} { puts "I'm empty" }
(CDEF) 52 % ""
I'm empty
(CDEF) 53 %

Any string can be a variable or command name, only :: is special as a 
namespace separator.

This only works because of the sparse syntax; to retrieve a variable's 
content, $ is used. For "strange" names quoting is required, therefore I 
had to use "" in the example.

It's a different matter how useful this actually is. One of the object 
systems in Tcl uses the empty variable to represent "self" as an array, 
so that you can write $(prop) for self.prop as it is in Python.

	Christian



From greg.ewing at canterbury.ac.nz  Sun Feb 20 17:17:53 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 21 Feb 2022 11:17:53 +1300
Subject: Why does not Python accept functions with no names?
In-Reply-To: <mailman.158.1645378047.7010.python-list@python.org>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
 <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>
 <mailman.158.1645378047.7010.python-list@python.org>
Message-ID: <j7ft0hF84ovU1@mid.individual.net>

On 21/02/22 6:27 am, Abdur-Rahmaan Janhangeer wrote:
> Well Python deliberately throws an exception if we do not
> pass in a function name. Just wanted to know why it should. As the
> above case is a 100% pure anonymous function.

The syntax for a function definition is defined in the grammar
as requiring an identifier. An identifier in turn is defined
as consisting of at least one character. So the grammar would
need to be changed to make the name optional. Also, the
compiler would need a special case to treat a missing name
there as though it were an empty string.

So again, why *should* it be allowed, given that the parser and
compiler would have to go out of their way to treat it as a
special case, only to create a function that there is no
easy way to call?

BTW, this is not what is usually meant by the term "anonymous
function". An anonymous function is one that is not bound
to *any* name. The thing you're proposing wouldn't be
anonymous -- it would have a name, that name being the empty
string.

-- 
Greg

From wlfraed at ix.netcom.com  Sun Feb 20 22:34:13 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sun, 20 Feb 2022 22:34:13 -0500
Subject: Long running process - how to speed up?
References: <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
 <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
 <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>
 <CA+i5JwbJVUg7gerHARxRCnkNpjP1gOdormZkAOT6TF6s4uN8uQ@mail.gmail.com>
 <31d366cd-2536-d528-6e6c-f22b18840dd4@wichmann.us>
 <CA+i5JwaryuvPmLudYU4Scog5nZeAw9Uk0STkxStuhiEafBaD=g@mail.gmail.com>
 <mailman.161.1645380345.7010.python-list@python.org>
Message-ID: <92161hpib4l2mv74gb0tome40o0erolq0r@4ax.com>

On Sun, 20 Feb 2022 18:05:33 +0000, Shaozhong SHI <shishaozhong at gmail.com>
declaimed the following:


>I am trying this approach,
>
>import multiprocessing as mp
>
>def my_func(x):
>  print(x**x)
>

	Not much of a processing load there, especially for your small set of
integers. I suspect you are consuming a significant time just having the OS
create and tear down each process.

	In truth, for that example, I suspect plain threads will run faster
because you don't have the overhead of setting up a process with new
stdin/stdout/stderr. The exponential operation probably runs within one
Python threading quantum, and the print() will trigger a thread switch to
allow the next one to compute.

>def main():
>  pool = mp.Pool(mp.cpu_count())
>  result = pool.map(my_func, [4,2,3])

-=-=-
>>> import multiprocessing as mp
>>> mp.cpu_count()
8
>>> 
-=-=-

	Really under-loaded on my Win10 system (hyperthreaded processors count
as 2 CPUs, so a quad-core HT reports as 8 CPUs).	Even an older
Raspberry-Pi 3B (quad core) reports

-=-=-
md_admin at microdiversity:~$ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing as mp
>>> mp.cpu_count()
4
>>> exit()
md_admin at microdiversity:~$
-=-=-


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From rosuav at gmail.com  Sun Feb 20 23:21:52 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 21 Feb 2022 15:21:52 +1100
Subject: Why does not Python accept functions with no names?
In-Reply-To: <sutqhf$p1$1@gioia.aioe.org>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <185C4014-65AA-4100-BCCA-3A66A50F8B0D@barrys-emacs.org>
 <mailman.155.1645374100.7010.python-list@python.org>
 <sutqhf$p1$1@gioia.aioe.org>
Message-ID: <CAPTjJmq0QncZ-O74taroD4g8ThOq2OrBfNT5r1HvoBrkCjRUyA@mail.gmail.com>

On Mon, 21 Feb 2022 at 14:36, Python <python at example.invalid> wrote:
>
> Barry wrote:
> >
> >
> >> On 20 Feb 2022, at 15:35, Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:
> >>
> >> ?Greetings list.
> >>
> >> Out of curiosity, why doesn't Python accept
> >> def ():
> >>     return '---'
> >>
> >> ()
> >>
> >> Where the function name is ''?
> >
> > Because there is no variable that is holding a ref to the code.
> > So it?s pointless.
>
> Fun fact, it can be done, but it's (afaik) then (almost) unusable...
>
>  >>> a
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> NameError: name 'a' is not defined
>  >>> locals()['a'] = 42
>  >>> a
> 42
>  >>> locals()[''] = 42
>  >>> locals()['']
> 42
>  >>> locals()[''] = (lambda x: x*42)
>  >>> locals()[''](1)
> 42
>

Minor nitpick: Please use globals() rather than locals() for this sort
of demo. At module level (including the REPL), they are the same, but
inside a function there's no guarantee that locals() can be mutated in
this way.

ChrisA

From grant.b.edwards at gmail.com  Sun Feb 20 23:45:32 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Sun, 20 Feb 2022 20:45:32 -0800 (PST)
Subject: Why does not Python accept functions with no names?
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <mailman.154.1645371155.7010.python-list@python.org>
 <sutnsc$lm1$1@gioia.aioe.org> <suu2ve$64h$1@dont-email.me>
Message-ID: <621318ec.1c69fb81.5d3c5.00a4@mx.google.com>

On 2022-02-20, Christian Gollwitzer <auriocus at gmx.de> wrote:

>> For the same reason an empty sequence of characters cannot
>> be a variable name. Do you know any language (or formal
>> theory) that allows that?
>
> Tcl allows that:

Interesting to know, but the fact that Tcl does something differnt is
more of an argument against it.

My one expereince trying to write a non-trivial application was a
study in frustration.  After spending days trying to get Tcl to do
something useful, I finally gave up and wrote the program in Scheme in
a few hours.

--
Grant

From avigross at verizon.net  Mon Feb 21 00:46:53 2022
From: avigross at verizon.net (Avi Gross)
Date: Mon, 21 Feb 2022 05:46:53 +0000 (UTC)
Subject: Why does not Python accept functions with no names?
In-Reply-To: <j7ft0hF84ovU1@mid.individual.net>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
 <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>
 <mailman.158.1645378047.7010.python-list@python.org>
 <j7ft0hF84ovU1@mid.individual.net>
Message-ID: <2097576864.290697.1645422413705@mail.yahoo.com>

Amusingly, Greg, if you had a system where the un-named anonymous function was to be named the unique value of the empty string, then a second such anonymous function definition would over-write it, as with any named function. The kind of anonymous function we are more used to might be something you can create as say elements of a list so you have a list of functions you can access as in f[0] but in a sense that has a name as it can be accessed as a component of the data structure called f.? I am not sure if python would trivially let you create that. But the point is if you want to be able to make many such pseudo-anonymous functions, in the end, there can only be one.


-----Original Message-----
From: Greg Ewing <greg.ewing at canterbury.ac.nz>
To: python-list at python.org
Sent: Sun, Feb 20, 2022 5:17 pm
Subject: Re: Why does not Python accept functions with no names?

On 21/02/22 6:27 am, Abdur-Rahmaan Janhangeer wrote:
> Well Python deliberately throws an exception if we do not
> pass in a function name. Just wanted to know why it should. As the
> above case is a 100% pure anonymous function.

The syntax for a function definition is defined in the grammar
as requiring an identifier. An identifier in turn is defined
as consisting of at least one character. So the grammar would
need to be changed to make the name optional. Also, the
compiler would need a special case to treat a missing name
there as though it were an empty string.

So again, why *should* it be allowed, given that the parser and
compiler would have to go out of their way to treat it as a
special case, only to create a function that there is no
easy way to call?

BTW, this is not what is usually meant by the term "anonymous
function". An anonymous function is one that is not bound
to *any* name. The thing you're proposing wouldn't be
anonymous -- it would have a name, that name being the empty
string.

-- 
Greg
-- 
https://mail.python.org/mailman/listinfo/python-list

From rosuav at gmail.com  Mon Feb 21 01:00:39 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 21 Feb 2022 17:00:39 +1100
Subject: Why does not Python accept functions with no names?
In-Reply-To: <2097576864.290697.1645422413705@mail.yahoo.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
 <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>
 <mailman.158.1645378047.7010.python-list@python.org>
 <j7ft0hF84ovU1@mid.individual.net>
 <2097576864.290697.1645422413705@mail.yahoo.com>
Message-ID: <CAPTjJmp=iKWgWhqL7tpnLrCJx4JJxOs8xyp6vjFw1q2+1qRCdA@mail.gmail.com>

On Mon, 21 Feb 2022 at 16:48, Avi Gross via Python-list
<python-list at python.org> wrote:
>
> Amusingly, Greg, if you had a system where the un-named anonymous function was to be named the unique value of the empty string, then a second such anonymous function definition would over-write it, as with any named function. The kind of anonymous function we are more used to might be something you can create as say elements of a list so you have a list of functions you can access as in f[0] but in a sense that has a name as it can be accessed as a component of the data structure called f.  I am not sure if python would trivially let you create that. But the point is if you want to be able to make many such pseudo-anonymous functions, in the end, there can only be one.
>

Functions in Python have two different concepts of "name", which
aren't always the same. One is the way that you refer to it; the other
is its __name__ attribute (and related attributes like __qualname__).
An anonymous function doesn't necessarily have to lack a name, per se;
it simply doesn't have a very interesting name:

>>> (lambda: 1).__name__
'<lambda>'
>>> (lambda: 2).__name__
'<lambda>'
>>> (lambda: 3).__name__
'<lambda>'

If you want to refer to them as f[0] etc, they can have arbitrary
names, which may or may not themselves be meaningful. I frequently
build a dictionary or list of functions by decorating standard 'def'
statements:

funcs = []

@funcs.append
def spam(): ...
@funcs.append
def ham(): ...

They still have __name__ attributes, but it's absolutely fine to
double them up, since they're going to be looked up via the list.
(Though there's still value in having at least *some* name on them,
since it shows up in backtraces.)

There's very little reason to have a function that actually doesn't
have a name. It's usually fine to give it an arbitrary and meaningless
name.

ChrisA

From eryksun at gmail.com  Mon Feb 21 02:35:17 2022
From: eryksun at gmail.com (Eryk Sun)
Date: Mon, 21 Feb 2022 01:35:17 -0600
Subject: Why does not Python accept functions with no names?
In-Reply-To: <j7ft0hF84ovU1@mid.individual.net>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
 <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>
 <mailman.158.1645378047.7010.python-list@python.org>
 <j7ft0hF84ovU1@mid.individual.net>
Message-ID: <CACL+1aup1BrEeuFgEpSGK33k8PNWoYQmQSbkfcq95oXEf6ycag@mail.gmail.com>

On 2/20/22, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> BTW, this is not what is usually meant by the term "anonymous
> function". An anonymous function is one that is not bound
> to *any* name. The thing you're proposing wouldn't be
> anonymous -- it would have a name, that name being the empty
> string.

Sorry. I read Avi's reply first, scanned the original post and didn't
consider the literal question. I doubt that Python would have become a
popular language if it allowed the empty string as an identifier. That
would let common syntax errors slip past the compiler with useless
results, e.g `= 1`, `c. = 1`. It's no less absurd in a `def`
statement.

From vanyp at skynet.be  Mon Feb 21 06:19:36 2022
From: vanyp at skynet.be (vanyp)
Date: Mon, 21 Feb 2022 12:19:36 +0100
Subject: Aw: PYT - How can I subscribe to a topic in the mailing list?
In-Reply-To: <trinity-98c1266a-c3a5-4533-a4bf-a30a939f79fc-1645302643381@3c-app-gmx-bap34>
References: <deb1ec0b-e8a4-cee7-b718-eb0425ca8d4f@skynet.be>
 <trinity-98c1266a-c3a5-4533-a4bf-a30a939f79fc-1645302643381@3c-app-gmx-bap34>
Message-ID: <bf328876-2176-29da-94e4-9854f166bb7f@skynet.be>

The option to filter by topic is offered in the mailing list 
subscription customization page, although no option list is given. 
Topics may have been a project that was never implemented.

Thank you for your help.


Le 19/02/2022 ? 21:30, Karsten Hilbert a ?crit?:
>> Betreff: PYT - How can I subscribe to a topic in the mailing list?
> Mailing lists don't subdivide by topic.
>
> Your mailer may allow you to filter by Subject:.
>
>> And h*ow do I set the topic in my messages?*
> not applicable
>
> The equivalent would be the Subject: header.
>
> Karsten

From avigross at verizon.net  Mon Feb 21 12:09:27 2022
From: avigross at verizon.net (Avi Gross)
Date: Mon, 21 Feb 2022 17:09:27 +0000 (UTC)
Subject: Why does not Python accept functions with no names?
In-Reply-To: <CACL+1aup1BrEeuFgEpSGK33k8PNWoYQmQSbkfcq95oXEf6ycag@mail.gmail.com>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
 <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>
 <mailman.158.1645378047.7010.python-list@python.org>
 <j7ft0hF84ovU1@mid.individual.net>
 <CACL+1aup1BrEeuFgEpSGK33k8PNWoYQmQSbkfcq95oXEf6ycag@mail.gmail.com>
Message-ID: <1789599238.380432.1645463367458@mail.yahoo.com>

Eric,
You bring up a related topic which I agree with. You need to be careful to make aspects of a language as consistent as possible and thus allowing no receiver of a function definition (somewhat different than no name) might result in anomalies in other parts of the language. Errors that could be caught would be accepted, even if teh resulting code was nonsense.
Some languages use a real placeholder such as "_" (single underscore) to represent an I DON'T CARE scenario meaning I do not want to provide a name and Python allows code like:
(a, _, c, _, e) = (1, 2, 3, 4, 5)
print(a, _, c, _, e)1 4 3 4 5
The above code snippet suggests that _ is allowed to be used multiple times and retains whatever is done last. But an experiment shows there is nothing special about underscore as a name in that replacing it with an x above gets the same results. It may not be implemented at the interpreter level as "throw away" or skip over, but I can imagine languages implementing just that. In the above, certainly, the reading in of the number 2 may have happened and the variable reused leaves it without one of the pointers to it.
But consider what happens if the matching above was to more complex items like function calls that may have all kinds of effects and side effects.? I may want to throw away the results returned, but may want the side effect.?
So in a sense the question being asked might have a partial suggestion, in Python, to simply provide a clue when creating the function to have the name _, albeit I still keep wondering WHY you want that.
Consider other things like a language in which just typing the name of a variable has no meaning. I mean it does not autoprint. If a line by itself says:
myvar
It typically starts a search for whatever method the language uses to find that name in their data structures to determine if it exists in the current context. If not found, you expect an error message. But even if found, what does one do with it? In some languages, there seems to be no point in even accessing it and it seems to be anything from harmless code to be ignored or an error because it is not a valid construct and the user may have meant to say myvar = 6 or something. But in a language like Python, it may have some impact just from being invoked such as incrementing a counter within an object. If you want to have such functionality in the language, you now have to allow it. And one way to support this is to make believe you typed print(myvar) which on my machine fails when myvar is not defined. When it is, it prints what it can. This works even when I ask for seemingly useless operations like "()" which creates and then (after the print) ignores an empty tuple.?
Anonymous functions arguably have an important place in many languages and I keep seeing languages adding some version of them or new syntax for them, as R recently did. But they are best used when made deliberately and the example we are given seems highly likely to be a mistake where the user forgot to put in a name where it was expected. At the very least it may merit a WARNING. Or, if allowed, I wonder if the interpreter should perhaps do something amusing like give it some long nonsense name and notify the user that in case they actually wanted a real function, it is now mxyzptlk666 ...


-----Original Message-----
From: Eryk Sun <eryksun at gmail.com>
To: python-list at python.org
Sent: Mon, Feb 21, 2022 2:35 am
Subject: Re: Why does not Python accept functions with no names?

On 2/20/22, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> BTW, this is not what is usually meant by the term "anonymous
> function". An anonymous function is one that is not bound
> to *any* name. The thing you're proposing wouldn't be
> anonymous -- it would have a name, that name being the empty
> string.

Sorry. I read Avi's reply first, scanned the original post and didn't
consider the literal question. I doubt that Python would have become a
popular language if it allowed the empty string as an identifier. That
would let common syntax errors slip past the compiler with useless
results, e.g `= 1`, `c. = 1`. It's no less absurd in a `def`
statement.
-- 
https://mail.python.org/mailman/listinfo/python-list

From martin.schoon at gmail.com  Mon Feb 21 14:52:08 2022
From: martin.schoon at gmail.com (Martin =?UTF-8?Q?Sch=C3=B6=C3=B6n?=)
Date: 21 Feb 2022 19:52:08 GMT
Subject: Saving/exporting plots from Jupyter-labs?
References: <j6vtr9F64mrU1@mid.individual.net>
Message-ID: <j7i8r8Fm182U1@mid.individual.net>

Den 2022-02-14 skrev Martin Sch??n <martin.schoon at gmail.com>:
>
> Now I am trying out Jupyter-labs. I like it. I have two head-
> scratchers for now:
>
<snip>
> 2) Why is Jupyter-labs hooking up to Google-analytics?

Now I can answer this one myself. In a tab I had been working my
way through a Holoviews tutorial. The tutorial demonstrated how
one can open the online Holoviews documentation inside a 
notebook cell. This is how google-analytics and twitter got
'involved'. Jupyter-labs was not guilty.

/Martin

From arj.python at gmail.com  Mon Feb 21 23:05:27 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Tue, 22 Feb 2022 07:05:27 +0300
Subject: Why does not Python accept functions with no names?
In-Reply-To: <j7ft0hF84ovU1@mid.individual.net>
References: <CADrxXXn+xkO_vZXwqXwL21iLF4LJE+avAnfoXkBe2xV1rZigsQ@mail.gmail.com>
 <CAPTjJmrNRa_Ut1NkEx4QDeVxXoXhQM7LETtdA6PZnZpfoOhFTQ@mail.gmail.com>
 <CADrxXXmGcWXdVtL3aJr-GXFyp=GD8asz6UuzD=Yi8JxDo9=VYA@mail.gmail.com>
 <mailman.158.1645378047.7010.python-list@python.org>
 <j7ft0hF84ovU1@mid.individual.net>
Message-ID: <CADrxXXnUgfZ-pvWWyQwy7FCn8U-F45xmuFsbCe8nPdR5LY6ZZA@mail.gmail.com>

> BTW, this is not what is usually meant by the term "anonymous
function". An anonymous function is one that is not bound
to *any* name. The thing you're proposing wouldn't be
anonymous -- it would have a name, that name being the empty
string.


Thanks for clarifying this point ???

From sasikanthreddy1998 at hotmail.com  Tue Feb 22 01:17:48 2022
From: sasikanthreddy1998 at hotmail.com (SASI KANTH REDDY GUJJULA)
Date: Tue, 22 Feb 2022 06:17:48 +0000
Subject: Python
Message-ID: <HK0PR06MB302821777E610D7E42B3DF2DD83B9@HK0PR06MB3028.apcprd06.prod.outlook.com>

Pip files are not installing after the python 3.10.2 version installing in my devise. Please solve this for me.

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows


From arj.python at gmail.com  Tue Feb 22 03:31:42 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Tue, 22 Feb 2022 12:31:42 +0400
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
Message-ID: <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>

A front end eng sent me this for how to check for the internet in JS

https://html.spec.whatwg.org/multipage/system-state.html#dom-navigator-online

But it also says:

"This attribute is inherently unreliable. A computer can be connected to a
network without having Internet access."

As discussed here but, it would have been nevertheless great to have this
tiny function instead of
nothing

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius


On Mon, Feb 7, 2022 at 1:17 PM Abdur-Rahmaan Janhangeer <
arj.python at gmail.com> wrote:

> Greetings,
>
> Using the standard library or 3rd party libraries, what's the
> best way to check if there is internet? Checking if google.com
> is reachable is good but I wonder if there is a more native,
> protocol-oriented
> way of knowing?
>
> Even this URL recommends checking if a domain is up as a way to check for
> internet connectivity:
>
> https://www.ibm.com/support/pages/checking-network-connectivity-when-using-python-and-ibm-resilient-circuits
>
> Kind Regards,
>
> Abdur-Rahmaan Janhangeer
> about <https://compileralchemy.github.io/> | blog
> <https://www.pythonkitchen.com>
> github <https://github.com/Abdur-RahmaanJ>
> Mauritius
>

From rosuav at gmail.com  Tue Feb 22 03:40:15 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 22 Feb 2022 19:40:15 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
Message-ID: <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>

On Tue, 22 Feb 2022 at 19:33, Abdur-Rahmaan Janhangeer
<arj.python at gmail.com> wrote:
>
> As discussed here but, it would have been nevertheless great to have this
> tiny function instead of
> nothing
>

Here's a function that determines whether or not you have an internet
connection. It's almost as reliable as some of the other examples
given - I know this, because I tried it ten times, and it gave the
correct result every time!

def has_internet():
    return True

Tell me, is it useful to have something that doesn't always give the
right answer, even if it usually does? Is there any value whatsoever
in a lie?

ChrisA

From rosuav at gmail.com  Tue Feb 22 03:40:54 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 22 Feb 2022 19:40:54 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
 <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
Message-ID: <CAPTjJmr-Lwhs09fqYVPEZmxAB2Ngq41BLaB1-mhOHUkfBqrqSw@mail.gmail.com>

On Tue, 22 Feb 2022 at 19:40, Chris Angelico <rosuav at gmail.com> wrote:
>
> On Tue, 22 Feb 2022 at 19:33, Abdur-Rahmaan Janhangeer
> <arj.python at gmail.com> wrote:
> >
> > As discussed here but, it would have been nevertheless great to have this
> > tiny function instead of
> > nothing
> >
>
> Here's a function that determines whether or not you have an internet
> connection. It's almost as reliable as some of the other examples
> given - I know this, because I tried it ten times, and it gave the
> correct result every time!
>
> def has_internet():
>     return True
>
> Tell me, is it useful to have something that doesn't always give the
> right answer, even if it usually does? Is there any value whatsoever
> in a lie?

Obligatory XKCD:

https://xkcd.com/937/

ChrisA

From arj.python at gmail.com  Tue Feb 22 04:00:54 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Tue, 22 Feb 2022 13:00:54 +0400
Subject: Best way to check if there is internet?
In-Reply-To: <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
 <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
Message-ID: <CADrxXXni3__uxmzFpNjQW=0BT-q=XiN6MAq7NheyN2Ca+vjuXQ@mail.gmail.com>

Thanks for the function but i think having the function
as described won't return True all time

Me: dipping my foot in hot water, yes we can go ...

In the sprit of "practicality beats purity", devs needs a function
XD

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From Karsten.Hilbert at gmx.net  Tue Feb 22 04:20:04 2022
From: Karsten.Hilbert at gmx.net (Karsten Hilbert)
Date: Tue, 22 Feb 2022 10:20:04 +0100
Subject: Aw: Re: Best way to check if there is internet?
In-Reply-To: <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
 <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
Message-ID: <trinity-cbb5f14e-44a6-4bc3-a1ff-80943dd543aa-1645521604952@3c-app-gmx-bs40>

> Is there any value whatsoever in a lie?

Do we _know_ it's a lie ?

Does a lie become a Falsed Truth once it becomes known ?

Karsten


From frank at chagford.com  Tue Feb 22 04:19:33 2022
From: frank at chagford.com (Frank Millman)
Date: Tue, 22 Feb 2022 11:19:33 +0200
Subject: One-liner to merge lists?
Message-ID: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>

Hi all

I think this should be a simple one-liner, but I cannot figure it out.

I have a dictionary with a number of keys, where each value is a single 
list -

 >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}

I want to combine all values into a single list -

 >>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']

I can do this -

 >>> a = []
 >>> for v in d.values():
...?? a.extend(v)
...
 >>> a
['aaa', 'bbb', 'ccc', 'fff', 'ggg']

I can also do this -

 >>> from itertools import chain
 >>> a = list(chain(*d.values()))
 >>> a
['aaa', 'bbb', 'ccc', 'fff', 'ggg']
 >>>

Is there a simpler way?

Thanks

Frank Millman



From rosuav at gmail.com  Tue Feb 22 04:30:13 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 22 Feb 2022 20:30:13 +1100
Subject: One-liner to merge lists?
In-Reply-To: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
Message-ID: <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>

On Tue, 22 Feb 2022 at 20:24, Frank Millman <frank at chagford.com> wrote:
>
> Hi all
>
> I think this should be a simple one-liner, but I cannot figure it out.
>
> I have a dictionary with a number of keys, where each value is a single
> list -
>
>  >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>
> I want to combine all values into a single list -
>
>  >>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>
> I can do this -
>
>  >>> a = []
>  >>> for v in d.values():
> ...   a.extend(v)
> ...
>  >>> a
> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>
> I can also do this -
>
>  >>> from itertools import chain
>  >>> a = list(chain(*d.values()))
>  >>> a
> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>  >>>
>
> Is there a simpler way?
>

itertools.chain is a good option, as it scales well to arbitrary
numbers of lists (and you're guaranteed to iterate over them all just
once as you construct the list). But if you know that the lists aren't
too large or too numerous, here's another method that works:

>>> sum(d.values(), [])
['aaa', 'bbb', 'ccc', 'fff', 'ggg']

It's simply adding all the lists together, though you have to tell it
that you don't want a numeric summation.

ChrisA

From frank at chagford.com  Tue Feb 22 04:44:11 2022
From: frank at chagford.com (Frank Millman)
Date: Tue, 22 Feb 2022 11:44:11 +0200
Subject: One-liner to merge lists?
In-Reply-To: <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
Message-ID: <274c2144-a996-5b2d-173e-0c51bcfc71c5@chagford.com>

On 2022-02-22 11:30 AM, Chris Angelico wrote:
> On Tue, 22 Feb 2022 at 20:24, Frank Millman <frank at chagford.com> wrote:
>>
>> Hi all
>>
>> I think this should be a simple one-liner, but I cannot figure it out.
>>
>> I have a dictionary with a number of keys, where each value is a single
>> list -
>>
>>   >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>>
>> I want to combine all values into a single list -
>>
>>   >>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>
>> I can do this -
>>
>>   >>> a = []
>>   >>> for v in d.values():
>> ...   a.extend(v)
>> ...
>>   >>> a
>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>
>> I can also do this -
>>
>>   >>> from itertools import chain
>>   >>> a = list(chain(*d.values()))
>>   >>> a
>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>   >>>
>>
>> Is there a simpler way?
>>
> 
> itertools.chain is a good option, as it scales well to arbitrary
> numbers of lists (and you're guaranteed to iterate over them all just
> once as you construct the list). But if you know that the lists aren't
> too large or too numerous, here's another method that works:
> 
>>>> sum(d.values(), [])
> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> 
> It's simply adding all the lists together, though you have to tell it
> that you don't want a numeric summation.
> 

Thanks, that is neat.

However, I did see this -

 >>> help(sum)
Help on built-in function sum in module builtins:

sum(iterable, /, start=0)
     Return the sum of a 'start' value (default: 0) plus an iterable of 
numbers

     When the iterable is empty, return the start value.
     This function is intended specifically for use with numeric values
and may reject non-numeric types.
 >>>

So it seems that it is not recommended.

I think I will stick with itertools.chain.

Frank


From rosuav at gmail.com  Tue Feb 22 04:52:28 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 22 Feb 2022 20:52:28 +1100
Subject: One-liner to merge lists?
In-Reply-To: <274c2144-a996-5b2d-173e-0c51bcfc71c5@chagford.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
 <274c2144-a996-5b2d-173e-0c51bcfc71c5@chagford.com>
Message-ID: <CAPTjJmr-JgGTzqxO-H65DGxE6SLGFdgyfrT-xz2s1Q6XaFsg5A@mail.gmail.com>

On Tue, 22 Feb 2022 at 20:46, Frank Millman <frank at chagford.com> wrote:
>
> On 2022-02-22 11:30 AM, Chris Angelico wrote:
> > On Tue, 22 Feb 2022 at 20:24, Frank Millman <frank at chagford.com> wrote:
> >>
> >> Hi all
> >>
> >> I think this should be a simple one-liner, but I cannot figure it out.
> >>
> >> I have a dictionary with a number of keys, where each value is a single
> >> list -
> >>
> >>   >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
> >>
> >> I want to combine all values into a single list -
> >>
> >>   >>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>
> >> I can do this -
> >>
> >>   >>> a = []
> >>   >>> for v in d.values():
> >> ...   a.extend(v)
> >> ...
> >>   >>> a
> >> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>
> >> I can also do this -
> >>
> >>   >>> from itertools import chain
> >>   >>> a = list(chain(*d.values()))
> >>   >>> a
> >> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>   >>>
> >>
> >> Is there a simpler way?
> >>
> >
> > itertools.chain is a good option, as it scales well to arbitrary
> > numbers of lists (and you're guaranteed to iterate over them all just
> > once as you construct the list). But if you know that the lists aren't
> > too large or too numerous, here's another method that works:
> >
> >>>> sum(d.values(), [])
> > ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >
> > It's simply adding all the lists together, though you have to tell it
> > that you don't want a numeric summation.
> >
>
> Thanks, that is neat.
>
> However, I did see this -
>
>  >>> help(sum)
> Help on built-in function sum in module builtins:
>
> sum(iterable, /, start=0)
>      Return the sum of a 'start' value (default: 0) plus an iterable of
> numbers
>
>      When the iterable is empty, return the start value.
>      This function is intended specifically for use with numeric values
> and may reject non-numeric types.
>  >>>
>
> So it seems that it is not recommended.
>
> I think I will stick with itertools.chain.
>

Yup, itertools.chain is definitely the recommended way to do things.
It's short, efficient, and only slightly unclear. If the clarity is a
problem, you can always wrap it into a function:

def sum_lists(iterable):
    return list(chain.from_iterable(iterable))


ChrisA

From antoon.pardon at vub.be  Tue Feb 22 09:04:54 2022
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Tue, 22 Feb 2022 15:04:54 +0100
Subject: Best way to check if there is internet?
In-Reply-To: <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
 <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
Message-ID: <cfc23612-62ce-b9d1-230f-b06066512904@vub.be>


Op 22/02/2022 om 09:40 schreef Chris Angelico:
> On Tue, 22 Feb 2022 at 19:33, Abdur-Rahmaan Janhangeer
> <arj.python at gmail.com>  wrote:
>> As discussed here but, it would have been nevertheless great to have this
>> tiny function instead of
>> nothing
>>
> Here's a function that determines whether or not you have an internet
> connection. It's almost as reliable as some of the other examples
> given - I know this, because I tried it ten times, and it gave the
> correct result every time!

So, you discovered a way of testing that is not very thorough.

> def has_internet():
>      return True
>
> Tell me, is it useful to have something that doesn't always give the
> right answer, even if it usually does? Is there any value whatsoever
> in a lie?

Yes that is useful. Live is full of that kind of situations. We in computerland
are spoiled with the accuracy we can enjoy. It seems even spoiled to the extend
that when offered a solution that is not 100% accurated we consider it a lie.

-- 
Antoon.

From mats at wichmann.us  Tue Feb 22 09:07:54 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Tue, 22 Feb 2022 07:07:54 -0700
Subject: Python
In-Reply-To: <HK0PR06MB302821777E610D7E42B3DF2DD83B9@HK0PR06MB3028.apcprd06.prod.outlook.com>
References: <HK0PR06MB302821777E610D7E42B3DF2DD83B9@HK0PR06MB3028.apcprd06.prod.outlook.com>
Message-ID: <8c0a6db6-8584-e398-554c-8dcbaa123eab@wichmann.us>

On 2/21/22 23:17, SASI KANTH REDDY GUJJULA wrote:
> Pip files are not installing after the python 3.10.2 version installing in my devise. Please solve this for me.

Please ask a clearer question.

Can you tell us what "are not installing" means? Are you getting
permission errors?  Are you installing and then unable to import what
you have installed?  Something else?


From rosuav at gmail.com  Tue Feb 22 10:21:36 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 23 Feb 2022 02:21:36 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <cfc23612-62ce-b9d1-230f-b06066512904@vub.be>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
 <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
 <cfc23612-62ce-b9d1-230f-b06066512904@vub.be>
Message-ID: <CAPTjJmrSZdQn_B1TBJgE0roPZ8quCoQKTKtGJMLGnYZ0EJ587g@mail.gmail.com>

On Wed, 23 Feb 2022 at 01:06, Antoon Pardon <antoon.pardon at vub.be> wrote:
>
>
> Op 22/02/2022 om 09:40 schreef Chris Angelico:
> > On Tue, 22 Feb 2022 at 19:33, Abdur-Rahmaan Janhangeer
> > <arj.python at gmail.com>  wrote:
> >> As discussed here but, it would have been nevertheless great to have this
> >> tiny function instead of
> >> nothing
> >>
> > Here's a function that determines whether or not you have an internet
> > connection. It's almost as reliable as some of the other examples
> > given - I know this, because I tried it ten times, and it gave the
> > correct result every time!
>
> So, you discovered a way of testing that is not very thorough.
>
> > def has_internet():
> >      return True
> >
> > Tell me, is it useful to have something that doesn't always give the
> > right answer, even if it usually does? Is there any value whatsoever
> > in a lie?
>
> Yes that is useful. Live is full of that kind of situations. We in computerland
> are spoiled with the accuracy we can enjoy. It seems even spoiled to the extend
> that when offered a solution that is not 100% accurated we consider it a lie.
>

Cool! Then you have a very useful internet-availability testing
function. Have fun!

ChrisA

From cl at isbd.net  Tue Feb 22 04:02:35 2022
From: cl at isbd.net (Chris Green)
Date: Tue, 22 Feb 2022 09:02:35 +0000
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
 <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
 <mailman.178.1645519228.7010.python-list@python.org>
Message-ID: <beleei-f754.ln1@esprimo.zbmc.eu>

Chris Angelico <rosuav at gmail.com> wrote:
> On Tue, 22 Feb 2022 at 19:33, Abdur-Rahmaan Janhangeer
> <arj.python at gmail.com> wrote:
> >
> > As discussed here but, it would have been nevertheless great to have this
> > tiny function instead of
> > nothing
> >
> 
> Here's a function that determines whether or not you have an internet
> connection. It's almost as reliable as some of the other examples
> given - I know this, because I tried it ten times, and it gave the
> correct result every time!
> 
> def has_internet():
>     return True
> 
> Tell me, is it useful to have something that doesn't always give the
> right answer, even if it usually does? Is there any value whatsoever
> in a lie?
> 
That's sort of in the same area as a stopped clock being right more
often than one that runs just a bit slow (or fast).  :-)

-- 
Chris Green
?

From David.Raymond at tomtom.com  Tue Feb 22 10:45:52 2022
From: David.Raymond at tomtom.com (David Raymond)
Date: Tue, 22 Feb 2022 15:45:52 +0000
Subject: One-liner to merge lists?
In-Reply-To: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
Message-ID: <AM0PR07MB5700068EE7683E9AACAE78C8873B9@AM0PR07MB5700.eurprd07.prod.outlook.com>

> Is there a simpler way?

>>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>>> [a for b in d.values() for a in b]
['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>


From rosuav at gmail.com  Tue Feb 22 11:01:25 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 23 Feb 2022 03:01:25 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <beleei-f754.ln1@esprimo.zbmc.eu>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
 <CAPTjJmpjY=KgxqUZv50HCO6JBjNs8k5koJ0ZdY_h=Nq9rUyXWQ@mail.gmail.com>
 <mailman.178.1645519228.7010.python-list@python.org>
 <beleei-f754.ln1@esprimo.zbmc.eu>
Message-ID: <CAPTjJmotmGZ6m51pOQbs9j+42mUkqNjoMCwO4azX1A72c1W5ag@mail.gmail.com>

On Wed, 23 Feb 2022 at 02:33, Chris Green <cl at isbd.net> wrote:
>
> Chris Angelico <rosuav at gmail.com> wrote:
> > On Tue, 22 Feb 2022 at 19:33, Abdur-Rahmaan Janhangeer
> > <arj.python at gmail.com> wrote:
> > >
> > > As discussed here but, it would have been nevertheless great to have this
> > > tiny function instead of
> > > nothing
> > >
> >
> > Here's a function that determines whether or not you have an internet
> > connection. It's almost as reliable as some of the other examples
> > given - I know this, because I tried it ten times, and it gave the
> > correct result every time!
> >
> > def has_internet():
> >     return True
> >
> > Tell me, is it useful to have something that doesn't always give the
> > right answer, even if it usually does? Is there any value whatsoever
> > in a lie?
> >
> That's sort of in the same area as a stopped clock being right more
> often than one that runs just a bit slow (or fast).  :-)
>

Oddly enough, if your options for clocks are "right twice a day",
"right once a week", "right once a month", and "never right", the most
useful is probably actually the one that's never right, because if it
ticks at the right rate but is set slightly wrong, it'll always be
exactly that same offset away from correct (which you can adjust for).

But a function to tell you whether you have the internet or not is
like a clock that tells you whether you have time or not. In a
technical (and vacuous) sense, yes, you do... in a practical and
useful sense, who knows?

ChrisA

From 2QdxY4RzWzUUiLuE at potatochowder.com  Tue Feb 22 10:57:02 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Tue, 22 Feb 2022 09:57:02 -0600
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
Message-ID: <YhUHzqDyTcAijuWG@scrozzle>

On 2022-02-09 at 11:15:34 +0400,
Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:

> I think for me having the internet means ability to request urls

You can always ask.

The real question is what will the response be?  ;-)

This entire exercise is a race condition, just like checking for that a
file exists before deleting it, or that it doesn't exist before creating
it.

If you "have internet" when you check, what assurance do you have that
you will still "have internet" when you actually want to use it?

And if you don't "have internet" when you check, when do you check
again?

I think someone said it way upthread:  don't check, just do whatever you
came to do, and it will work or it will fail (presumably, your program
can tell the difference, regardless of a past snapshot of being able to
retrieve data from an arbitrary URL).

EAFP, anyone?

From avigross at verizon.net  Tue Feb 22 12:07:45 2022
From: avigross at verizon.net (Avi Gross)
Date: Tue, 22 Feb 2022 17:07:45 +0000 (UTC)
Subject: One-liner to merge lists?
In-Reply-To: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
Message-ID: <1749653924.602429.1645549665669@mail.yahoo.com>

Frank,

Given this kind of data: d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}

You seem focused on a one-liner, however ridiculous or inefficient. Does this count as a somewhat weird one liner?

comb = [];   _ = [comb.append(v) for v in d.values()]

If you run the code and ignore the returned result, the value in comb is:

print(comb)

[['aaa', 'bbb', 'ccc'], ['fff', 'ggg']]

As discussed the other day here, this is an example of a variant on the anonymous function discussion as what is wanted is a side effect. I could omit assigning the result to anything and run this code after the previous like this:

[comb.append(v) for v in d.values()]
[None, None]
comb
[['aaa', 'bbb', 'ccc'], ['fff', 'ggg'], ['aaa', 'bbb', 'ccc'], ['fff', 'ggg']]

As shown the list comprehension itself is not returning anything of value and need not be assigned to anything. But it then generally is set to autoprint and that can be supressed by assigning the value to _ or anything you can ignore.

I am NOT saying this is a good way but that it is sort of a one-liner. There are good reasons I do not see people doing things this way!

-----Original Message-----
From: Frank Millman <frank at chagford.com>
To: python-list at python.org
Sent: Tue, Feb 22, 2022 4:19 am
Subject: One-liner to merge lists?


Hi all



I think this should be a simple one-liner, but I cannot figure it out.



I have a dictionary with a number of keys, where each value is a single 

list -



 >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}



I want to combine all values into a single list -



 >>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']



I can do this -



 >>> a = []

 >>> for v in d.values():

...?? a.extend(v)

...

 >>> a

['aaa', 'bbb', 'ccc', 'fff', 'ggg']



I can also do this -



 >>> from itertools import chain

 >>> a = list(chain(*d.values()))

 >>> a

['aaa', 'bbb', 'ccc', 'fff', 'ggg']

 >>>



Is there a simpler way?



Thanks



Frank Millman





-- 

https://mail.python.org/mailman/listinfo/python-list


From arj.python at gmail.com  Tue Feb 22 12:32:34 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Tue, 22 Feb 2022 20:32:34 +0300
Subject: Best way to check if there is internet?
In-Reply-To: <YhUHzqDyTcAijuWG@scrozzle>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <YhUHzqDyTcAijuWG@scrozzle>
Message-ID: <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>

Well, nice perspective.

It's a valid consideration, sound theory
but poor practicality according to me.

It you view it like this then between the moment
we press run or enter key on the terminal,
our Python interpreter might get deleted.

We never know, it can happen.

Though it's nice to go in and catch exceptions,
if you have a long operation upcoming it might be nice
to see if your key element is present.

Much like checking if goal posts are present before
starting a football game. You can of course start
the match, pass the ball around and when shooting
you stop the match as you realise that the goal posts
are not around.

Checking by exceptions is what the snippet
i shared does. But we check for the ability to do it
before we start.

Of course, it can change in the seconds that follow. But it's too much pure
logic at work.

From avigross at verizon.net  Tue Feb 22 13:37:15 2022
From: avigross at verizon.net (Avi Gross)
Date: Tue, 22 Feb 2022 18:37:15 +0000 (UTC)
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <YhUHzqDyTcAijuWG@scrozzle>
 <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
Message-ID: <679863926.632554.1645555035148@mail.yahoo.com>

As usual, his discussion is wandering a bit.

Yes, some things are not reliable but sometimes a heuristic answer is better than none.

Internet connections are beyond flaky and worse the connection you want may be blocked for many reasons even if other connections you try are working. Any number of routers can lose power or you personally may be blocked by something if they deem the place you want unsafe and much more.

In Statistics, you often work not with TRUE and FALSE but with ranges of probability and you may deem something as tentatively true if it is 95% or 99% likely it did not happen by chance given your sample.

But for a long-running program, checking if the internet is up ONCE is clearly not good enough if it runs for months at a time. Checking right before making a connection may be a bit much too.

But computers often spend a majority of their time doing things like checking. If I am using a GUI and the process is waiting to see if I type some key (or even am depressing the Shift key which normally does not generate any symbols by itself) and also testing if my mouse is moving or has moved over an area or one of several buttons has been clicked, then what may be happening is something in a loop that keeps checking and when an event has happened, notifies something to continue and perhaps provides the results. Something along those lines can happen when say a region of memory/data is being shared and one part wants to know if the memory is still locked or has finally been written to by another part and can be read and so on.

For something important, it is not enough to check if a file system is mounted and available and then write the file and move on. You may want to not only test the exit status of the write, or arrange to catch exceptions, but to then re-open the file and read in what you wrote and verify that it seems correct. In some cases, you get even more paranoid and save some kind of log entry that can be used to recreate the scenario if something bad happens and store that in another location in case there is a fire or something. 

So the answer is that it is up to you to decide how much effort is warranted. Can you just try a connection and react if it fails or must you at least get a decent guess that there is an internet connection of some kind up and running or must it be ever more bulletproof?

A scenario might be one where you are providing info and can adjust depending on various settings. If the user has an internet connection on, you can use video and graphics and so on that result in something current. But if they have settings suggesting they want to minimize the data they send and receive, the same request may result in the browser you invoked showing them just some alternative text. You have no way of knowing. Again, if the internet is not up, you may just present your own canned text, perhaps dated. Or, you can suggest to the user they turn on the internet and try again, or ...

It sounds to me like your main issue is not whether the internet is up right now but whether the user has an internet connection. If they run the program on a laptop at home or work then they may well be connected, but sitting in their car in some parking lot, may well not be.




-----Original Message-----
From: Abdur-Rahmaan Janhangeer <arj.python at gmail.com>
To: Python <python-list at python.org>
Sent: Tue, Feb 22, 2022 12:32 pm
Subject: Re: Best way to check if there is internet?


Well, nice perspective.

It's a valid consideration, sound theory
but poor practicality according to me.

It you view it like this then between the moment
we press run or enter key on the terminal,
our Python interpreter might get deleted.

We never know, it can happen.

Though it's nice to go in and catch exceptions,
if you have a long operation upcoming it might be nice
to see if your key element is present.

Much like checking if goal posts are present before
starting a football game. You can of course start
the match, pass the ball around and when shooting
you stop the match as you realise that the goal posts
are not around.

Checking by exceptions is what the snippet
i shared does. But we check for the ability to do it
before we start.

Of course, it can change in the seconds that follow. But it's too much pure
logic at work.

-- 
https://mail.python.org/mailman/listinfo/python-list


From rosuav at gmail.com  Tue Feb 22 13:47:22 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 23 Feb 2022 05:47:22 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <679863926.632554.1645555035148@mail.yahoo.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <YhUHzqDyTcAijuWG@scrozzle>
 <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
 <679863926.632554.1645555035148@mail.yahoo.com>
Message-ID: <CAPTjJmrtTKis-16S1WYNpVJ6Xdb2YB-A9w7PZS+x1QQsSb+VJA@mail.gmail.com>

On Wed, 23 Feb 2022 at 05:38, Avi Gross via Python-list
<python-list at python.org> wrote:
> It sounds to me like your main issue is not whether the internet is up right now but whether the user has an internet connection. If they run the program on a laptop at home or work then they may well be connected, but sitting in their car in some parking lot, may well not be.
>

Very seldom, a computer actually has *no* network connection. Much
more often, you have a network connection, but it's not very useful.
For example, you might have full network connectivity... but only on
localhost. Or you might be on a LAN, and fully able to access its
resources, but your uplink to the rest of the internet is down. Or
maybe you're a server inside Facebook, and the rest of the internet is
slowly getting disconnected from you. Do you have an internet
connection? Do you have what you need?

*There is no way* to answer the question of whether you have internet
access, other than to try the exact thing you want to do, and see if
it fails. People who think that there's a binary state of "have
internet" // "do not have internet" have clearly never worked in
networking....

(No, I've never accidentally misconfigured a network so that we had
full access to everything *except* the local network. Why do you ask?
Anyway, I fixed it within a few minutes.)

ChrisA

From arj.python at gmail.com  Tue Feb 22 14:04:43 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Tue, 22 Feb 2022 22:04:43 +0300
Subject: Best way to check if there is internet?
In-Reply-To: <679863926.632554.1645555035148@mail.yahoo.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <YhUHzqDyTcAijuWG@scrozzle>
 <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
 <679863926.632554.1645555035148@mail.yahoo.com>
Message-ID: <CADrxXX=VFCOPZSw5ibcLrbCdiGT15SdOxDjXVLr2BANCd=0_SQ@mail.gmail.com>

I've got my answers but the 'wandering' a bit
on this thread is at least connected to the topic ^^.

Last question: If we check for wifi or ethernet cable connection?

Wifi sometimes tell of connected but no internet connection.
So it detected that there is or there is no internet ...

From barry at barrys-emacs.org  Tue Feb 22 16:03:49 2022
From: barry at barrys-emacs.org (Barry)
Date: Tue, 22 Feb 2022 21:03:49 +0000
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
References: <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
Message-ID: <E68CCD84-AF7E-42DA-BC50-0CB43DB7CF98@barrys-emacs.org>



> On 22 Feb 2022, at 17:44, Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:
> 
> ?Well, nice perspective.
> 
> It's a valid consideration, sound theory
> but poor practicality according to me.
> 
> It you view it like this then between the moment
> we press run or enter key on the terminal,
> our Python interpreter might get deleted.

This is under your control usually so you do not need to check for python to be installed.
> 
> We never know, it can happen.
> 
> Though it's nice to go in and catch exceptions,
> if you have a long operation upcoming it might be nice
> to see if your key element is present.

You might check for enough disk space before starting to ensure that you can write the results, again this is usually under your control.

But you do not control the internet as an end user.
As it is not under your control you have to code to survive its failure modes.
For example email delivery is retried until the MTA gives up or succeeds.

> 
> Much like checking if goal posts are present before
> starting a football game. You can of course start
> the match, pass the ball around and when shooting
> you stop the match as you realise that the goal posts
> are not around.

Check before you start. But you are going to notice people attempting to take goal posts away while you play.

> Checking by exceptions is what the snippet
> i shared does. But we check for the ability to do it
> before we start.
> 
> Of course, it can change in the seconds that follow. But it's too much pure
> logic at work.

Since you have to deal with things that you do not control changing after you check what is the point in checking? You have to write the code to recover anyway.

Barry

> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From drsalists at gmail.com  Tue Feb 22 23:02:57 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 22 Feb 2022 20:02:57 -0800
Subject: One-liner to merge lists?
In-Reply-To: <AM0PR07MB5700068EE7683E9AACAE78C8873B9@AM0PR07MB5700.eurprd07.prod.outlook.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <AM0PR07MB5700068EE7683E9AACAE78C8873B9@AM0PR07MB5700.eurprd07.prod.outlook.com>
Message-ID: <CAGGBd_q-i9UdEpywpR3WxRJVKCMa2Dn+ULRPdzb5cT5F4S0C9g@mail.gmail.com>

On Tue, Feb 22, 2022 at 7:46 AM David Raymond <David.Raymond at tomtom.com>
wrote:

> > Is there a simpler way?
>
> >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
> >>> [a for b in d.values() for a in b]
> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>>
>

I like that way best.

But I'd still:
1) use a little more descriptive identifiers
2) put it in a function with a descriptive name
3) give the function a decent docstring

From arj.python at gmail.com  Tue Feb 22 23:08:27 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Wed, 23 Feb 2022 07:08:27 +0300
Subject: Best way to check if there is internet?
In-Reply-To: <E68CCD84-AF7E-42DA-BC50-0CB43DB7CF98@barrys-emacs.org>
References: <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
 <E68CCD84-AF7E-42DA-BC50-0CB43DB7CF98@barrys-emacs.org>
Message-ID: <CADrxXX=xHQNBt=wGBJCxjy1sGpwC51=A6BLz1nniRbvFXg4WLg@mail.gmail.com>

>
> > Since you have to deal with things that you do not control changing
> after you check what is the point in checking? You have to write the code
> to recover anyway.
>


Well foe my usecase, the operation running is not
in months or a really long time. As Avi mentionned above,
that would be pointless.

The upcoming operation is short enough and
though we cannot assume that it will be the case
when we are running the operation, we sensibly
assume it will still be the case.

This is more an extra check to not start an operation than
either relying to check at the start or when query.
It's both.

>

From rosuav at gmail.com  Tue Feb 22 23:42:04 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 23 Feb 2022 15:42:04 +1100
Subject: One-liner to merge lists?
In-Reply-To: <CAGGBd_q-i9UdEpywpR3WxRJVKCMa2Dn+ULRPdzb5cT5F4S0C9g@mail.gmail.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <AM0PR07MB5700068EE7683E9AACAE78C8873B9@AM0PR07MB5700.eurprd07.prod.outlook.com>
 <CAGGBd_q-i9UdEpywpR3WxRJVKCMa2Dn+ULRPdzb5cT5F4S0C9g@mail.gmail.com>
Message-ID: <CAPTjJmqL34dz5CcBfU8Ygd0v7fVfebRtyCRk5U5qgjw8k4Eg-A@mail.gmail.com>

On Wed, 23 Feb 2022 at 15:04, Dan Stromberg <drsalists at gmail.com> wrote:
>
> On Tue, Feb 22, 2022 at 7:46 AM David Raymond <David.Raymond at tomtom.com>
> wrote:
>
> > > Is there a simpler way?
> >
> > >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
> > >>> [a for b in d.values() for a in b]
> > ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> > >>>
> >
>
> I like that way best.
>
> But I'd still:
> 1) use a little more descriptive identifiers
> 2) put it in a function with a descriptive name
> 3) give the function a decent docstring

If you're going to do that, then you may as well use list(chain()) and
let the work be done in C.

ChrisA

From avigross at verizon.net  Wed Feb 23 01:02:00 2022
From: avigross at verizon.net (Avi Gross)
Date: Wed, 23 Feb 2022 06:02:00 +0000 (UTC)
Subject: One-liner to merge lists?
In-Reply-To: <CAGGBd_q-i9UdEpywpR3WxRJVKCMa2Dn+ULRPdzb5cT5F4S0C9g@mail.gmail.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <AM0PR07MB5700068EE7683E9AACAE78C8873B9@AM0PR07MB5700.eurprd07.prod.outlook.com>
 <CAGGBd_q-i9UdEpywpR3WxRJVKCMa2Dn+ULRPdzb5cT5F4S0C9g@mail.gmail.com>
Message-ID: <602763320.753970.1645596120909@mail.yahoo.com>

I had to think about it too, David.

Renaming it lets it make a bit more sense:

[item for sublist in d.values() for item in sublist]

This spells it out for me that you are taking one list within the main list at a time and then taking one item at a time from the list. Since the nesting is consistent in this case it makes sense.

Sadly, I could break it easily by just adding a non-list item like this:

d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg'], 3: 666}

And, of course, if you nest it any deeper, it is not completely flattened.

Chris makes the point that using the chain version may be better as it is compiled in C. Really? I would prefer a solution that even if slower, is more robust. Is there some function already written that will flatten what is given to it by checking the type of what is supplied, and based on the type, returning any simple objects like numbers or a character string, and converting many other composite things to something like a list and after popping off an entry, calling itself recursively on the rest until the list is empty? 

This would be a good start:

def flatten2list(object):
    gather = []
    for item in object:
        if isinstance(item, (list, tuple, set)):
            gather.extend(flatten2list(item))            
        else:
            gather.append(item)
    return gather

Of course I had to call the above as:

flatten2list(d.values())
['aaa', 'bbb', 'ccc', 'fff', 'ggg']

Here is a more complex version of the dictionary:

d = {1: ['aaa', 'bbb', 'ccc'], 2: [['fff', ['ggg']]], 3: 666, 4: set(["a", "e", "i", "o", "u"])}
d.values()
dict_values([['aaa', 'bbb', 'ccc'], [['fff', ['ggg']]], 666, {'e', 'u', 'a', 'o', 'i'}])

flatten2list(d.values())

['aaa', 'bbb', 'ccc', 'fff', 'ggg', 666, 'e', 'u', 'a', 'o', 'i']


The quest for a one-liner sometimes forgets that a typical function call is also a one-liner, with the work hidden elsewhere, often in a module written in python or mainly in C and so on. I suspect much more detailed functions like the above are available with a little search that handle more including just about any iterable that terminates.


-----Original Message-----
From: Dan Stromberg <drsalists at gmail.com>
To: David Raymond <David.Raymond at tomtom.com>
Cc: python-list at python.org <python-list at python.org>
Sent: Tue, Feb 22, 2022 11:02 pm
Subject: Re: One-liner to merge lists?


On Tue, Feb 22, 2022 at 7:46 AM David Raymond <David.Raymond at tomtom.com>
wrote:

> > Is there a simpler way?
>
> >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
> >>> [a for b in d.values() for a in b]
> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>>
>

I like that way best.

But I'd still:
1) use a little more descriptive identifiers
2) put it in a function with a descriptive name
3) give the function a decent docstring

-- 
https://mail.python.org/mailman/listinfo/python-list


From frank at chagford.com  Wed Feb 23 01:15:26 2022
From: frank at chagford.com (Frank Millman)
Date: Wed, 23 Feb 2022 08:15:26 +0200
Subject: One-liner to merge lists?
In-Reply-To: <AM0PR07MB5700068EE7683E9AACAE78C8873B9@AM0PR07MB5700.eurprd07.prod.outlook.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <AM0PR07MB5700068EE7683E9AACAE78C8873B9@AM0PR07MB5700.eurprd07.prod.outlook.com>
Message-ID: <e0ab7c83-faa2-4e3a-966d-967fdfbf552f@chagford.com>

On 2022-02-22 5:45 PM, David Raymond wrote:
>> Is there a simpler way?
> 
>>>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>>>> [a for b in d.values() for a in b]
> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>>
> 

Now that's what I was looking for.

I am not saying that I will use it, but as an academic exercise I felt 
sure that there had to be a one-liner in pure python.

I had forgotten about nested comprehensions. Thanks for the reminder.

Frank

From wlfraed at ix.netcom.com  Tue Feb 22 23:34:24 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Tue, 22 Feb 2022 23:34:24 -0500
Subject: Best way to check if there is internet?
References: <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
 <E68CCD84-AF7E-42DA-BC50-0CB43DB7CF98@barrys-emacs.org>
 <CADrxXX=xHQNBt=wGBJCxjy1sGpwC51=A6BLz1nniRbvFXg4WLg@mail.gmail.com>
 <mailman.5.1645589322.2329.python-list@python.org>
Message-ID: <lidb1hp5vuk8k0iemf09bbe146virag49c@4ax.com>

On Wed, 23 Feb 2022 07:08:27 +0300, Abdur-Rahmaan Janhangeer
<arj.python at gmail.com> declaimed the following:


>The upcoming operation is short enough and
>though we cannot assume that it will be the case
>when we are running the operation, we sensibly
>assume it will still be the case.
>
>This is more an extra check to not start an operation than
>either relying to check at the start or when query.
>It's both.

	So you have a success/fail tree of...

1	attempt a connection to some IP/port
1a		success -- attempt connection for desired operation
1a1			success -- operation completes
1a2			failure -- report that the operation can not be completed
1b		failure -- report that you will not attempt the operation
				because your test connection to some site failed
				EVEN IF THE OPERATION COULD SUCCEED
				(if the problem is with "some IP/port", not
				the rest of the network)

vs

1	attempt connection for desired operation
1a		success -- operation completes
1b		failure -- report that the operation cannot be completed

	Personally the second tree is simpler -- AND NEEDS TO BE PART OF THE
OTHER TREE ANYWAY! [short tree 1/1a/1b are long tree 1a/1a1/1a2] The only
way the longer tree makes sense is by making step 1 a connection to the
IP/port requested in the desired operation -- and if that succeeds you've
basically performed the shorter tree.

>
>>


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From __peter__ at web.de  Wed Feb 23 06:18:19 2022
From: __peter__ at web.de (Peter Otten)
Date: Wed, 23 Feb 2022 12:18:19 +0100
Subject: One-liner to merge lists?
In-Reply-To: <274c2144-a996-5b2d-173e-0c51bcfc71c5@chagford.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
 <274c2144-a996-5b2d-173e-0c51bcfc71c5@chagford.com>
Message-ID: <843155aa-8536-07ab-3050-ad79b0eb281a@web.de>

On 22/02/2022 10:44, Frank Millman wrote:
> On 2022-02-22 11:30 AM, Chris Angelico wrote:
>> On Tue, 22 Feb 2022 at 20:24, Frank Millman <frank at chagford.com> wrote:
>>>
>>> Hi all
>>>
>>> I think this should be a simple one-liner, but I cannot figure it out.
>>>
>>> I have a dictionary with a number of keys, where each value is a single
>>> list -
>>>
>>> ? >>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>>>
>>> I want to combine all values into a single list -
>>>
>>> ? >>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>
>>> I can do this -
>>>
>>> ? >>> a = []
>>> ? >>> for v in d.values():
>>> ...?? a.extend(v)
>>> ...
>>> ? >>> a
>>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>
>>> I can also do this -
>>>
>>> ? >>> from itertools import chain
>>> ? >>> a = list(chain(*d.values()))
>>> ? >>> a
>>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>> ? >>>
>>>
>>> Is there a simpler way?
>>>
>>
>> itertools.chain is a good option, as it scales well to arbitrary
>> numbers of lists (and you're guaranteed to iterate over them all just
>> once as you construct the list). But if you know that the lists aren't
>> too large or too numerous, here's another method that works:
>>
>>>>> sum(d.values(), [])
>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>
>> It's simply adding all the lists together, though you have to tell it
>> that you don't want a numeric summation.
>>
>
> Thanks, that is neat.
>
> However, I did see this -
>
>  >>> help(sum)
> Help on built-in function sum in module builtins:
>
> sum(iterable, /, start=0)
>  ??? Return the sum of a 'start' value (default: 0) plus an iterable of
> numbers
>
>  ??? When the iterable is empty, return the start value.
>  ??? This function is intended specifically for use with numeric values
> and may reject non-numeric types.
>  >>>
>
> So it seems that it is not recommended.

If you don't like the idea of 'adding' strings you can 'concat'enate:

 >>> items = [[1,2,3], [4,5], [6]]
 >>> functools.reduce(operator.concat, items)
[1, 2, 3, 4, 5, 6]
 >>> functools.reduce(operator.iconcat, items, [])
[1, 2, 3, 4, 5, 6]

The latter is the functional way to spell your for... extend() loop.
Don't forget to provide the initial value in that case lest you modify
the input:

 >> functools.reduce(operator.iconcat, items)  # wrong
[1, 2, 3, 4, 5, 6]
 >>> items
[[1, 2, 3, 4, 5, 6], [4, 5], [6]]  # oops



From akkana at shallowsky.com  Wed Feb 23 11:28:40 2022
From: akkana at shallowsky.com (Akkana Peck)
Date: Wed, 23 Feb 2022 09:28:40 -0700
Subject: Best way to check if there is internet?
In-Reply-To: <YhUHzqDyTcAijuWG@scrozzle>
Message-ID: <YhZguEKPO/QFbsS6@shallowsky.com>

2QdxY4RzWzUUiLuE at potatochowder.com writes:
> I think someone said it way upthread:  don't check, just do whatever you
> came to do, and it will work or it will fail (presumably, your program
> can tell the difference, regardless of a past snapshot of being able to
> retrieve data from an arbitrary URL).
> 
> EAFP, anyone?

Yes, but the code being discussed is still helpful, if only for
error handling: yes, the network isn't fully up, but *why" isn't it?

while True:
    try:
        do_whatever_I_came_to_do()
    except NetworkError:
        net_config_with_good_error_detection()

Aside from error handling, it's useful in a network-up script:
when you've just enabled a network, it's good to check right then if
there's a captive portal and deal with it. If you just assume that
the first thing the user wants is to go to an http: page in a
browser, and so don't bother to check for a captive portal,
that'll be annoying for people who want to fetch IMAP mail,
or run ssh, or load an https: page.

        ...Akkana

From rosuav at gmail.com  Wed Feb 23 12:31:11 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 24 Feb 2022 04:31:11 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <YhZguEKPO/QFbsS6@shallowsky.com>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
Message-ID: <CAPTjJmqiovxnxpTwqJO+hv5cqrMiprwb1bZcd6-ZNURAXMs1Dg@mail.gmail.com>

On Thu, 24 Feb 2022 at 03:39, Akkana Peck <akkana at shallowsky.com> wrote:
>
> 2QdxY4RzWzUUiLuE at potatochowder.com writes:
> > I think someone said it way upthread:  don't check, just do whatever you
> > came to do, and it will work or it will fail (presumably, your program
> > can tell the difference, regardless of a past snapshot of being able to
> > retrieve data from an arbitrary URL).
> >
> > EAFP, anyone?
>
> Yes, but the code being discussed is still helpful, if only for
> error handling: yes, the network isn't fully up, but *why" isn't it?
>
> while True:
>     try:
>         do_whatever_I_came_to_do()
>     except NetworkError:
>         net_config_with_good_error_detection()

That's fair, although the NetworkError should at least be able to
distinguish between "no DNS server", "DNS server returned error",
"timeout connecting", "host not found", "network not found",
"connection refused", etc.

> Aside from error handling, it's useful in a network-up script:
> when you've just enabled a network, it's good to check right then if
> there's a captive portal and deal with it.

That's a little harder to recognize, since there are so many ways that
it can be done. Do all HTTP requests return 302 redirects? Do all
requests return a static HTML page? What do non-HTTP requests return?
I don't know that there's a generic way to recognize those kinds of
login screens.

> If you just assume that
> the first thing the user wants is to go to an http: page in a
> browser, and so don't bother to check for a captive portal,
> that'll be annoying for people who want to fetch IMAP mail,
> or run ssh, or load an https: page.

Agreed, it is annoying, especially when those logins time out after
fifteen minutes and force you to watch another ad before they let you
use the connection. But it's still extremely hard to recognize
reliably, so you'd probably end up coding something for the specific
portal that you are experienced with.

ChrisA

From 2QdxY4RzWzUUiLuE at potatochowder.com  Wed Feb 23 15:04:25 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Wed, 23 Feb 2022 14:04:25 -0600
Subject: Best way to check if there is internet?
In-Reply-To: <YhZguEKPO/QFbsS6@shallowsky.com>
References: <YhUHzqDyTcAijuWG@scrozzle>
 <YhZguEKPO/QFbsS6@shallowsky.com>
Message-ID: <YhaTSeVHRDMAHLmr@scrozzle>

On 2022-02-23 at 09:28:40 -0700,
Akkana Peck <akkana at shallowsky.com> wrote:

> 2QdxY4RzWzUUiLuE at potatochowder.com writes:
> > I think someone said it way upthread:  don't check, just do whatever you
> > came to do, and it will work or it will fail (presumably, your program
> > can tell the difference, regardless of a past snapshot of being able to
> > retrieve data from an arbitrary URL).
> > 
> > EAFP, anyone?
>
> Yes, but the code being discussed is still helpful, if only for
> error handling: yes, the network isn't fully up, but *why" isn't it?
>
> while True:
>     try:
>         do_whatever_I_came_to_do()
>     except NetworkError:
>         net_config_with_good_error_detection()

Sure, after the fact, once things fail, diagnostics are useful.  The
questions that remain are when should you run them, how complex do they
have to be, and how reliable can they be?

> Aside from error handling, it's useful in a network-up script:
> when you've just enabled a network, it's good to check right then if
> there's a captive portal and deal with it. If you just assume that
> the first thing the user wants is to go to an http: page in a
> browser, and so don't bother to check for a captive portal,
> that'll be annoying for people who want to fetch IMAP mail,
> or run ssh, or load an https: page.

Honest questions:  If I'm captured by a portal, do I "have internet"?
If I don't "have internet," who and/or what should detect, report,
and/or fix it?  When?

(As a matter of fact, I'm one of those people who annoyingly has to deal
with captive portals when my only use for the internet is IMAP or SMTP.)

From boblatest at yahoo.com  Wed Feb 23 16:07:42 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 23 Feb 2022 21:07:42 GMT
Subject: Long running process - how to speed up?
References: <CA+i5Jwa3uqoeLcK-rGESYj1C4kuZV1cSr1aGnPu+qDvr91647w@mail.gmail.com>
 <CAPTjJmoD=pBi7Zo2LOmEFxzhesn-uoUehnwHzgyzmOarwroj_g@mail.gmail.com>
 <trinity-fd6f7ab2-c8ae-406f-ac70-9b9b8049030c-1645271957895@3c-app-gmx-bap34>
 <CAPTjJmo2PipiD9kv-2TnRnovi6HSzvoBxF488kOWOhtOKt8oZg@mail.gmail.com>
 <CA+i5JwbJVUg7gerHARxRCnkNpjP1gOdormZkAOT6TF6s4uN8uQ@mail.gmail.com>
 <mailman.142.1645272584.7010.python-list@python.org>
Message-ID: <j7nm0uFn39mU2@mid.individual.net>

Shaozhong SHI wrote:
> Can it be divided into several processes?

I'd do it like this:

from time import sleep
from threading import Thread

t = Thread(target=lambda: sleep(1))
t.run()

# do your work here

t.wait()


From grandsirerich at googlemail.com  Wed Feb 23 17:02:46 2022
From: grandsirerich at googlemail.com (Richard Pullin)
Date: Wed, 23 Feb 2022 22:02:46 +0000
Subject: Coding help
Message-ID: <CAEmEco_7HHUfv2D1+4ppmbLC3b=1+ehZ4dOXFMA_xygVPy_sEg@mail.gmail.com>

I know next to nothing about computer coding nor Python.

However, I am working on a mathematical challenge in which coding is
required to calculate and generate different potential solutions.

Can anyone help? If so, please private message me and we can discuss in
more detail.

Many thanks,
Richard

From boblatest at yahoo.com  Wed Feb 23 16:35:29 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 23 Feb 2022 21:35:29 GMT
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <YhUHzqDyTcAijuWG@scrozzle>
 <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
 <mailman.7.1645551728.12653.python-list@python.org>
Message-ID: <j7nnl1Fn39mU3@mid.individual.net>

Abdur-Rahmaan Janhangeer wrote:
> Well, nice perspective.
>
> It's a valid consideration, sound theory
> but poor practicality according to me.

On the contrary: It is absolutely the right and only way to do it.

> It you view it like this then between the moment
> we press run or enter key on the terminal,
> our Python interpreter might get deleted.

Yeah, but by your logic you'd have to check the existence of /usr/bin/python
each time before you run it.

> Though it's nice to go in and catch exceptions,
> if you have a long operation upcoming it might be nice
> to see if your key element is present.

That is true, but you still must make sure that your long operation fails in a
controlled manner if the necessary ressource becomes unavailable in the
meantime.

> Checking by exceptions is what the snippet
> i shared does. But we check for the ability to do it
> before we start.

Nothing wrong with checking before.

> Of course, it can change in the seconds that follow. But it's too much pure
> logic at work.

No. It's the correct way of doing it.

From boblatest at yahoo.com  Wed Feb 23 16:37:28 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 23 Feb 2022 21:37:28 GMT
Subject: Best way to check if there is internet?
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXnjKOhk5tzWMoSSFV49f+2dCVEGEJr=6LfJWOoOoCFE3w@mail.gmail.com>
 <YhUHzqDyTcAijuWG@scrozzle>
 <CADrxXXmQfoYF_ZQaViaUK=+RCjka3Wd2R9VSTbRoMezye12VFQ@mail.gmail.com>
 <679863926.632554.1645555035148@mail.yahoo.com>
 <CADrxXX=VFCOPZSw5ibcLrbCdiGT15SdOxDjXVLr2BANCd=0_SQ@mail.gmail.com>
 <mailman.2.1645556701.2329.python-list@python.org>
Message-ID: <j7nnooFn39mU4@mid.individual.net>

Abdur-Rahmaan Janhangeer wrote:
> I've got my answers but the 'wandering' a bit
> on this thread is at least connected to the topic ^^.
>
> Last question: If we check for wifi or ethernet cable connection?
>
> Wifi sometimes tell of connected but no internet connection.
> So it detected that there is or there is no internet ...

Your application obviously needs to make a certain network connection. Why
don't you just check if the connection can be made? Why would you care if it's
via cable or Wifi?

From tdldev at gmail.com  Thu Feb 24 13:00:54 2022
From: tdldev at gmail.com (Jack Dangler)
Date: Thu, 24 Feb 2022 13:00:54 -0500
Subject: Coding help
In-Reply-To: <CAEmEco_7HHUfv2D1+4ppmbLC3b=1+ehZ4dOXFMA_xygVPy_sEg@mail.gmail.com>
References: <CAEmEco_7HHUfv2D1+4ppmbLC3b=1+ehZ4dOXFMA_xygVPy_sEg@mail.gmail.com>
Message-ID: <2591af5b-bd5f-8f1d-b2f7-fc8c6d902bd2@gmail.com>


On 2/23/22 17:02, Richard Pullin via Python-list wrote:
> I know next to nothing about computer coding nor Python.
>
> However, I am working on a mathematical challenge in which coding is
> required to calculate and generate different potential solutions.
>
> Can anyone help? If so, please private message me and we can discuss in
> more detail.
>
> Many thanks,
> Richard

Sure

What do you have in mind?


From boblatest at yahoo.com  Thu Feb 24 07:08:50 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 24 Feb 2022 12:08:50 GMT
Subject: Threading question .. am I doing this right?
Message-ID: <j7paqiF2b3vU1@mid.individual.net>

I have a multi-threaded application (a web service) where several threads need
data from an external database. That data is quite a lot, but it is almost
always the same. Between incoming requests, timestamped records get added to
the DB.

So I decided to keep an in-memory cache of the DB records that gets only
"topped up" with the most recent records on each request:


    from threading import Lock, Thread


    class MyCache():
        def __init__(self):
            self.cache = None
            self.cache_lock = Lock()

        def _update(self):
            new_records = query_external_database()
            if self.cache is None:
                self.cache = new_records
            else:
                self.cache.extend(new_records)

        def get_data(self):
            with self.cache_lock:
                self._update()

            return self.cache

    my_cache = MyCache() # module level


This works, but even those "small" queries can sometimes hang for a long time,
causing incoming requests to pile up at the "with self.cache_lock" block.

Since it is better to quickly serve the client with slightly outdated data than
not at all, I came up with the "impatient" solution below. The idea is that an
incoming request triggers an update query in another thread, waits for a short
timeout for that thread to finish and then returns either updated or old data.

class MyCache():
    def __init__(self):
        self.cache = None
        self.thread_lock = Lock()
        self.update_thread = None

    def _update(self):
        new_records = query_external_database()
        if self.cache is None:
            self.cache = new_records
        else:
            self.cache.extend(new_records)

    def get_data(self):
        if self.cache is None:
            timeout = 10 # allow more time to get initial batch of data
        else:
            timeout = 0.5
        with self.thread_lock:
            if self.update_thread is None or not self.update_thread.is_alive():
                self.update_thread = Thread(target=self._update)
                self.update_thread.start()
                self.update_thread.join(timeout)

        return self.cache

    my_cache = MyCache()

My question is: Is this a solid approach? Am I forgetting something? For
instance, I believe that I don't need another lock to guard self.cache.append()
because _update() can ever only run in one thread at a time. But maybe I'm
overlooking something.


From smitalfulzele38 at gmail.com  Thu Feb 24 09:26:44 2022
From: smitalfulzele38 at gmail.com (Smital Fulzele)
Date: Thu, 24 Feb 2022 06:26:44 -0800 (PST)
Subject: Extract specific no of data from nedCDF files using python
Message-ID: <3cccad26-c237-40a2-a5c8-554652ad0f9dn@googlegroups.com>

Hi.I am learning python and I am working with some netCDF files. Suppose I have temperature data from 1950-2020 and I want data for only 1960-2015. How should I extract it. 

From rosuav at gmail.com  Thu Feb 24 15:03:54 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 25 Feb 2022 07:03:54 +1100
Subject: Threading question .. am I doing this right?
In-Reply-To: <j7paqiF2b3vU1@mid.individual.net>
References: <j7paqiF2b3vU1@mid.individual.net>
Message-ID: <CAPTjJmo3sn7_hoWMb+UghrNd0dJ-AimKb315hOtFqAhqWOLk3w@mail.gmail.com>

On Fri, 25 Feb 2022 at 06:54, Robert Latest via Python-list
<python-list at python.org> wrote:
>
> I have a multi-threaded application (a web service) where several threads need
> data from an external database. That data is quite a lot, but it is almost
> always the same. Between incoming requests, timestamped records get added to
> the DB.
>
> So I decided to keep an in-memory cache of the DB records that gets only
> "topped up" with the most recent records on each request:

Depending on your database, this might be counter-productive. A
PostgreSQL database running on localhost, for instance, has its own
caching, and data transfers between two apps running on the same
computer can be pretty fast. The complexity you add in order to do
your own caching might be giving you negligible benefit, or even a
penalty. I would strongly recommend benchmarking the naive "keep going
back to the database" approach first, as a baseline, and only testing
these alternatives when you've confirmed that the database really is a
bottleneck.

> Since it is better to quickly serve the client with slightly outdated data than
> not at all, I came up with the "impatient" solution below. The idea is that an
> incoming request triggers an update query in another thread, waits for a short
> timeout for that thread to finish and then returns either updated or old data.
>
> class MyCache():
>     def __init__(self):
>         self.cache = None
>         self.thread_lock = Lock()
>         self.update_thread = None
>
>     def _update(self):
>         new_records = query_external_database()
>         if self.cache is None:
>             self.cache = new_records
>         else:
>             self.cache.extend(new_records)
>
>     def get_data(self):
>         if self.cache is None:
>             timeout = 10 # allow more time to get initial batch of data
>         else:
>             timeout = 0.5
>         with self.thread_lock:
>             if self.update_thread is None or not self.update_thread.is_alive():
>                 self.update_thread = Thread(target=self._update)
>                 self.update_thread.start()
>                 self.update_thread.join(timeout)
>
>         return self.cache
>
>     my_cache = MyCache()
>
> My question is: Is this a solid approach? Am I forgetting something? For
> instance, I believe that I don't need another lock to guard self.cache.append()
> because _update() can ever only run in one thread at a time. But maybe I'm
> overlooking something.

Hmm, it's complicated. There is another approach, and that's to
completely invert your thinking: instead of "request wants data, so
let's get data", have a thread that periodically updates your cache
from the database, and then all requests return from the cache,
without pinging the requester. Downside: It'll be requesting fairly
frequently. Upside: Very simple, very easy, no difficulties debugging.

How many requests per second does your service process? (By
"requests", I mean things that require this particular database
lookup.) What's average throughput, what's peak throughput? And
importantly, what sorts of idle times do you have? For instance, if
you might have to handle 100 requests/second, but there could be
hours-long periods with no requests at all (eg if your clients are all
in the same timezone and don't operate at night), that's a very
different workload from 10 r/s constantly throughout the day.

ChrisA

From david at lowryduda.com  Thu Feb 24 15:03:17 2022
From: david at lowryduda.com (David Lowry-Duda)
Date: Thu, 24 Feb 2022 15:03:17 -0500
Subject: Extract specific no of data from nedCDF files using python
In-Reply-To: <3cccad26-c237-40a2-a5c8-554652ad0f9dn@googlegroups.com>
References: <3cccad26-c237-40a2-a5c8-554652ad0f9dn@googlegroups.com>
Message-ID: <YhfkhVdGj5xwV806@icerm-dld>

> Hi.I am learning python and I am working with some netCDF files. 
> Suppose I have temperature data from 1950-2020 and I want data for 
> only 1960-2015. How should I extract it. -- 

Alternately, use https://unidata.github.io/netcdf4-python/ or gdal.

It might also be possible to read
https://stackoverflow.com/questions/14035148/import-netcdf-file-to-pandas-dataframe
and using pandas dataframes to organize your data, if desired.

Good luck!

- DLD

From arj.python at gmail.com  Fri Feb 25 07:49:33 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 25 Feb 2022 15:49:33 +0300
Subject: Best way to check if there is internet?
In-Reply-To: <YhaTSeVHRDMAHLmr@scrozzle>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
Message-ID: <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>

Avi solved the captive portal problem above.
It's by comparing the page content to a dynamic
output.

I was asking for wifi just in case i missed
anything we did not yet discuss.

I consider the thread closed, thanks everybody,
it was very fruitful for me.

Thanks for the in-between. I really like the Python comunity as,
even though it's a 'scripting' language, the scrutiny
of issues and general discussions are currenly up to
a very nice level.

See you on another thread soon!

From abdelkader.belahcene at enst.dz  Fri Feb 25 04:12:10 2022
From: abdelkader.belahcene at enst.dz (BELAHCENE Abdelkader)
Date: Fri, 25 Feb 2022 10:12:10 +0100
Subject: C is it always faster than nump?
Message-ID: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>

Hi,
a lot of people think that C (or C++) is faster than python, yes I agree,
but I think that's not the case with numpy, I believe numpy is faster than
C, at least in some cases.


*Is there another explanation ?Or where can find  a doc speaking  about the
subject?*Thanks a lot
Regards
Numpy implements vectorization for arrays, or I'm wrong. Anyway here is an
example Let's look at the following case:
Here is the result on my laptop i3:

Labs$ *python3 tempsExe.py  50000*
  sum with Python: 1250025000 and NumPy 1250025000
      time used Python Sum: * 37.28 sec *
      time used  Numpy Sum:  *1.85 sec*

Labs$ *./tt    50000 *

*   CPU  time :7.521730*

*    The value : 1250025000 *





















*This is the Python3 program :import timeit as itimport numpy as npimport
systry : n=eval(sys.argv[1])except: print ("needs integer as argument") ;
exit() a=range(1,n+1)b=np.array(a)def func1():     return sum(a)def
func2(): return np.sum(b)print(f"sum with Python: {func1()} and NumPy
{func2()} ")tm1=it.timeit(stmt=func1, number=n)print(f"time used Python
Sum: {round(tm1,2)} sec")tm2=it.timeit(stmt=func2, number=n)print(f"time
used  Numpy Sum: {round(tm2,2)} sec")*


















*and Here the C program:#include <time.h>#include <stdio.h>#include
<stdlib.h>long func1(int n){         long  r=0;        for (int  i=1; i<=
n;i++) r+= i;         return r;}int main(int argc, char* argv[]){
clock_t c0, c1;         long v,count; int n;       if ( argc < 2)
{              printf("Please give an argument");             return
-1;      }    n=atoi(argv[1]);     c0 = clock();*




*   for (int j=0;j < n;j++) v=func1(n);     c1 = clock();     printf
("\tCPU  time :%.2f sec", (float)(c1 - c0)/CLOCKS_PER_SEC);
printf("\n\tThe value : %ld\n",  v);}*

From greg.ewing at canterbury.ac.nz  Thu Feb 24 23:59:23 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 25 Feb 2022 17:59:23 +1300
Subject: Threading question .. am I doing this right?
In-Reply-To: <j7paqiF2b3vU1@mid.individual.net>
References: <j7paqiF2b3vU1@mid.individual.net>
Message-ID: <j7r61eFdf8oU1@mid.individual.net>

On 25/02/22 1:08 am, Robert Latest wrote:
> My question is: Is this a solid approach? Am I forgetting something?

I can see a few of problems:

* If more than one thread calls get_data() during the initial
cache filling, it looks like only one of them will wait for
the thread -- the others will skip waiting altogether and
immediately return None.

* Also if the first call to get_data() times out it will
return None (although maybe that's acceptable if the caller
is expecting it).

* The caller of get_data() is getting an object that could
be changed under it by a future update.

-- 
Greg

From loris.bennett at fu-berlin.de  Fri Feb 25 05:32:02 2022
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Fri, 25 Feb 2022 11:32:02 +0100
Subject: When to use SQLAlchemy listen events
Message-ID: <87lexz6xul.fsf@hornfels.zedat.fu-berlin.de>

Hi,

I am wondering whether SQLAlchemy listen events are appropriate for the
following situation:

I have a table containing users and a table for events related to users

  class User(Base):
      __tablename__ = "users"

      uid = Column('uid', String(64), primary_key=True)
      gid = Column('gid', String(64), ForeignKey('groups.gid'), nullable=False)
      lang = Column('lang', String(2))


  class UserEvent(Base):
      __tablename__ = "user_events"

      id = Column('id', Integer, primary_key=True)
      date = Column('date', Date, nullable=False)
      uid = Column('gid', String(64), ForeignKey('users.uid'), nullable=False)
      comment = Column('comment', String(256))

(There are also analogous tables for groups and group events).

The functions provided by the interface are things like the following

  add_user(user, group, lang)
  move_user(user, group)
  delete_user(user)
  warn_user(user, reason)

Whereas 'add/move/delete_user' result in changes to the table 'users',
'warn_user' does not.  All should produce entries in the table
'user_events'. 

There could be more functions similar to 'warn_user' that only create an
entry in 'user_events'.  Potentially there could be a lot more of
these than the 'user'-table-changing type.

It seems like for the first three functions, capturing the resulting
database changes in the table 'user_events' would be a standard use-case
for listen event.  However, the 'warn_user' function is different.

So can/should I shoehorn the 'warn_user' function to being like the
others three and use listen events, or should I just come up with my own
mechanism which will allow any function just to add an entry to the
events table?

Cheers,

Loris

-- 
This signature is currently under construction.

From michael.stemper at gmail.com  Fri Feb 25 09:50:49 2022
From: michael.stemper at gmail.com (Michael F. Stemper)
Date: Fri, 25 Feb 2022 08:50:49 -0600
Subject: Best way to check if there is internet?
In-Reply-To: <mailman.17.1645793390.2329.python-list@python.org>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
Message-ID: <svaqc9$vr$2@dont-email.me>

On 25/02/2022 06.49, Abdur-Rahmaan Janhangeer wrote:

> Thanks for the in-between. I really like the Python comunity as,
> even though it's a 'scripting' language,

To me, it's a programming language. In my usage, a "script" is
a bunch of OS commands.

-- 
Michael F. Stemper
Life's too important to take seriously.

From grant.b.edwards at gmail.com  Fri Feb 25 11:52:51 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 25 Feb 2022 08:52:51 -0800 (PST)
Subject: Best way to check if there is internet?
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me>
Message-ID: <62190963.1c69fb81.18d22.89f4@mx.google.com>

On 2022-02-25, Michael F. Stemper <michael.stemper at gmail.com> wrote:
> On 25/02/2022 06.49, Abdur-Rahmaan Janhangeer wrote:
>
>> Thanks for the in-between. I really like the Python comunity as,
>> even though it's a 'scripting' language,

And we like you, even though you're only a ...

In English, a statement like that is considered rather insulting.

> To me, it's a programming language. In my usage, a "script" is a
> bunch of OS commands.

Exactly. A script is a set of commands that you would have normally
entered by hand at a command prompt. But, after having gone through
the process a few times, you decided to shove them into a file to save
on typing.

Python is a programming language. Period.

[Yes, I know there's some sort of Python command interpreter intended
to be used for normal day-to-day use, but that's not what we're
talking about here.]

I classify some files that start with '#!/usr/bin/bash' as scripts,
but some are programs that just happen to be written in bash because
bash provides good facilities for doing filesystem manipulation and
running other executables.

I've never seen a Python program that I would consider a script.

But maybe that's just me...

--
Grant



From bschollnick at schollnick.net  Fri Feb 25 12:10:07 2022
From: bschollnick at schollnick.net (Benjamin Schollnick)
Date: Fri, 25 Feb 2022 12:10:07 -0500
Subject: Best way to check if there is internet?
In-Reply-To: <62190963.1c69fb81.18d22.89f4@mx.google.com>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me> <62190963.1c69fb81.18d22.89f4@mx.google.com>
Message-ID: <4F2A3FE7-8919-4553-A1CC-68807640263B@schollnick.net>

>>> Thanks for the in-between. I really like the Python comunity as,
>>> even though it's a 'scripting' language,
> 
> And we like you, even though you're only a ...
> 
> In English, a statement like that is considered rather insulting.
> 
>> To me, it's a programming language. In my usage, a "script" is a
>> bunch of OS commands.
> 
> Exactly. A script is a set of commands that you would have normally
> entered by hand at a command prompt. But, after having gone through
> the process a few times, you decided to shove them into a file to save
> on typing.
> 
> Python is a programming language. Period.

Exactly.  In my opinion ?It?s a scripting language?, is a devaluation of the language.
It?s an attempt to throw python in to a ?trivial? classification.  

Heck, the same argument can be made of Bash, PHP, Perl, and a few other languages as well.

How many ?scripts? have been throw quickly together in Perl, or PHP?  

Quite a damn few, yet, would anyone call Wordpress a ?script??

It?s effectively damning with faint praise.

		- Benjamin



From arj.python at gmail.com  Fri Feb 25 13:07:50 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 25 Feb 2022 21:07:50 +0300
Subject: Best way to check if there is internet?
In-Reply-To: <62190963.1c69fb81.18d22.89f4@mx.google.com>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me>
 <62190963.1c69fb81.18d22.89f4@mx.google.com>
Message-ID: <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>

Normally people put Python in the scripting category.
I learnt typed languages like C++ and Java at first.
People who learn languages like these tend to put
Python in a non-serious category. The post was
more ironic than litteral.

After 5 years of following the mailing lists i realised
that there is more than the eyes meet. My hobby has
always been language engineering / compiler theory
and just recently i realised that Python is pretty serious about
language features.

I have been following language feature proposals from various
languages. Some decide to avoid Python's route,
but others have been trying hard to catch up with Python.
One gleaming example is the switch case. JS recently proposed pattern
matching, referencing Python and explaining why the proposal
is a cool treatment of the usecase.

As a side note, if by scripting we mean OS commands,
then Python started as a sysadmin language.

From boblatest at yahoo.com  Fri Feb 25 12:17:25 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 25 Feb 2022 17:17:25 GMT
Subject: Threading question .. am I doing this right?
References: <j7paqiF2b3vU1@mid.individual.net>
 <CAPTjJmo3sn7_hoWMb+UghrNd0dJ-AimKb315hOtFqAhqWOLk3w@mail.gmail.com>
 <mailman.15.1645733047.2329.python-list@python.org>
Message-ID: <j7sh95Flcr9U1@mid.individual.net>

Chris Angelico wrote:
> Depending on your database, this might be counter-productive. A
> PostgreSQL database running on localhost, for instance, has its own
> caching, and data transfers between two apps running on the same
> computer can be pretty fast. The complexity you add in order to do
> your own caching might be giving you negligible benefit, or even a
> penalty. I would strongly recommend benchmarking the naive "keep going
> back to the database" approach first, as a baseline, and only testing
> these alternatives when you've confirmed that the database really is a
> bottleneck.

"Depending on your database" is the key phrase. This is not "my" database that
is running on localhost. It is an external MSSQL server that I have no control
over and whose requests frequently time out.

> Hmm, it's complicated. There is another approach, and that's to
> completely invert your thinking: instead of "request wants data, so
> let's get data", have a thread that periodically updates your cache
> from the database, and then all requests return from the cache,
> without pinging the requester. Downside: It'll be requesting fairly
> frequently. Upside: Very simple, very easy, no difficulties debugging.

I'm using a similar approach in other places, but there I actually have a
separate process that feeds my local, fast DB with unwieldy data. But that is
not merely replicating, it actually preprocesses and "adds value" to the data,
and the data is worth retaining on my server. I didn't want to take that
approach in this instance because it is a bit too much overhead for essentially
"throwaway" stuff. I like the idea of starting a separated "timed" thread in
the same application. Need to think about that.

Background: The clients are SBCs that display data on screens distributed
throughout a manufacturing facility. They periodically refresh every few
minutes. Occasionally the requests would pile up waiting for the databsase, so
that some screens displayed error messages for a minute or two. Nobody cares
but my pride was piqued and the error logs filled up.

I've had my proposed solution running for a few days now without errors. For me
that's enough but I wanted to ask you guys if I made some logical mistakes.

From boblatest at yahoo.com  Fri Feb 25 12:19:32 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 25 Feb 2022 17:19:32 GMT
Subject: Threading question .. am I doing this right?
References: <j7paqiF2b3vU1@mid.individual.net>
 <j7r61eFdf8oU1@mid.individual.net>
Message-ID: <j7shd4Flcr9U2@mid.individual.net>

Greg Ewing wrote:
> * If more than one thread calls get_data() during the initial
> cache filling, it looks like only one of them will wait for
> the thread -- the others will skip waiting altogether and
> immediately return None.

Right. But that needs to be dealt with somehow. No data is no data.

> * Also if the first call to get_data() times out it will
> return None (although maybe that's acceptable if the caller
> is expecting it).

Right. Needs to be dealt with.

> * The caller of get_data() is getting an object that could
> be changed under it by a future update.

I don't think that's a problem. If it turns out to be one I'll create a copy of
the data while I hold the lock and pass that back.
>

From Richard at Damon-Family.org  Fri Feb 25 13:48:13 2022
From: Richard at Damon-Family.org (Richard Damon)
Date: Fri, 25 Feb 2022 13:48:13 -0500
Subject: C is it always faster than nump?
In-Reply-To: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
Message-ID: <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>

On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> Hi,
> a lot of people think that C (or C++) is faster than python, yes I agree,
> but I think that's not the case with numpy, I believe numpy is faster than
> C, at least in some cases.
>
My understanding is that numpy is written in C, so for it to be faster 
than C, you are saying that C is faster that C.

The key point is that numpy was written by skilled programmers who 
carefully optimized their code to be as fast as possible for the major 
cases. Thus it is quite possible for the numpy code to be faster in C 
than code written by a person without that level of care and effort.

There are similar package available for many languages, including C/C++ 
to let mere mortals get efficient numerical processing.

-- 
Richard Damon


From rosuav at gmail.com  Fri Feb 25 14:28:42 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 06:28:42 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me>
 <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
Message-ID: <CAPTjJmpo386pX03DUyphA6mjPm2shE3ZqaN7_vOovBj3x2LhVw@mail.gmail.com>

On Sat, 26 Feb 2022 at 05:09, Abdur-Rahmaan Janhangeer
<arj.python at gmail.com> wrote:
>
> Normally people put Python in the scripting category.

You have a very interesting definition of "normal".

ChrisA

From rosuav at gmail.com  Fri Feb 25 14:31:08 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 06:31:08 +1100
Subject: Threading question .. am I doing this right?
In-Reply-To: <j7sh95Flcr9U1@mid.individual.net>
References: <j7paqiF2b3vU1@mid.individual.net>
 <CAPTjJmo3sn7_hoWMb+UghrNd0dJ-AimKb315hOtFqAhqWOLk3w@mail.gmail.com>
 <mailman.15.1645733047.2329.python-list@python.org>
 <j7sh95Flcr9U1@mid.individual.net>
Message-ID: <CAPTjJmra6m=JjACbXoJjSJU+OCVFXsS9Pby=uY4eY5L9q8a+vQ@mail.gmail.com>

On Sat, 26 Feb 2022 at 05:16, Robert Latest via Python-list
<python-list at python.org> wrote:
>
> Chris Angelico wrote:
> > Depending on your database, this might be counter-productive. A
> > PostgreSQL database running on localhost, for instance, has its own
> > caching, and data transfers between two apps running on the same
> > computer can be pretty fast. The complexity you add in order to do
> > your own caching might be giving you negligible benefit, or even a
> > penalty. I would strongly recommend benchmarking the naive "keep going
> > back to the database" approach first, as a baseline, and only testing
> > these alternatives when you've confirmed that the database really is a
> > bottleneck.
>
> "Depending on your database" is the key phrase. This is not "my" database that
> is running on localhost. It is an external MSSQL server that I have no control
> over and whose requests frequently time out.
>

Okay, cool. That's crucial to know.

I'm still curious as to the workload (requests per second), as it
might still be worth going for the feeder model. But if your current
system works, then it may be simplest to debug that rather than
change.

ChrisA

From rosuav at gmail.com  Fri Feb 25 14:41:36 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 06:41:36 +1100
Subject: C is it always faster than nump?
In-Reply-To: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
Message-ID: <CAPTjJmokOs3q_JLhsEy2caa8Q5xGu5__-xh+V+nM+xxJs6T1Bw@mail.gmail.com>

On Sat, 26 Feb 2022 at 03:13, BELAHCENE Abdelkader
<abdelkader.belahcene at enst.dz> wrote:
> *This is the Python3 program :import timeit as itimport numpy as npimport
> systry : n=eval(sys.argv[1])except: print ("needs integer as argument") ;
> exit() a=range(1,n+1)b=np.array(a)def func1():     return sum(a)def
> func2(): return np.sum(b)print(f"sum with Python: {func1()} and NumPy
> {func2()} ")tm1=it.timeit(stmt=func1, number=n)print(f"time used Python
> Sum: {round(tm1,2)} sec")tm2=it.timeit(stmt=func2, number=n)print(f"time
> used  Numpy Sum: {round(tm2,2)} sec")*

This is terrible code. Even aside from the messed-up formatting (for
which your mail client is probably to blame), using eval() and a bare
"except:" clause is not a good way to use Python. And then you get
timeit to do as many iterations as the length of the array, which is
hardly indicative or meaningful for small values.

> *and Here the C program:#include <time.h>#include <stdio.h>#include
> <stdlib.h>long func1(int n){         long  r=0;        for (int  i=1; i<=
> n;i++) r+= i;         return r;}int main(int argc, char* argv[]){
> clock_t c0, c1;         long v,count; int n;       if ( argc < 2)
> {              printf("Please give an argument");             return
> -1;      }    n=atoi(argv[1]);     c0 = clock();*
>
>
>
>
> *   for (int j=0;j < n;j++) v=func1(n);     c1 = clock();     printf
> ("\tCPU  time :%.2f sec", (float)(c1 - c0)/CLOCKS_PER_SEC);
> printf("\n\tThe value : %ld\n",  v);}*

At least you're consistent, using an iteration count equal to the
length of the array again. But that just means that it's equally
meaningless.

Did you know that Python's timeit and C's clock don't even measure the
same thing?

ChrisA

From avigross at verizon.net  Fri Feb 25 14:43:04 2022
From: avigross at verizon.net (Avi Gross)
Date: Fri, 25 Feb 2022 19:43:04 +0000 (UTC)
Subject: C is it always faster than nump?
In-Reply-To: <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
Message-ID: <375047121.1338435.1645818184031@mail.yahoo.com>

I agree with Richard.

Some people may be confused and think c is the speed of light and relativistically speaking, nothing can be faster. (OK, just joking. The uses of the same letter of the alphabet are not at all related. One is named for the language that came after the one named B, while the other may be short for celeritas meaning speed.)

There is no such thing as C. C does nothing. It is a combination of a language specification and some pieces of software called compilers that implement it well or less well.

There is such a thing as a PROGRAM. A program completely written in C is a thing. It can run fast or slow based on a combination of how it was written and on what data it operates on, which hardware and OS and so on. AND some of it may likely be running code from libraries written in other languages like FORTRAN that get linked into it in some way at compile time or runtime, and hooks into the local OS and so on.

So your program written supposedly in pure C, may run faster or slower. If you program a "sort" algorithm in C, it may matter if it is an implementation of a merge sort or at bubble sort or ...

As noted, numpy is largely written in C. It may well be optimized in some places but there are constraints that may well make it hard to optimize compared to some other implementation without those constraints. In particular, it interfaces with standard Python data structures at times such as when initializing from a Python List, or List of Lists, or needing to hold on to various attributes so it can be converted back, or things I am not even aware of.

So, I suspect it may well be possible to make a pure C library similar to numpy in many ways but that can only be used within a C program that only uses native C data structures. It also is possible to write such a program that is horribly slow. And it is possible to write a less complex version of numpy that does not support some current numpy functionality and overall runs much faster on what it does support.

I do wonder at the reason numpy and pandas and lots of other modules have to exist. Other languages like R made design choices that built in ideas of vectorization from the start. Python has lots of object-oriented extensibility that can allow you to create interpreted code that may easily extend it in areas to have some similar features. You can create an array-like data structure that holds only one object type and is extended so adding two together (or multiplying) ends up doing it componentwise. But attempts to do some such things often run into problems as they tend to be slow. So numpy was not written in python, mostly, albeit it could have been even more impressive if it took advantage of more pythonic abilities, at a cost.

But now that numpy is in C, pretty much, it is somewhat locked in when and if other things in Python change. 

The reality is that many paradigms carried too far end up falling short.


-----Original Message-----
From: Richard Damon <Richard at Damon-Family.org>
To: python-list at python.org
Sent: Fri, Feb 25, 2022 1:48 pm
Subject: Re: C is it always faster than nump?


On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> Hi,
> a lot of people think that C (or C++) is faster than python, yes I agree,
> but I think that's not the case with numpy, I believe numpy is faster than
> C, at least in some cases.
>
My understanding is that numpy is written in C, so for it to be faster 
than C, you are saying that C is faster that C.

The key point is that numpy was written by skilled programmers who 
carefully optimized their code to be as fast as possible for the major 
cases. Thus it is quite possible for the numpy code to be faster in C 
than code written by a person without that level of care and effort.

There are similar package available for many languages, including C/C++ 
to let mere mortals get efficient numerical processing.

-- 
Richard Damon


-- 
https://mail.python.org/mailman/listinfo/python-list


From rosuav at gmail.com  Fri Feb 25 14:47:00 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 06:47:00 +1100
Subject: C is it always faster than nump?
In-Reply-To: <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
Message-ID: <CAPTjJmrVS1Ee5+hZLawz8r4ojN2N_a41-ORKJ7Lza879a-9Vkg@mail.gmail.com>

On Sat, 26 Feb 2022 at 05:49, Richard Damon <Richard at damon-family.org> wrote:
>
> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> > Hi,
> > a lot of people think that C (or C++) is faster than python, yes I agree,
> > but I think that's not the case with numpy, I believe numpy is faster than
> > C, at least in some cases.
> >
> My understanding is that numpy is written in C, so for it to be faster
> than C, you are saying that C is faster that C.

Fortran actually, but ultimately, they both get compiled to machine code.

Really, what the OP has demonstrated is that good, well-written code
called from bad code produces meaningless numbers that are not the
same as bad code written in C. I can't even use that to prove that
good code is faster than bad code, since the measurements aren't
comparable; but if that were what was being measured, it wouldn't be
very surprising.

To do a fair comparison of C and Numpy, you'd have to:

1) Use the same type of timer
2) Use the same algorithm (unless you're benchmarking "naive C code"
against "well-written Fortran code")
3) Have comparable levels of compile-time optimization
4) Cope with the vagaries of CPU caching
5) Ensure that the same data types are being used everywhere
6) Probably several other things that I didn't think of.

The precise way you run the test could easily skew it by orders of
magnitude in either direction.

> The key point is that numpy was written by skilled programmers who
> carefully optimized their code to be as fast as possible for the major
> cases. Thus it is quite possible for the numpy code to be faster in C
> than code written by a person without that level of care and effort.

This is clearly true.

ChrisA

From rosuav at gmail.com  Fri Feb 25 14:58:01 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 06:58:01 +1100
Subject: C is it always faster than nump?
In-Reply-To: <375047121.1338435.1645818184031@mail.yahoo.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <375047121.1338435.1645818184031@mail.yahoo.com>
Message-ID: <CAPTjJmozkL3U2CvimvQFhkU8QwpJyRL2ORiS3AseHr1e=8vLKw@mail.gmail.com>

On Sat, 26 Feb 2022 at 06:44, Avi Gross via Python-list
<python-list at python.org> wrote:
>
> I agree with Richard.
>
> Some people may be confused and think c is the speed of light and relativistically speaking, nothing can be faster. (OK, just joking. The uses of the same letter of the alphabet are not at all related. One is named for the language that came after the one named B, while the other may be short for celeritas meaning speed.)
>
> There is no such thing as C. C does nothing. It is a combination of a language specification and some pieces of software called compilers that implement it well or less well.
>

Uhh, that's taking it a little bit TOO far.... I agree with your
point, but saying that there's no such thing as C is slightly unfair
:)

> There is such a thing as a PROGRAM. A program completely written in C is a thing. It can run fast or slow based on a combination of how it was written and on what data it operates on, which hardware and OS and so on. AND some of it may likely be running code from libraries written in other languages like FORTRAN that get linked into it in some way at compile time or runtime, and hooks into the local OS and so on.
>
> So your program written supposedly in pure C, may run faster or slower. If you program a "sort" algorithm in C, it may matter if it is an implementation of a merge sort or at bubble sort or ...
>

More specifically: You're benchmarking a particular *implementation*
of a particular *algorithm*. Depending on what you're trying to
demonstrate, either could be significant.

Performance testing between two things written in C is a huge job.
Performance testing across languages has a strong tendency to be
meaningless (like benchmarking Python's integers against JavaScript's
numbers).

> As noted, numpy is largely written in C. It may well be optimized in some places but there are constraints that may well make it hard to optimize compared to some other implementation without those constraints. In particular, it interfaces with standard Python data structures at times such as when initializing from a Python List, or List of Lists, or needing to hold on to various attributes so it can be converted back, or things I am not even aware of.
>

(Fortran)

In theory, summing a Numpy array should be incredibly fast, but in
practice, there's a lot of variation, and it can be quite surprising.
For instance, integers are faster than floats, everyone knows that.
And it's definitely faster to sum smaller integers than larger ones.

rosuav at sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(1000000), dtype=numpy.float64)' 'numpy.sum(x)'
1000 loops, best of 5: 325 usec per loop
rosuav at sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(1000000), dtype=numpy.int64)' 'numpy.sum(x)'
500 loops, best of 5: 551 usec per loop
rosuav at sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(1000000), dtype=numpy.int32)' 'numpy.sum(x)'
500 loops, best of 5: 680 usec per loop

... Or not.

Summing arrays isn't necessarily the best test of numpy anyway, but as
you can see, testing is an incredibly difficult thing to get right.
The easiest thing to prove is that you have no idea how to prove
anything usefully, and most of us achieve that every time :)

ChrisA


> So, I suspect it may well be possible to make a pure C library similar to numpy in many ways but that can only be used within a C program that only uses native C data structures. It also is possible to write such a program that is horribly slow. And it is possible to write a less complex version of numpy that does not support some current numpy functionality and overall runs much faster on what it does support.
>
> I do wonder at the reason numpy and pandas and lots of other modules have to exist. Other languages like R made design choices that built in ideas of vectorization from the start. Python has lots of object-oriented extensibility that can allow you to create interpreted code that may easily extend it in areas to have some similar features. You can create an array-like data structure that holds only one object type and is extended so adding two together (or multiplying) ends up doing it componentwise. But attempts to do some such things often run into problems as they tend to be slow. So numpy was not written in python, mostly, albeit it could have been even more impressive if it took advantage of more pythonic abilities, at a cost.
>
> But now that numpy is in C, pretty much, it is somewhat locked in when and if other things in Python change.
>
> The reality is that many paradigms carried too far end up falling short.
>
>
> -----Original Message-----
> From: Richard Damon <Richard at Damon-Family.org>
> To: python-list at python.org
> Sent: Fri, Feb 25, 2022 1:48 pm
> Subject: Re: C is it always faster than nump?
>
>
> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> > Hi,
> > a lot of people think that C (or C++) is faster than python, yes I agree,
> > but I think that's not the case with numpy, I believe numpy is faster than
> > C, at least in some cases.
> >
> My understanding is that numpy is written in C, so for it to be faster
> than C, you are saying that C is faster that C.
>
> The key point is that numpy was written by skilled programmers who
> carefully optimized their code to be as fast as possible for the major
> cases. Thus it is quite possible for the numpy code to be faster in C
> than code written by a person without that level of care and effort.
>
> There are similar package available for many languages, including C/C++
> to let mere mortals get efficient numerical processing.
>
> --
> Richard Damon
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
> --
> https://mail.python.org/mailman/listinfo/python-list

From michael.stemper at gmail.com  Fri Feb 25 14:48:32 2022
From: michael.stemper at gmail.com (Michael F. Stemper)
Date: Fri, 25 Feb 2022 13:48:32 -0600
Subject: Best way to check if there is internet?
In-Reply-To: <mailman.21.1645812487.2329.python-list@python.org>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me> <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <mailman.21.1645812487.2329.python-list@python.org>
Message-ID: <svbbqo$f58$1@dont-email.me>

On 25/02/2022 12.07, Abdur-Rahmaan Janhangeer wrote:

> I have been following language feature proposals from various
> languages. Some decide to avoid Python's route,
> but others have been trying hard to catch up with Python.
> One gleaming example is the switch case. JS recently proposed pattern
> matching, referencing Python and explaining why the proposal
> is a cool treatment of the usecase.

I'm not clear on what you mean here. JavaScript has had a
switch/case construct since 1.2, in the late 1990s. As far
as I can determine, python has no such thing, since PEP-3103
was rejected in 2007.

> As a side note, if by scripting we mean OS commands,
> then Python started as a sysadmin language.

I never knew this. Where can I read more about this origin?

-- 
Michael F. Stemper
If it isn't running programs and it isn't fusing atoms, it's just bending space.

From 2QdxY4RzWzUUiLuE at potatochowder.com  Fri Feb 25 15:30:59 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Fri, 25 Feb 2022 14:30:59 -0600
Subject: Best way to check if there is internet?
In-Reply-To: <svbbqo$f58$1@dont-email.me>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me>
 <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <mailman.21.1645812487.2329.python-list@python.org>
 <svbbqo$f58$1@dont-email.me>
Message-ID: <Yhk8g6LAoo9H8m0a@scrozzle>

On 2022-02-25 at 13:48:32 -0600,
"Michael F. Stemper" <michael.stemper at gmail.com> wrote:

> On 25/02/2022 12.07, Abdur-Rahmaan Janhangeer wrote:

> > I have been following language feature proposals from various
> > languages. Some decide to avoid Python's route, but others have been
> > trying hard to catch up with Python.  One gleaming example is the
> > switch case. JS recently proposed pattern matching, referencing
> > Python and explaining why the proposal is a cool treatment of the
> > usecase.

> I'm not clear on what you mean here. JavaScript has had a switch/case
> construct since 1.2, in the late 1990s. As far as I can determine,
> python has no such thing, since PEP-3103 was rejected in 2007.

Python has a relatively new (as of version 3.10) "match" statement:

    https://docs.python.org/3/reference/compound_stmts.html#the-match-statement

From grant.b.edwards at gmail.com  Fri Feb 25 16:12:18 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Fri, 25 Feb 2022 13:12:18 -0800 (PST)
Subject: C is it always faster than nump?
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
Message-ID: <62194632.1c69fb81.395dc.a204@mx.google.com>

On 2022-02-25, Richard Damon <Richard at Damon-Family.org> wrote:
> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
>> Hi,
>> a lot of people think that C (or C++) is faster than python, yes I agree,
>> but I think that's not the case with numpy, I believe numpy is faster than
>> C, at least in some cases.
>
> My understanding is that numpy is written in C,

Sort of. IIRC a lot of the heavly lifting is done by libraries like
BLAS and LAPAK that were written in FORTRAN the last time I checked.

Has that changed?

Or am I conflating numpy with some other scientific-python stuff. Back
when I did a lot of numerical stuff in Python, I remember spending a
lot of time watching FORTRAN compiles. Admittedly, that's getting to
be 10+ years ago...

--
Grant



From rosuav at gmail.com  Fri Feb 25 16:23:44 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 08:23:44 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <Yhk8g6LAoo9H8m0a@scrozzle>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me>
 <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <mailman.21.1645812487.2329.python-list@python.org>
 <svbbqo$f58$1@dont-email.me> <Yhk8g6LAoo9H8m0a@scrozzle>
Message-ID: <CAPTjJmrHhuiH39CQRHw4MvVMRnuFeLVsN1znf=hZt14dezUvgw@mail.gmail.com>

On Sat, 26 Feb 2022 at 07:32, <2QdxY4RzWzUUiLuE at potatochowder.com> wrote:
>
> On 2022-02-25 at 13:48:32 -0600,
> "Michael F. Stemper" <michael.stemper at gmail.com> wrote:
>
> > On 25/02/2022 12.07, Abdur-Rahmaan Janhangeer wrote:
>
> > > I have been following language feature proposals from various
> > > languages. Some decide to avoid Python's route, but others have been
> > > trying hard to catch up with Python.  One gleaming example is the
> > > switch case. JS recently proposed pattern matching, referencing
> > > Python and explaining why the proposal is a cool treatment of the
> > > usecase.
>
> > I'm not clear on what you mean here. JavaScript has had a switch/case
> > construct since 1.2, in the late 1990s. As far as I can determine,
> > python has no such thing, since PEP-3103 was rejected in 2007.
>
> Python has a relatively new (as of version 3.10) "match" statement:
>
>     https://docs.python.org/3/reference/compound_stmts.html#the-match-statement

Every language learns from every other. Python has learned from
JavaScript too. Both languages have learned from Haskell. Pike has
learned from Python; and Python has learned from Pike. Nearly every
language learns from C and Smalltalk (sometimes indirectly). Of
course, not everything that is learned is copied - sometimes what you
learn is "let's not do it that way".

Python's match statement is not a switch/case block, and if JS is
borrowing the idea, then that's strong evidence that, even with
switch/case, match/case is a valuable addition. They behave very
differently in usage.

ChrisA

From michael.stemper at gmail.com  Fri Feb 25 16:05:29 2022
From: michael.stemper at gmail.com (Michael F. Stemper)
Date: Fri, 25 Feb 2022 15:05:29 -0600
Subject: Best way to check if there is internet?
In-Reply-To: <mailman.29.1645821070.2329.python-list@python.org>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me> <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <mailman.21.1645812487.2329.python-list@python.org>
 <svbbqo$f58$1@dont-email.me> <Yhk8g6LAoo9H8m0a@scrozzle>
 <mailman.29.1645821070.2329.python-list@python.org>
Message-ID: <svbgb5$hso$1@dont-email.me>

On 25/02/2022 14.30, 2QdxY4RzWzUUiLuE at potatochowder.com wrote:
> On 2022-02-25 at 13:48:32 -0600,
> "Michael F. Stemper" <michael.stemper at gmail.com> wrote:
> 
>> On 25/02/2022 12.07, Abdur-Rahmaan Janhangeer wrote:
> 
>>> I have been following language feature proposals from various
>>> languages. Some decide to avoid Python's route, but others have been
>>> trying hard to catch up with Python.  One gleaming example is the
>>> switch case. JS recently proposed pattern matching, referencing
>>> Python and explaining why the proposal is a cool treatment of the
>>> usecase.
> 
>> I'm not clear on what you mean here. JavaScript has had a switch/case
>> construct since 1.2, in the late 1990s. As far as I can determine,
>> python has no such thing, since PEP-3103 was rejected in 2007.
> 
> Python has a relatively new (as of version 3.10) "match" statement:
> 
>      https://docs.python.org/3/reference/compound_stmts.html#the-match-statement

Looks as if I have some reading to do. Thanks.


-- 
Michael F. Stemper
Always use apostrophe's and "quotation marks" properly.

From barry at barrys-emacs.org  Fri Feb 25 17:03:24 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Fri, 25 Feb 2022 22:03:24 +0000
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me> <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
Message-ID: <0C1D4B1A-CD61-499D-A376-3DA2B39E56DC@barrys-emacs.org>



> On 25 Feb 2022, at 18:07, Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:
> 
> Normally people put Python in the scripting category.
> I learnt typed languages like C++ and Java at first.
> People who learn languages like these tend to put
> Python in a non-serious category. The post was
> more ironic than litteral.

I think that python is a serious language, as I do C and C++.
What is key to a problem is which language is best to solve the
problem (given you have a free choice).

My day job is writing and maintaining a large python application.
Python allows us to use best engineering technics.

Python is used because it out performs C/C++/etc in the speed that
we can implement new features and generate revenue from them.

> 
> After 5 years of following the mailing lists i realised
> that there is more than the eyes meet. My hobby has
> always been language engineering / compiler theory
> and just recently i realised that Python is pretty serious about
> language features.
> 
> I have been following language feature proposals from various
> languages. Some decide to avoid Python's route,
> but others have been trying hard to catch up with Python.
> One gleaming example is the switch case. JS recently proposed pattern
> matching, referencing Python and explaining why the proposal
> is a cool treatment of the usecase.
> 
> As a side note, if by scripting we mean OS commands,
> then Python started as a sysadmin language.

It was started as a research language to teach children programming.
Python's simplicity, that we benefit from to this day, comes from that original purpose.

And indeed one of the pleasures of the UK PYCON is seeing children present
their projects to a room full of adult programers. Seeing a 6 year old present their
working project is wonderful to behold.

Barry


> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From rosuav at gmail.com  Fri Feb 25 17:22:13 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 09:22:13 +1100
Subject: Best way to check if there is internet?
In-Reply-To: <0C1D4B1A-CD61-499D-A376-3DA2B39E56DC@barrys-emacs.org>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me>
 <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <0C1D4B1A-CD61-499D-A376-3DA2B39E56DC@barrys-emacs.org>
Message-ID: <CAPTjJmp9GLjYHuCww6m0ptuqRmNYie2_U3Pg041FnBwp3BS6Lw@mail.gmail.com>

On Sat, 26 Feb 2022 at 09:04, Barry Scott <barry at barrys-emacs.org> wrote:
>
>
>
> > On 25 Feb 2022, at 18:07, Abdur-Rahmaan Janhangeer <arj.python at gmail.com> wrote:
> >
> > Normally people put Python in the scripting category.
> > I learnt typed languages like C++ and Java at first.
> > People who learn languages like these tend to put
> > Python in a non-serious category. The post was
> > more ironic than litteral.
>
> I think that python is a serious language, as I do C and C++.
> What is key to a problem is which language is best to solve the
> problem (given you have a free choice).
>

Exactly. What language is best for a situation depends on myriad factors:

* Development convenience
* Execution performance
* Language/library tools
* Interaction with other parts of a system
* Corporate politics
* The ability to use other people's code
* Etcetera, etcetera, etcetera.

What's the best language for modding Kerbal Space Program? C#, because
ultimately, anything else will require a C# wrapper.

What's the best language for building a scientific simulation of
particle collisions? Probably something high level like Python, backed
by some hard number-crunching work in Fortran.

What's the best language for writing a browser-based game? JavaScript.

What's the best language for swearing in? Ah, that one I can't help
you with, although I've heard good things about French. :)

The only languages that are "not serious languages" in that sense are
deliberate toys. Chef, Piet, Whitespace, Ook, these are non-serious
languages that you shouldn't be writing production code in. Anything
else? If it's right for your purposes, use it, and don't let anyone
shame you for it.

There are almost no "right" and "wrong" design decisions. Your
decision has consequences, and if you accept those consequences, the
decision was not wrong.

Dismissing a mainstream language as "not serious" is, in effect,
telling everyone who uses that language "you're doing it wrong". And
that's an insult that should not be made lightly.

ChrisA

From Richard at Damon-Family.org  Fri Feb 25 17:57:32 2022
From: Richard at Damon-Family.org (Richard Damon)
Date: Fri, 25 Feb 2022 17:57:32 -0500
Subject: C is it always faster than nump?
In-Reply-To: <CAPTjJmrVS1Ee5+hZLawz8r4ojN2N_a41-ORKJ7Lza879a-9Vkg@mail.gmail.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <CAPTjJmrVS1Ee5+hZLawz8r4ojN2N_a41-ORKJ7Lza879a-9Vkg@mail.gmail.com>
Message-ID: <dcccefa7-4c88-ce82-1426-3bc19029a30a@Damon-Family.org>

On 2/25/22 2:47 PM, Chris Angelico wrote:
> On Sat, 26 Feb 2022 at 05:49, Richard Damon <Richard at damon-family.org> wrote:
>> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
>>> Hi,
>>> a lot of people think that C (or C++) is faster than python, yes I agree,
>>> but I think that's not the case with numpy, I believe numpy is faster than
>>> C, at least in some cases.
>>>
>> My understanding is that numpy is written in C, so for it to be faster
>> than C, you are saying that C is faster that C.
> Fortran actually, but ultimately, they both get compiled to machine code.

Looking at the Github repo I see:

Languages:
Python.? 62.5%
C??? ?????? 35.3%
C++.?????? 1.0%
Cython.?? 0.9%
Shell.?????? 0.2%
Fortran.?? 0.1%

So there is a bit of Fortan in there, but it looks like most of the 
heavy lifting is in C.

My guess is the Fortran is likely some hooks to add Fortran modules into 
the program with numpy.

...
>> The key point is that numpy was written by skilled programmers who
>> carefully optimized their code to be as fast as possible for the major
>> cases. Thus it is quite possible for the numpy code to be faster in C
>> than code written by a person without that level of care and effort.
> This is clearly true.
>
> ChrisA


-- 
Richard Damon


From rosuav at gmail.com  Fri Feb 25 18:02:16 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 10:02:16 +1100
Subject: C is it always faster than nump?
In-Reply-To: <dcccefa7-4c88-ce82-1426-3bc19029a30a@Damon-Family.org>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <CAPTjJmrVS1Ee5+hZLawz8r4ojN2N_a41-ORKJ7Lza879a-9Vkg@mail.gmail.com>
 <dcccefa7-4c88-ce82-1426-3bc19029a30a@Damon-Family.org>
Message-ID: <CAPTjJmpgmuSYWKANxt=CF-522yAC+G9Up7u9O=mODzGfFdHSUA@mail.gmail.com>

On Sat, 26 Feb 2022 at 09:58, Richard Damon <Richard at damon-family.org> wrote:
>
> On 2/25/22 2:47 PM, Chris Angelico wrote:
> > On Sat, 26 Feb 2022 at 05:49, Richard Damon <Richard at damon-family.org> wrote:
> >> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> >>> Hi,
> >>> a lot of people think that C (or C++) is faster than python, yes I agree,
> >>> but I think that's not the case with numpy, I believe numpy is faster than
> >>> C, at least in some cases.
> >>>
> >> My understanding is that numpy is written in C, so for it to be faster
> >> than C, you are saying that C is faster that C.
> > Fortran actually, but ultimately, they both get compiled to machine code.
>
> Looking at the Github repo I see:
>
> Languages:
> Python.  62.5%
> C           35.3%
> C++.       1.0%
> Cython.   0.9%
> Shell.       0.2%
> Fortran.   0.1%
>
> So there is a bit of Fortan in there, but it looks like most of the
> heavy lifting is in C.
>
> My guess is the Fortran is likely some hooks to add Fortran modules into
> the program with numpy.
>

GitHub's analysis isn't always very meaningful. I think it's pretty
clear that Numpy isn't implemented in Python :) In any case, point is
that the implementation language is mostly irrelevant.

ChrisA

From sjeik_appie at hotmail.com  Fri Feb 25 18:04:00 2022
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sat, 26 Feb 2022 00:04:00 +0100
Subject: One-liner to merge lists?
In-Reply-To: <843155aa-8536-07ab-3050-ad79b0eb281a@web.de>
Message-ID: <DB6PR01MB38957B5E5C8782D98A521039833E9@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

     If you don't like the idea of 'adding' strings you can 'concat'enate:

     >>> items = [[1,2,3], [4,5], [6]]
     >>> functools.reduce(operator.concat, items)
     [1, 2, 3, 4, 5, 6]
     >>> functools.reduce(operator.iconcat, items, [])
     [1, 2, 3, 4, 5, 6]

     The latter is the functional way to spell your for... extend() loop.
     Don't forget to provide the initial value in that case lest you modify
     the input:

     >> functools.reduce(operator.iconcat, items)? # wrong
     [1, 2, 3, 4, 5, 6]
     >>> items
     [[1, 2, 3, 4, 5, 6], [4, 5], [6]]? # oops

     --
     https://mail.python.org/mailman/listinfo/python-list

   ====
   Was also thinking about reduce, though this one uses a dunder method:
   from functools import reduce
   d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
   print(reduce(list.__add__, list(d.values())))

From avigross at verizon.net  Fri Feb 25 18:06:57 2022
From: avigross at verizon.net (Avi Gross)
Date: Fri, 25 Feb 2022 23:06:57 +0000 (UTC)
Subject: C is it always faster than nump?
In-Reply-To: <62194632.1c69fb81.395dc.a204@mx.google.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <62194632.1c69fb81.395dc.a204@mx.google.com>
Message-ID: <1895471735.1373151.1645830417417@mail.yahoo.com>

Is that fair, Grant?

I go back far enough that in my earliest years I was submitting FORTRAN programs written on punched cards and often getting them back the next day. The compiling was not the major factor in how long it took.

For many cases, a compiled language only needs to be compiled once and can be run many times. Other than during development timeframes, the major concept of speed is how well it runs not how long it takes to compile, especially if the compiler is busy doing all kinds of optimizations to your code or checking for possible obscure errors or making sure the code does not try something funny like access memory not allowed and so on.

An interpreted language often has to do the same things every time, albeit some have come up with ways to partially compile modules and only redo if they have been changed. Some of what they do every time is in some sense wasted effort but as a tradeoff, they can do many things in a dynamic way that compiled programs may not do easily or at all. 

Another argument I find is unfair, is some comparisons with what I consider "mixed" effort. If you have a program that calls various numpy routines using native python in between, then clearly a decent amount of the time spent is not in numpy. It may for example suddenly import a module and that takes time spent doing nothing about the problem other than loading what is needed. Another example, it is common to place the numbers or other values you get from numpy operations into dictionaries, lists and so on, or to make graphs. If you had done the same work in a purely C (or FORTRAN or whatever environment) and had access to similar other functionality, the latter two would all be in C or some compiled library. 

With exceptions aplenty, speed is not always a primary consideration. Often programmer time and productivity matter more. I have seen many projects though that were first implemented in a language like Python and when they had figured out a decent way to do the task reliably, they handed it over to people to redo much or all of it using languages like C++, compiles and all, as a reasonable translation from a working application may take much less time to implement and once done, way work better. At least it may until you want to change it! Prototyping often works better in some languages. Python has a compromise that modules can use to speed up important parts of the process by substituting a compiled function in C/C++/whatever for a local Python function but not necessarily switching entirely to C and suffering under the negatives of that environment.

I do wonder if we are already on a path where a language that handles concepts like parallelism (or even weirder, quantum computations) well may be a better candidate for doing some projects well as even though it may not run at top speed, it can make use of many processors or a network of machines in the cloud, to handle things in a more flexible way that may even get the job done faster.

Next discussion is whether pandas is faster than C, followed by SciPy ...

I do have to wonder if anyone ever considered adding back enough functionality into base Python to make some additions less needed. Is there any reason the kind of structures used by so many languages cannot be made part of python such as a vector/array that holds exactly one kind of data structure and not force use of things like a list when that is more than is needed?


-----Original Message-----
From: Grant Edwards <grant.b.edwards at gmail.com>
To: python-list at python.org
Sent: Fri, Feb 25, 2022 4:12 pm
Subject: Re: C is it always faster than nump?


On 2022-02-25, Richard Damon <Richard at Damon-Family.org> wrote:

> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:

>> Hi,

>> a lot of people think that C (or C++) is faster than python, yes I agree,

>> but I think that's not the case with numpy, I believe numpy is faster than

>> C, at least in some cases.

>

> My understanding is that numpy is written in C,



Sort of. IIRC a lot of the heavly lifting is done by libraries like

BLAS and LAPAK that were written in FORTRAN the last time I checked.



Has that changed?



Or am I conflating numpy with some other scientific-python stuff. Back

when I did a lot of numerical stuff in Python, I remember spending a

lot of time watching FORTRAN compiles. Admittedly, that's getting to

be 10+ years ago...



--

Grant





-- 

https://mail.python.org/mailman/listinfo/python-list


From barry at barrys-emacs.org  Fri Feb 25 18:05:54 2022
From: barry at barrys-emacs.org (Barry)
Date: Fri, 25 Feb 2022 23:05:54 +0000
Subject: C is it always faster than nump?
In-Reply-To: <dcccefa7-4c88-ce82-1426-3bc19029a30a@Damon-Family.org>
References: <dcccefa7-4c88-ce82-1426-3bc19029a30a@Damon-Family.org>
Message-ID: <52B63131-742D-47AD-AA5C-2D045F96D855@barrys-emacs.org>



> On 25 Feb 2022, at 23:00, Richard Damon <Richard at damon-family.org> wrote:
> 
> ?On 2/25/22 2:47 PM, Chris Angelico wrote:
>>> On Sat, 26 Feb 2022 at 05:49, Richard Damon <Richard at damon-family.org> wrote:
>>> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
>>>> Hi,
>>>> a lot of people think that C (or C++) is faster than python, yes I agree,
>>>> but I think that's not the case with numpy, I believe numpy is faster than
>>>> C, at least in some cases.
>>>> 
>>> My understanding is that numpy is written in C, so for it to be faster
>>> than C, you are saying that C is faster that C.
>> Fortran actually, but ultimately, they both get compiled to machine code.
> 
> Looking at the Github repo I see:
> 
> Languages:
> Python.  62.5%
> C           35.3%
> C++.       1.0%
> Cython.   0.9%
> Shell.       0.2%
> Fortran.   0.1%

I assume that this is just for bumpy and not for all its dependencies.
That will add a lot of Fortran and c++ I expect.

> 
> So there is a bit of Fortan in there, but it looks like most of the heavy lifting is in C.
> 
> My guess is the Fortran is likely some hooks to add Fortran modules into the program with numpy.
> 
> ...
>>> The key point is that numpy was written by skilled programmers who
>>> carefully optimized their code to be as fast as possible for the major
>>> cases. Thus it is quite possible for the numpy code to be faster in C
>>> than code written by a person without that level of care and effort.
>> This is clearly true.
>> 
>> ChrisA
> 
> 
> -- 
> Richard Damon
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From rosuav at gmail.com  Fri Feb 25 18:11:30 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 10:11:30 +1100
Subject: One-liner to merge lists?
In-Reply-To: <DB6PR01MB38957B5E5C8782D98A521039833E9@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
References: <843155aa-8536-07ab-3050-ad79b0eb281a@web.de>
 <DB6PR01MB38957B5E5C8782D98A521039833E9@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
Message-ID: <CAPTjJmovzRg0SezbKqh9sW7H=EzAn=h47WLUpW-weTmz9KGaHg@mail.gmail.com>

On Sat, 26 Feb 2022 at 10:05, Albert-Jan Roskam <sjeik_appie at hotmail.com> wrote:
>    Was also thinking about reduce, though this one uses a dunder method:
>    from functools import reduce
>    d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>    print(reduce(list.__add__, list(d.values())))

If you don't want to use the dunder, just use the operator module:

reduce(operator.add, d.values())

But ultimately, that's still the same as sum(), and it's no more
readable than chain(), so I'd still be inclined to go with chain and
then wrap it in a function if you need the clarity.

ChrisA

From greg.ewing at canterbury.ac.nz  Fri Feb 25 19:40:35 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 26 Feb 2022 13:40:35 +1300
Subject: Best way to check if there is internet?
In-Reply-To: <mailman.33.1645827746.2329.python-list@python.org>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me> <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <0C1D4B1A-CD61-499D-A376-3DA2B39E56DC@barrys-emacs.org>
 <CAPTjJmp9GLjYHuCww6m0ptuqRmNYie2_U3Pg041FnBwp3BS6Lw@mail.gmail.com>
 <mailman.33.1645827746.2329.python-list@python.org>
Message-ID: <j7tb83Fqd5dU1@mid.individual.net>

On 26/02/22 11:22 am, Chris Angelico wrote:
> What's the best language for swearing in? Ah, that one I can't help
> you with, although I've heard good things about French. :)

Russian seems to be quite good for it too, judging by
certain dashcam footage channels.

-- 
Greg

From oscar.j.benjamin at gmail.com  Fri Feb 25 21:21:29 2022
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 26 Feb 2022 02:21:29 +0000
Subject: C is it always faster than nump?
In-Reply-To: <52B63131-742D-47AD-AA5C-2D045F96D855@barrys-emacs.org>
References: <dcccefa7-4c88-ce82-1426-3bc19029a30a@Damon-Family.org>
 <52B63131-742D-47AD-AA5C-2D045F96D855@barrys-emacs.org>
Message-ID: <CAHVvXxTFLnWyOSmctRiKvvk2fBXzpzumk_fYoOr1e7orAUA8rQ@mail.gmail.com>

On Fri, 25 Feb 2022 at 23:13, Barry <barry at barrys-emacs.org> wrote:
>
> > On 25 Feb 2022, at 23:00, Richard Damon <Richard at damon-family.org> wrote:
> >
> > ?On 2/25/22 2:47 PM, Chris Angelico wrote:
> >>> On Sat, 26 Feb 2022 at 05:49, Richard Damon <Richard at damon-family.org> wrote:
> >>> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> >>>> Hi,
> >>>> a lot of people think that C (or C++) is faster than python, yes I agree,
> >>>> but I think that's not the case with numpy, I believe numpy is faster than
> >>>> C, at least in some cases.
> >>>>
> >>> My understanding is that numpy is written in C, so for it to be faster
> >>> than C, you are saying that C is faster that C.
> >> Fortran actually, but ultimately, they both get compiled to machine code.
> >
> > Looking at the Github repo I see:
> >
> > Languages:
> > Python.  62.5%
> > C           35.3%
> > C++.       1.0%
> > Cython.   0.9%
> > Shell.       0.2%
> > Fortran.   0.1%
>
> I assume that this is just for bumpy and not for all its dependencies.
> That will add a lot of Fortran and c++ I expect.

NumPy links with BLAS/LAPACK that will do the heavy lifting for common
linear algebra operations. Multiple different BLAS libraries can be
used with NumPy and those libraries might be written in Fortran and
might also involve some hand-crafted assembly for particular
architectures. Some explanation of NumPy's BLAS/LAPACK support is
here:
https://numpy.org/devdocs/user/building.html

By default if you install NumPy from conda (or at least if you install
the Anaconda distribution) then I think that NumPy will use the Intel
MKL library:
https://en.wikipedia.org/wiki/Math_Kernel_Library
As I understand it the core of MKL is Fortran and is compiled with
Intel's ifortran compiler but some parts might be C/C++. MKL is also
the same library that is used by Matlab for its own heavy lifting.
It's not sufficiently free to be used in NumPy's PyPI wheels though.

If you install the precompiled NumPy wheels with pip from PyPI then I
think those are statically linked with OpenBLAS:
https://github.com/xianyi/OpenBLAS
Again I think the core of OpenBLAS is Fortran but there's some C in there.

In the pre-wheel days the situation was that NumPy provided installer
files for Windows that would give binaries linked with ATLAS (also
Fortran):
https://en.wikipedia.org/wiki/Automatically_Tuned_Linear_Algebra_Software

I think at some point NumPy used to use the OSX Accelerate library but
the page I linked above says that's now deprecated. I don't know
anything about Accelerate but I wouldn't be surprised to hear that it
was a bunch of old Fortran code!

If you build NumPy from source without having any BLAS/LAPACK
libraries then I think it uses its own backup version of these that is
written in C but not as well optimised. This used to be the default
for a pip install on Linux in pre-wheel times.

Many operations in NumPy don't actually use BLAS/LAPACK and for those
parts the heavy lifting is all done in NumPy's own C code.

Lastly SciPy which is very often used together with NumPy does have a
load of Fortran code. As I understand it at some point NumPy and SciPy
were divided from the previous numerical Python libraries and there
was a deliberate split so that the Fortran code all ended up in SciPy
rather than NumPy.

--
Oscar

From wlfraed at ix.netcom.com  Fri Feb 25 21:59:00 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Fri, 25 Feb 2022 21:59:00 -0500
Subject: C is it always faster than nump?
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <62194632.1c69fb81.395dc.a204@mx.google.com>
 <1895471735.1373151.1645830417417@mail.yahoo.com>
 <mailman.37.1645830361.2329.python-list@python.org>
Message-ID: <1q5j1htr37sbq9pp2umnl6ogtqndtru6rd@4ax.com>

On Fri, 25 Feb 2022 23:06:57 +0000 (UTC), Avi Gross <avigross at verizon.net>
declaimed the following:

>I do have to wonder if anyone ever considered adding back enough functionality into base Python to make some additions less needed. Is there any reason the kind of structures used by so many languages cannot be made part of python such as a vector/array that holds exactly one kind of data structure and not force use of things like a list when that is more than is needed?
>
	https://docs.python.org/3/library/array.html

seems to fit the criteria...


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From arj.python at gmail.com  Fri Feb 25 22:29:49 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sat, 26 Feb 2022 06:29:49 +0300
Subject: Best way to check if there is internet?
In-Reply-To: <svbbqo$f58$1@dont-email.me>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me>
 <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <mailman.21.1645812487.2329.python-list@python.org>
 <svbbqo$f58$1@dont-email.me>
Message-ID: <CADrxXXnzezwF=Ya=3Fsi1dFhq7Tw_pbZgCkfRTLbDMx-cdtniQ@mail.gmail.com>

Yes i know that JS has switch from before

I was referring to the pattern matching proposal.

I cannot find the original place i read it from but here's the github one

https://github.com/tc39/proposal-pattern-matching

From oscar.j.benjamin at gmail.com  Fri Feb 25 22:33:06 2022
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Sat, 26 Feb 2022 03:33:06 +0000
Subject: C is it always faster than nump?
In-Reply-To: <1q5j1htr37sbq9pp2umnl6ogtqndtru6rd@4ax.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <62194632.1c69fb81.395dc.a204@mx.google.com>
 <1895471735.1373151.1645830417417@mail.yahoo.com>
 <mailman.37.1645830361.2329.python-list@python.org>
 <1q5j1htr37sbq9pp2umnl6ogtqndtru6rd@4ax.com>
Message-ID: <CAHVvXxSH-z0hpFZbYBSrd87vB3ajcCLVUMTpuesk3BO=JvSQyQ@mail.gmail.com>

On Sat, 26 Feb 2022 at 03:10, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>
> On Fri, 25 Feb 2022 23:06:57 +0000 (UTC), Avi Gross <avigross at verizon.net>
> declaimed the following:
>
> >I do have to wonder if anyone ever considered adding back enough functionality into base Python to make some additions less needed. Is there any reason the kind of structures used by so many languages cannot be made part of python such as a vector/array that holds exactly one kind of data structure and not force use of things like a list when that is more than is needed?
> >
>         https://docs.python.org/3/library/array.html
>
> seems to fit the criteria...

The stdlib array module is basically unused in comparison to NumPy.
The capabilities of the array module do not meet the needs for most
users who want to do anything useful with arrays.

The intention in creating NumPy  (in the NumPy/SciPy split) was that
it might be possible that NumPy could be merged into core Python.
Unfortunately that didn't come to be.

--
Oscar

From avigross at verizon.net  Fri Feb 25 22:33:58 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 26 Feb 2022 03:33:58 +0000 (UTC)
Subject: C is it always faster than nump?
In-Reply-To: <1q5j1htr37sbq9pp2umnl6ogtqndtru6rd@4ax.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <62194632.1c69fb81.395dc.a204@mx.google.com>
 <1895471735.1373151.1645830417417@mail.yahoo.com>
 <mailman.37.1645830361.2329.python-list@python.org>
 <1q5j1htr37sbq9pp2umnl6ogtqndtru6rd@4ax.com>
Message-ID: <6601923.1400966.1645846438538@mail.yahoo.com>

Dennis,

What you describe may be a start but is it anything I might not have easily created myself? https://docs.python.org/3/library/array.html

I can see creating my own object and adding those methods and attributes while gaining very little, except perhaps storage. 

Can I add or multiply two such items efficiently if it contains a numeric value? Can I offer them as an argument to all kinds of functions which can now handle it well? How does it work if a second operand is a scalar or an array of another data type. Can two be compared and result in an array of boolean (not seen in the list of types). Numpy does quite a bit of that kind of thing but perhaps better is a language like R where all that and more are built in. 

But with numpy and more available anyway, it may not be necessary to reinvent much of that. I was just wondering if it ever made sense to simply include it in the base python, perhaps as a second executable with a name like pythonn to signify that it is more numeric. So if you run that, you know you do not need to add an assortment of modules. I keep seeing programs that just automatically add numpy and pandas and various graphic modules and other scientific and machine learning modules. Of course not everyone needs or even wants this. Many simply use base Python techniques even if they are low for larger amounts of data.




From arj.python at gmail.com  Fri Feb 25 22:36:57 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sat, 26 Feb 2022 06:36:57 +0300
Subject: Best way to check if there is internet?
In-Reply-To: <svbbqo$f58$1@dont-email.me>
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me>
 <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <mailman.21.1645812487.2329.python-list@python.org>
 <svbbqo$f58$1@dont-email.me>
Message-ID: <CADrxXXkc63kt5JqLwd1WkVw=YoXZx8-bDtxwUjFqiWB2BGA-Zg@mail.gmail.com>

> I never knew this. Where can I read more about this origin?

https://python-history.blogspot.com/2009/01/personal-history-part-1-cwi.html?m=1

<<So there was a need for a language that would ?bridge the gap between C
and the shell.? For a long time, this was Python?s main catchphrase.>>


There are more sources which i cannot remember
from the top of my head

From rosuav at gmail.com  Fri Feb 25 23:16:35 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 15:16:35 +1100
Subject: C is it always faster than nump?
In-Reply-To: <6601923.1400966.1645846438538@mail.yahoo.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <62194632.1c69fb81.395dc.a204@mx.google.com>
 <1895471735.1373151.1645830417417@mail.yahoo.com>
 <mailman.37.1645830361.2329.python-list@python.org>
 <1q5j1htr37sbq9pp2umnl6ogtqndtru6rd@4ax.com>
 <6601923.1400966.1645846438538@mail.yahoo.com>
Message-ID: <CAPTjJmrg7KAZzyHh+XVAfwLcLCOg4jNDARgfKv02D8-sodZ0KQ@mail.gmail.com>

On Sat, 26 Feb 2022 at 14:35, Avi Gross via Python-list
<python-list at python.org> wrote:
> But with numpy and more available anyway, it may not be necessary to reinvent much of that. I was just wondering if it ever made sense to simply include it in the base python, perhaps as a second executable with a name like pythonn to signify that it is more numeric. So if you run that, you know you do not need to add an assortment of modules. I keep seeing programs that just automatically add numpy and pandas and various graphic modules and other scientific and machine learning modules. Of course not everyone needs or even wants this. Many simply use base Python techniques even if they are low for larger amounts of data.
>

How would that be different from getting one of the numeric/scientific
distributions of Python? Why should it be a different Python
executable?!?

ChrisA

From avigross at verizon.net  Fri Feb 25 23:37:34 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 26 Feb 2022 04:37:34 +0000 (UTC)
Subject: C is it always faster than nump?
In-Reply-To: <CAPTjJmozkL3U2CvimvQFhkU8QwpJyRL2ORiS3AseHr1e=8vLKw@mail.gmail.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <375047121.1338435.1645818184031@mail.yahoo.com>
 <CAPTjJmozkL3U2CvimvQFhkU8QwpJyRL2ORiS3AseHr1e=8vLKw@mail.gmail.com>
Message-ID: <1735230060.1402275.1645850254409@mail.yahoo.com>

Yes, Chris, C is real as a somewhat abstract concept. There are a whole slew of different variations each time it is released anew with changes and then some people at various times built actual compilers that implement a varying subset of what is possible, and not necessarily in quite the same way.

As you gathered, I am saying that comparing languages is not so effective as comparing implementations and even better specific programs on specific data. And yet, you can still get odd results if you cherry pick what to test. Consider a sorting algorithm that rapidly checks if the data is already sorted, and if so, does not bother sorting it. It will quite possibly be the fastest one in a comparison if the data is chosen to be already in order! But on many other sets of data it will have wasted some time checking if it is in order while other algorithms have started sorting it!

Bad example, maybe, but there are better ones. Consider an algorithm that does no checking for one of many errors that can happen. It does not see if the arguments it gets are within expected ranges of types or values. It does not intercept attempts to divide by zero and much more. Another algorithm is quite bulletproof and thus has lots more code and maybe runs much slower. Is it shocking if it tests slower . But the other code may end up failing faster in the field and need a rewrite.

A really fair comparison is often really hard. Languages are abstract and sometimes a new implementation makes a huge change.

Take interpreted languages including Python and R that specify all kinds of functions that may be written within the language at first. Someone may implement a function like sum() (just an example) that looks like the sum of a long list of items is the first item added to a slightly longer sum of the remaining items. It stops when the final recursive sum is about to be called with no remaining arguments. Clearly this implementation may be a tad slow. But does Python require this version of sum() or will it allow any version that can be called the same way and returns the same results every time? Does it even matter if the function is written in C or C++ or FORTRAN or even assembler of some kind, as long as it is placed in an accessible library and there is some interface that allows you to make the call in python notation and it is fed to the function in the way it requires, and similarly deals with returned values? A wrapper, sort of.

The use of such a shortcut is not against the spirit of the language. You can still specify you want the sum() function from some module, or write your own. This is true most places. I remember way back when how early UNIX shells did silly things like call /bin/echo to do trivial things, or call an external program to do something as trivial as i=i+1 and then they started building in such functionality and your shell scripts suddenly really speeded up. A non-programmer I once worked for wrote some truly humongous shell scripts that brought machines it was run on remotely in places like Japan during their day-time to their knees. Collecting billing data from all over by running a pipeline with 9 processes per line/row was a bit much. 

At first I sped it up quite a bit by using newer built-in features like I described, or doing more with fewer elements in pipelines. But I saw how much was caused by using the wrong tools for the job and there were programs designed to analyze data in various ways.

I replaced almost all of it with an AWK script that speeded things up many orders of magnitude. And, yes, AWK was not as fast as C but more trivial to program in for this need as it had so  many needed aspects built-in or happening automagically.

Would we do the entire project differently today? Definitely. All the billing records would not be sitting in an assortment of flat files all over the place but rather be fed into some database that made retrieval of all kinds of reports straightforward without needing to write much code at all.

How many modules or "packages" were once written largely using the language and then gradually "improved" by replacing parts, especially slower parts, with external content as we have been discussing? In a sense, some Python applications run on older versions of Python may be running faster as newer versions have improved some of the "same" code while to the user, they see it running on the same language, Python?

-----Original Message-----
From: Chris Angelico <rosuav at gmail.com>
To: python-list at python.org <python-list at python.org>
Sent: Fri, Feb 25, 2022 2:58 pm
Subject: Re: C is it always faster than nump?


On Sat, 26 Feb 2022 at 06:44, Avi Gross via Python-list
<python-list at python.org> wrote:
>
> I agree with Richard.
>
> Some people may be confused and think c is the speed of light and relativistically speaking, nothing can be faster. (OK, just joking. The uses of the same letter of the alphabet are not at all related. One is named for the language that came after the one named B, while the other may be short for celeritas meaning speed.)
>
> There is no such thing as C. C does nothing. It is a combination of a language specification and some pieces of software called compilers that implement it well or less well.
>

Uhh, that's taking it a little bit TOO far.... I agree with your
point, but saying that there's no such thing as C is slightly unfair
:)

> There is such a thing as a PROGRAM. A program completely written in C is a thing. It can run fast or slow based on a combination of how it was written and on what data it operates on, which hardware and OS and so on. AND some of it may likely be running code from libraries written in other languages like FORTRAN that get linked into it in some way at compile time or runtime, and hooks into the local OS and so on.
>
> So your program written supposedly in pure C, may run faster or slower. If you program a "sort" algorithm in C, it may matter if it is an implementation of a merge sort or at bubble sort or ...
>

More specifically: You're benchmarking a particular *implementation*
of a particular *algorithm*. Depending on what you're trying to
demonstrate, either could be significant.

Performance testing between two things written in C is a huge job.
Performance testing across languages has a strong tendency to be
meaningless (like benchmarking Python's integers against JavaScript's
numbers).

> As noted, numpy is largely written in C. It may well be optimized in some places but there are constraints that may well make it hard to optimize compared to some other implementation without those constraints. In particular, it interfaces with standard Python data structures at times such as when initializing from a Python List, or List of Lists, or needing to hold on to various attributes so it can be converted back, or things I am not even aware of.
>

(Fortran)

In theory, summing a Numpy array should be incredibly fast, but in
practice, there's a lot of variation, and it can be quite surprising.
For instance, integers are faster than floats, everyone knows that.
And it's definitely faster to sum smaller integers than larger ones.

rosuav at sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(1000000), dtype=numpy.float64)' 'numpy.sum(x)'
1000 loops, best of 5: 325 usec per loop
rosuav at sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(1000000), dtype=numpy.int64)' 'numpy.sum(x)'
500 loops, best of 5: 551 usec per loop
rosuav at sikorsky:~$ python3 -m timeit -s 'import numpy; x =
numpy.array(range(1000000), dtype=numpy.int32)' 'numpy.sum(x)'
500 loops, best of 5: 680 usec per loop

... Or not.

Summing arrays isn't necessarily the best test of numpy anyway, but as
you can see, testing is an incredibly difficult thing to get right.
The easiest thing to prove is that you have no idea how to prove
anything usefully, and most of us achieve that every time :)

ChrisA



> So, I suspect it may well be possible to make a pure C library similar to numpy in many ways but that can only be used within a C program that only uses native C data structures. It also is possible to write such a program that is horribly slow. And it is possible to write a less complex version of numpy that does not support some current numpy functionality and overall runs much faster on what it does support.
>
> I do wonder at the reason numpy and pandas and lots of other modules have to exist. Other languages like R made design choices that built in ideas of vectorization from the start. Python has lots of object-oriented extensibility that can allow you to create interpreted code that may easily extend it in areas to have some similar features. You can create an array-like data structure that holds only one object type and is extended so adding two together (or multiplying) ends up doing it componentwise. But attempts to do some such things often run into problems as they tend to be slow. So numpy was not written in python, mostly, albeit it could have been even more impressive if it took advantage of more pythonic abilities, at a cost.
>
> But now that numpy is in C, pretty much, it is somewhat locked in when and if other things in Python change.
>
> The reality is that many paradigms carried too far end up falling short.
>
>
> -----Original Message-----
> From: Richard Damon <Richard at Damon-Family.org>
> To: python-list at python.org
> Sent: Fri, Feb 25, 2022 1:48 pm
> Subject: Re: C is it always faster than nump?
>
>
> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> > Hi,
> > a lot of people think that C (or C++) is faster than python, yes I agree,
> > but I think that's not the case with numpy, I believe numpy is faster than
> > C, at least in some cases.
> >
> My understanding is that numpy is written in C, so for it to be faster
> than C, you are saying that C is faster that C.
>
> The key point is that numpy was written by skilled programmers who
> carefully optimized their code to be as fast as possible for the major
> cases. Thus it is quite possible for the numpy code to be faster in C
> than code written by a person without that level of care and effort.
>
> There are similar package available for many languages, including C/C++
> to let mere mortals get efficient numerical processing.
>
> --
> Richard Damon
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
> --
> https://mail.python.org/mailman/listinfo/python-list
-- 
https://mail.python.org/mailman/listinfo/python-list

From avigross at verizon.net  Fri Feb 25 23:41:15 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 26 Feb 2022 04:41:15 +0000 (UTC)
Subject: C is it always faster than nump?
In-Reply-To: <CAPTjJmrg7KAZzyHh+XVAfwLcLCOg4jNDARgfKv02D8-sodZ0KQ@mail.gmail.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <62194632.1c69fb81.395dc.a204@mx.google.com>
 <1895471735.1373151.1645830417417@mail.yahoo.com>
 <mailman.37.1645830361.2329.python-list@python.org>
 <1q5j1htr37sbq9pp2umnl6ogtqndtru6rd@4ax.com>
 <6601923.1400966.1645846438538@mail.yahoo.com>
 <CAPTjJmrg7KAZzyHh+XVAfwLcLCOg4jNDARgfKv02D8-sodZ0KQ@mail.gmail.com>
Message-ID: <314249599.1407364.1645850475320@mail.yahoo.com>

Agreed, Chris. There are many ways to get something done. I often use the Anaconda distribution because it tends to bundle many of the modules I need and more.

Not that it is such a big deal to load the ones you need, but if you share your program, others trying to use it may have some problems.


-----Original Message-----
From: Chris Angelico <rosuav at gmail.com>
To: python-list at python.org <python-list at python.org>
Sent: Fri, Feb 25, 2022 11:16 pm
Subject: Re: C is it always faster than nump?


On Sat, 26 Feb 2022 at 14:35, Avi Gross via Python-list
<python-list at python.org> wrote:
> But with numpy and more available anyway, it may not be necessary to reinvent much of that. I was just wondering if it ever made sense to simply include it in the base python, perhaps as a second executable with a name like pythonn to signify that it is more numeric. So if you run that, you know you do not need to add an assortment of modules. I keep seeing programs that just automatically add numpy and pandas and various graphic modules and other scientific and machine learning modules. Of course not everyone needs or even wants this. Many simply use base Python techniques even if they are low for larger amounts of data.
>

How would that be different from getting one of the numeric/scientific
distributions of Python? Why should it be a different Python
executable?!?

ChrisA

-- 
https://mail.python.org/mailman/listinfo/python-list


From rosuav at gmail.com  Sat Feb 26 00:02:36 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 26 Feb 2022 16:02:36 +1100
Subject: C is it always faster than nump?
In-Reply-To: <1735230060.1402275.1645850254409@mail.yahoo.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <375047121.1338435.1645818184031@mail.yahoo.com>
 <CAPTjJmozkL3U2CvimvQFhkU8QwpJyRL2ORiS3AseHr1e=8vLKw@mail.gmail.com>
 <1735230060.1402275.1645850254409@mail.yahoo.com>
Message-ID: <CAPTjJmrd08h+9yXUM47-8ADH1-fEGj0E8Sv-D9jpeCH9DoRo9w@mail.gmail.com>

On Sat, 26 Feb 2022 at 15:39, Avi Gross via Python-list
<python-list at python.org> wrote:
> Take interpreted languages including Python and R that specify all kinds of functions that may be written within the language at first. Someone may implement a function like sum() (just an example) that looks like the sum of a long list of items is the first item added to a slightly longer sum of the remaining items. It stops when the final recursive sum is about to be called with no remaining arguments. Clearly this implementation may be a tad slow. But does Python require this version of sum() or will it allow any version that can be called the same way and returns the same results every time?
>

That's also true of C and pretty much every language I know of. They
define semantics, not implementation.

ChrisA

From drsalists at gmail.com  Sat Feb 26 00:17:21 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Fri, 25 Feb 2022 21:17:21 -0800
Subject: C is it always faster than nump?
In-Reply-To: <CAPTjJmrd08h+9yXUM47-8ADH1-fEGj0E8Sv-D9jpeCH9DoRo9w@mail.gmail.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <2c0d93a8-5ea5-9b63-c3cd-183446d21446@Damon-Family.org>
 <375047121.1338435.1645818184031@mail.yahoo.com>
 <CAPTjJmozkL3U2CvimvQFhkU8QwpJyRL2ORiS3AseHr1e=8vLKw@mail.gmail.com>
 <1735230060.1402275.1645850254409@mail.yahoo.com>
 <CAPTjJmrd08h+9yXUM47-8ADH1-fEGj0E8Sv-D9jpeCH9DoRo9w@mail.gmail.com>
Message-ID: <CAGGBd_rRs_s6h6ts0GKL=NVqsn4eNEphOmiD+fhNn+D1iPL6MQ@mail.gmail.com>

On Fri, Feb 25, 2022 at 9:03 PM Chris Angelico <rosuav at gmail.com> wrote:

> On Sat, 26 Feb 2022 at 15:39, Avi Gross via Python-list
> <python-list at python.org> wrote:
> > Take interpreted languages including Python and R that specify all kinds
> of functions that may be written within the language at first. Someone may
> implement a function like sum() (just an example) that looks like the sum
> of a long list of items is the first item added to a slightly longer sum of
> the remaining items. It stops when the final recursive sum is about to be
> called with no remaining arguments. Clearly this implementation may be a
> tad slow. But does Python require this version of sum() or will it allow
> any version that can be called the same way and returns the same results
> every time?
> >
>
> That's also true of C and pretty much every language I know of. They
> define semantics, not implementation.
>

This comes back to something we've discussed before.

A language that is described primarily by a reference implementation rather
than a standard, runs the risk of being defined by that implementation.

From drsalists at gmail.com  Sat Feb 26 00:44:14 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Fri, 25 Feb 2022 21:44:14 -0800
Subject: C is it always faster than nump?
In-Reply-To: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
Message-ID: <CAGGBd_qEQjDHFwghs6226_jxtjV4AEDDjnoVc2duuqMizLFDBw@mail.gmail.com>

On Fri, Feb 25, 2022 at 8:12 AM BELAHCENE Abdelkader <
abdelkader.belahcene at enst.dz> wrote:

> Hi,
> a lot of people think that C (or C++) is faster than python, yes I agree,
> but I think that's not the case with numpy, I believe numpy is faster than
> C, at least in some cases.
>

This is all "last time I heard".

numpy is written, in significant part, in Fortran.

Fortran, especially for matrix math with variable dimensions, can be faster
than C.

Fortran, (still last I heard) did not support pointers, which gives Fortran
compilers the chance to exploit a very nice class of optimizations you
can't use nearly as well in languages with pointers.

I used to code C to be built with the "noalias" optimization, to get much
of the speed of Fortran in C.  But it required using an error prone subset
of C without good error detection.

From abdelkader.belahcene at enst.dz  Sat Feb 26 02:19:10 2022
From: abdelkader.belahcene at enst.dz (BELAHCENE Abdelkader)
Date: Sat, 26 Feb 2022 08:19:10 +0100
Subject: C is it always faster than nump?
In-Reply-To: <CAGGBd_qEQjDHFwghs6226_jxtjV4AEDDjnoVc2duuqMizLFDBw@mail.gmail.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <CAGGBd_qEQjDHFwghs6226_jxtjV4AEDDjnoVc2duuqMizLFDBw@mail.gmail.com>
Message-ID: <CABcwF5=aR39w2LcfS8=+FBZc4CVaRcmwLrs1uMDJ5sE0ANvEFA@mail.gmail.com>

Thanks every body,
I want to close the subject, but just a naive question:
Does numpy use  a* vectorization *for arrays?
I mean when I add 2 arrays ( or in sum function) how it is done,
in an other word
b=np.arange(100);
 t=np.sum(b)
is equivalent or not to
 s=0
for i in range(100): s +=b[i]
thanks a lot


Le sam. 26 f?vr. 2022 ? 06:44, Dan Stromberg <drsalists at gmail.com> a ?crit :

>
> On Fri, Feb 25, 2022 at 8:12 AM BELAHCENE Abdelkader <
> abdelkader.belahcene at enst.dz> wrote:
>
>> Hi,
>> a lot of people think that C (or C++) is faster than python, yes I agree,
>> but I think that's not the case with numpy, I believe numpy is faster than
>> C, at least in some cases.
>>
>
> This is all "last time I heard".
>
> numpy is written, in significant part, in Fortran.
>
> Fortran, especially for matrix math with variable dimensions, can be
> faster than C.
>
> Fortran, (still last I heard) did not support pointers, which gives
> Fortran compilers the chance to exploit a very nice class of optimizations
> you can't use nearly as well in languages with pointers.
>
> I used to code C to be built with the "noalias" optimization, to get much
> of the speed of Fortran in C.  But it required using an error prone subset
> of C without good error detection.
>
>
>

From hjp-python at hjp.at  Sat Feb 26 06:59:50 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 26 Feb 2022 12:59:50 +0100
Subject: Best way to check if there is internet?
In-Reply-To: <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <CADrxXXmBYz7+NyUG-D_pDK4peRzmsHi0+iwXjs-UWG26qg5wpA@mail.gmail.com>
Message-ID: <20220226115950.tset7kjbvcbgng2o@hjp.at>

On 2022-02-22 12:31:42 +0400, Abdur-Rahmaan Janhangeer wrote:
> A front end eng sent me this for how to check for the internet in JS
> 
> https://html.spec.whatwg.org/multipage/system-state.html#dom-navigator-online
> 
> But it also says:
> 
> "This attribute is inherently unreliable. A computer can be connected to a
> network without having Internet access."

That actually checks whether the browser is in offline mode. You can set
this mode (in Firefox at least) via File -> Work offline. The browser
will also monitor network interfaces and switch to offline mode if none
(except loopback) are up. But fundamentally it's about the state of the
browser ("... must return false if the user agent will not contact the
network ...") not about whether your computer has "internet access" in
any meaningful sense.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220226/8e5e206a/attachment.sig>

From hjp-python at hjp.at  Sat Feb 26 07:16:10 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 26 Feb 2022 13:16:10 +0100
Subject: PYT - How can I subscribe to a topic in the mailing list?
In-Reply-To: <bf328876-2176-29da-94e4-9854f166bb7f@skynet.be>
References: <deb1ec0b-e8a4-cee7-b718-eb0425ca8d4f@skynet.be>
 <trinity-98c1266a-c3a5-4533-a4bf-a30a939f79fc-1645302643381@3c-app-gmx-bap34>
 <bf328876-2176-29da-94e4-9854f166bb7f@skynet.be>
Message-ID: <20220226121610.mvk4mcssqi6b2fsd@hjp.at>

On 2022-02-21 12:19:36 +0100, vanyp wrote:
> The option to filter by topic is offered in the mailing list subscription
> customization page, although no option list is given. Topics may have been a
> project that was never implemented.

Topics are a rarely used feature of the mailman mailing list software.
The administrator can configure topics and regular expression to
recognize them. Then mailman will scan each message it received to
assign one or more topics to it. As far as I know I have never been on a
mailinglist where the administrator had configured topics. It seems like
one of those features that seemed like a good idea to the developers but
aren't really used by anyone in practice.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220226/bf7a55a6/attachment.sig>

From hjp-python at hjp.at  Sat Feb 26 09:42:16 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 26 Feb 2022 15:42:16 +0100
Subject: PYT - The expressions described in the Python language reference
 yield only boolean values
In-Reply-To: <b4f0af4a-ce4e-d8d1-4ffa-d01ee6625e1a@skynet.be>
References: <b4f0af4a-ce4e-d8d1-4ffa-d01ee6625e1a@skynet.be>
Message-ID: <20220226144216.ewvfvu2rknfn6mts@hjp.at>

On 2022-02-19 23:28:28 +0100, vanyp wrote:
> *I am trying to learn Python from the grammar given in the Python language
> reference and I am surprised.*
> 
> *Lets start here:*
> 
> *"*
> *6.3.4.?Calls*
> 
> A call calls a callable object (e.g., a function
> <https://docs.python.org/3/glossary.html#term-function>) with a possibly
> empty series of arguments
> <https://docs.python.org/3/glossary.html#term-argument>:
> 
> *call *::= |primary <https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-primary>|"("
> [|argument_list <https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-argument_list>|[","]
> | |comprehension <https://docs.python.org/3/reference/expressions.html#grammar-token-python-grammar-comprehension>|]

With all those links this is very hard to read. Please try to format
your mails in a way that makes them easy to follow.

[...][

> *The first or_test is strange, I assume it should be replaced by expression.*

Nope.

> *But even so I think that the only ways out of the recursion are the or_test
> or the lambda_expr.*
> 
> *And by the grammar, those evaluate to booleans as follows:*

No. A grammar specifies how to parse the text, it doesn't say anything
about the types of these expressions (especially not in a dynamically
typed language like python). 

> *Where did I, or the language reference, go wrong?*

First, you neglected the difference between parsing an expression and
evaluating it.

Secondly, you didn't see the way out. 

For example, let's parse the expression

?f(1+2, 3)?

Can this be a call?

To confirm this, is has to match 

    call                 ::=  primary "(" [argument_list [","] | comprehension] ")"

which looks plausible, so let's dive in:

    primary ::=  atom | attributeref | subscription | slicing | call

    atom      ::=  identifier | literal | enclosure

    identifier   ::=  xid_start xid_continue*

?f? is an identier which is an atom which is a primary.

Good so far. Now for the argument-list:

    argument_list        ::=  positional_arguments ["," starred_and_keywords] ...

    positional_arguments ::=  positional_item ("," positional_item)*

We have two comma-separated items, good so far. Check ?1+2? first

    positional_item      ::=  assignment_expression | "*" expression

    assignment_expression ::=  [identifier ":="] expression

    expression             ::=  conditional_expression | lambda_expr

    conditional_expression ::=  or_test ["if" or_test "else" expression]

    or_test  ::=  and_test | or_test "or" and_test

    and_test ::=  not_test | and_test "and" not_test

    not_test ::=  comparison | "not" not_test

    comparison    ::=  or_expr (comp_operator or_expr)*

    or_expr  ::=  xor_expr | or_expr "|" xor_expr

    xor_expr ::=  and_expr | xor_expr "^" and_expr

    and_expr ::=  shift_expr | and_expr "&" shift_expr

    shift_expr ::=  a_expr | shift_expr ("<<" | ">>") a_expr

    a_expr ::=  m_expr | a_expr "+" m_expr | a_expr "-" m_expr

Oof. That was a deep recursion, but we finally found a ?+?. 
So 1 must be an a_expr and 2 an m_expr. Actually recursing once more reveals
that both can be m_expr:

    m_expr ::=  u_expr | m_expr "*" u_expr | m_expr "@" m_expr | ...

    u_expr ::=  power | "-" u_expr | "+" u_expr | "~" u_expr

    power ::=  (await_expr | primary) ["**" u_expr]

    primary ::=  atom | attributeref | subscription | slicing | call

    atom      ::=  identifier | literal | enclosure

    literal ::=  stringliteral | bytesliteral | integer | floatnumber | imagnumber

Ok, they are both integer. 

Then we do basically the same for the second argument and we arrive at
the parse tree:

[warning: fixed width font required]

                                    call
                                      |
   -----------------------------------+--------------------------
  /        /                          |                           \
  |        |                    argument_list                      |
  |        |                    positional_arguments               |
  |        |                   /          |         \              |
  |        |       positional_item        | positional_item        |
  |        |       assignment_expression  | assignment_expression  |
  |        |       expression             | expression             |
  |        |       conditional_expression | conditional_expression |
  |        |       or_test                | or_test                |
  |        |       and_test               | and_test               |
  |        |       not_test               | not_test               |
  |        |       comparison             | comparison             |
  |        |       or_expr                | or_expr                |
  |        |       xor_expr               | xor_expr               |
  |        |       and_expr               | and_expr               |
  |        |       shift_expr             | shift_expr             |
  |        |       a_expr                 | a_expr                 |
  |        |      /  |   \                |                        |
  |        | m_expr  | m_expr             | m_expr                 |
  |        | u_expr  | u_expr             | u_expr                 |
  |        | power   | power              | power                  |
  |        | primary | primary            | primary                |
primary    | atom    | atom               | atom                   |
atom       | literal | literal            | literal                |
identifier | integer | integer            | integer                |
f          ( 1       + 2                  , 3                      )

(Of course, most real parsers start at the bottom and build up, but I
think for humans the top-down approach is more convenient.)
        hp


-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220226/139577ef/attachment.sig>

From boblatest at yahoo.com  Sat Feb 26 05:13:13 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 26 Feb 2022 10:13:13 GMT
Subject: Best way to check if there is internet?
References: <YhUHzqDyTcAijuWG@scrozzle> <YhZguEKPO/QFbsS6@shallowsky.com>
 <YhaTSeVHRDMAHLmr@scrozzle>
 <CADrxXXkg+0MBPjawN-1FqGQBCe9T43Cpi1h1fk7vqcfd8+QALg@mail.gmail.com>
 <mailman.17.1645793390.2329.python-list@python.org>
 <svaqc9$vr$2@dont-email.me> <62190963.1c69fb81.18d22.89f4@mx.google.com>
 <CADrxXXki4iVcgW-nYzjcxcn2mAwFuOnzJwOuQJPEqfczns=wOg@mail.gmail.com>
 <mailman.21.1645812487.2329.python-list@python.org>
 <svbbqo$f58$1@dont-email.me> <Yhk8g6LAoo9H8m0a@scrozzle>
 <CAPTjJmrHhuiH39CQRHw4MvVMRnuFeLVsN1znf=hZt14dezUvgw@mail.gmail.com>
 <mailman.31.1645824237.2329.python-list@python.org>
Message-ID: <j7ucppF1pppU1@mid.individual.net>

Chris Angelico wrote:
> Every language learns from every other.

Except Visual Basic, which didn't learn anything from anywhere, and all that
can be learned from it is how not to do it. Ugh.


From nddtwentyone at gmail.com  Sat Feb 26 06:19:01 2022
From: nddtwentyone at gmail.com (Neil)
Date: Sat, 26 Feb 2022 11:19:01 -0000 (UTC)
Subject: C is it always faster than nump?
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <CAGGBd_qEQjDHFwghs6226_jxtjV4AEDDjnoVc2duuqMizLFDBw@mail.gmail.com>
 <mailman.50.1645854268.2329.python-list@python.org>
Message-ID: <svd2b3$el1$1@dont-email.me>

Dan Stromberg <drsalists at gmail.com> wrote:
> On Fri, Feb 25, 2022 at 8:12 AM BELAHCENE Abdelkader <
> abdelkader.belahcene at enst.dz> wrote:
> 
>> Hi,
>> a lot of people think that C (or C++) is faster than python, yes I agree,
>> but I think that's not the case with numpy, I believe numpy is faster than
>> C, at least in some cases.
>>
> 
> This is all "last time I heard".
> 
> numpy is written, in significant part, in Fortran.
> 
> Fortran, especially for matrix math with variable dimensions, can be faster
> than C.
> 
> Fortran, (still last I heard) did not support pointers, which gives Fortran
> compilers the chance to exploit a very nice class of optimizations you
> can't use nearly as well in languages with pointers.
> 
> I used to code C to be built with the "noalias" optimization, to get much
> of the speed of Fortran in C.  But it required using an error prone subset
> of C without good error detection.

Pointers were introduced in Fortran 90.

Neil.

From wlfraed at ix.netcom.com  Sat Feb 26 13:24:12 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sat, 26 Feb 2022 13:24:12 -0500
Subject: How to solve the given problem?
References: <1323924283.239248.1644533825969@mail.yahoo.com>
 <028b56df-5ff5-4fd7-9205-eea6bdc569d7n@googlegroups.com>
 <mailman.106.1644533830.7010.python-list@python.org>
 <1d8c2edf-6d36-4af5-81d5-b60d74a951a3n@googlegroups.com>
 <vtbq0hh47arejt7m9ogncl9gig2be7hmjn@4ax.com>
 <d089e36b-99c5-43a2-a3a1-49a599f0d90dn@googlegroups.com>
 <1g1t0hdrj3tn1jqvnevicare8r2i26bhbb@4ax.com>
 <ed03b9df-c8a2-47d3-ab91-3cc538710d0en@googlegroups.com>
Message-ID: <28rk1h9q9rbougnf6djlr13eg9gd8e2b42@4ax.com>

On Sat, 26 Feb 2022 02:49:15 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>Its better to adjust the feed in the coming next two feeds that is the third and fourth one. Thirty or thirty one units in the third feed and the remaining units which are nine or ten in the fourth feed.  

	Is there a question in that?

	It's your assignment -- you will have to justify your design to whoever
gave you that assignment. So far all you have is a statement with no
justification.

	And you really do need to justify the part where you seem to ignore
"distribute the remaining 40 unit in the rest of the day" and "Try to keep
the distribution similar to the current feeding pattern."

	If you really wanted to provide a /generalized/ solution then "40
units" will never appear IN THE CODE. A generalized solution would accept
two lines of input: the original/planned schedule (for the whole day) and,
the implemented schedule up to when the mistake was detected. It would then
generate a new schedule for the rest of the day taking into account what
had already been done. (PLANNED and ACTUAL are input, ADJUSTED is output)

PLANNED:	150    100    30    30    30    20    20    10    5    5
ACTUAL:	150    60

ADJUSTED:	<changes made to rest of the day>

Note that a generalized solution would handle

PLANNED:	150    100    30    30    30    20    20    10    5    5
ACTUAL:	150    60      30

ADJUSTED:	<changes made to rest of the day>

where the mistake was on the second feed, but not discovered until after
the third feed.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From wlfraed at ix.netcom.com  Sat Feb 26 13:40:35 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sat, 26 Feb 2022 13:40:35 -0500
Subject: C is it always faster than nump?
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <CAGGBd_qEQjDHFwghs6226_jxtjV4AEDDjnoVc2duuqMizLFDBw@mail.gmail.com>
 <mailman.50.1645854268.2329.python-list@python.org>
Message-ID: <rcsk1h1tinpp57b0satj5vlrgtspb76al9@4ax.com>

On Fri, 25 Feb 2022 21:44:14 -0800, Dan Stromberg <drsalists at gmail.com>
declaimed the following:

>Fortran, (still last I heard) did not support pointers, which gives Fortran
>compilers the chance to exploit a very nice class of optimizations you
>can't use nearly as well in languages with pointers.
>
	Haven't looked much at Fortran-90/95 then... 

	Variable declaration gained a POINTER qualifier, and there is an
ALLOCATE intrinsic to obtain memory.

	And with difficulty one could get the result in DEC/VMS FORTRAN-77
since DEC implemented (across all their language compilers) intrinsics
controlling how arguments are passed -- overriding the language native
passing: 
		CALL XYZ(%val(M))
would actually pass the value of M, not Fortran default address-of, with
the result that XYZ would use that value /as/ the address of the actual
argument. (Others were %ref() and %descr() -- descriptor being a small
structure with the address reference along with, say, upper/lower bounds;
often used for strings).



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From drsalists at gmail.com  Sun Feb 27 00:35:26 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Sat, 26 Feb 2022 21:35:26 -0800
Subject: One-liner to merge lists?
In-Reply-To: <CAPTjJmovzRg0SezbKqh9sW7H=EzAn=h47WLUpW-weTmz9KGaHg@mail.gmail.com>
References: <843155aa-8536-07ab-3050-ad79b0eb281a@web.de>
 <DB6PR01MB38957B5E5C8782D98A521039833E9@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
 <CAPTjJmovzRg0SezbKqh9sW7H=EzAn=h47WLUpW-weTmz9KGaHg@mail.gmail.com>
Message-ID: <CAGGBd_o5i=s_WCx_gqJ1i2AFB+cfC9W=BpHsqOcVH9NkV78QKQ@mail.gmail.com>

On Fri, Feb 25, 2022 at 3:15 PM Chris Angelico <rosuav at gmail.com> wrote:

> But ultimately, that's still the same as sum(), and it's no more
> readable than chain(), so I'd still be inclined to go with chain and
> then wrap it in a function if you need the clarity.
>

"Need" the clarity?  Please tell me you don't think clarity is for the
feeble minded.

From rosuav at gmail.com  Sun Feb 27 01:18:40 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 27 Feb 2022 17:18:40 +1100
Subject: One-liner to merge lists?
In-Reply-To: <CAGGBd_o5i=s_WCx_gqJ1i2AFB+cfC9W=BpHsqOcVH9NkV78QKQ@mail.gmail.com>
References: <843155aa-8536-07ab-3050-ad79b0eb281a@web.de>
 <DB6PR01MB38957B5E5C8782D98A521039833E9@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
 <CAPTjJmovzRg0SezbKqh9sW7H=EzAn=h47WLUpW-weTmz9KGaHg@mail.gmail.com>
 <CAGGBd_o5i=s_WCx_gqJ1i2AFB+cfC9W=BpHsqOcVH9NkV78QKQ@mail.gmail.com>
Message-ID: <CAPTjJmrLvdsdiKHHWtm+JhzOtegKrc-Rtx-u+OOO+187p1ubRw@mail.gmail.com>

On Sun, 27 Feb 2022 at 16:35, Dan Stromberg <drsalists at gmail.com> wrote:
>
>
> On Fri, Feb 25, 2022 at 3:15 PM Chris Angelico <rosuav at gmail.com> wrote:
>>
>> But ultimately, that's still the same as sum(), and it's no more
>> readable than chain(), so I'd still be inclined to go with chain and
>> then wrap it in a function if you need the clarity.
>
>
> "Need" the clarity?  Please tell me you don't think clarity is for the feeble minded.
>

It's more about whether you consider that list(chain(...)) is a clear
and readable idiom, or you prefer to give a name to it. Not every
two-operation action needs its own name.

ChrisA

From barry at barrys-emacs.org  Sun Feb 27 03:51:36 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 27 Feb 2022 08:51:36 +0000
Subject: One-liner to merge lists?
In-Reply-To: <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
Message-ID: <B8A2DE85-B994-4040-B35E-6ED598C5FBCA@barrys-emacs.org>



> On 22 Feb 2022, at 09:30, Chris Angelico <rosuav at gmail.com> wrote:
> 
> On Tue, 22 Feb 2022 at 20:24, Frank Millman <frank at chagford.com <mailto:frank at chagford.com>> wrote:
>> 
>> Hi all
>> 
>> I think this should be a simple one-liner, but I cannot figure it out.
>> 
>> I have a dictionary with a number of keys, where each value is a single
>> list -
>> 
>>>>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>> 
>> I want to combine all values into a single list -
>> 
>>>>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>> 
>> I can do this -
>> 
>>>>> a = []
>>>>> for v in d.values():
>> ...   a.extend(v)
>> ...
>>>>> a
>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>> 
>> I can also do this -
>> 
>>>>> from itertools import chain
>>>>> a = list(chain(*d.values()))
>>>>> a
>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>>> 
>> 
>> Is there a simpler way?
>> 
> 
> itertools.chain is a good option, as it scales well to arbitrary
> numbers of lists (and you're guaranteed to iterate over them all just
> once as you construct the list). But if you know that the lists aren't
> too large or too numerous, here's another method that works:
> 
>>>> sum(d.values(), [])
> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> 
> It's simply adding all the lists together, though you have to tell it
> that you don't want a numeric summation.

If you code is performance sensitive do not use sum() as it creates lots of tmp list that are deleted.

I have an outstanding ticket at work to replace all use of sum() on lists as when we profiled it
stands out as a slow operation. We have a lots of list of list that we need to flatten.

Barry


> 
> ChrisA
> -- 
> https://mail.python.org/mailman/listinfo/python-list <https://mail.python.org/mailman/listinfo/python-list>

From soyeomul at doraji.xyz  Sun Feb 27 08:53:27 2022
From: soyeomul at doraji.xyz (Byung-Hee HWANG)
Date: Sun, 27 Feb 2022 22:53:27 +0900
Subject: ping script
In-Reply-To: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
Message-ID: <20220227135327.2807-1-soyeomul@doraji.xyz>

simple ping check script with python3 (Python 3.9.2)
tested under Debian 11 Bullseye:

soyeomul at penguin:~/gitlab/test$ ./fping.py localhost
ok
soyeomul at penguin:~/gitlab/test$ ./fping.py python.org
ok
soyeomul at penguin:~/gitlab/test$ ./fping.py python3.org
python3.org: No address associated with hostname
something is wrong...
soyeomul at penguin:~/gitlab/test$ 

Signed-off-by: Byung-Hee HWANG <soyeomul at doraji.xyz>
---
 fping.py | 31 +++++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100755 fping.py

diff --git a/fping.py b/fping.py
new file mode 100755
index 0000000..ccc1e58
--- /dev/null
+++ b/fping.py
@@ -0,0 +1,31 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+from subprocess import PIPE, Popen
+from os import path
+import sys
+
+"""
+REFERENCE:
+<https://mail.python.org/pipermail/python-list/2022-February/thread.html#905156>
+"""
+
+if not path.isfile('/usr/bin/fping'):
+    sys.exit("you first install fping, then try again...")
+    
+def check(xyz):
+    cmd = "fping %s" % (xyz)
+    try_ = Popen(cmd, stdout=PIPE, shell=True)
+    output = try_.communicate()[0].decode("utf-8")
+
+    return output
+
+
+if __name__ == "__main__":
+    xyz = sys.argv[1]
+    if "alive" in check(xyz):
+        print("ok")
+    else:
+        print("something is wrong...")
+
+# 2022-02-27, GNU Emacs 27.1 (Debian 11 Bullseye)
-- 
2.30.2


From barry at barrys-emacs.org  Sun Feb 27 10:46:53 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 27 Feb 2022 15:46:53 +0000
Subject: ping script
In-Reply-To: <20220227135327.2807-1-soyeomul@doraji.xyz>
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <20220227135327.2807-1-soyeomul@doraji.xyz>
Message-ID: <FADD0109-6145-4380-BF0F-451119867820@barrys-emacs.org>



> On 27 Feb 2022, at 13:53, Byung-Hee HWANG <soyeomul at doraji.xyz> wrote:
> 
> simple ping check script with python3 (Python 3.9.2)
> tested under Debian 11 Bullseye:
> 
> soyeomul at penguin:~/gitlab/test$ ./fping.py localhost
> ok
> soyeomul at penguin:~/gitlab/test$ ./fping.py python.org
> ok
> soyeomul at penguin:~/gitlab/test$ ./fping.py python3.org
> python3.org: No address associated with hostname
> something is wrong...

This is correct python3.org <http://python3.org/> is only setup for email.
Use the host and dig commands to check for yourself.

Compare

$ host python.org <http://python.org/>

with

$ host python3.org

And compare:

$ dig -t A python.org <http://python.org/>
$ dig -t MX python.org <http://python.org/>

with

$ dig -t A python3.org <http://python.org/>
$ dig -t MX python3.org <http://python.org/>

Barry

> soyeomul at penguin:~/gitlab/test$ 
> 
> Signed-off-by: Byung-Hee HWANG <soyeomul at doraji.xyz>
> ---
> fping.py | 31 +++++++++++++++++++++++++++++++
> 1 file changed, 31 insertions(+)
> create mode 100755 fping.py
> 
> diff --git a/fping.py b/fping.py
> new file mode 100755
> index 0000000..ccc1e58
> --- /dev/null
> +++ b/fping.py
> @@ -0,0 +1,31 @@
> +#!/usr/bin/env python3
> +# -*- coding: utf-8 -*-
> +
> +from subprocess import PIPE, Popen
> +from os import path
> +import sys
> +
> +"""
> +REFERENCE:
> +<https://mail.python.org/pipermail/python-list/2022-February/thread.html#905156>
> +"""
> +
> +if not path.isfile('/usr/bin/fping'):
> +    sys.exit("you first install fping, then try again...")
> +    
> +def check(xyz):
> +    cmd = "fping %s" % (xyz)
> +    try_ = Popen(cmd, stdout=PIPE, shell=True)
> +    output = try_.communicate()[0].decode("utf-8")
> +
> +    return output
> +
> +
> +if __name__ == "__main__":
> +    xyz = sys.argv[1]
> +    if "alive" in check(xyz):
> +        print("ok")
> +    else:
> +        print("something is wrong...")
> +
> +# 2022-02-27, GNU Emacs 27.1 (Debian 11 Bullseye)
> -- 
> 2.30.2
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From python at mrabarnett.plus.com  Sun Feb 27 11:20:12 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 27 Feb 2022 16:20:12 +0000
Subject: One-liner to merge lists?
In-Reply-To: <B8A2DE85-B994-4040-B35E-6ED598C5FBCA@barrys-emacs.org>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
 <B8A2DE85-B994-4040-B35E-6ED598C5FBCA@barrys-emacs.org>
Message-ID: <11886612-41aa-c93e-bfb7-119f1b38da1e@mrabarnett.plus.com>

On 2022-02-27 08:51, Barry Scott wrote:
> 
> 
>> On 22 Feb 2022, at 09:30, Chris Angelico <rosuav at gmail.com> wrote:
>> 
>> On Tue, 22 Feb 2022 at 20:24, Frank Millman <frank at chagford.com <mailto:frank at chagford.com>> wrote:
>>> 
>>> Hi all
>>> 
>>> I think this should be a simple one-liner, but I cannot figure it out.
>>> 
>>> I have a dictionary with a number of keys, where each value is a single
>>> list -
>>> 
>>>>>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>>> 
>>> I want to combine all values into a single list -
>>> 
>>>>>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>> 
>>> I can do this -
>>> 
>>>>>> a = []
>>>>>> for v in d.values():
>>> ...   a.extend(v)
>>> ...
>>>>>> a
>>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>> 
>>> I can also do this -
>>> 
>>>>>> from itertools import chain
>>>>>> a = list(chain(*d.values()))
>>>>>> a
>>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>>>> 
>>> 
>>> Is there a simpler way?
>>> 
>> 
>> itertools.chain is a good option, as it scales well to arbitrary
>> numbers of lists (and you're guaranteed to iterate over them all just
>> once as you construct the list). But if you know that the lists aren't
>> too large or too numerous, here's another method that works:
>> 
>>>>> sum(d.values(), [])
>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>> 
>> It's simply adding all the lists together, though you have to tell it
>> that you don't want a numeric summation.
> 
> If you code is performance sensitive do not use sum() as it creates lots of tmp list that are deleted.
> 
> I have an outstanding ticket at work to replace all use of sum() on lists as when we profiled it
> stands out as a slow operation. We have a lots of list of list that we need to flatten.
> 
I think that 'sum' uses '__add__' but not '__iadd__'.

If it copied the given value, if present, and then used '__iadd__', if 
present, wouldn't that speed it up?

From rosuav at gmail.com  Sun Feb 27 11:28:23 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 28 Feb 2022 03:28:23 +1100
Subject: One-liner to merge lists?
In-Reply-To: <11886612-41aa-c93e-bfb7-119f1b38da1e@mrabarnett.plus.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
 <B8A2DE85-B994-4040-B35E-6ED598C5FBCA@barrys-emacs.org>
 <11886612-41aa-c93e-bfb7-119f1b38da1e@mrabarnett.plus.com>
Message-ID: <CAPTjJmqeQuHVRAtHgw+Yw35S1t9CevT7YCwoOSVZcb1r72JSaQ@mail.gmail.com>

On Mon, 28 Feb 2022 at 03:24, MRAB <python at mrabarnett.plus.com> wrote:
>
> On 2022-02-27 08:51, Barry Scott wrote:
> >
> >
> >> On 22 Feb 2022, at 09:30, Chris Angelico <rosuav at gmail.com> wrote:
> >>
> >> On Tue, 22 Feb 2022 at 20:24, Frank Millman <frank at chagford.com <mailto:frank at chagford.com>> wrote:
> >>>
> >>> Hi all
> >>>
> >>> I think this should be a simple one-liner, but I cannot figure it out.
> >>>
> >>> I have a dictionary with a number of keys, where each value is a single
> >>> list -
> >>>
> >>>>>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
> >>>
> >>> I want to combine all values into a single list -
> >>>
> >>>>>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>>
> >>> I can do this -
> >>>
> >>>>>> a = []
> >>>>>> for v in d.values():
> >>> ...   a.extend(v)
> >>> ...
> >>>>>> a
> >>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>>
> >>> I can also do this -
> >>>
> >>>>>> from itertools import chain
> >>>>>> a = list(chain(*d.values()))
> >>>>>> a
> >>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>>>>>
> >>>
> >>> Is there a simpler way?
> >>>
> >>
> >> itertools.chain is a good option, as it scales well to arbitrary
> >> numbers of lists (and you're guaranteed to iterate over them all just
> >> once as you construct the list). But if you know that the lists aren't
> >> too large or too numerous, here's another method that works:
> >>
> >>>>> sum(d.values(), [])
> >> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
> >>
> >> It's simply adding all the lists together, though you have to tell it
> >> that you don't want a numeric summation.
> >
> > If you code is performance sensitive do not use sum() as it creates lots of tmp list that are deleted.
> >
> > I have an outstanding ticket at work to replace all use of sum() on lists as when we profiled it
> > stands out as a slow operation. We have a lots of list of list that we need to flatten.
> >
> I think that 'sum' uses '__add__' but not '__iadd__'.
>
> If it copied the given value, if present, and then used '__iadd__', if
> present, wouldn't that speed it up?

It's hardly relevant. If you care about speed, use chain(), like
everyone's been saying all along :)

ChrisA

From morphex at gmail.com  Sun Feb 27 05:16:56 2022
From: morphex at gmail.com (Morten W. Petersen)
Date: Sun, 27 Feb 2022 11:16:56 +0100
Subject: Timezone for datetime.date objects
Message-ID: <CAEtLV=pV-TP6WuBvoLYNfReHUyjKaJqpRqj0W8kkjeT5eLMJdQ@mail.gmail.com>

Hi.

I'm working on a project where I need to query data for a given year, in
the correct timezone.  For accounting purposes.

The code I worked on today is here, see also the version history:

https://github.com/morphex/ethereum-classic-taxman/blob/main/generate_csv.py

I was initially using the date object to get the right timespan, but then
found that using the right timezone with that was a bit of a pain.  So I
went for the datetime object instead, specifying 0 on hour, minute and
second.

What's the thinking behind this with the date object?  Wouldn't it be nice
to be able to specify a timezone?

Regards,

Morten

-- 
I am https://leavingnorway.info
Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue
Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen
Also playing music and podcasting here:
http://www.mixcloud.com/morten-w-petersen/
On Google+ here https://plus.google.com/107781930037068750156
On Instagram at https://instagram.com/morphexx/

From philip.bloom at applovin.com  Sun Feb 27 11:34:21 2022
From: philip.bloom at applovin.com (Philip Bloom)
Date: Sun, 27 Feb 2022 08:34:21 -0800
Subject: Getting Syslog working on OSX Monterey
Message-ID: <CADoNktU6JPX00wDf9asFVr=NpTAy1ruRn_pHnLunc91M9_sokA@mail.gmail.com>

Hello,

First time mailing and looking for help/guidance.  Hopefully not too in the
wrong place.

We've had an app with logging to a /var/log for many years through
logging.sysloghandler.  Recently, though, I noticed that it suddenly was
getting no logs whatsoever over there and investigated, believing the
recent triggering change was upgrading to Mac OSX 12 (Monterey).

My understanding is a bit primitive, but our config had never been hard to
follow previously and I may be missing something fundamental.  I've looked
over a number of online examples but they don't seem to be running
correctly either.  I'm on Python 3.6.8 (though we're about to start an
upgrade to the latest stable, if that in theory may have changes to syslog
handling).  I'm mostly ending up here since I'm finding such differences in
response between python modules I'd expect to be fairly similar.

Separating into a test script I've been testing with logging.SysLogHandler,
syslog, and the command line 'syslog -s'.  Replaced all below with a
placeholder appName but it's normally a fairly unique setupCDNGUI name.

Example with syslog (python module):  - This will show up in System.log but
not in Apple's Console for all messages to device, though won't get all the
way to file.
import syslog
syslog.syslog(syslog.LOG_NOTICE, 'AppName: THIS IS A TEST - Processing
started')
syslog.syslog(syslog.LOG_ERR, 'AppName: THIS IS A TEST ERROR - Processing
started')

Example with SyslogHandler: - This doesn't seem to show up anywhere besides
the screen log, which is odd to me as reading the docs, I understood this
to be a logging integration into the syslog module, so expected similar
behavior in at least what was sent out.  Syslog handler is definitely
picked up in logging.handlers

root_logger = logging.getLogger()
root_logger.setLevel(logging.DEBUG)
basic_datefmt = '%m/%d/%Y %I:%M:%S %p'

formatter = logging.Formatter(fmt='%(asctime)s %(levelname)s: %(message)s',
datefmt=basic_datefmt)
syslog_format = logging.Formatter(fmt='SetupCDNGUI: %(message)s',
datefmt=basic_datefmt)

scrhandler = logging.StreamHandler()
scrhandler.setFormatter(formatter)
root_logger.addHandler(scrhandler)

sys_handler = SysLogHandler(address='/var/run/syslog')
#sys_handler.encodePriority(SysLogHandler.LOG_USER, SysLogHandler.LOG_ALERT)
# Tried with the above, but didn't make a difference.  Neither did not
defining the address and letting it go to local host.
sys_handler.setFormatter(syslog_format)
root_logger.addHandler(sys_handler)

logging.critical("This is a test")
logging.error("This is a test error")
logging.info("Still a test")
logging.debug("Testy test")

Using command line: Oddly, this gets to Console.Apps messages from device
reliably, though doesn't get picked up by syslog -w or get received by the
ASL config redirect:
syslog -s -l error -k Message "appName: this is a test2"
syslog -s -l notice -k Message "appName: this is a test3"

ASL configuration, which is loaded according to syslog -config:
> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
file_max=50M all_max=500M
? [CA= Sender appName] file /var/log/appName/appName.log

My end goal is really to get just a working python logging ->
var/log/appname/appname.log again so glad to just be pointed in the right
direction if way off base.

-- 
Philip Bloom
Director, Services Engineering
*AppLovin Corporation*
M: (786)-338-1439 <786-338-1439>
[image: LinkedIn] <https://www.linkedin.com/company/applovin> [image:
Twitter] <https://twitter.com/AppLovin> [image: Facebook]
<https://facebook.com/AppLovin> [image: Instagram]
<https://www.instagram.com/applovin/>
[image: AppLovin] <https://www.applovin.com/>

From wlfraed at ix.netcom.com  Sun Feb 27 14:17:39 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sun, 27 Feb 2022 14:17:39 -0500
Subject: Getting Syslog working on OSX Monterey
References: <CADoNktU6JPX00wDf9asFVr=NpTAy1ruRn_pHnLunc91M9_sokA@mail.gmail.com>
 <mailman.63.1645986822.2329.python-list@python.org>
Message-ID: <ddjn1h1lja4p0q95llcah2ii6290r7h2e5@4ax.com>

On Sun, 27 Feb 2022 08:34:21 -0800, Philip Bloom
<philip.bloom at applovin.com> declaimed the following:

>
>sys_handler = SysLogHandler(address='/var/run/syslog')

	As I read the documentation, the address is supposed to be the NETWORK
address of the machine to receive the log messages...

https://docs.python.org/3/library/logging.handlers.html#sysloghandler

>#sys_handler.encodePriority(SysLogHandler.LOG_USER, SysLogHandler.LOG_ALERT)
># Tried with the above, but didn't make a difference.  Neither did not
>defining the address and letting it go to local host.
>sys_handler.setFormatter(syslog_format)
>root_logger.addHandler(sys_handler)
>
>logging.critical("This is a test")
>logging.error("This is a test error")
>logging.info("Still a test")
>logging.debug("Testy test")
>
>Using command line: Oddly, this gets to Console.Apps messages from device
>reliably, though doesn't get picked up by syslog -w or get received by the
>ASL config redirect:
>syslog -s -l error -k Message "appName: this is a test2"
>syslog -s -l notice -k Message "appName: this is a test3"
>
>ASL configuration, which is loaded according to syslog -config:
>> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
>file_max=50M all_max=500M
>? [CA= Sender appName] file /var/log/appName/appName.log
>
>My end goal is really to get just a working python logging ->
>var/log/appname/appname.log again so glad to just be pointed in the right
>direction if way off base.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From philip.bloom at applovin.com  Sun Feb 27 15:17:53 2022
From: philip.bloom at applovin.com (Philip Bloom)
Date: Sun, 27 Feb 2022 12:17:53 -0800
Subject: Getting Syslog working on OSX Monterey
In-Reply-To: <ddjn1h1lja4p0q95llcah2ii6290r7h2e5@4ax.com>
References: <CADoNktU6JPX00wDf9asFVr=NpTAy1ruRn_pHnLunc91M9_sokA@mail.gmail.com>
 <mailman.63.1645986822.2329.python-list@python.org>
 <ddjn1h1lja4p0q95llcah2ii6290r7h2e5@4ax.com>
Message-ID: <CADoNktW9vS_Xi7sba=DtRAGz7Qziro4ZMM9OLbJoJDyq6h4YZA@mail.gmail.com>

Thanks.  Had tried it with no address, which defaults to ('localhost',
'514') as well as address='/var/run/syslog' which had been working
previously, and the doc recommends as:
For example, on Linux it?s usually ?/dev/log? but on OS/X it?s
?/var/run/syslog?. You?ll need to check your platform and use the
appropriate address (you may need to do this check at runtime if your
application needs to run on several platforms)

Aiming for this to be being picked up on the same computer I'm running the
test script (Python -> Logging -> SyslogHandler -> Mac OS Syslogd -> parsed
ASL config -> /var/log/Appname/Appname.log).   It's why I think I may be
missing something fundamental, but it feels like something subtle changed
in the latest OSX.

On Sun, Feb 27, 2022 at 11:26 AM Dennis Lee Bieber <wlfraed at ix.netcom.com>
wrote:

> On Sun, 27 Feb 2022 08:34:21 -0800, Philip Bloom
> <philip.bloom at applovin.com> declaimed the following:
>
> >
> >sys_handler = SysLogHandler(address='/var/run/syslog')
>
>         As I read the documentation, the address is supposed to be the
> NETWORK
> address of the machine to receive the log messages...
>
> https://docs.python.org/3/library/logging.handlers.html#sysloghandler
>
> >#sys_handler.encodePriority(SysLogHandler.LOG_USER,
> SysLogHandler.LOG_ALERT)
> ># Tried with the above, but didn't make a difference.  Neither did not
> >defining the address and letting it go to local host.
> >sys_handler.setFormatter(syslog_format)
> >root_logger.addHandler(sys_handler)
> >
> >logging.critical("This is a test")
> >logging.error("This is a test error")
> >logging.info("Still a test")
> >logging.debug("Testy test")
> >
> >Using command line: Oddly, this gets to Console.Apps messages from device
> >reliably, though doesn't get picked up by syslog -w or get received by the
> >ASL config redirect:
> >syslog -s -l error -k Message "appName: this is a test2"
> >syslog -s -l notice -k Message "appName: this is a test3"
> >
> >ASL configuration, which is loaded according to syslog -config:
> >> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
> >file_max=50M all_max=500M
> >? [CA= Sender appName] file /var/log/appName/appName.log
> >
> >My end goal is really to get just a working python logging ->
> >var/log/appname/appname.log again so glad to just be pointed in the right
> >direction if way off base.
>
>
> --
>         Wulfraed                 Dennis Lee Bieber         AF6VN
>         wlfraed at ix.netcom.com
> http://wlfraed.microdiversity.freeddns.org/
> --
> https://mail.python.org/mailman/listinfo/python-list
>


-- 
Philip Bloom
Director, Services Engineering
*AppLovin Corporation*
M: (786)-338-1439 <786-338-1439>
[image: LinkedIn] <https://www.linkedin.com/company/applovin> [image:
Twitter] <https://twitter.com/AppLovin> [image: Facebook]
<https://facebook.com/AppLovin> [image: Instagram]
<https://www.instagram.com/applovin/>
[image: AppLovin] <https://www.applovin.com/>

From wlfraed at ix.netcom.com  Sun Feb 27 15:02:03 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sun, 27 Feb 2022 15:02:03 -0500
Subject: Getting Syslog working on OSX Monterey
References: <CADoNktU6JPX00wDf9asFVr=NpTAy1ruRn_pHnLunc91M9_sokA@mail.gmail.com>
 <mailman.63.1645986822.2329.python-list@python.org>
 <ddjn1h1lja4p0q95llcah2ii6290r7h2e5@4ax.com>
Message-ID: <s4mn1htd6fkar0ogscjtm7g0ev24tus6vc@4ax.com>

On Sun, 27 Feb 2022 14:17:39 -0500, Dennis Lee Bieber
<wlfraed at ix.netcom.com> declaimed the following:

	APOLOGIES -- I thought I KILLED the draft message, not sent it...


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From ethan at stoneleaf.us  Sun Feb 27 16:10:24 2022
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sun, 27 Feb 2022 13:10:24 -0800
Subject: Aw: PYT - How can I subscribe to a topic in the mailing list?
In-Reply-To: <bf328876-2176-29da-94e4-9854f166bb7f@skynet.be>
References: <deb1ec0b-e8a4-cee7-b718-eb0425ca8d4f@skynet.be>
 <trinity-98c1266a-c3a5-4533-a4bf-a30a939f79fc-1645302643381@3c-app-gmx-bap34>
 <bf328876-2176-29da-94e4-9854f166bb7f@skynet.be>
Message-ID: <156a9e9d-40f5-7a1b-dfd0-a26caf2391a1@stoneleaf.us>

On 2/21/22 3:19 AM, vanyp wrote:

 > The option to filter by topic is offered in the mailing list subscription customization page, although
 > no option list is given. Topics may have been a project that was never implemented.

Topics are not currently enabled.  I suspect for them to be useful, people would have to be diligent about setting them, 
which, quite frankly, I don't see happening -- some don't even set a useful subject line.

--
~Ethan~

From __peter__ at web.de  Sun Feb 27 16:49:24 2022
From: __peter__ at web.de (Peter Otten)
Date: Sun, 27 Feb 2022 22:49:24 +0100
Subject: One-liner to merge lists?
In-Reply-To: <CAPTjJmqeQuHVRAtHgw+Yw35S1t9CevT7YCwoOSVZcb1r72JSaQ@mail.gmail.com>
References: <c42a2547-e35b-51e9-110b-d2adda7b9fd9@chagford.com>
 <CAPTjJmp_U06NVZC8kfh6_gznU_aZBA9tHsc40rrK+4KHjBLdxg@mail.gmail.com>
 <B8A2DE85-B994-4040-B35E-6ED598C5FBCA@barrys-emacs.org>
 <11886612-41aa-c93e-bfb7-119f1b38da1e@mrabarnett.plus.com>
 <CAPTjJmqeQuHVRAtHgw+Yw35S1t9CevT7YCwoOSVZcb1r72JSaQ@mail.gmail.com>
Message-ID: <59a72dd9-3c1e-3fc5-bcc3-31f6419855b0@web.de>

On 27/02/2022 17:28, Chris Angelico wrote:
> On Mon, 28 Feb 2022 at 03:24, MRAB <python at mrabarnett.plus.com> wrote:
>>
>> On 2022-02-27 08:51, Barry Scott wrote:
>>>
>>>
>>>> On 22 Feb 2022, at 09:30, Chris Angelico <rosuav at gmail.com> wrote:
>>>>
>>>> On Tue, 22 Feb 2022 at 20:24, Frank Millman <frank at chagford.com <mailto:frank at chagford.com>> wrote:
>>>>>
>>>>> Hi all
>>>>>
>>>>> I think this should be a simple one-liner, but I cannot figure it out.
>>>>>
>>>>> I have a dictionary with a number of keys, where each value is a single
>>>>> list -
>>>>>
>>>>>>>> d = {1: ['aaa', 'bbb', 'ccc'], 2: ['fff', 'ggg']}
>>>>>
>>>>> I want to combine all values into a single list -
>>>>>
>>>>>>>> ans = ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>>>
>>>>> I can do this -
>>>>>
>>>>>>>> a = []
>>>>>>>> for v in d.values():
>>>>> ...   a.extend(v)
>>>>> ...
>>>>>>>> a
>>>>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>>>
>>>>> I can also do this -
>>>>>
>>>>>>>> from itertools import chain
>>>>>>>> a = list(chain(*d.values()))
>>>>>>>> a
>>>>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>>>>>>
>>>>>
>>>>> Is there a simpler way?
>>>>>
>>>>
>>>> itertools.chain is a good option, as it scales well to arbitrary
>>>> numbers of lists (and you're guaranteed to iterate over them all just
>>>> once as you construct the list). But if you know that the lists aren't
>>>> too large or too numerous, here's another method that works:
>>>>
>>>>>>> sum(d.values(), [])
>>>> ['aaa', 'bbb', 'ccc', 'fff', 'ggg']
>>>>
>>>> It's simply adding all the lists together, though you have to tell it
>>>> that you don't want a numeric summation.
>>>
>>> If you code is performance sensitive do not use sum() as it creates lots of tmp list that are deleted.
>>>
>>> I have an outstanding ticket at work to replace all use of sum() on lists as when we profiled it
>>> stands out as a slow operation. We have a lots of list of list that we need to flatten.
>>>
>> I think that 'sum' uses '__add__' but not '__iadd__'.
>>
>> If it copied the given value, if present, and then used '__iadd__', if
>> present, wouldn't that speed it up?
>
> It's hardly relevant. If you care about speed, use chain(), like
> everyone's been saying all along :)

Not everyone. I believe (*) that __iadd__ is faster, and again, you can
spell it

functools.reduce(operator.iconcat, list_of_lists, [])

(*) Actual measurements left as an exercise ;)


From cs at cskk.id.au  Sun Feb 27 16:17:57 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Mon, 28 Feb 2022 08:17:57 +1100
Subject: Timezone for datetime.date objects
In-Reply-To: <CAEtLV=pV-TP6WuBvoLYNfReHUyjKaJqpRqj0W8kkjeT5eLMJdQ@mail.gmail.com>
References: <CAEtLV=pV-TP6WuBvoLYNfReHUyjKaJqpRqj0W8kkjeT5eLMJdQ@mail.gmail.com>
Message-ID: <YhvqhXTRobKsoECi@cskk.homeip.net>

On 27Feb2022 11:16, Morten W. Petersen <morphex at gmail.com> wrote:
>I was initially using the date object to get the right timespan, but 
>then
>found that using the right timezone with that was a bit of a pain.  So I
>went for the datetime object instead, specifying 0 on hour, minute and
>second.
>
>What's the thinking behind this with the date object?  Wouldn't it be nice
>to be able to specify a timezone?

This has come up before. My own opinion is that no, it would be a bad 
idea. You're giving subday resolution to an object which is inherently 
"days". Leaving aside the many complications it brings (compare two 
dates, now requiring timezone context?) you've already hit on the easy 
and simple solution: datetimes.

I'd even go so far as to suggest that if you needed a timezone for 
precision, then dates are the _wrong_ precision to work in.

Anyway, personally I am -1 on the idea.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From rosuav at gmail.com  Sun Feb 27 17:09:08 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 28 Feb 2022 09:09:08 +1100
Subject: Timezone for datetime.date objects
In-Reply-To: <YhvqhXTRobKsoECi@cskk.homeip.net>
References: <CAEtLV=pV-TP6WuBvoLYNfReHUyjKaJqpRqj0W8kkjeT5eLMJdQ@mail.gmail.com>
 <YhvqhXTRobKsoECi@cskk.homeip.net>
Message-ID: <CAPTjJmqLXE+=A6GLLt200Zo0o-E9bS10L1N+5HGchm9DOxrR5A@mail.gmail.com>

On Mon, 28 Feb 2022 at 08:51, Cameron Simpson <cs at cskk.id.au> wrote:
>
> On 27Feb2022 11:16, Morten W. Petersen <morphex at gmail.com> wrote:
> >I was initially using the date object to get the right timespan, but
> >then
> >found that using the right timezone with that was a bit of a pain.  So I
> >went for the datetime object instead, specifying 0 on hour, minute and
> >second.
> >
> >What's the thinking behind this with the date object?  Wouldn't it be nice
> >to be able to specify a timezone?
>
> This has come up before. My own opinion is that no, it would be a bad
> idea. You're giving subday resolution to an object which is inherently
> "days". Leaving aside the many complications it brings (compare two
> dates, now requiring timezone context?) you've already hit on the easy
> and simple solution: datetimes.
>
> I'd even go so far as to suggest that if you needed a timezone for
> precision, then dates are the _wrong_ precision to work in.
>

I would agree. If you have timestamps and you're trying to determine
whether they're within a certain range, and timezones matter, then
your range is not days; it begins at a specific point in time and ends
at a specific point in time. Is that point midnight? 2AM? Start/close
of business? It could be anything, and I don't see a problem with
requiring that it be specified.

ChrisA

From barry at barrys-emacs.org  Sun Feb 27 17:16:54 2022
From: barry at barrys-emacs.org (Barry)
Date: Sun, 27 Feb 2022 22:16:54 +0000
Subject: Getting Syslog working on OSX Monterey
In-Reply-To: <CADoNktU6JPX00wDf9asFVr=NpTAy1ruRn_pHnLunc91M9_sokA@mail.gmail.com>
References: <CADoNktU6JPX00wDf9asFVr=NpTAy1ruRn_pHnLunc91M9_sokA@mail.gmail.com>
Message-ID: <0D40B574-2975-49BD-9E97-9055317169E5@barrys-emacs.org>



> On 27 Feb 2022, at 18:36, Philip Bloom via Python-list <python-list at python.org> wrote:
> 
> ?Hello,
> 
> First time mailing and looking for help/guidance.  Hopefully not too in the
> wrong place.
> 
> We've had an app with logging to a /var/log for many years through
> logging.sysloghandler.  Recently, though, I noticed that it suddenly was
> getting no logs whatsoever over there and investigated, believing the
> recent triggering change was upgrading to Mac OSX 12 (Monterey).
> 
> My understanding is a bit primitive, but our config had never been hard to
> follow previously and I may be missing something fundamental.  I've looked
> over a number of online examples but they don't seem to be running
> correctly either.  I'm on Python 3.6.8 (though we're about to start an
> upgrade to the latest stable, if that in theory may have changes to syslog
> handling).  I'm mostly ending up here since I'm finding such differences in
> response between python modules I'd expect to be fairly similar.
> 
> Separating into a test script I've been testing with logging.SysLogHandler,
> syslog, and the command line 'syslog -s'.  Replaced all below with a
> placeholder appName but it's normally a fairly unique setupCDNGUI name.
> 
> Example with syslog (python module):  - This will show up in System.log but
> not in Apple's Console for all messages to device, though won't get all the
> way to file.
> import syslog
> syslog.syslog(syslog.LOG_NOTICE, 'AppName: THIS IS A TEST - Processing
> started')
> syslog.syslog(syslog.LOG_ERR, 'AppName: THIS IS A TEST ERROR - Processing
> started')
> 
> Example with SyslogHandler: - This doesn't seem to show up anywhere besides
> the screen log, which is odd to me as reading the docs, I understood this
> to be a logging integration into the syslog module, so expected similar
> behavior in at least what was sent out.  Syslog handler is definitely
> picked up in logging.handlers
> 
> root_logger = logging.getLogger()
> root_logger.setLevel(logging.DEBUG)
> basic_datefmt = '%m/%d/%Y %I:%M:%S %p'
> 
> formatter = logging.Formatter(fmt='%(asctime)s %(levelname)s: %(message)s',
> datefmt=basic_datefmt)
> syslog_format = logging.Formatter(fmt='SetupCDNGUI: %(message)s',
> datefmt=basic_datefmt)
> 
> scrhandler = logging.StreamHandler()
> scrhandler.setFormatter(formatter)
> root_logger.addHandler(scrhandler)
> 
> sys_handler = SysLogHandler(address='/var/run/syslog')
> #sys_handler.encodePriority(SysLogHandler.LOG_USER, SysLogHandler.LOG_ALERT)
> # Tried with the above, but didn't make a difference.  Neither did not
> defining the address and letting it go to local host.
> sys_handler.setFormatter(syslog_format)
> root_logger.addHandler(sys_handler)
> 
> logging.critical("This is a test")
> logging.error("This is a test error")
> logging.info("Still a test")
> logging.debug("Testy test")
> 
> Using command line: Oddly, this gets to Console.Apps messages from device
> reliably, though doesn't get picked up by syslog -w or get received by the
> ASL config redirect:
> syslog -s -l error -k Message "appName: this is a test2"
> syslog -s -l notice -k Message "appName: this is a test3"
> 
> ASL configuration, which is loaded according to syslog -config:
>> /var/log/appName/appName.log mode=0640 compress format=std rotate=seq
> file_max=50M all_max=500M
> ? [CA= Sender appName] file /var/log/appName/appName.log
> 
> My end goal is really to get just a working python logging ->
> var/log/appname/appname.log again so glad to just be pointed in the right
> direction if way off base.

If you look at the code of the logging modules syslog handle you will see that
it does not use syslog. It?s assuming that it can network to a syslog listener.
Such a listener is not running on my systems as far as I know.

I have always assumed that if I want a logger syslog handler that I would have
to implement it myself. So far I have code that uses syslog directly and have
not written that code yet.

Barry

> 
> -- 
> Philip Bloom
> Director, Services Engineering
> *AppLovin Corporation*
> M: (786)-338-1439 <786-338-1439>
> [image: LinkedIn] <https://www.linkedin.com/company/applovin> [image:
> Twitter] <https://twitter.com/AppLovin> [image: Facebook]
> <https://facebook.com/AppLovin> [image: Instagram]
> <https://www.instagram.com/applovin/>
> [image: AppLovin] <https://www.applovin.com/>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From soyeomul at doraji.xyz  Sun Feb 27 23:03:37 2022
From: soyeomul at doraji.xyz (=?utf-8?B?7Zmp67OR7Z2s?=)
Date: Mon, 28 Feb 2022 13:03:37 +0900
Subject: ping script
In-Reply-To: <FADD0109-6145-4380-BF0F-451119867820@barrys-emacs.org> (Barry
 Scott's message of "Sun, 27 Feb 2022 15:46:53 +0000")
References: <CADrxXXmDuGo7-JmpeEwSpecL9BO-K_umudE7hOCcyC+V78UqJA@mail.gmail.com>
 <20220227135327.2807-1-soyeomul@doraji.xyz>
 <FADD0109-6145-4380-BF0F-451119867820@barrys-emacs.org>
Message-ID: <874k4j3aee.fsf@penguin>

Dear Barry,

Barry Scott <barry at barrys-emacs.org> writes:

>> [...]
> This is correct python3.org <http://python3.org/> is only setup for email.
> Use the host and dig commands to check for yourself.

It is very amazing! Thanks for explanation!

> Compare
>
> $ host python.org <http://python.org/>
>
> with
>
> $ host python3.org
>
> And compare:
>
> $ dig -t A python.org <http://python.org/>
> $ dig -t MX python.org <http://python.org/>
>
> with
>
> $ dig -t A python3.org <http://python.org/>
> $ dig -t MX python3.org <http://python.org/>
>
> Barry
>

Sincerely, Linux fan Byung-Hee

-- 
^????? _????_ ?????_^))//

From loris.bennett at fu-berlin.de  Mon Feb 28 03:41:55 2022
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Mon, 28 Feb 2022 09:41:55 +0100
Subject: When to use SQLAlchemy listen events
References: <87lexz6xul.fsf@hornfels.zedat.fu-berlin.de>
Message-ID: <87sfs38jsc.fsf@hornfels.zedat.fu-berlin.de>

"Loris Bennett" <loris.bennett at fu-berlin.de> writes:

> Hi,
>
> I am wondering whether SQLAlchemy listen events are appropriate for the
> following situation:
>
> I have a table containing users and a table for events related to users
>
>   class User(Base):
>       __tablename__ = "users"
>
>       uid = Column('uid', String(64), primary_key=True)
>       gid = Column('gid', String(64), ForeignKey('groups.gid'), nullable=False)
>       lang = Column('lang', String(2))
>
>
>   class UserEvent(Base):
>       __tablename__ = "user_events"
>
>       id = Column('id', Integer, primary_key=True)
>       date = Column('date', Date, nullable=False)
>       uid = Column('gid', String(64), ForeignKey('users.uid'), nullable=False)
>       comment = Column('comment', String(256))
>
> (There are also analogous tables for groups and group events).
>
> The functions provided by the interface are things like the following
>
>   add_user(user, group, lang)
>   move_user(user, group)
>   delete_user(user)
>   warn_user(user, reason)
>
> Whereas 'add/move/delete_user' result in changes to the table 'users',
> 'warn_user' does not.  All should produce entries in the table
> 'user_events'. 
>
> There could be more functions similar to 'warn_user' that only create an
> entry in 'user_events'.  Potentially there could be a lot more of
> these than the 'user'-table-changing type.
>
> It seems like for the first three functions, capturing the resulting
> database changes in the table 'user_events' would be a standard use-case
> for listen event.  However, the 'warn_user' function is different.
>
> So can/should I shoehorn the 'warn_user' function to being like the
> others three and use listen events, or should I just come up with my own
> mechanism which will allow any function just to add an entry to the
> events table?

So I just ended up writing my own decorator.  That seems more
appropriate and flexible in this instance.

-- 
This signature is currently under construction.

From loris.bennett at fu-berlin.de  Mon Feb 28 04:11:06 2022
From: loris.bennett at fu-berlin.de (Loris Bennett)
Date: Mon, 28 Feb 2022 10:11:06 +0100
Subject: SQLAlchemy: JSON vs. PickleType vs. raw string for serialised data
Message-ID: <87o82r8ifp.fsf@hornfels.zedat.fu-berlin.de>

Hi,

I have an SQLAlchemy class for an event:

  class UserEvent(Base):
      __tablename__ = "user_events"

      id = Column('id', Integer, primary_key=True)
      date = Column('date', Date, nullable=False)
      uid = Column('gid', String(64), ForeignKey('users.uid'), nullable=False)
      info = ??

The event may have arbitrary, but dict-like data associated with it,
which I want to add in the field 'info'.  This data never needs to be
modified, once the event has been inserted into the DB.

What type should the info field have?  JSON, PickleType, String, or
something else?

I couldn't find any really reliable sounding information about the relative
pros and cons, apart from a Reddit thread claiming that pickled dicts
are larger than dicts converted to JSON or String.

Cheers,

Loris

-- 
This signature is currently under construction.

From edmondo.giovannozzi at gmail.com  Mon Feb 28 07:20:10 2022
From: edmondo.giovannozzi at gmail.com (Edmondo Giovannozzi)
Date: Mon, 28 Feb 2022 04:20:10 -0800 (PST)
Subject: C is it always faster than nump?
In-Reply-To: <rcsk1h1tinpp57b0satj5vlrgtspb76al9@4ax.com>
References: <CABcwF5=bLWRKgv7M2CRom5LwBCLojxnEq9SYeEDrFiEn5Gfxog@mail.gmail.com>
 <CAGGBd_qEQjDHFwghs6226_jxtjV4AEDDjnoVc2duuqMizLFDBw@mail.gmail.com>
 <mailman.50.1645854268.2329.python-list@python.org>
 <rcsk1h1tinpp57b0satj5vlrgtspb76al9@4ax.com>
Message-ID: <6a3623dd-b401-462e-b27e-0b098d43a7f7n@googlegroups.com>

Il giorno sabato 26 febbraio 2022 alle 19:41:37 UTC+1 Dennis Lee Bieber ha scritto:
> On Fri, 25 Feb 2022 21:44:14 -0800, Dan Stromberg <drsa... at gmail.com> 
> declaimed the following:
> >Fortran, (still last I heard) did not support pointers, which gives Fortran 
> >compilers the chance to exploit a very nice class of optimizations you 
> >can't use nearly as well in languages with pointers. 
> >
> Haven't looked much at Fortran-90/95 then... 
> 
> Variable declaration gained a POINTER qualifier, and there is an 
> ALLOCATE intrinsic to obtain memory. 
> 
> And with difficulty one could get the result in DEC/VMS FORTRAN-77 
> since DEC implemented (across all their language compilers) intrinsics 
> controlling how arguments are passed -- overriding the language native 
> passing: 
> CALL XYZ(%val(M)) 
> would actually pass the value of M, not Fortran default address-of, with 
> the result that XYZ would use that value /as/ the address of the actual 
> argument. (Others were %ref() and %descr() -- descriptor being a small 
> structure with the address reference along with, say, upper/lower bounds; 
> often used for strings).
> -- 
> Wulfraed Dennis Lee Bieber AF6VN 
> wlf... at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

The latest Fortran revision is the 2018.
A variable can also have the VALUE attribute even though nowhere in the standard is written that it means passing the data by value. It just means that if a variable is changed in a procedure the changes don't propagate back to the caller.
With the iso_c_binding one can directly call a C function or let a Fortran procedure appear as a C function. There is the C_LOC that gives the C address of a variable if needed. Of course from 2003 it is fully object oriented.
The claim that it was faster then C is mostly related to the aliasing rule that is forbidden in Fortran. The C introduced the "restrict" qualifier for the same reason.
In Fortran you also have array operation like you have in numpy. 

From boblatest at yahoo.com  Mon Feb 28 10:05:03 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 28 Feb 2022 15:05:03 GMT
Subject: Threading question .. am I doing this right?
References: <j7paqiF2b3vU1@mid.individual.net>
 <CAPTjJmo3sn7_hoWMb+UghrNd0dJ-AimKb315hOtFqAhqWOLk3w@mail.gmail.com>
 <mailman.15.1645733047.2329.python-list@python.org>
 <j7sh95Flcr9U1@mid.individual.net>
 <CAPTjJmra6m=JjACbXoJjSJU+OCVFXsS9Pby=uY4eY5L9q8a+vQ@mail.gmail.com>
 <mailman.24.1645817480.2329.python-list@python.org>
Message-ID: <j846kvF5bk6U1@mid.individual.net>

Chris Angelico wrote:
> I'm still curious as to the workload (requests per second), as it might still
> be worth going for the feeder model. But if your current system works, then
> it may be simplest to debug that rather than change.

It is by all accounts a low-traffic situation, maybe one request/second. But
the view in question opens four plots on one page, generating four separate
requests. So with only two clients and a blocking DB connection, the whole
application with eight uwsgi worker threads comes down. Now with the "extra
load thread" modification, the app worked fine for several days with only two
threads.

Out of curiosity I tried the "feeder thread" approach with a dummy thread that
just sleeps and logs something every few seconds, ten times total. For some
reason it sometimes hangs after eight or nine loops, and then uwsgi cannot
restart gracefully probably because it is still waiting for that thread to
finish. Also my web app is built around setting up the DB connections in the
request context, so using an extra thread outside that context would require
doubling some DB infrastructure. Probably not worth it at this point.

From sjeik_appie at hotmail.com  Mon Feb 28 12:13:32 2022
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Mon, 28 Feb 2022 18:13:32 +0100
Subject: SQLAlchemy: JSON vs. PickleType vs. raw string for serialised data
In-Reply-To: <87o82r8ifp.fsf@hornfels.zedat.fu-berlin.de>
Message-ID: <DB6PR01MB3895965D87653721A0E8444183019@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   On Feb 28, 2022 10:11, Loris Bennett <loris.bennett at fu-berlin.de> wrote:

     Hi,

     I have an SQLAlchemy class for an event:

     ? class UserEvent(Base):
     ????? __tablename__ = "user_events"

     ????? id = Column('id', Integer, primary_key=True)
     ????? date = Column('date', Date, nullable=False)
     ????? uid = Column('gid', String(64), ForeignKey('users.uid'),
     nullable=False)
     ????? info = ??

     The event may have arbitrary, but dict-like data associated with it,
     which I want to add in the field 'info'.? This data never needs to be
     modified, once the event has been inserted into the DB.

     What type should the info field have?? JSON, PickleType, String, or
     something else?

     I couldn't find any really reliable sounding information about the
     relative
     pros and cons, apart from a Reddit thread claiming that pickled dicts
     are larger than dicts converted to JSON or String.

     Cheers,

     Loris

   ====
   I think you need a
   BLOB.?https://docs.sqlalchemy.org/en/14/core/type_basics.html#sqlalchemy.types.LargeBinary

From boblatest at yahoo.com  Mon Feb 28 16:28:39 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 28 Feb 2022 21:28:39 GMT
Subject: SQLAlchemy: JSON vs. PickleType vs. raw string for serialised data
References: <87o82r8ifp.fsf@hornfels.zedat.fu-berlin.de>
 <DB6PR01MB3895965D87653721A0E8444183019@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
 <mailman.71.1646068426.2329.python-list@python.org>
Message-ID: <j84t47F9lg2U1@mid.individual.net>

Albert-Jan Roskam wrote:
>      The event may have arbitrary, but dict-like data associated with it,
>      which I want to add in the field 'info'.? This data never needs to be
>      modified, once the event has been inserted into the DB.
>
>      What type should the info field have?? JSON, PickleType, String, or
>      something else?
>
>      I couldn't find any really reliable sounding information about the
>      relative
>      pros and cons, apart from a Reddit thread claiming that pickled dicts
>      are larger than dicts converted to JSON or String.

I've done exactly this. Since my data was strictly ASCII I decided to go for
JSON. But in the end you're the only one who can decide this because only you
know the data. That's why you won't find any hard and fast rule for this.

From hjp-python at hjp.at  Mon Feb 28 16:41:09 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Mon, 28 Feb 2022 22:41:09 +0100
Subject: Getting Syslog working on OSX Monterey
In-Reply-To: <0D40B574-2975-49BD-9E97-9055317169E5@barrys-emacs.org>
References: <CADoNktU6JPX00wDf9asFVr=NpTAy1ruRn_pHnLunc91M9_sokA@mail.gmail.com>
 <0D40B574-2975-49BD-9E97-9055317169E5@barrys-emacs.org>
Message-ID: <20220228214109.ntoihdobzf5naqrx@hjp.at>

On 2022-02-27 22:16:54 +0000, Barry wrote:
> If you look at the code of the logging modules syslog handle you will see that
> it does not use syslog. It?s assuming that it can network to a syslog listener.
> Such a listener is not running on my systems as far as I know.
> 
> I have always assumed that if I want a logger syslog handler that I would have
> to implement it myself. So far I have code that uses syslog directly and have
> not written that code yet.

What do you mean by using syslog directly? The syslog(3) library
function also just sends messages to a "syslog listener" (more commonly
called a syslog daemon) - at least on any unix-like system I'm familiar
with (which doesn't include MacOS). It will, however, always use the
*local* syslog daemon - AFAIK there is no standard way to open a remote
connection (many syslog daemons can be configured to forward messages to
a remote server, however).

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220228/c24b768d/attachment.sig>

From barry at barrys-emacs.org  Mon Feb 28 17:05:05 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Mon, 28 Feb 2022 22:05:05 +0000
Subject: Getting Syslog working on OSX Monterey
In-Reply-To: <20220228214109.ntoihdobzf5naqrx@hjp.at>
References: <CADoNktU6JPX00wDf9asFVr=NpTAy1ruRn_pHnLunc91M9_sokA@mail.gmail.com>
 <0D40B574-2975-49BD-9E97-9055317169E5@barrys-emacs.org>
 <20220228214109.ntoihdobzf5naqrx@hjp.at>
Message-ID: <4EAF5AC0-446D-4493-AFF0-2FD8A8889439@barrys-emacs.org>



> On 28 Feb 2022, at 21:41, Peter J. Holzer <hjp-python at hjp.at> wrote:
> 
> On 2022-02-27 22:16:54 +0000, Barry wrote:
>> If you look at the code of the logging modules syslog handle you will see that
>> it does not use syslog. It?s assuming that it can network to a syslog listener.
>> Such a listener is not running on my systems as far as I know.
>> 
>> I have always assumed that if I want a logger syslog handler that I would have
>> to implement it myself. So far I have code that uses syslog directly and have
>> not written that code yet.
> 
> What do you mean by using syslog directly? The syslog(3) library
> function also just sends messages to a "syslog listener" (more commonly
> called a syslog daemon) - at least on any unix-like system I'm familiar
> with (which doesn't include MacOS). It will, however, always use the
> *local* syslog daemon - AFAIK there is no standard way to open a remote
> connection (many syslog daemons can be configured to forward messages to
> a remote server, however).

I'm re-reading the code to check on what I'm seeing. (Its been a long
time since I last look deeply at this code).

You can write to /dev/log if you pass that to
SysLogHandler(address='/dev/log'), but the default is to use a socket
to talk to a network listener on localhost:514. There are no deamons
listening on port 514 on my Fedora systems or mac OS.

That is not what you would expect as the default if you are using the C
API.

What you do not see used in the SyslogHandler() is the import syslog
and hence its nor using openlog() etc from syslog API.

Barry



>        hp
> 
> -- 
>   _  | Peter J. Holzer    | Story must make more sense than reality.
> |_|_) |                    |
> | |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |       challenge!"
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From morphex at gmail.com  Mon Feb 28 17:28:23 2022
From: morphex at gmail.com (Morten W. Petersen)
Date: Mon, 28 Feb 2022 23:28:23 +0100
Subject: Timezone for datetime.date objects
In-Reply-To: <CAPTjJmqLXE+=A6GLLt200Zo0o-E9bS10L1N+5HGchm9DOxrR5A@mail.gmail.com>
References: <CAEtLV=pV-TP6WuBvoLYNfReHUyjKaJqpRqj0W8kkjeT5eLMJdQ@mail.gmail.com>
 <YhvqhXTRobKsoECi@cskk.homeip.net>
 <CAPTjJmqLXE+=A6GLLt200Zo0o-E9bS10L1N+5HGchm9DOxrR5A@mail.gmail.com>
Message-ID: <CAEtLV=qYmx9+n15rqbaX7i8RZPs7hnGu_prhYHqdZ671PdTVaw@mail.gmail.com>

Hi Chris, Cameron.

Well, let's say I specify the datetime 2022-02-22 02:02 (AM). I think
everyone could agree that it also means 2022-02-22 02:02:00:00, to
2022-02-22 02:02:59:59.

And I think the same applies for a date. If the pipes are clogged and I
can't take (give) a shit, a shower or do anything else involving fluids, I
can just leave the keys under the doormat, and agree a date with the
plumber, and go off to a friend of relatives' place for a couple of days
while waiting for the plumber to do the necessary work.

Usually that would imply that the plumber visits from 07:00 to 15:00 on the
given date, with an implicit timezone. It could also mean that the plumber
shows up at 01:00 at night and fixes it, or at 18:00 in the evening.

If a newspaper talks about new years celebrations, and specifically talks
about what happens on the 1st of January, this could mean at 00:01, or a
later dinner party at 20:00. But the celebration that starts at midnight,
doesn't start at the same moment all over the world.  So context, or
location and implicit timezone does matter.

I was also thinking of specifying some range objects for my needs, as that
makes sense from what I've worked with earlier, this makes sense for
example for a month or a year (or even decade) as well.

But the point is that a date is not just a flat, one-dimensional object.

Regards,

Morten

On Sun, Feb 27, 2022 at 11:11 PM Chris Angelico <rosuav at gmail.com> wrote:

> On Mon, 28 Feb 2022 at 08:51, Cameron Simpson <cs at cskk.id.au> wrote:
> >
> > On 27Feb2022 11:16, Morten W. Petersen <morphex at gmail.com> wrote:
> > >I was initially using the date object to get the right timespan, but
> > >then
> > >found that using the right timezone with that was a bit of a pain.  So I
> > >went for the datetime object instead, specifying 0 on hour, minute and
> > >second.
> > >
> > >What's the thinking behind this with the date object?  Wouldn't it be
> nice
> > >to be able to specify a timezone?
> >
> > This has come up before. My own opinion is that no, it would be a bad
> > idea. You're giving subday resolution to an object which is inherently
> > "days". Leaving aside the many complications it brings (compare two
> > dates, now requiring timezone context?) you've already hit on the easy
> > and simple solution: datetimes.
> >
> > I'd even go so far as to suggest that if you needed a timezone for
> > precision, then dates are the _wrong_ precision to work in.
> >
>
> I would agree. If you have timestamps and you're trying to determine
> whether they're within a certain range, and timezones matter, then
> your range is not days; it begins at a specific point in time and ends
> at a specific point in time. Is that point midnight? 2AM? Start/close
> of business? It could be anything, and I don't see a problem with
> requiring that it be specified.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>


-- 
I am https://leavingnorway.info
Videos at https://www.youtube.com/user/TheBlogologue
Twittering at http://twitter.com/blogologue
Blogging at http://blogologue.com
Playing music at https://soundcloud.com/morten-w-petersen
Also playing music and podcasting here:
http://www.mixcloud.com/morten-w-petersen/
On Google+ here https://plus.google.com/107781930037068750156
On Instagram at https://instagram.com/morphexx/

From greg.ewing at canterbury.ac.nz  Mon Feb 28 17:16:32 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 1 Mar 2022 11:16:32 +1300
Subject: SQLAlchemy: JSON vs. PickleType vs. raw string for serialised data
In-Reply-To: <mailman.71.1646068426.2329.python-list@python.org>
References: <87o82r8ifp.fsf@hornfels.zedat.fu-berlin.de>
 <DB6PR01MB3895965D87653721A0E8444183019@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>
 <mailman.71.1646068426.2329.python-list@python.org>
Message-ID: <j84vu3Fa7avU1@mid.individual.net>

On 1/03/22 6:13 am, Albert-Jan Roskam wrote:
>     I think you need a
>     BLOB.?https://docs.sqlalchemy.org/en/14/core/type_basics.html#sqlalchemy.types.LargeBinary

That won't help on its own, since you still need to choose a
serialisation format to store in the blob.

I'd be inclined to use JSON if the data is something that can
be easily represented that way.

-- 
Greg


From cs at cskk.id.au  Mon Feb 28 16:52:03 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 1 Mar 2022 08:52:03 +1100
Subject: SQLAlchemy: JSON vs. PickleType vs. raw string for serialised data
In-Reply-To: <87o82r8ifp.fsf@hornfels.zedat.fu-berlin.de>
References: <87o82r8ifp.fsf@hornfels.zedat.fu-berlin.de>
Message-ID: <Yh1EAwa4a6xaNtaR@cskk.homeip.net>

On 28Feb2022 10:11, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>I have an SQLAlchemy class for an event:
>
>  class UserEvent(Base):
>      __tablename__ = "user_events"
>
>      id = Column('id', Integer, primary_key=True)
>      date = Column('date', Date, nullable=False)
>      uid = Column('gid', String(64), ForeignKey('users.uid'), nullable=False)
>      info = ??
>
>The event may have arbitrary, but dict-like data associated with it,
>which I want to add in the field 'info'.  This data never needs to be
>modified, once the event has been inserted into the DB.
>
>What type should the info field have?  JSON, PickleType, String, or
>something else?

I would use JSON, it expresses dicts well provided the dicts contain 
only basic types (strings, numbers, other dicts/lists of basic types 
recursively).

I have personal problems with pickle because nonPython code can't read 
it.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From rosuav at gmail.com  Mon Feb 28 17:55:33 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 1 Mar 2022 09:55:33 +1100
Subject: Timezone for datetime.date objects
In-Reply-To: <CAEtLV=qYmx9+n15rqbaX7i8RZPs7hnGu_prhYHqdZ671PdTVaw@mail.gmail.com>
References: <CAEtLV=pV-TP6WuBvoLYNfReHUyjKaJqpRqj0W8kkjeT5eLMJdQ@mail.gmail.com>
 <YhvqhXTRobKsoECi@cskk.homeip.net>
 <CAPTjJmqLXE+=A6GLLt200Zo0o-E9bS10L1N+5HGchm9DOxrR5A@mail.gmail.com>
 <CAEtLV=qYmx9+n15rqbaX7i8RZPs7hnGu_prhYHqdZ671PdTVaw@mail.gmail.com>
Message-ID: <CAPTjJmrNOrqG+FD7V6SjVAr-t2713LOzmqjMv0OkE6MVJ2K+GQ@mail.gmail.com>

On Tue, 1 Mar 2022 at 09:28, Morten W. Petersen <morphex at gmail.com> wrote:
>
> Hi Chris, Cameron.
>
> Well, let's say I specify the datetime 2022-02-22 02:02 (AM). I think everyone could agree that it also means 2022-02-22 02:02:00:00, to 2022-02-22 02:02:59:59.
>

Not sure how many :59s you want there :) I'm going to assume you mean
"02:02:00" to "02:02:59".

> And I think the same applies for a date. If the pipes are clogged and I can't take (give) a shit, a shower or do anything else involving fluids, I can just leave the keys under the doormat, and agree a date with the plumber, and go off to a friend of relatives' place for a couple of days while waiting for the plumber to do the necessary work.
>

That is one of the fundamental differences between humans and
computers. Humans are very VERY sloppy with time descriptions. With
computers, it's much better to be clear about time ranges; a time does
not imply a specific window size. (And autistic humans are more like
computers.)

> Usually that would imply that the plumber visits from 07:00 to 15:00 on the given date, with an implicit timezone. It could also mean that the plumber shows up at 01:00 at night and fixes it, or at 18:00 in the evening.
>

Around here, that means the plumber might visit between 09:00 and
17:00, but also might not. Humans are sloppy.

> If a newspaper talks about new years celebrations, and specifically talks about what happens on the 1st of January, this could mean at 00:01, or a later dinner party at 20:00. But the celebration that starts at midnight, doesn't start at the same moment all over the world.  So context, or location and implicit timezone does matter.
>

Yes, it matters... and you can't depend on newspapers to get these
things correct, because humans are sloppy. For instance, a natural
disaster in some other place in the world might be reported with the
timezone where it happened ("the hurricane that hit New Orleans on
Thursday"), but also might use the timezone where the newspaper is
being published ("last night, a hurricane devastated New Orleans"). So
the implicit timezone matters, but is also very unclear.

> I was also thinking of specifying some range objects for my needs, as that makes sense from what I've worked with earlier, this makes sense for example for a month or a year (or even decade) as well.
>
> But the point is that a date is not just a flat, one-dimensional object.
>

If your needs require a range, then use a range (or a pair of
datetimes). A date does not imply a range.

Side point: It's actually not that uncommon for a day to start or end
at some time other than midnight. For instance, here's the timetable
for one of Melbourne's railway lines:

https://d309ul1fvo6zfp.cloudfront.net/1645676503802/train-4-2022-02-28-2022-03-02.pdf

The last outbound train on a Friday night (scroll down to page 8)
departs at 1:20AM from Dandenong Station and arrives at 1:34AM at
Cranbourne Station. It's the same train that departed Westall Station
at 1:03AM. Whichever way you measure it, that service most definitely
runs entirely after midnight, but it counts as a Friday service, not a
Saturday one. So if I were to ask you "how many services run on a
Friday?", you should count this one, regardless of what your phone
says the day is. The best way to define the day would be 3AM to 3AM.

But that's the railways. Here's a NightRider bus timetable:

https://d309ul1fvo6zfp.cloudfront.net/1645676503802/bus-15131-2021-10-29-2022-12-31.pdf

How many services does THIS route run on a Saturday? Four or five
(depending on direction). It would be best to define THIS day from
midnight, or maybe even noon. The definition is context-sensitive, so
if you need a range, *be explicit*. It's okay for the range to be
implicit to your users, but make it explicit in your code:

day = datetime.datetime(y, m, d, 4, 0, 0), datetime.timedelta(days=1)

Python doesn't have a datetimerange object, so I'd use either a
datetime,timedelta pair, or a tuple of two datetimes. In some
contexts, it might be safe to let the timedelta be implicit, but at
very least, it's worth being clear that "this day" means "starting at
4AM on this day", or whatever it be.

Even when you describe a minute, it is most definitely not 100% clear
whether it means 02:02:00 to 02:03:00 (inclusive/exclusive), 02:02:00
to 02:02:01 (inc/exc), or the exact instant at 02:02:00. All three are
valid meanings, and a full timerange is the only way to be clear.

ChrisA