From avigross at verizon.net  Sat Jan  1 00:18:16 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 1 Jan 2022 00:18:16 -0500
Subject: builtins.TypeError: catching classes that do not inherit from
 BaseException is not allowed
In-Reply-To: <Yc7y2uhvqI9BTxhv@hermes.hilbert.loc>
References: <CAPTjJmq22Xez9SuNys971C_xmX+xE70HfxOSX_AVqVWeuoNOkA@mail.gmail.com>
 <d7cd019a-9a61-43e5-a3aa-8256a528580bn@googlegroups.com>
 <mailman.70.1640877797.3079.python-list@python.org>
 <CAPTjJmodUF0_pZjpSVPa6RtuYuCUVOcJu69nY-yqU8etqT2v2g@mail.gmail.com>
 <170b346e-2923-4519-b337-1db7b1978684n@googlegroups.com>
 <mailman.73.1640905447.3079.python-list@python.org>
 <2218d5fc-8862-9525-4c75-c2225e6247ac@mrabarnett.plus.com>
 <df8487f1-2a28-48a1-9c9d-dafae37abf26n@googlegroups.com>
 <mailman.75.1640908231.3079.python-list@python.org>
 <661ece76-fe30-49a9-85be-efc70d7dc395n@googlegroups.com>
 <Yc7y2uhvqI9BTxhv@hermes.hilbert.loc>
Message-ID: <055a01d7fece$fbb721c0$f3256540$@verizon.net>

I am sure some people have a sense of humor, but anyone on this forum who
actually does not have some idea of what various "tree" data structures are
in computer science, probably won't get any replies from me when asking such
questions.

But indeed there are things closer to classical trees that are traversed in
various ways including depth-first versus breadth first. Some have special
names like a parse tree or a decision tree and they can even be collected
into a random forest.

But fundamentally, the idea of using a recursion that at each node may take
multiple next steps by calling an instance of itself at the new node, is
fairly common, as are using it to search many kinds of data structures.

The discussion though suggests that no single idea about computer languages
need be used constantly and sometimes recursion is not the only method or
even the best method. You can design tree structures in ways that allow them
to be traversed in an iterative way such as having bi-directional links
along with flags that mark where you entered a node from or already visited.
Tools should be tools, not religions. 

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Karsten Hilbert
Sent: Friday, December 31, 2021 7:09 AM
To: python-list at python.org
Subject: Re: builtins.TypeError: catching classes that do not inherit from
BaseException is not allowed

Am Thu, Dec 30, 2021 at 03:57:25PM -0800 schrieb hongy... at gmail.com:

> > > Then what cases/scenarios can demonstrate the beauty of recursion?
> > >
> > Walking a tree.
>
> There are many type of trees. Do you mean all of them?

Palm trees don't lend themselves to recursion all that much.

Karsten
--
GPG  40BE 5B0E C98E 1713 AFA6  5BC0 3BEA AC80 7D4F C89B
-- 
https://mail.python.org/mailman/listinfo/python-list


From barry at barrys-emacs.org  Sat Jan  1 08:09:16 2022
From: barry at barrys-emacs.org (Barry)
Date: Sat, 1 Jan 2022 13:09:16 +0000
Subject: recover pickled data: pickle data was truncated
In-Reply-To: <1549b721-f6b6-4754-a5ff-ead72eb78d3an@googlegroups.com>
References: <1549b721-f6b6-4754-a5ff-ead72eb78d3an@googlegroups.com>
Message-ID: <E4782BE5-1B73-46E2-9066-9EF5F255902B@barrys-emacs.org>



> On 31 Dec 2021, at 17:53, iMath <redstone-cold at 163.com> wrote:
> 
> ?? 2021?12?30???? UTC+8 03:13:21?<Marco Sulla> ???
>>> On Wed, 29 Dec 2021 at 18:33, iMath <redsto... at 163.com> wrote: 
>>> But I found the size of the file of the shelve data didn't change much, so I guess the data are still in it , I just wonder any way to recover my data.
>> I agree with Barry, Chris and Avi. IMHO your data is lost. Unpickling 
>> it by hand is a harsh work and maybe unreliable. 
>> 
>> Is there any reason you can't simply add a semaphore to avoid writing 
>> at the same time and re-run the code and regenerate the data?
> 
> Thanks for your replies! I didn't have a sense of adding a semaphore on writing to pickle data before, so  corrupted the data.
> Since my data was colleted in the daily usage, so cannot re-run the code and regenerate the data.
> In order to avoid corrupting my data again and the complicity of using  a semaphore, now I am using json text to store my data.

That will not fix the problem. You will end up with corrupt json.

If you have one writer and one read then may be you can use the fact that a rename is atomic.

Writer does this:
1. Creat new json file in the same folder but with a tmp name
2. Rename the file from its tmp name to the public name.

The read will just read the public name.

I am not sure what happens in your world if the writer runs a second time before the data is read.

In that case you need to create a queue of files to be read.

But if the problem is two process racing against each other you MUST use locking.
It cannot be avoided for robust operations.

Barry


> -- 
> https://mail.python.org/mailman/listinfo/python-list


From Marco.Sulla.Python at gmail.com  Sat Jan  1 11:13:13 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sat, 1 Jan 2022 17:13:13 +0100
Subject: recover pickled data: pickle data was truncated
In-Reply-To: <E4782BE5-1B73-46E2-9066-9EF5F255902B@barrys-emacs.org>
References: <1549b721-f6b6-4754-a5ff-ead72eb78d3an@googlegroups.com>
 <E4782BE5-1B73-46E2-9066-9EF5F255902B@barrys-emacs.org>
Message-ID: <CABbU2U_KFSRmXjxky7HUmC8Qr8OU+JTmrxcjjpPimKDG1ax7Mg@mail.gmail.com>

I agree with Barry. You can create a folder or a file with
pseudo-random names. I recommend you to use str(uuid.uuid4())

On Sat, 1 Jan 2022 at 14:11, Barry <barry at barrys-emacs.org> wrote:
>
>
>
> > On 31 Dec 2021, at 17:53, iMath <redstone-cold at 163.com> wrote:
> >
> > ?? 2021?12?30???? UTC+8 03:13:21?<Marco Sulla> ???
> >>> On Wed, 29 Dec 2021 at 18:33, iMath <redsto... at 163.com> wrote:
> >>> But I found the size of the file of the shelve data didn't change much, so I guess the data are still in it , I just wonder any way to recover my data.
> >> I agree with Barry, Chris and Avi. IMHO your data is lost. Unpickling
> >> it by hand is a harsh work and maybe unreliable.
> >>
> >> Is there any reason you can't simply add a semaphore to avoid writing
> >> at the same time and re-run the code and regenerate the data?
> >
> > Thanks for your replies! I didn't have a sense of adding a semaphore on writing to pickle data before, so  corrupted the data.
> > Since my data was colleted in the daily usage, so cannot re-run the code and regenerate the data.
> > In order to avoid corrupting my data again and the complicity of using  a semaphore, now I am using json text to store my data.
>
> That will not fix the problem. You will end up with corrupt json.
>
> If you have one writer and one read then may be you can use the fact that a rename is atomic.
>
> Writer does this:
> 1. Creat new json file in the same folder but with a tmp name
> 2. Rename the file from its tmp name to the public name.
>
> The read will just read the public name.
>
> I am not sure what happens in your world if the writer runs a second time before the data is read.
>
> In that case you need to create a queue of files to be read.
>
> But if the problem is two process racing against each other you MUST use locking.
> It cannot be avoided for robust operations.
>
> Barry
>
>
> > --
> > https://mail.python.org/mailman/listinfo/python-list
>
> --
> https://mail.python.org/mailman/listinfo/python-list

From kvratkin at gmail.com  Sat Jan  1 13:41:48 2022
From: kvratkin at gmail.com (Kirill Ratkin)
Date: Sat, 1 Jan 2022 21:41:48 +0300
Subject: using OpenGL on windows 11
In-Reply-To: <CADNO+12UZv+YPqb+_CFzyVh=jVjaFT-_EH7XS6Y6Qcw9LaXR9Q@mail.gmail.com>
References: <CADNO+12UZv+YPqb+_CFzyVh=jVjaFT-_EH7XS6Y6Qcw9LaXR9Q@mail.gmail.com>
Message-ID: <6487b4e1-f040-7133-8563-bbd859c96cb8@gmail.com>

Hi!

Your machine remembers but you probably use different virtual envs.

PyCharm, as I remember, creates its own venv (per project maybe).

It's better to remove python interpreter installed from MS application 
market and download python installer from python.org.

Install it as user or system. If you install python as user the 
interpreter files and standard libs will be in your profile 
(%USERPROFILE%\AppData\Local\Programs\...)

Then add path to python.exe to system or user PATH environment variable 
(using Windows GUI or thru CMD (command setx)).

(as I remember installer can do this task for you)

Now you have python interpreter installed and visible from command line 
(btw it's better to install visual studio build tools as well).

Open CMD or PowerShell and try command 'python'. If it works - Ok.

Then create virtual environment somewhere (for example in your home 
profile (%USERPROFILE%)): python -m venv venv

The directory C:\Users\<YourName>\venv appears.

 From CMD or PowerShell do: venv\Scripts\activate

You see command prompt changed and python virtual environment activated.

Now you can install any python stuff using pip. All new packages will be 
in 'venv\Lib\site-packages' but not in standard Lib folder.

Start PyCharm, select recently created venv (unfortunately I can't help 
you with menu path of python interpreter settings inside PyCharm because 
don't use it now).

But as for me VS Code is better. Try it. :)


// BR (KR)

31.12.2021 2:23, vrg bls ?????:
>    Hi! I am very new to Python admittedly, but did try several options
> regarding troubleshooting to get OpenGL to run with Python. I am using
> PyCharm as my interface, did try installing setup tools and have
> reinstalled using pip install function a few times. I did notice my machine
> seems to be having trouble remembering that I did install python and keeps
> prompting me to install from MS app store. I did try to do this but again
> did not register the installation of Python. I am able to open a window in
> PyCharm and get other functions to work but then get the Module not
> Installed error for OpenGL. I even tried the setuptools install and
> easy_install never popped up as an option. is this a windows 11 issue? does
> anything about this sound familiar?
> Thank you, and sorry if this has already been answered in a forum
> somewhere, I did look and could not find it
>
>

From Marco.Sulla.Python at gmail.com  Sat Jan  1 13:49:55 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sat, 1 Jan 2022 19:49:55 +0100
Subject: How to make a type of a C extension compatible with mypy
Message-ID: <CABbU2U_jAVbRPjx5qqogs8_GiWEti4kvSfvS9X7uZPNg7MQRDw@mail.gmail.com>

I created a type in a C extension, that is an immutable dict. If I do:

a: mydict[str, str]

it works. But it doesn't work with mypy, as signalled to me by an user:

https://github.com/Marco-Sulla/python-frozendict/issues/39

How can I make it work? I don't know what he means with annotating
methods, and furthermore I suppose I can't do this in C.

From Marco.Sulla.Python at gmail.com  Sat Jan  1 13:50:43 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sat, 1 Jan 2022 19:50:43 +0100
Subject: How to implement freelists in dict 3.10 for previous versions?
In-Reply-To: <CAEfz+Tz-0VJBD4EX3FqiAbRuLpiRAU4avS3DmkS7Jcib9uH1qA@mail.gmail.com>
References: <CABbU2U8nqnFTjF9FUp_YcWgmQSPK4c52pu+Gahw9GO2mPGhhBw@mail.gmail.com>
 <CAEfz+Tz-0VJBD4EX3FqiAbRuLpiRAU4avS3DmkS7Jcib9uH1qA@mail.gmail.com>
Message-ID: <CABbU2U8zxnmUAKbrr_p5+48NujuwtUj0b1gM+uVw5gCk9D_G2A@mail.gmail.com>

Ooookay, I suppose I have to study a little the thing :D

On Thu, 30 Dec 2021 at 07:59, Inada Naoki <songofacandy at gmail.com> wrote:
>
> On Wed, Dec 29, 2021 at 7:25 PM Marco Sulla
> <Marco.Sulla.Python at gmail.com> wrote:
> >
> > I noticed that now freelists in dict use _Py_dict_state. I suppose
> > this is done for thread safety.
> >
>
> Some core-dev are working on per-interpreter GIL. But it is not done yet.
> So you don't need to follow it soon. Your extension module will work
> well in Python 3.11.
>
> > I would implement it also for a C extension that uses CPython < 3.10.
> > How can I achieve this?
>
> See PyModule_GetState() to have per-interpreter module state instead
> of static variables.
> https://docs.python.org/3/c-api/module.html#c.PyModule_GetState
>
>
> --
> Inada Naoki  <songofacandy at gmail.com>

From kvratkin at gmail.com  Sat Jan  1 14:20:20 2022
From: kvratkin at gmail.com (Kirill Ratkin)
Date: Sat, 1 Jan 2022 22:20:20 +0300
Subject: How to make a type of a C extension compatible with mypy
In-Reply-To: <CABbU2U_jAVbRPjx5qqogs8_GiWEti4kvSfvS9X7uZPNg7MQRDw@mail.gmail.com>
References: <CABbU2U_jAVbRPjx5qqogs8_GiWEti4kvSfvS9X7uZPNg7MQRDw@mail.gmail.com>
Message-ID: <0bded175-cf0a-482e-3e98-fae21f8e2d52@gmail.com>

Hi Marco,

It seems issue is about file py.typed and PEP-561. As I understand 
issue's description you don't need to do it in C but some work in Python 
files need to done. Maybe this link helps 
(https://blog.whtsky.me/tech/2021/dont-forget-py.typed-for-your-typed-python-package/)

// BR (KR)

01.01.2022 21:49, Marco Sulla ?????:
> I created a type in a C extension, that is an immutable dict. If I do:
>
> a: mydict[str, str]
>
> it works. But it doesn't work with mypy, as signalled to me by an user:
>
> https://github.com/Marco-Sulla/python-frozendict/issues/39
>
> How can I make it work? I don't know what he means with annotating
> methods, and furthermore I suppose I can't do this in C.

From d9e9r9e9k at gmail.com  Sat Jan  1 14:20:04 2022
From: d9e9r9e9k at gmail.com (the derek team)
Date: Sat, 1 Jan 2022 12:20:04 -0700
Subject: PYTHON NOT WORKING
Message-ID: <CAG39XqbfZw8g9gm4Ht2CLLXOVaGh+1kLa40MHUSmR95z0XUKdQ@mail.gmail.com>

HI, I am trying to use python 3.10-1 on windows but, When I try to open
python, it crashes. Anaconda also does not work. When I try to use the
powershell, it gives me an error message saying that this is not recognized
as a valid cmdlet. Please help.

From drsalists at gmail.com  Sat Jan  1 17:53:31 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Sat, 1 Jan 2022 14:53:31 -0800
Subject: PYTHON NOT WORKING
In-Reply-To: <CAG39XqbfZw8g9gm4Ht2CLLXOVaGh+1kLa40MHUSmR95z0XUKdQ@mail.gmail.com>
References: <CAG39XqbfZw8g9gm4Ht2CLLXOVaGh+1kLa40MHUSmR95z0XUKdQ@mail.gmail.com>
Message-ID: <CAGGBd_o28LfHOirLTsikB3pcCmxa1-3YfW0_WBzJ4wnzS2cBjw@mail.gmail.com>

On Sat, Jan 1, 2022 at 1:52 PM the derek team <d9e9r9e9k at gmail.com> wrote:

> HI, I am trying to use python 3.10-1 on windows but, When I try to open
> python, it crashes. Anaconda also does not work. When I try to use the
> powershell, it gives me an error message saying that this is not recognized
> as a valid cmdlet. Please help.
>

Please describe where you obtained your CPython.

Also, please cut-and-paste the full text of the errors you're receiving.

HTH.

From barry at barrys-emacs.org  Sun Jan  2 07:37:16 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 2 Jan 2022 12:37:16 +0000
Subject: recover pickled data: pickle data was truncated
In-Reply-To: <CABbU2U_KFSRmXjxky7HUmC8Qr8OU+JTmrxcjjpPimKDG1ax7Mg@mail.gmail.com>
References: <1549b721-f6b6-4754-a5ff-ead72eb78d3an@googlegroups.com>
 <E4782BE5-1B73-46E2-9066-9EF5F255902B@barrys-emacs.org>
 <CABbU2U_KFSRmXjxky7HUmC8Qr8OU+JTmrxcjjpPimKDG1ax7Mg@mail.gmail.com>
Message-ID: <ADF38103-1A68-4086-B3E3-2A77930AB916@barrys-emacs.org>



> On 1 Jan 2022, at 16:13, Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
> 
> I agree with Barry. You can create a folder or a file with
> pseudo-random names. I recommend you to use str(uuid.uuid4())

At work and personally I use iso-8601 timestamps to make the files unique and easy to
find out when they where created.

:>>> t = datetime.datetime.now()
:>>> t
datetime.datetime(2022, 1, 2, 12, 34, 1, 267935)
:>>> t.strftime('%Y-%m-%dT%H-%M-%S')
'2022-01-02T12-34-01'
:>>>

That is good enough as long as you create the files slower than once a second.

Oh and yes use JSON, it is far better as a way of exchanging data than pickle.
Easy to read and check, can be processes in many languages.

Barry


> 
> On Sat, 1 Jan 2022 at 14:11, Barry <barry at barrys-emacs.org> wrote:
>> 
>> 
>> 
>>> On 31 Dec 2021, at 17:53, iMath <redstone-cold at 163.com> wrote:
>>> 
>>> ?? 2021?12?30???? UTC+8 03:13:21?<Marco Sulla> ???
>>>>> On Wed, 29 Dec 2021 at 18:33, iMath <redsto... at 163.com> wrote:
>>>>> But I found the size of the file of the shelve data didn't change much, so I guess the data are still in it , I just wonder any way to recover my data.
>>>> I agree with Barry, Chris and Avi. IMHO your data is lost. Unpickling
>>>> it by hand is a harsh work and maybe unreliable.
>>>> 
>>>> Is there any reason you can't simply add a semaphore to avoid writing
>>>> at the same time and re-run the code and regenerate the data?
>>> 
>>> Thanks for your replies! I didn't have a sense of adding a semaphore on writing to pickle data before, so  corrupted the data.
>>> Since my data was colleted in the daily usage, so cannot re-run the code and regenerate the data.
>>> In order to avoid corrupting my data again and the complicity of using  a semaphore, now I am using json text to store my data.
>> 
>> That will not fix the problem. You will end up with corrupt json.
>> 
>> If you have one writer and one read then may be you can use the fact that a rename is atomic.
>> 
>> Writer does this:
>> 1. Creat new json file in the same folder but with a tmp name
>> 2. Rename the file from its tmp name to the public name.
>> 
>> The read will just read the public name.
>> 
>> I am not sure what happens in your world if the writer runs a second time before the data is read.
>> 
>> In that case you need to create a queue of files to be read.
>> 
>> But if the problem is two process racing against each other you MUST use locking.
>> It cannot be avoided for robust operations.
>> 
>> Barry
>> 
>> 
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>> 
>> --
>> https://mail.python.org/mailman/listinfo/python-list
> 


From Marco.Sulla.Python at gmail.com  Sun Jan  2 11:17:23 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sun, 2 Jan 2022 17:17:23 +0100
Subject: Who wrote Py_UNREACHABLE?
Message-ID: <CABbU2U_ddA5p+ooaXtTBphZWQ+fjoz8dxkbPMtaH6T4OdY0gXw@mail.gmail.com>

#if defined(RANDALL_WAS_HERE)
#  define Py_UNREACHABLE() \
    Py_FatalError( \
        "If you're seeing this, the code is in what I thought was\n" \
        "an unreachable state.\n\n" \
        "I could give you advice for what to do, but honestly, why\n" \
        "should you trust me?  I clearly screwed this up.  I'm writing\n" \
        "a message that should never appear, yet I know it will\n" \
        "probably appear someday.\n\n" \
        "On a deep level, I know I'm not up to this task.\n" \
        "I'm so sorry.\n" \
        "https://xkcd.com/2200")
#elif defined(Py_DEBUG)
#  define Py_UNREACHABLE() \
    Py_FatalError( \
        "We've reached an unreachable state. Anything is possible.\n" \
        "The limits were in our heads all along. Follow your dreams.\n" \
        "https://xkcd.com/2200")

etc

From rosuav at gmail.com  Sun Jan  2 11:25:41 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 3 Jan 2022 03:25:41 +1100
Subject: Who wrote Py_UNREACHABLE?
In-Reply-To: <CABbU2U_ddA5p+ooaXtTBphZWQ+fjoz8dxkbPMtaH6T4OdY0gXw@mail.gmail.com>
References: <CABbU2U_ddA5p+ooaXtTBphZWQ+fjoz8dxkbPMtaH6T4OdY0gXw@mail.gmail.com>
Message-ID: <CAPTjJmoDbW1NWcCb+fb=TRs+qNhuQ=cw6-391uJ1sYD7v88Byw@mail.gmail.com>

On Mon, Jan 3, 2022 at 3:19 AM Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
>
> #if defined(RANDALL_WAS_HERE)
> #  define Py_UNREACHABLE() \
>     Py_FatalError( \
>         "If you're seeing this, the code is in what I thought was\n" \
>         "an unreachable state.\n\n" \
>         "I could give you advice for what to do, but honestly, why\n" \
>         "should you trust me?  I clearly screwed this up.  I'm writing\n" \
>         "a message that should never appear, yet I know it will\n" \
>         "probably appear someday.\n\n" \
>         "On a deep level, I know I'm not up to this task.\n" \
>         "I'm so sorry.\n" \
>         "https://xkcd.com/2200")
> #elif defined(Py_DEBUG)
> #  define Py_UNREACHABLE() \
>     Py_FatalError( \
>         "We've reached an unreachable state. Anything is possible.\n" \
>         "The limits were in our heads all along. Follow your dreams.\n" \
>         "https://xkcd.com/2200")
>
> etc

Zachary Ware, 2019-09-12, commit #3ab614
https://github.com/python/cpython/pull/16032

ChrisA

From barry at barrys-emacs.org  Sun Jan  2 12:06:34 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 2 Jan 2022 17:06:34 +0000
Subject: Who wrote Py_UNREACHABLE?
In-Reply-To: <CABbU2U_ddA5p+ooaXtTBphZWQ+fjoz8dxkbPMtaH6T4OdY0gXw@mail.gmail.com>
References: <CABbU2U_ddA5p+ooaXtTBphZWQ+fjoz8dxkbPMtaH6T4OdY0gXw@mail.gmail.com>
Message-ID: <3226FD48-E66E-4E3A-A0A8-C6621E49692D@barrys-emacs.org>



> On 2 Jan 2022, at 16:17, Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
> 
> #if defined(RANDALL_WAS_HERE)
> #  define Py_UNREACHABLE() \
>    Py_FatalError( \
>        "If you're seeing this, the code is in what I thought was\n" \
>        "an unreachable state.\n\n" \
>        "I could give you advice for what to do, but honestly, why\n" \
>        "should you trust me?  I clearly screwed this up.  I'm writing\n" \
>        "a message that should never appear, yet I know it will\n" \
>        "probably appear someday.\n\n" \
>        "On a deep level, I know I'm not up to this task.\n" \
>        "I'm so sorry.\n" \
>        "https://xkcd.com/2200")
> #elif defined(Py_DEBUG)
> #  define Py_UNREACHABLE() \
>    Py_FatalError( \
>        "We've reached an unreachable state. Anything is possible.\n" \
>        "The limits were in our heads all along. Follow your dreams.\n" \
>        "https://xkcd.com/2200")

use git blame to find out the commit then use git log to read the commit message.

Barry


> 
> etc
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From hongyi.zhao at gmail.com  Sun Jan  2 07:23:46 2022
From: hongyi.zhao at gmail.com (hongy...@gmail.com)
Date: Sun, 2 Jan 2022 04:23:46 -0800 (PST)
Subject: ModuleNotFoundError: No module named 'DistUtilsExtra'
Message-ID: <2912cd7c-2fc3-4038-8c30-64412729e344n@googlegroups.com>

On Ubuntu 20.04.3 LTS, I try to install pdfarranger [1] as follows but failed:

$ sudo apt-get install python3-pip python3-distutils-extra \
                                      python3-wheel python3-gi python3-gi-cairo \
                                      gir1.2-gtk-3.0 gir1.2-poppler-0.18 python3-setuptools
$ git clone https://github.com/pdfarranger/pdfarranger.git pdfarranger.git
$ cd pdfarranger.git
$ pyenv shell 3.8.3
$ pyenv virtualenv --system-site-packages pdfarranger
$ pyenv shell pdfarranger 
$ pip install -U pip
$ ./setup.py build
Traceback (most recent call last):
  File "./setup.py", line 24, in <module>
    from DistUtilsExtra.command import (
ModuleNotFoundError: No module named 'DistUtilsExtra'


See the following for the package list installed in this virtualenv:

$ pip list 
Package    Version
---------- ------------
pip        21.3.1
pyfiglet   0.8.post1
setuptools 41.2.0
vtk        9.0.20200612

Any hints for fixing this problem? Also see here [2-3] for relevant discussions.

[1] https://github.com/pdfarranger/pdfarranger
[2] https://github.com/pdfarranger/pdfarranger/issues/604
[3] https://discuss.python.org/t/modulenotfounderror-no-module-named-distutilsextra/12834

Regards,
HZ

From Marco.Sulla.Python at gmail.com  Sun Jan  2 17:14:55 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sun, 2 Jan 2022 23:14:55 +0100
Subject: ModuleNotFoundError: No module named 'DistUtilsExtra'
In-Reply-To: <2912cd7c-2fc3-4038-8c30-64412729e344n@googlegroups.com>
References: <2912cd7c-2fc3-4038-8c30-64412729e344n@googlegroups.com>
Message-ID: <CABbU2U-QMA2XQUBA_9UeQRF8Pat=wKBr1yE741S8QE6TmPB+Pw@mail.gmail.com>

https://askubuntu.com/questions/584857/distutilsextra-problem

On Sun, 2 Jan 2022 at 18:52, hongy... at gmail.com <hongyi.zhao at gmail.com> wrote:
>
> On Ubuntu 20.04.3 LTS, I try to install pdfarranger [1] as follows but failed:
>
> $ sudo apt-get install python3-pip python3-distutils-extra \
>                                       python3-wheel python3-gi python3-gi-cairo \
>                                       gir1.2-gtk-3.0 gir1.2-poppler-0.18 python3-setuptools
> $ git clone https://github.com/pdfarranger/pdfarranger.git pdfarranger.git
> $ cd pdfarranger.git
> $ pyenv shell 3.8.3
> $ pyenv virtualenv --system-site-packages pdfarranger
> $ pyenv shell pdfarranger
> $ pip install -U pip
> $ ./setup.py build
> Traceback (most recent call last):
>   File "./setup.py", line 24, in <module>
>     from DistUtilsExtra.command import (
> ModuleNotFoundError: No module named 'DistUtilsExtra'
>
>
> See the following for the package list installed in this virtualenv:
>
> $ pip list
> Package    Version
> ---------- ------------
> pip        21.3.1
> pyfiglet   0.8.post1
> setuptools 41.2.0
> vtk        9.0.20200612
>
> Any hints for fixing this problem? Also see here [2-3] for relevant discussions.
>
> [1] https://github.com/pdfarranger/pdfarranger
> [2] https://github.com/pdfarranger/pdfarranger/issues/604
> [3] https://discuss.python.org/t/modulenotfounderror-no-module-named-distutilsextra/12834
>
> Regards,
> HZ
> --
> https://mail.python.org/mailman/listinfo/python-list

From hongyi.zhao at gmail.com  Sun Jan  2 19:30:38 2022
From: hongyi.zhao at gmail.com (hongy...@gmail.com)
Date: Sun, 2 Jan 2022 16:30:38 -0800 (PST)
Subject: ModuleNotFoundError: No module named 'DistUtilsExtra'
In-Reply-To: <mailman.92.1641161734.3079.python-list@python.org>
References: <CABbU2U-QMA2XQUBA_9UeQRF8Pat=wKBr1yE741S8QE6TmPB+Pw@mail.gmail.com>
 <2912cd7c-2fc3-4038-8c30-64412729e344n@googlegroups.com>
 <mailman.92.1641161734.3079.python-list@python.org>
Message-ID: <05cc00f3-7c0c-403a-8713-7dead16d153dn@googlegroups.com>

On Monday, January 3, 2022 at 6:15:53 AM UTC+8, Marco Sulla wrote:
> https://askubuntu.com/questions/584857/distutilsextra-problem

I found its Git repository using the following method:

$ for i in python-distutils-extra python3-distutils-extra; do apt showsrc $i |& grep git; done | sort -u
Vcs-Git: https://salsa.debian.org/python-team/modules/python-distutils-extra.git

Therefore, for developers, the more robust and reasonable pdfarranger installation steps should be as follows:

#https://github.com/pdfarranger/pdfarranger#install-from-source
$ sudo apt install gir1.2-gtk-3.0 gir1.2-poppler-0.18 
$ git clone https://github.com/pdfarranger/pdfarranger.git pdfarranger.git 
$ cd pdfarranger.git 
$ pyenv shell 3.8.3
$ pyenv virtualenv --system-site-packages pdfarranger
$ pyenv shell pdfarranger 
$ pip install -U pip
$ pip install git+https://salsa.debian.org/python-team/modules/python-distutils-extra.git
$ pip install img2pdf pygobject
$ pip install .

Another note: the python-distutils-extra or python3-distutils-extra is only a dependency in the installation process, once the package is installed, it will no longer be needed.

For a more detailed discussion, please see here [1].

[1] https://github.com/pdfarranger/pdfarranger/issues/604#issuecomment-1003801361

Regards,
HZ

From vuducdong at gmail.com  Sun Jan  2 21:36:42 2022
From: vuducdong at gmail.com (=?UTF-8?B?xJDDtG5nIFbFqQ==?=)
Date: Mon, 3 Jan 2022 09:36:42 +0700
Subject: KeyError: 'payload' while running core.py
Message-ID: <CAF-CctwYbcpXZ+ULYG-V2Kz4xx-3QQqkHkoodfV3yaLaCu5V4g@mail.gmail.com>

That is error, please help me fix it.

Traceback (most recent call last):
  File
"C:\Users\vuduc\AppData\Local\Programs\Python\Python39\lib\site-packages\frida\core.py",
line 450, in _on_message
    callback(message, data)
  File "C:\NordLynx-main\nordlynx.py", line 43, in onMessage
    if 'PrivateKey' not in message["payload"]: return
KeyError: 'payload'

Signature powered by
<http://www.wisestamp.com/email-install?utm_source=extension&utm_medium=email&utm_campaign=footer>
WiseStamp
<http://www.wisestamp.com/email-install?utm_source=extension&utm_medium=email&utm_campaign=footer>

From rosuav at gmail.com  Sun Jan  2 21:53:46 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 3 Jan 2022 13:53:46 +1100
Subject: KeyError: 'payload' while running core.py
In-Reply-To: <CAF-CctwYbcpXZ+ULYG-V2Kz4xx-3QQqkHkoodfV3yaLaCu5V4g@mail.gmail.com>
References: <CAF-CctwYbcpXZ+ULYG-V2Kz4xx-3QQqkHkoodfV3yaLaCu5V4g@mail.gmail.com>
Message-ID: <CAPTjJmrULD1vedE+n8T4yvKtkmBNVa8+1QkUj-e50r1aky+0bQ@mail.gmail.com>

On Mon, Jan 3, 2022 at 1:46 PM ??ng V? <vuducdong at gmail.com> wrote:
>
> That is error, please help me fix it.
>
> Traceback (most recent call last):
>   File
> "C:\Users\vuduc\AppData\Local\Programs\Python\Python39\lib\site-packages\frida\core.py",
> line 450, in _on_message
>     callback(message, data)
>   File "C:\NordLynx-main\nordlynx.py", line 43, in onMessage
>     if 'PrivateKey' not in message["payload"]: return
> KeyError: 'payload'

The problem is with NordLynx, not Python.

> Signature powered by
> WiseStamp
> <url trimmed>

Quite impressive that a blank signature needs a "powered by" advertisement.

ChrisA

From jnmarques2002 at gmail.com  Mon Jan  3 07:45:50 2022
From: jnmarques2002 at gmail.com (Joao Marques)
Date: Mon, 3 Jan 2022 12:45:50 +0000
Subject: Starting using Python
Message-ID: <CALdZMUFDBHQz4PFHO9MS9pm1=sKSnjCg3aUsXdQsgjOnR36yyA@mail.gmail.com>

Good morning: I have a very simple question: I want to start writing
programs in Python so I went to the Microsoft Store and installed
Python3.9. No problem so far. I would prefer to have a gui interface, an
interface that I can use file-->Open and File-->Save as, as I see it on
different videos. How can I get it? Because my problem is to run the
programs I have already written and saved on a *.py file in my own working
directory, not in the Python's CWD directory.
Can you please help?
I am running Windows 10 Pro version 20H2

Regards,
Joao

From jnmarques2002 at gmail.com  Mon Jan  3 07:50:03 2022
From: jnmarques2002 at gmail.com (Joao Marques)
Date: Mon, 3 Jan 2022 12:50:03 +0000
Subject: Starting Python
Message-ID: <CALdZMUHAKKR7S7DKLA77uW9-OiyUXsHJvuxkO+tHyUXKJwSz8g@mail.gmail.com>

Hi everyone:
Ignore my previous post.
I was starting Python3.9 instead of starting IDLE. With idle I already get
the options I wanted
REgards,
Joao

From Marco.Sulla.Python at gmail.com  Tue Jan  4 13:27:31 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Tue, 4 Jan 2022 19:27:31 +0100
Subject: Why operations between dict views return a set and not a frozenset?
Message-ID: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>

$ python
Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
[GCC 10.1.1 20200718] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = {1:2}
>>> c = {1:2, 3:4}
>>> c.keys() - a.keys()
{3}
>>>


Why not frozenset({3})?

From rosuav at gmail.com  Tue Jan  4 13:36:00 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Jan 2022 05:36:00 +1100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
Message-ID: <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>

On Wed, Jan 5, 2022 at 5:29 AM Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
>
> $ python
> Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
> [GCC 10.1.1 20200718] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> a = {1:2}
> >>> c = {1:2, 3:4}
> >>> c.keys() - a.keys()
> {3}
> >>>
>

Let's start with this.

>>> a = {1}
>>> c = {1, 3}
>>> c - a
{3}

Do you agree that this should be a set?

If so, then the next question is: should the keys view be considered
frozen or not? Remember the set of keys can change (when the
underlying dict changes).

It's not difficult to construct a frozenset from a set.

ChrisA

From Marco.Sulla.Python at gmail.com  Tue Jan  4 15:03:58 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Tue, 4 Jan 2022 21:03:58 +0100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
Message-ID: <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>

On Tue, 4 Jan 2022 at 19:38, Chris Angelico <rosuav at gmail.com> wrote:
> [...] should the keys view be considered
> frozen or not? Remember the set of keys can change (when the
> underlying dict changes).

Well, also the items can change, but they are returned as tuples with
2 elements.

It seems to me that the stdlib, when something should return a
sequence, prefers to return a tuple. So I expected the same preference
for frozenset over set.

> It's not difficult to construct a frozenset from a set.

This sentence has the commutative property :)

From ijbrewster at alaska.edu  Tue Jan  4 17:57:26 2022
From: ijbrewster at alaska.edu (Israel Brewster)
Date: Tue, 4 Jan 2022 13:57:26 -0900
Subject: Shapely Polygon creating empty polygon
Message-ID: <2DB2FC25-8FEC-4063-BAFD-068AFC6DC89D@alaska.edu>

I?m running into an issue with shapely that is baffling me. Perhaps someone here can help out?

When running shapely directly from a python 3.8 interpreter, it works as expected:

>>> import shapely
>>> shapely.__version__
'1.8.0'
>>> from shapely.geometry import Polygon
>>> bounds = [-164.29635821669632, 54.64251856269729, -163.7631779798799, 54.845450778742546]
>>> print(Polygon.from_bounds(*bounds))
POLYGON ((-164.2963582166963 54.64251856269729, -164.2963582166963 54.84545077874255, -163.7631779798799 54.84545077874255, -163.7631779798799 54.64251856269729, -164.2963582166963 54.64251856269729))

However, if I put this exact same code into my Flask app (currently running under the Flask development environment) as part of handling a request, I get an empty polygon:

>>>    import shapely
>>>    print(shapely.__version__)
>>>    from shapely.geometry import Polygon
>>>    print(Polygon.from_bounds(*bounds))

Output:

1.8.0
POLYGON EMPTY

In fact, *any* attempt to create a polygon gives the same result:
>>> test = Polygon(((1, 1), (2, 1), (2, 2)))
>>> print(test)
POLYGON EMPTY

What am I missing here? Why doesn?t it work as part of a Flask request call?
---
Israel Brewster
Software Engineer
Alaska Volcano Observatory 
Geophysical Institute - UAF 
2156 Koyukuk Drive 
Fairbanks AK 99775-7320
Work: 907-474-5172
cell:  907-328-9145


From cs at cskk.id.au  Tue Jan  4 16:30:30 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Wed, 5 Jan 2022 08:30:30 +1100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
References: <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
Message-ID: <YdS8dt853evDS8II@cskk.homeip.net>

On 04Jan2022 21:03, Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
>On Tue, 4 Jan 2022 at 19:38, Chris Angelico <rosuav at gmail.com> wrote:
>> [...] should the keys view be considered
>> frozen or not? Remember the set of keys can change (when the
>> underlying dict changes).
>
>Well, also the items can change, but they are returned as tuples with
>2 elements.
>
>It seems to me that the stdlib, when something should return a
>sequence, prefers to return a tuple. So I expected the same preference
>for frozenset over set.
>
>> It's not difficult to construct a frozenset from a set.
>
>This sentence has the commutative property :)

Indeed.

But speaking for myself, I may well want to perform additional work on 
the object returned. Making a copy of it for tht purpose seems very 
wasteful (imagine the set is quite large). A modifiable version can be 
used immediately with no time or space cost. And it can be left alone if 
it is to be unchanged.  If I got a frozenset back I would inherently 
have to copy it to do "modifying work".

So I prefer getting a modifiable object back.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From rosuav at gmail.com  Tue Jan  4 18:52:41 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 5 Jan 2022 10:52:41 +1100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
Message-ID: <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>

On Wed, Jan 5, 2022 at 7:04 AM Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
>
> On Tue, 4 Jan 2022 at 19:38, Chris Angelico <rosuav at gmail.com> wrote:
> > [...] should the keys view be considered
> > frozen or not? Remember the set of keys can change (when the
> > underlying dict changes).
>
> Well, also the items can change, but they are returned as tuples with
> 2 elements.

That's because a tuple is the correct data type when returning two
distinct items. It's not a list that has two elements in it; it's a
tuple of (key, value). Immutability is irrelevant.

> It seems to me that the stdlib, when something should return a
> sequence, prefers to return a tuple. So I expected the same preference
> for frozenset over set.

Got any examples of variable-length sequences? Usually a tuple is a
structure, not just a sequence. If something is just returning a
sequence, it'll most often return a dedicated sequence type (like
range in Py3) or a list (like lots of things in Py2).

ChrisA

From python at mrabarnett.plus.com  Tue Jan  4 19:47:02 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 5 Jan 2022 00:47:02 +0000
Subject: Shapely Polygon creating empty polygon
In-Reply-To: <2DB2FC25-8FEC-4063-BAFD-068AFC6DC89D@alaska.edu>
References: <2DB2FC25-8FEC-4063-BAFD-068AFC6DC89D@alaska.edu>
Message-ID: <74793275-1a23-dbef-4079-425bfaae7237@mrabarnett.plus.com>

On 2022-01-04 22:57, Israel Brewster wrote:
> I?m running into an issue with shapely that is baffling me. Perhaps someone here can help out?
> 
> When running shapely directly from a python 3.8 interpreter, it works as expected:
> 
>>>> import shapely
>>>> shapely.__version__
> '1.8.0'
>>>> from shapely.geometry import Polygon
>>>> bounds = [-164.29635821669632, 54.64251856269729, -163.7631779798799, 54.845450778742546]
>>>> print(Polygon.from_bounds(*bounds))
> POLYGON ((-164.2963582166963 54.64251856269729, -164.2963582166963 54.84545077874255, -163.7631779798799 54.84545077874255, -163.7631779798799 54.64251856269729, -164.2963582166963 54.64251856269729))
> 
> However, if I put this exact same code into my Flask app (currently running under the Flask development environment) as part of handling a request, I get an empty polygon:
> 
>>>>    import shapely
>>>>    print(shapely.__version__)
>>>>    from shapely.geometry import Polygon
>>>>    print(Polygon.from_bounds(*bounds))

In that piece of code you didn't set 'bounds'. It might be worth setting 
it or at least printing it out to double-check.
> 
> Output:
> 
> 1.8.0
> POLYGON EMPTY
> 
> In fact, *any* attempt to create a polygon gives the same result:
>>>> test = Polygon(((1, 1), (2, 1), (2, 2)))
>>>> print(test)
> POLYGON EMPTY
> 
Did you try that piece of code both in Flask and from the command line, 
again to double-check.

> What am I missing here? Why doesn?t it work as part of a Flask request call?
> 

From 2QdxY4RzWzUUiLuE at potatochowder.com  Tue Jan  4 23:09:42 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Tue, 4 Jan 2022 20:09:42 -0800
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <YdS8dt853evDS8II@cskk.homeip.net>
References: <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <YdS8dt853evDS8II@cskk.homeip.net>
Message-ID: <YdUaBuw/RLmUgcNO@scrozzle>

On 2022-01-05 at 08:30:30 +1100,
Cameron Simpson <cs at cskk.id.au> wrote:

> On 04Jan2022 21:03, Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
> >On Tue, 4 Jan 2022 at 19:38, Chris Angelico <rosuav at gmail.com> wrote:
> >> [...] should the keys view be considered
> >> frozen or not? Remember the set of keys can change (when the
> >> underlying dict changes).
> >
> >Well, also the items can change, but they are returned as tuples with
> >2 elements.
> >
> >It seems to me that the stdlib, when something should return a
> >sequence, prefers to return a tuple. So I expected the same preference
> >for frozenset over set.
> >
> >> It's not difficult to construct a frozenset from a set.
> >
> >This sentence has the commutative property :)
> 
> Indeed.
> 
> But speaking for myself, I may well want to perform additional work on 
> the object returned. Making a copy of it for tht purpose seems very 
> wasteful (imagine the set is quite large). A modifiable version can be 
> used immediately with no time or space cost. And it can be left alone if 
> it is to be unchanged.  If I got a frozenset back I would inherently 
> have to copy it to do "modifying work".

Unless the additional work is to use it as a dictionary key, or to add
it to an existing [necessarily mutable!] set.  Then again, that's not
work *on* the object returned, that's additional work *with* the object
returned.

We could go around and around on this all day.  :-)

Python began with the premise of mutability.  IIRC, many attempts at
immutable class instances (for use in sets or as dictionary keys) have
run into the same issues Marco Sulla is having.

From drsalists at gmail.com  Tue Jan  4 23:47:18 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 4 Jan 2022 20:47:18 -0800
Subject: Starting using Python
In-Reply-To: <CALdZMUFDBHQz4PFHO9MS9pm1=sKSnjCg3aUsXdQsgjOnR36yyA@mail.gmail.com>
References: <CALdZMUFDBHQz4PFHO9MS9pm1=sKSnjCg3aUsXdQsgjOnR36yyA@mail.gmail.com>
Message-ID: <CAGGBd_q9==JkrnnLzVSi7Td6=xeq6Z3K-onHhfbr7yDX9dj_Mw@mail.gmail.com>

On Mon, Jan 3, 2022 at 8:06 AM Joao Marques <jnmarques2002 at gmail.com> wrote:

> Good morning: I have a very simple question: I want to start writing
> programs in Python so I went to the Microsoft Store and installed
> Python3.9. No problem so far. I would prefer to have a gui interface, an
> interface that I can use file-->Open and File-->Save as, as I see it on
> different videos. How can I get it? Because my problem is to run the
> programs I have already written and saved on a *.py file in my own working
> directory, not in the Python's CWD directory.
> Can you please help?
> I am running Windows 10 Pro version 20H2
>

I'm thinking you probably want PyCharm or VSCode.  Give them each a google
search and compare a little.

From Marco.Sulla.Python at gmail.com  Wed Jan  5 08:04:48 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Wed, 5 Jan 2022 14:04:48 +0100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
Message-ID: <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>

On Wed, 5 Jan 2022 at 00:54, Chris Angelico <rosuav at gmail.com> wrote:
> That's because a tuple is the correct data type when returning two
> distinct items. It's not a list that has two elements in it; it's a
> tuple of (key, value). Immutability is irrelevant.

Immutability is irrelevant, speed no. A tuple is faster than a list
and more compact. Also frozenset is faster than set. Indeed CPython
optimises internally a

for x in {1, 2, 3}

transforming the set in a frozenset for a matter of speed. That's why
tuple is usually preferred. I expected the same for frozenset

> Got any examples of variable-length sequences?

function positional args are tuples, for example.

> Usually a tuple is a
> structure, not just a sequence.

....eh? Are you talking about the underlying C code?

> If something is just returning a
> sequence, it'll most often return a dedicated sequence type (like
> range in Py3) or a list (like lots of things in Py2).

Python 2 is now obsolete, I don't think is relevant for the discussion.

About your sentence, yes, usually a dedicated view, sequence or
generator is returned, but tuples too are really much used. A list is
returned very sporadically, for what I remember.

From rosuav at gmail.com  Wed Jan  5 08:14:10 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 6 Jan 2022 00:14:10 +1100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
Message-ID: <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>

On Thu, Jan 6, 2022 at 12:05 AM Marco Sulla
<Marco.Sulla.Python at gmail.com> wrote:
>
> On Wed, 5 Jan 2022 at 00:54, Chris Angelico <rosuav at gmail.com> wrote:
> > That's because a tuple is the correct data type when returning two
> > distinct items. It's not a list that has two elements in it; it's a
> > tuple of (key, value). Immutability is irrelevant.
>
> Immutability is irrelevant, speed no. A tuple is faster than a list
> and more compact. Also frozenset is faster than set. Indeed CPython
> optimises internally a
>
> for x in {1, 2, 3}
>
> transforming the set in a frozenset for a matter of speed. That's why
> tuple is usually preferred. I expected the same for frozenset

That's an entirely invisible optimization, but it's more than just
"frozenset is faster than set". It's that a frozenset or tuple can be
stored as a function's constants, which is a massive difference.

In fact, the two data types are virtually identical in performance once created:

rosuav at sikorsky:~$ python3 -m timeit -s "stuff = {1,2,3}" "for x in stuff: pass"
5000000 loops, best of 5: 46.2 nsec per loop
rosuav at sikorsky:~$ python3 -m timeit -s "stuff = frozenset({1,2,3})"
"for x in stuff: pass"
5000000 loops, best of 5: 46.7 nsec per loop
rosuav at sikorsky:~$ python3 -m timeit -s "stuff = set(range(10000))"
"for x in stuff: pass"
5000 loops, best of 5: 82.1 usec per loop
rosuav at sikorsky:~$ python3 -m timeit -s "stuff =
frozenset(range(10000))" "for x in stuff: pass"
5000 loops, best of 5: 81.3 usec per loop

Mutability is irrelevant, and so is the speed of the data type. Having
set operations on keys views return frozensets wouldn't improve
anything.

> > Got any examples of variable-length sequences?
>
> function positional args are tuples, for example.
>
> > Usually a tuple is a
> > structure, not just a sequence.
>
> ....eh? Are you talking about the underlying C code?

No, I'm talking about purpose. In general, a list contains a sequence
of things whose order matters but which can be removed from - you can
remove one item from your shopping list and the rest still mean what
they were. A tuple has a specific set of things in a specific order,
like Cartesian coordinates. You can't just remove the x coordinate
from an (x,y,z) tuple without fundamentally changing what it is.

Function positional arguments aren't interchangeable, so it makes
sense to have them as a tuple. Removing the first argument would
redefine what all the others mean, so a tuple is correct - it's not
just a list that's been made immutable for performance's sake.
(Function *keyword* arguments, on the other hand, are different; as
long as the mapping from keys to values is maintained, you can remove
some of them and pass the rest on, without fundamentally changing
their meaning.)

Do you have any examples of actually variable-length sequences that
are tuples for speed?

Measure before claiming a speed difference.

ChrisA

From antoon.pardon at vub.be  Wed Jan  5 09:06:32 2022
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Wed, 5 Jan 2022 15:06:32 +0100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
Message-ID: <47d7aa9d-de05-6c50-01d0-990f16262f07@vub.be>



Op 4/01/2022 om 19:27 schreef Marco Sulla:
> $ python
> Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
> [GCC 10.1.1 20200718] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> a = {1:2}
>>>> c = {1:2, 3:4}
>>>> c.keys() - a.keys()
> {3}
>
> Why not frozenset({3})?

My 2 cents worths: Because dictviews mutate with the directory. That makes
them more like set than like frozenset. So operations on them produce a set.

-- 
Antoon Pardon.

From lickta19 at gmail.com  Wed Jan  5 03:29:06 2022
From: lickta19 at gmail.com (Malick Lickta Keita)
Date: Wed, 5 Jan 2022 16:29:06 +0800
Subject: Pip not working
Message-ID: <4E8E0D9E-EC51-4AFC-B28B-FB41B36B3A3B@gmail.com>

?Hello I?ve installed the latest version of python but pip is not working 

Sent from my iPhone

From narshad.380 at gmail.com  Wed Jan  5 02:34:20 2022
From: narshad.380 at gmail.com (NArshad)
Date: Tue, 4 Jan 2022 23:34:20 -0800 (PST)
Subject: AttributeError: 'NoneType' object has no attribute 'get'
Message-ID: <9f86242d-81cf-4deb-b681-ce760aeac69an@googlegroups.com>

How to correct what is written below:

Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Users\Dani Brothers\Anaconda3\lib\tkinter\__init__.py", line 1705, in __call__
    return self.func(*args)
  File "D:/Python/Book Bank/New folder/PyCharm/Final/Excel.py", line 57, in SaveBook
    e_pissue.get(),
AttributeError: 'NoneType' object has no attribute 'get'

Process finished with exit code 0

From redstone-cold at 163.com  Wed Jan  5 03:44:11 2022
From: redstone-cold at 163.com (iMath)
Date: Wed, 5 Jan 2022 00:44:11 -0800 (PST)
Subject: recover pickled data: pickle data was truncated
In-Reply-To: <mailman.88.1641127045.3079.python-list@python.org>
References: <1549b721-f6b6-4754-a5ff-ead72eb78d3an@googlegroups.com>
 <E4782BE5-1B73-46E2-9066-9EF5F255902B@barrys-emacs.org>
 <ADF38103-1A68-4086-B3E3-2A77930AB916@barrys-emacs.org>
 <CABbU2U_KFSRmXjxky7HUmC8Qr8OU+JTmrxcjjpPimKDG1ax7Mg@mail.gmail.com>
 <mailman.88.1641127045.3079.python-list@python.org>
Message-ID: <4be15827-6a87-4389-a545-838d1581a521n@googlegroups.com>

Thanks for all your kind help, wish you a promising year!

From ijbrewster at alaska.edu  Wed Jan  5 12:48:14 2022
From: ijbrewster at alaska.edu (Israel Brewster)
Date: Wed, 5 Jan 2022 08:48:14 -0900
Subject: Shapely Polygon creating empty polygon
In-Reply-To: <2DB2FC25-8FEC-4063-BAFD-068AFC6DC89D@alaska.edu>
References: <2DB2FC25-8FEC-4063-BAFD-068AFC6DC89D@alaska.edu>
Message-ID: <9CCDAE3E-AF6F-4287-B98E-A4AE58D6CC07@alaska.edu>

Found it! Apparently, it?s an import order issue. This works:

>>> from shapely.geometry import Polygon
>>> from osgeo import osr
>>> bounds = [-164.29635821669632, 54.64251856269729, -163.7631779798799, 54.845450778742546]
>>> print(Polygon.from_bounds(*bounds))
POLYGON ((-164.2963582166963 54.64251856269729, -164.2963582166963 54.84545077874255, -163.7631779798799 54.84545077874255, -163.7631779798799 54.64251856269729, -164.2963582166963 54.64251856269729))

But this doesn?t:

>>> from osgeo import osr
>>> from shapely.geometry import Polygon
>>> bounds = [-164.29635821669632, 54.64251856269729, -163.7631779798799, 54.845450778742546]
>>> print(Polygon.from_bounds(*bounds))
POLYGON EMPTY

?So apparently I have to make sure to import shapely *before* I import anything from osgeo. Why? I have no idea...
---
Israel Brewster
Software Engineer
Alaska Volcano Observatory 
Geophysical Institute - UAF 
2156 Koyukuk Drive 
Fairbanks AK 99775-7320
Work: 907-474-5172
cell:  907-328-9145

> On Jan 4, 2022, at 1:57 PM, Israel Brewster <ijbrewster at alaska.edu> wrote:
> 
> I?m running into an issue with shapely that is baffling me. Perhaps someone here can help out?
> 
> When running shapely directly from a python 3.8 interpreter, it works as expected:
> 
> >>> import shapely
> >>> shapely.__version__
> '1.8.0'
> >>> from shapely.geometry import Polygon
> >>> bounds = [-164.29635821669632, 54.64251856269729, -163.7631779798799, 54.845450778742546]
> >>> print(Polygon.from_bounds(*bounds))
> POLYGON ((-164.2963582166963 54.64251856269729, -164.2963582166963 54.84545077874255, -163.7631779798799 54.84545077874255, -163.7631779798799 54.64251856269729, -164.2963582166963 54.64251856269729))
> 
> However, if I put this exact same code into my Flask app (currently running under the Flask development environment) as part of handling a request, I get an empty polygon:
> 
> >>>    import shapely
> >>>    print(shapely.__version__)
> >>>    from shapely.geometry import Polygon
> >>>    print(Polygon.from_bounds(*bounds))
> 
> Output:
> 
> 1.8.0
> POLYGON EMPTY
> 
> In fact, *any* attempt to create a polygon gives the same result:
> >>> test = Polygon(((1, 1), (2, 1), (2, 2)))
> >>> print(test)
> POLYGON EMPTY
> 
> What am I missing here? Why doesn?t it work as part of a Flask request call?
> ---
> Israel Brewster
> Software Engineer
> Alaska Volcano Observatory 
> Geophysical Institute - UAF 
> 2156 Koyukuk Drive 
> Fairbanks AK 99775-7320
> Work: 907-474-5172
> cell:  907-328-9145
> 


From Marco.Sulla.Python at gmail.com  Wed Jan  5 16:00:34 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Wed, 5 Jan 2022 22:00:34 +0100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
 <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
Message-ID: <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>

On Wed, 5 Jan 2022 at 14:16, Chris Angelico <rosuav at gmail.com> wrote:
> That's an entirely invisible optimization, but it's more than just
> "frozenset is faster than set". It's that a frozenset or tuple can be
> stored as a function's constants, which is a massive difference.

Can you explain this?

> In fact, the two data types are virtually identical in performance once created [...]

This is really strange, since in theory frozenset should not have to
check if itself is mutated during the iteration, on each cycle. So the
speed should be noticeable faster. Maybe frozenset was not optimised,
because the use case is really little and will add potentially useless
C code? Furthermore, more the code, more the memory consumption and
less the speed. I have to check setobject.c.

> Function positional arguments aren't interchangeable, so it makes
> sense to have them as a tuple.

You are wrong, since kwarg is a dict. Indeed I proposed to use
frozendict for kwargs, and Guido said that it's a pity that this will
break a lot of existing Python code :D, since the fact that args is
_immutable_ and kwargs not always bothered him.

Anyway, I'm starting to think that neither set nor frozenset are good
for dict items:

(venv_3_10) marco at buzz:~$ python
Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
[GCC 10.1.1 20200718] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = {1: 2}
>>> b = {3: []}
>>> a | b
{1: 2, 3: []}
>>> a.items() | b.items()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'
>>>

From rosuav at gmail.com  Wed Jan  5 17:00:13 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 6 Jan 2022 09:00:13 +1100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
 <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
 <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>
Message-ID: <CAPTjJmqJ+nZfO-T6bkCbSERVko2TbQP5g11y_oB0+ZgUx06cfw@mail.gmail.com>

On Thu, Jan 6, 2022 at 8:01 AM Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
>
> On Wed, 5 Jan 2022 at 14:16, Chris Angelico <rosuav at gmail.com> wrote:
> > That's an entirely invisible optimization, but it's more than just
> > "frozenset is faster than set". It's that a frozenset or tuple can be
> > stored as a function's constants, which is a massive difference.
>
> Can you explain this?

Play around with dis.dis and timeit.

> > Function positional arguments aren't interchangeable, so it makes
> > sense to have them as a tuple.
>
> You are wrong, since kwarg is a dict. Indeed I proposed to use
> frozendict for kwargs, and Guido said that it's a pity that this will
> break a lot of existing Python code :D, since the fact that args is
> _immutable_ and kwargs not always bothered him.

Excuse me? I mentioned kwargs in the part that you removed from the
quote, and the part you're quoting explicitly says "positional
arguments".

> Anyway, I'm starting to think that neither set nor frozenset are good
> for dict items:
>
> (venv_3_10) marco at buzz:~$ python
> Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
> [GCC 10.1.1 20200718] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> a = {1: 2}
> >>> b = {3: []}
> >>> a | b
> {1: 2, 3: []}
> >>> a.items() | b.items()
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: unhashable type: 'list'
> >>>

Well yes. Only dict keys can be considered to be set-like. I don't
know WHAT you think you're trying to do here, but if you ever thought
of set operations on dict values, you may want to completely rethink
what you're doing.

Performance is not an automatic result of immutability. That simply
isn't how it works.

ChrisA

From jarrodkg at gmail.com  Wed Jan  5 17:26:16 2022
From: jarrodkg at gmail.com (Jarrod G)
Date: Wed, 5 Jan 2022 17:26:16 -0500
Subject: How best to handle SystemError in ctypes callback
Message-ID: <CAJMuovqyCgmnLpJ06t=aFvfjaW4=A+VAyfNpyCz6KCz8Y7=0ag@mail.gmail.com>

With the following code:

import ctypes
> import readline
> from ctypes.util import find_library
>
> rl = ctypes.cdll.LoadLibrary(find_library('readline'))
>
> rl_redisplay = rl.rl_redisplay
> rl_redisplay.restype = None
> rl_redisplay.argtypes = None
>
> rl_redisplay_function = ctypes.c_void_p.in_dll(rl, 'rl_redisplay_function')
>
> def _wrapper():
>     rl_redisplay()
>
> fp = ctypes.CFUNCTYPE(None)(_wrapper)
> rl_redisplay_function.value = ctypes.cast(fp, ctypes.c_void_p).value
>
> input('enter ctrl-c now')
>

What is the best or correct way to handle SystemError when a Ctrl-c happens
at the input prompt? Is a try/except around rl_redisplay the best or
correct approach?

Based on the docs for SystemError (
https://docs.python.org/3/library/exceptions.html#SystemError), I don't
want to mask/hide an internal error.

Jarrod

From kushal at locationd.net  Thu Jan  6 13:02:59 2022
From: kushal at locationd.net (Kushal Kumaran)
Date: Thu, 06 Jan 2022 10:02:59 -0800
Subject: AttributeError: 'NoneType' object has no attribute 'get'
In-Reply-To: <9f86242d-81cf-4deb-b681-ce760aeac69an@googlegroups.com>
 (NArshad's message of "Tue, 4 Jan 2022 23:34:20 -0800 (PST)")
References: <9f86242d-81cf-4deb-b681-ce760aeac69an@googlegroups.com>
Message-ID: <87r19kbv3g.fsf@locationd.net>

On Tue, Jan 04 2022 at 11:34:20 PM, NArshad <narshad.380 at gmail.com> wrote:
> How to correct what is written below:
>
> Exception in Tkinter callback
> Traceback (most recent call last):
>   File "C:\Users\Dani Brothers\Anaconda3\lib\tkinter\__init__.py", line 1705, in __call__
>     return self.func(*args)
>   File "D:/Python/Book Bank/New folder/PyCharm/Final/Excel.py", line 57, in SaveBook
>     e_pissue.get(),
> AttributeError: 'NoneType' object has no attribute 'get'
>
> Process finished with exit code 0

The error means that e_pissue is None, so your code cannot call the get
method on it.  You need to fix the code so that e_pissue is not None and
is instead referring to an object that has a suitable get method.

Without looking at the rest of your code, it is not possible to get any
more specific.  Looking at the path of the file, it looks like you may
be following some programming exercises from a book.  If so, perhaps you
need to look up the section about understanding exception tracebacks.

-- 
regards,
kushal

From mats at wichmann.us  Thu Jan  6 16:21:48 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Thu, 6 Jan 2022 14:21:48 -0700
Subject: AttributeError: 'NoneType' object has no attribute 'get'
In-Reply-To: <87r19kbv3g.fsf@locationd.net>
References: <9f86242d-81cf-4deb-b681-ce760aeac69an@googlegroups.com>
 <87r19kbv3g.fsf@locationd.net>
Message-ID: <17b830d4-d2f9-0a62-41a6-eff38a85c66a@wichmann.us>

On 1/6/22 11:02, Kushal Kumaran wrote:
> On Tue, Jan 04 2022 at 11:34:20 PM, NArshad <narshad.380 at gmail.com> wrote:
>> How to correct what is written below:
>>
>> Exception in Tkinter callback
>> Traceback (most recent call last):
>>   File "C:\Users\Dani Brothers\Anaconda3\lib\tkinter\__init__.py", line 1705, in __call__
>>     return self.func(*args)
>>   File "D:/Python/Book Bank/New folder/PyCharm/Final/Excel.py", line 57, in SaveBook
>>     e_pissue.get(),
>> AttributeError: 'NoneType' object has no attribute 'get'
>>
>> Process finished with exit code 0
> 
> The error means that e_pissue is None, so your code cannot call the get
> method on it.  You need to fix the code so that e_pissue is not None and
> is instead referring to an object that has a suitable get method.


And at a more meta level:  many functions in the Python world return
None as an indication that the operation did not succeed.  It's useful
because in many circumstances None is an "out of band" value - one that
could not happen naturally - and thus returning it provides an easy way
for the caller to check for success or failure.

However... if that is the case, you have to actually do that check.  If
you just proceed as if a vaild value had come back, you're defeating the
purpose - and are sure to fall into a hole like the one you've encountered.

From 2QdxY4RzWzUUiLuE at potatochowder.com  Thu Jan  6 16:45:52 2022
From: 2QdxY4RzWzUUiLuE at potatochowder.com (2QdxY4RzWzUUiLuE at potatochowder.com)
Date: Thu, 6 Jan 2022 13:45:52 -0800
Subject: AttributeError: 'NoneType' object has no attribute 'get'
In-Reply-To: <17b830d4-d2f9-0a62-41a6-eff38a85c66a@wichmann.us>
References: <9f86242d-81cf-4deb-b681-ce760aeac69an@googlegroups.com>
 <87r19kbv3g.fsf@locationd.net>
 <17b830d4-d2f9-0a62-41a6-eff38a85c66a@wichmann.us>
Message-ID: <YddjEPaiF9pfLXf7@scrozzle>

On 2022-01-06 at 14:21:48 -0700,
Mats Wichmann <mats at wichmann.us> wrote:

> And at a more meta level:  many functions in the Python world return
> None as an indication that the operation did not succeed.  It's useful
> because in many circumstances None is an "out of band" value - one
> that could not happen naturally - and thus returning it provides an
> easy way for the caller to check for success or failure.

Errors should never pass silently.

Unless explicitly silenced.

https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare/

Exceptions aren't perfect, but when something fails, I'd much rather
have an exception raised and thrown at me than to get back None.  An
exception is immediate,? but None often ends up causing trouble far
away? from where the actual failure occurred.

? in space and in time

From rosuav at gmail.com  Thu Jan  6 19:37:47 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 7 Jan 2022 11:37:47 +1100
Subject: AttributeError: 'NoneType' object has no attribute 'get'
In-Reply-To: <YddjEPaiF9pfLXf7@scrozzle>
References: <9f86242d-81cf-4deb-b681-ce760aeac69an@googlegroups.com>
 <87r19kbv3g.fsf@locationd.net>
 <17b830d4-d2f9-0a62-41a6-eff38a85c66a@wichmann.us>
 <YddjEPaiF9pfLXf7@scrozzle>
Message-ID: <CAPTjJmp0dX15vR54JXQH6JFciTutg7w4FmFaTL5rubDDPx6UFQ@mail.gmail.com>

On Fri, Jan 7, 2022 at 8:47 AM <2QdxY4RzWzUUiLuE at potatochowder.com> wrote:
>
> On 2022-01-06 at 14:21:48 -0700,
> Mats Wichmann <mats at wichmann.us> wrote:
>
> > And at a more meta level:  many functions in the Python world return
> > None as an indication that the operation did not succeed.  It's useful
> > because in many circumstances None is an "out of band" value - one
> > that could not happen naturally - and thus returning it provides an
> > easy way for the caller to check for success or failure.
>
> Errors should never pass silently.
>
> Unless explicitly silenced.
>
> https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare/
>

Python is not C, and returning None is not the same as returning a
null pointer. Also, I dispute that null pointers are the
"billion-dollar mistake" described there, but of course, everyone
loves to talk about figures like that.

Returning None is most definitely not a flaw.

ChrisA

From mats at wichmann.us  Fri Jan  7 09:42:45 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Fri, 7 Jan 2022 07:42:45 -0700
Subject: Pip not working
In-Reply-To: <4E8E0D9E-EC51-4AFC-B28B-FB41B36B3A3B@gmail.com>
References: <4E8E0D9E-EC51-4AFC-B28B-FB41B36B3A3B@gmail.com>
Message-ID: <d8f12dbf-fa2a-d389-8edf-4a9d1c522815@wichmann.us>

On 1/5/22 01:29, Malick Lickta Keita wrote:
> ?Hello I?ve installed the latest version of python but pip is not working 

If you want some help with that, you're going to have to provide some
details - no person on earth who isn't sitting looking at your computer
screen with you would know what is going on. What does "is not working"
mean?  How does it manifest?  Error messages? Things you've tried?

From skip.montanaro at gmail.com  Fri Jan  7 13:51:48 2022
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Fri, 7 Jan 2022 12:51:48 -0600
Subject: Gunicorn - HTTP and HTTPS in the same instance?
Message-ID: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>

Hopefully some Pythonistas are also Gunicornistas. I've had little success
finding help with a small dilemma in the docs or in other more specific
sources.

I'm testing out a new, small website. It is just Gunicorn+Flask. I'd like
to both listen for HTTP and HTTPS connections. Accordingly, in my config, I
have the Gunicorn process bind to both ports 80 and 443 if running as root:

if IAM_ROOT:
    bind = [
        '0.0.0.0:443',
        '0.0.0.0:80',
    ]
else:
    bind = [
        '0.0.0.0:8080',
    ]

Gunicorn listens on both ports, but insists on SSL/TLS chit chat over port
80, not just port 443 (which seems to work okay). Is there some magic
incantation to get it to just talk HTTP on port 80, or will I need to spin
up two instances? (The non-root config works fine - plain old HTTP over
port 8080.)

Thx,

Skip

From Marco.Sulla.Python at gmail.com  Fri Jan  7 14:27:17 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Fri, 7 Jan 2022 20:27:17 +0100
Subject: Pickle segfaults with custom type
Message-ID: <CABbU2U_1juvW8fye8K+U31tWgNX9myN+eDqqM0g7GtJG7HP-ng@mail.gmail.com>

I have a custom implementation of dict using a C extension. All works but
the pickling of views and iter types. Python segfaults if I try to pickle
them.

For example, I have:


static PyTypeObject PyFrozenDictIterKey_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    "frozendict.keyiterator",                   /* tp_name */
    sizeof(dictiterobject),                     /* tp_basicsize */
    0,                                          /* tp_itemsize */
    /* methods */
    (destructor)dictiter_dealloc,               /* tp_dealloc */
    0,                                          /* tp_vectorcall_offset */
    0,                                          /* tp_getattr */
    0,                                          /* tp_setattr */
    0,                                          /* tp_as_async */
    0,                                          /* tp_repr */
    0,                                          /* tp_as_number */
    0,                                          /* tp_as_sequence */
    0,                                          /* tp_as_mapping */
    PyObject_HashNotImplemented,                /* tp_hash */
    0,                                          /* tp_call */
    0,                                          /* tp_str */
    PyObject_GenericGetAttr,                    /* tp_getattro */
    0,                                          /* tp_setattro */
    0,                                          /* tp_as_buffer */
    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,    /* tp_flags */
    0,                                          /* tp_doc */
    (traverseproc)dictiter_traverse,            /* tp_traverse */
    0,                                          /* tp_clear */
    0,                                          /* tp_richcompare */
    0,                                          /* tp_weaklistoffset */
    PyObject_SelfIter,                          /* tp_iter */
    (iternextfunc)frozendictiter_iternextkey,   /* tp_iternext */
    dictiter_methods,                           /* tp_methods */
    0,
};

This is the backtrace I get with gdb:

#0  PyObject_Hash (v=0x7f043ce15540 <PyFrozenDictIterKey_Type>) at
../cpython_3_10/Objects/object.c:788
#1  0x000000000048611c in PyDict_GetItemWithError (op=0x7f043e1f4900,
key=key at entry=0x7f043ce15540 <PyFrozenDictIterKey_Type>)
    at ../cpython_3_10/Objects/dictobject.c:1520
#2  0x00007f043ce227f6 in save (self=self at entry=0x7f043d8507d0,
obj=obj at entry=0x7f043e1fb0b0, pers_save=pers_save at entry=0)
    at /home/marco/sources/cpython_3_10/Modules/_pickle.c:4381
#3  0x00007f043ce2534d in dump (self=self at entry=0x7f043d8507d0,
obj=obj at entry=0x7f043e1fb0b0) at
/home/marco/sources/cpython_3_10/Modules/_pickle.c:4515
#4  0x00007f043ce2567f in _pickle_dumps_impl (module=<optimized out>,
buffer_callback=<optimized out>, fix_imports=<optimized out>,
protocol=<optimized out>,
    obj=0x7f043e1fb0b0) at
/home/marco/sources/cpython_3_10/Modules/_pickle.c:1203
#5  _pickle_dumps (module=<optimized out>, args=<optimized out>,
nargs=<optimized out>, kwnames=<optimized out>)
    at /home/marco/sources/cpython_3_10/Modules/clinic/_pickle.c.h:619

and so on. The problematic part is in the second frame. Indeed the code of
_pickle.c here is:


        reduce_func = PyDict_GetItemWithError(st->dispatch_table,
                                              (PyObject *)type);

The problem is that type is NULL. It tries to get the attribute tp_hash and
it segfaults.

I tried to change the header of the type to:

PyVarObject_HEAD_INIT(&PyType_Type, 0)

This way it works but, as known, it does not compile on Windows.

The strange fact is that pickling the main type works, even if the type is
NULL, as suggested for a custom type. This is the main type:

PyTypeObject PyFrozenDict_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
    "frozendict." FROZENDICT_CLASS_NAME,        /* tp_name */
    sizeof(PyFrozenDictObject),                 /* tp_basicsize */
    0,                                          /* tp_itemsize */
    (destructor)dict_dealloc,                   /* tp_dealloc */
    0,                                          /* tp_vectorcall_offset */
    0,                                          /* tp_getattr */
    0,                                          /* tp_setattr */
    0,                                          /* tp_as_async */
    (reprfunc)frozendict_repr,                  /* tp_repr */
    &frozendict_as_number,                      /* tp_as_number */
    &dict_as_sequence,                          /* tp_as_sequence */
    &frozendict_as_mapping,                     /* tp_as_mapping */
    (hashfunc)frozendict_hash,                  /* tp_hash */
    0,                                          /* tp_call */
    0,                                          /* tp_str */
    PyObject_GenericGetAttr,                    /* tp_getattro */
    0,                                          /* tp_setattro */
    0,                                          /* tp_as_buffer */
    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC
        | Py_TPFLAGS_BASETYPE
        | _Py_TPFLAGS_MATCH_SELF
        | Py_TPFLAGS_MAPPING,                   /* tp_flags */
    frozendict_doc,                             /* tp_doc */
    dict_traverse,                              /* tp_traverse */
    0,                                          /* tp_clear */
    dict_richcompare,                           /* tp_richcompare */
    0,                                          /* tp_weaklistoffset */
    (getiterfunc)frozendict_iter,               /* tp_iter */
    0,                                          /* tp_iternext */
    frozendict_mapp_methods,                    /* tp_methods */
    0,                                          /* tp_members */
    0,                                          /* tp_getset */
    0,                                          /* tp_base */
    0,                                          /* tp_dict */
    0,                                          /* tp_descr_get */
    0,                                          /* tp_descr_set */
    0,                                          /* tp_dictoffset */
    0,                                          /* tp_init */
    PyType_GenericAlloc,                        /* tp_alloc */
    frozendict_new,                             /* tp_new */
    PyObject_GC_Del,                            /* tp_free */
    .tp_vectorcall = frozendict_vectorcall,
};

From narshad.380 at gmail.com  Thu Jan  6 13:55:30 2022
From: narshad.380 at gmail.com (NArshad)
Date: Thu, 6 Jan 2022 10:55:30 -0800 (PST)
Subject: What to write or search on github to get the code for what is written
 below:
Message-ID: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>

All this is going to be in python?s flask and HTML only

1. First, I have to check in the Excel sheet or table whether the book user has entered is present in the book bank or not.

2. If a book is present and the quantity of the required book is greater than 0 (COPIES_LEFT column in excel file) and if the user wants the book, it will be assigned to the user which he will take from the book bank physically. When COPIES_LEFT will is less than or equal to 0 the message will be ?Book finished or not present?.

3. The quantity of the book in the Excel file will be reduced by 1 in the COPIES_LEFT column and the name of the borrower or user will be entered/added in the Excel file table or sheet already made and the column name is BORROWER?S NAME.

4. The borrower?s or user name can be more than one so they will be separated with a comma in the Excel file BORROWER?S NAME column.


- All functions mentioned above are to be deployed on the website pythonhow.com so make according to https://pythonhow.com/python-tutorial/flask/web-development-with-python-and-flask/

- Do you know any other websites to deploy a python web application??

- No time to switch from Excel to anywhere else. Please do not make any changes to the Excel file.

- Tutorials and repositories of the web for such problems are also required. The same is required for python (flask, Django...) also.


From alister.ware at ntlworld.com  Thu Jan  6 16:58:59 2022
From: alister.ware at ntlworld.com (alister)
Date: Thu, 6 Jan 2022 21:58:59 -0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
Message-ID: <sr7on3$1ji0$1@gioia.aioe.org>

On Thu, 6 Jan 2022 10:55:30 -0800 (PST), NArshad wrote:

> All this is going to be in python?s flask and HTML only
> 
> 1. First, I have to check in the Excel sheet or table whether the book
> user has entered is present in the book bank or not.
> 
Excel is the wrong application for storing this data - it should be in a 
database

> 2. If a book is present and the quantity of the required book is greater
> than 0 (COPIES_LEFT column in excel file) and if the user wants the
> book, it will be assigned to the user which he will take from the book
> bank physically. When COPIES_LEFT will is less than or equal to 0 the
> message will be ?Book finished or not present?.
Again Excel is not the correct application for processing this data
> 
> 3. The quantity of the book in the Excel file will be reduced by 1 in
> the COPIES_LEFT column and the name of the borrower or user will be
> entered/added in the Excel file table or sheet already made and the
> column name is BORROWER?S NAME.
> 
Database!
> 4. The borrower?s or user name can be more than one so they will be
> separated with a comma in the Excel file BORROWER?S NAME column.
> 
Database
> 
> - All functions mentioned above are to be deployed on the website
> pythonhow.com so make according to
> https://pythonhow.com/python-tutorial/flask/web-development-with-python-
and-flask/
> 
> - Do you know any other websites to deploy a python web application??
> 
> - No time to switch from Excel to anywhere else. Please do not make any
> changes to the Excel file.
> 
> - Tutorials and repositories of the web for such problems are also
> required. The same is required for python (flask, Django...) also.

Sorry did not spot that this was a homework assignment
data should still be imported into a DB (a trivial task) It can be 
exported back to a compatible format just as easily if hard copy output is 
required




-- 
"Rembrandt's first name was Beauregard, which is why he never used it."
		-- Dave Barry

From rosuav at gmail.com  Fri Jan  7 14:57:28 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 8 Jan 2022 06:57:28 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
Message-ID: <CAPTjJmqiDjwrWrbC6F74ZPoKsUWUhsOsLESzbczKtwVrCbLq7w@mail.gmail.com>

On Sat, Jan 8, 2022 at 6:50 AM NArshad <narshad.380 at gmail.com> wrote:
> - All functions mentioned above are to be deployed on the website pythonhow.com so make according to https://pythonhow.com/python-tutorial/flask/web-development-with-python-and-flask/
>
> - Do you know any other websites to deploy a python web application??
>
> - No time to switch from Excel to anywhere else. Please do not make any changes to the Excel file.
>
> - Tutorials and repositories of the web for such problems are also required. The same is required for python (flask, Django...) also.
>

Instead of asking how to search GitHub for the code you want, start by
actually writing code.

ChrisA

From python at mrabarnett.plus.com  Fri Jan  7 15:22:54 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 7 Jan 2022 20:22:54 +0000
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
Message-ID: <152d2d00-87de-e836-9884-4fe0a81bb6bb@mrabarnett.plus.com>

On 2022-01-06 18:55, NArshad wrote:
> All this is going to be in python?s flask and HTML only
> 
> 1. First, I have to check in the Excel sheet or table whether the book user has entered is present in the book bank or not.
> 
> 2. If a book is present and the quantity of the required book is greater than 0 (COPIES_LEFT column in excel file) and if the user wants the book, it will be assigned to the user which he will take from the book bank physically. When COPIES_LEFT will is less than or equal to 0 the message will be ?Book finished or not present?.
> 
> 3. The quantity of the book in the Excel file will be reduced by 1 in the COPIES_LEFT column and the name of the borrower or user will be entered/added in the Excel file table or sheet already made and the column name is BORROWER?S NAME.
> 
> 4. The borrower?s or user name can be more than one so they will be separated with a comma in the Excel file BORROWER?S NAME column.
> 
> 
> - All functions mentioned above are to be deployed on the website pythonhow.com so make according to https://pythonhow.com/python-tutorial/flask/web-development-with-python-and-flask/
> 
> - Do you know any other websites to deploy a python web application??
> 
> - No time to switch from Excel to anywhere else. Please do not make any changes to the Excel file.
> 
> - Tutorials and repositories of the web for such problems are also required. The same is required for python (flask, Django...) also.
> 
As someone else has already pointed, Excel is the wrong way to do it, 
yet it remains a popular method.

Given that, you can read Excel spreadsheets with 'openpyxl', which is on 
PyPI.

From avigross at verizon.net  Fri Jan  7 16:12:53 2022
From: avigross at verizon.net (Avi Gross)
Date: Fri, 7 Jan 2022 21:12:53 +0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <CAPTjJmqiDjwrWrbC6F74ZPoKsUWUhsOsLESzbczKtwVrCbLq7w@mail.gmail.com>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <CAPTjJmqiDjwrWrbC6F74ZPoKsUWUhsOsLESzbczKtwVrCbLq7w@mail.gmail.com>
Message-ID: <563105087.1701352.1641589973481@mail.yahoo.com>

This entire thread seems a bit IFFY to me.
It does seme like HW to me but also a bit peripheral.
The fact that the data is in EXCEL is a detail. And unless a spreadheet is complex, it may be trivial to save the file as a .CSV and from then on read from there into Python (or anything) and when done making modifications, save it out again. EXCEL can also be used to open such a CSV and look at it. And, of course, as has been pointed out, there are ways to save data into EXCEL or read directly from it.
But the overall programming ideas required, once the data are in memory, are another story, especially if some fields are in ormats like a comma separated text string. Searching these is not trivial while expanding the data so each such "person" is on their own may simplify things, albeit not make it so easy to save it out.
I do note that unless there is a guarantee that reads and updates to where the data is stored are guaranteed to not collide with other things, there lurks danger here.
So if the request is not made more clear and focused on what is needed in mostly native Python, and accompanied by some evidence the user has shown some code they hope will do most of the work and is just stuck somewhere, I, personally, feel no need to offer any guidance.?


-----Original Message-----
From: Chris Angelico <rosuav at gmail.com>
To: python-list at python.org
Sent: Fri, Jan 7, 2022 2:57 pm
Subject: Re: What to write or search on github to get the code for what is written below:

On Sat, Jan 8, 2022 at 6:50 AM NArshad <narshad.380 at gmail.com> wrote:
> - All functions mentioned above are to be deployed on the website pythonhow.com so make according to https://pythonhow.com/python-tutorial/flask/web-development-with-python-and-flask/
>
> - Do you know any other websites to deploy a python web application??
>
> - No time to switch from Excel to anywhere else. Please do not make any changes to the Excel file.
>
> - Tutorials and repositories of the web for such problems are also required. The same is required for python (flask, Django...) also.
>

Instead of asking how to search GitHub for the code you want, start by
actually writing code.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

From nt_mahmood at yahoo.com  Fri Jan  7 18:07:49 2022
From: nt_mahmood at yahoo.com (Mahmood Naderan)
Date: Sat, 8 Jan 2022 00:07:49 +0100
Subject: Extracting dataframe column with multiple conditions on row values
References: <92704ABE-B6C4-4F7C-BF40-1883C61A3964.ref@hxcore.ol>
Message-ID: <92704ABE-B6C4-4F7C-BF40-1883C61A3964@hxcore.ol>

   Hi



   I have a csv file like this



   V0,V1,V2,V3

   4,1,1,1

   6,4,5,2

   2,3,6,7





   And I want to search two rows for a match and find the column. For
   example, I want to search row[0] for 1 and row[1] for 5. The corresponding
   column is V2 (which is the third column). Then I want to return the value
   at row[2] and the found column. The result should be 6 then.



   I can manually extract the specified rows (with index 0 and 1 which are
   fixed) and manually iterate over them like arrays to find a match. Then I



   key1 = 1

   key2 = 5

   row1 = df.iloc[0]              # row=[4,1,1,1]

   row2 = df.iloc[1]              # row=[6,4,5,2]

   for i in range(len(row1)):

       if row1[i] == key1:

           for j in range(len(row2)):

                if row2[j] == key2:

                    res = df.iloc[:,j]

                    print(res)    # 6





   Is there any way to use built-in function for a more efficient code?





   Regards,

   Mahmood



From PythonList at DancesWithMice.info  Fri Jan  7 20:21:05 2022
From: PythonList at DancesWithMice.info (dn)
Date: Sat, 8 Jan 2022 14:21:05 +1300
Subject: Extracting dataframe column with multiple conditions on row values
In-Reply-To: <92704ABE-B6C4-4F7C-BF40-1883C61A3964@hxcore.ol>
References: <92704ABE-B6C4-4F7C-BF40-1883C61A3964.ref@hxcore.ol>
 <92704ABE-B6C4-4F7C-BF40-1883C61A3964@hxcore.ol>
Message-ID: <7530730e-b793-3075-2ec5-ee2e3160e0bb@DancesWithMice.info>

Salaam Mahmood,


On 08/01/2022 12.07, Mahmood Naderan via Python-list wrote:
>    I have a csv file like this
>    V0,V1,V2,V3
>    4,1,1,1
>    6,4,5,2
>    2,3,6,7
> 
>    And I want to search two rows for a match and find the column. For
>    example, I want to search row[0] for 1 and row[1] for 5. The corresponding
>    column is V2 (which is the third column). Then I want to return the value
>    at row[2] and the found column. The result should be 6 then.

Not quite: isn't the "found column" also required?


>    I can manually extract the specified rows (with index 0 and 1 which are
>    fixed) and manually iterate over them like arrays to find a match. Then I

Perhaps this idea has been influenced by a similar solution in another
programming language. May I suggest that the better-answer you seek lies
in using Python idioms (as well as Python's tools)...


>    key1 = 1
>    key2 = 5

Fine, so far - excepting that this 'problem' is likely to be a small
part of some larger system. Accordingly, consider writing it as a
function. In which case, these two "keys" will become
function-parameters (and the two 'results' become return-values).


>    row1 = df.iloc[0]              # row=[4,1,1,1]
>    row2 = df.iloc[1]              # row=[6,4,5,2]

This is likely not native-Python. Let's create lists for 'everything',
just-because:

>>> headings = [ "V0","V1","V2","V3" ]
>>> row1 = [4,1,1,1]
>>> row2 = [6,4,5,2]
>>> results = [ 2,3,6,7 ]


Note how I'm using the Python REPL (in a "terminal", type "python" (as
appropriate to your OpSys) at the command-line). IMHO the REPL is a
grossly under-rated tool, and is a very good means towards
trial-and-error, and learning by example. Highly recommended!


>    for i in range(len(row1)):

This construction is very much a "code smell" for thinking that it is
not "pythonic". (and perhaps the motivation for this post)

In Python (compared with many other languages) the "for" loop should
actually be pronounced "for-each". In other words when we pair the
code-construct with a list (for example):

for each item in the list the computer should perform some suite of
commands.

(the "suite" is everything 'inside' the for-each-loop - NB my
'Python-betters' will quickly point-out that this feature is not limited
to Python-lists, but will work with any :iterable" - ref:
https://docs.python.org/3/tutorial/controlflow.html#for-statements)


Thus:

> for item in headings: print( item )
...
V0
V1
V2
V3


The problem is that when working with matrices/matrixes, a math
background equips one with the idea of indices/indexes, eg the
ubiquitous subscript-i. Accordingly, when reading 'math' where a formula
uses the upper-case Greek "sigma" character, remember that it means "for
all" or "for each"!

So, if Python doesn't use indexing or "pointers", how do we deal with
the problem?

Unfortunately, at first glance, the pythonic approach may seem
more-complicated or even somewhat convoluted, but once the concepts
(and/or the Python idioms) are learned, it is quite manageable (and
applicable to many more applications than matrices/matrixes!)...


>        if row1[i] == key1:
>            for j in range(len(row2)):
>                 if row2[j] == key2:
>                     res = df.iloc[:,j]
>                     print(res)    # 6
> 
>    Is there any way to use built-in function for a more efficient code?


This is where your idea bears fruit!

There is a Python "built-in function": zip(), which will 'join' lists.
NB do not become confused between zip() and zip archive/compressed files!

Most of the time reference book and web-page examples show zip() being
used to zip-together two lists into a single data-construct (which is an
iterable(!)). However, zip() will actually zip-together multiple (more
than two) "iterables". As the manual says:

?zip() returns an iterator of tuples, where the i-th tuple contains the
i-th element from each of the argument iterables.?

Ah, so that's where the math-idea of subscript-i went! It has become
'hidden' in Python's workings - or putting that another way: Python
looks after the subscripting for us (and given that 'out by one' errors
in pointers is a major source of coding-error in other languages,
thank-you very much Python!)

First re-state the source-data as Python lists, (per above) - except
that I recommend the names be better-chosen to be more meaningful (to
your application)!


Now, (in the REPL) try using zip():

>>> zip( headings, row1, row2, results )
<zip object at 0x7f655cca6bc0>

Does that seem a very good illustration? Not really, but re-read the
quotation from the manual (above) where it says that zip returns an
iterator. If we want to see the values an iterator will produce, then
turn it into an iterable data-structure, eg:

>>> list( zip( headings, row1, row2, results ) )
[('V0', 4, 6, 2), ('V1', 1, 4, 3), ('V2', 1, 5, 6), ('V3', 1, 2, 7)]

or, to see things more clearly, let me re-type it as:

[
('V0', 4, 6, 2),
('V1', 1, 4, 3),
('V2', 1, 5, 6),
('V3', 1, 2, 7)
]


What we now see is actually a "transpose" of the original 'matrix'
presented in the post/question!

(NB Python will perform this layout for us - read about the pprint library)


Another method which can also be employed (and which will illustrate the
loop required to code the eventual-solution(!)) is that Python's next()
will extract the first row of the transpose:

>>> row = next( zip( headings, row1, row2, results ) )
>>> row
('V0', 4, 6, 2)


This is all-well-and-good, but that result is a tuple of four items
(corresponding to one column in the way the source-data was explained).

If we need to consider the four individual data-items, that can be
improved using a Python feature called "tuple unpacking". Instead of the
above delivering a tuple which is then assigned to "row", the tuple can
be assigned to four "identifiers", eg

>>> heading, row1_item, row2_item, result= next( zip( headings, row1,
row2, results ) )

(apologies about email word-wrapping - this is a single line of Python-code)


Which, to prove the case, could be printed:

>>> heading, row1_item, row2_item, result
('V0', 4, 6, 2)


(ref:
https://docs.python.org/3/tutorial/datastructures.html?highlight=tuple%20unpacking#tuples-and-sequences)


Thus, if we repeatedly ask for the next() row from the zip-ped
transpose, eventually it will respond with the row starting 'V2' - which
is the desired-result, ie the row containing the 1, the 5, and the 6 -
and if you follow-through using the REPL, will be clearly visible.


Finally, 'all' that is required, is a for-each-loop which will iterate
across/down the zip object, one tuple (row of the transpose) at a time,
AND perform the "tuple-unpacking" all in one command, with an
if-statement to detect the correct row/column:

>>> for *tuple-unpacking* in *zip() etc*:
...     if row1_item == *what?* and row2_item == *what?*
...         print( *which* and *which identifier* )
...
V2 6

Yes, three lines. It's as easy as that!
(when you know how)

Worse: when you become more expert, you'll be able to compress all of
that down into a single-line solution - but it won't be as "readable" as
is this!


NB this question has a 'question-smell' of 'homework', so I'll not
complete the code for you - this is something *you* asked to learn and
the best way to learn is by 'doing' (not by 'reading').

However, please respond with your solution, or any further question
(with the next version of the code so-far, per this first-post - which
we appreciate!)

Regardless, you asked 'the right question' (curiosity is the key to
learning) and in the right way/manner. Well done!


NBB the above code-outline does not consider the situation where the
search fails/the keys are not found!


For further information, please review:
https://docs.python.org/3/library/functions.html?highlight=zip#zip

Also, further to the above discussion of combining lists and loops:
https://docs.python.org/3/tutorial/datastructures.html?highlight=zip#looping-techniques

and with a similar application (to this post):
https://docs.python.org/3/faq/programming.html?highlight=zip#how-can-i-sort-one-list-by-values-from-another-list

-- 
Regards,
=dn

From kushal at locationd.net  Fri Jan  7 20:54:25 2022
From: kushal at locationd.net (Kushal Kumaran)
Date: Fri, 07 Jan 2022 17:54:25 -0800
Subject: Gunicorn - HTTP and HTTPS in the same instance?
In-Reply-To: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>
 (Skip Montanaro's message of "Fri, 7 Jan 2022 12:51:48 -0600")
References: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>
Message-ID: <87mtk7at66.fsf@locationd.net>

On Fri, Jan 07 2022 at 12:51:48 PM, Skip Montanaro <skip.montanaro at gmail.com> wrote:
> Hopefully some Pythonistas are also Gunicornistas. I've had little success
> finding help with a small dilemma in the docs or in other more specific
> sources.
>
> I'm testing out a new, small website. It is just Gunicorn+Flask. I'd like
> to both listen for HTTP and HTTPS connections. Accordingly, in my config, I
> have the Gunicorn process bind to both ports 80 and 443 if running as root:
>
> if IAM_ROOT:
>     bind = [
>         '0.0.0.0:443',
>         '0.0.0.0:80',
>     ]
> else:
>     bind = [
>         '0.0.0.0:8080',
>     ]
>
> Gunicorn listens on both ports, but insists on SSL/TLS chit chat over port
> 80, not just port 443 (which seems to work okay). Is there some magic
> incantation to get it to just talk HTTP on port 80, or will I need to spin
> up two instances? (The non-root config works fine - plain old HTTP over
> port 8080.)
>

It is not possible to do this.  The ssl-ness is a global configuration,
and will apply to all of the listening sockets gunicorn creates.  To get
what you want, you need to run multiple instances, as you say, if you
can run those safely.

The recommended way to deploy gunicorn, though, is to front it with a
reverse proxy such as nginx.  You'd configure nginx (or whatever proxy
you choose) to listen on the interfaces/ports you want, and
enable/disable TLS as required.  Example for configuring nginx is at
https://docs.gunicorn.org/en/latest/deploy.html, although that
particular example does not talk about TLS.

-- 
regards,
kushal

From sjeik_appie at hotmail.com  Sat Jan  8 08:00:06 2022
From: sjeik_appie at hotmail.com (Albert-Jan Roskam)
Date: Sat, 08 Jan 2022 14:00:06 +0100
Subject: Gunicorn - HTTP and HTTPS in the same instance?
In-Reply-To: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>
Message-ID: <DB6PR01MB3895E8A8C5DD5A54E5C325FF834E9@DB6PR01MB3895.eurprd01.prod.exchangelabs.com>

   I always use NGINX for this. Run Flask/Gunicorn on localhost:5000 and have
   NGINX rewrite https requests to localhost requests. In nginx.conf I
   automatically redirect every http request to https. Static files are
   served by NGINX, not by Gunicorn, which is faster. NGINX also allows you
   to easily set the transfer encoding to gzip
   Albert-Jan

From skip.montanaro at gmail.com  Sat Jan  8 11:25:20 2022
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sat, 8 Jan 2022 10:25:20 -0600
Subject: Gunicorn - HTTP and HTTPS in the same instance?
In-Reply-To: <20220108161727.HRP0X993@iva4-b3ebd202b141.qloud-c.yandex.net>
References: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>
 <20220108161727.HRP0X993@iva4-b3ebd202b141.qloud-c.yandex.net>
Message-ID: <CANc-5UxwNO9u_QdwZZn2YROTUXiA4x9MnqHvBPu6Qt5ePKTLMg@mail.gmail.com>

Thanks all. I was hoping to get away without something more
sophisticated like NGINX. This is just a piddly little archive of an
old mailing list running on a single-core Ubuntu VM somewhere on the
East Coast. Speed is not a real requirement. Load balancing seemed
like overkill to me. Still, I guess if it has to be, then it has to
be.

Skip

From kvratkin at yandex.ru  Sat Jan  8 08:17:25 2022
From: kvratkin at yandex.ru (Kirill Ratkin)
Date: Sat, 8 Jan 2022 16:17:25 +0300
Subject: Gunicorn - HTTP and HTTPS in the same instance?
In-Reply-To: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>
References: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>
Message-ID: <20220108161727.HRP0X993@iva4-b3ebd202b141.qloud-c.yandex.net>


Hi.
You probably can solve issue on Gunicorn side. But afaik better solution is to use http proxy before Gunicorn. This proxy accepts https connection and proxy requests to Gunicorn instances as plain http.

This approach gives you:
a) Monitoring on network layer (tcpdump/wireshark shows you req/res on Gunicorn instance)
b) Scalability (proxy can spread traffic on several Gunicorn instances)
c) Maintenance (you can gracefully shutdown/restart Gunicorn instances one by one) 
d) Security (For example SSL certificate is configured in proxy only. There are another useful features which such proxy can do: simple authentication, ddos/fail2ban and so on)

Quite often NGINX is better choice for such proxy. But apache is good as well.

Best regards.
Kirill

??: Skip Montanaro
??????????: 7 ?????? 2022 ?. ? 21:54
????: Python
????: Gunicorn - HTTP and HTTPS in the same instance?

Hopefully some Pythonistas are also Gunicornistas. I've had little success
finding help with a small dilemma in the docs or in other more specific
sources.

I'm testing out a new, small website. It is just Gunicorn+Flask. I'd like
to both listen for HTTP and HTTPS connections. Accordingly, in my config, I
have the Gunicorn process bind to both ports 80 and 443 if running as root:

if IAM_ROOT:
    bind = [
        '0.0.0.0:443',
        '0.0.0.0:80',
    ]
else:
    bind = [
        '0.0.0.0:8080',
    ]

Gunicorn listens on both ports, but insists on SSL/TLS chit chat over port
80, not just port 443 (which seems to work okay). Is there some magic
incantation to get it to just talk HTTP on port 80, or will I need to spin
up two instances? (The non-root config works fine - plain old HTTP over
port 8080.)

Thx,

Skip
-- 
https://mail.python.org/mailman/listinfo/python-list


From edmondo.giovannozzi at gmail.com  Sat Jan  8 08:00:43 2022
From: edmondo.giovannozzi at gmail.com (Edmondo Giovannozzi)
Date: Sat, 8 Jan 2022 05:00:43 -0800 (PST)
Subject: Extracting dataframe column with multiple conditions on row values
In-Reply-To: <mailman.125.1641604882.3079.python-list@python.org>
References: <92704ABE-B6C4-4F7C-BF40-1883C61A3964.ref@hxcore.ol>
 <7530730e-b793-3075-2ec5-ee2e3160e0bb@DancesWithMice.info>
 <92704ABE-B6C4-4F7C-BF40-1883C61A3964@hxcore.ol>
 <mailman.125.1641604882.3079.python-list@python.org>
Message-ID: <6bc2ce03-75e3-48f7-8d7c-402345c6cb1cn@googlegroups.com>

Il giorno sabato 8 gennaio 2022 alle 02:21:40 UTC+1 dn ha scritto:
> Salaam Mahmood,
> On 08/01/2022 12.07, Mahmood Naderan via Python-list wrote: 
> > I have a csv file like this 
> > V0,V1,V2,V3 
> > 4,1,1,1 
> > 6,4,5,2 
> > 2,3,6,7 
> > 
> > And I want to search two rows for a match and find the column. For 
> > example, I want to search row[0] for 1 and row[1] for 5. The corresponding 
> > column is V2 (which is the third column). Then I want to return the value 
> > at row[2] and the found column. The result should be 6 then.
> Not quite: isn't the "found column" also required?
> > I can manually extract the specified rows (with index 0 and 1 which are 
> > fixed) and manually iterate over them like arrays to find a match. Then I
> Perhaps this idea has been influenced by a similar solution in another 
> programming language. May I suggest that the better-answer you seek lies 
> in using Python idioms (as well as Python's tools)...
> > key1 = 1 
> > key2 = 5
> Fine, so far - excepting that this 'problem' is likely to be a small 
> part of some larger system. Accordingly, consider writing it as a 
> function. In which case, these two "keys" will become 
> function-parameters (and the two 'results' become return-values).
> > row1 = df.iloc[0] # row=[4,1,1,1] 
> > row2 = df.iloc[1] # row=[6,4,5,2]
> This is likely not native-Python. Let's create lists for 'everything', 
> just-because: 
> 
> >>> headings = [ "V0","V1","V2","V3" ] 
> >>> row1 = [4,1,1,1] 
> >>> row2 = [6,4,5,2] 
> >>> results = [ 2,3,6,7 ] 
> 
> 
> Note how I'm using the Python REPL (in a "terminal", type "python" (as 
> appropriate to your OpSys) at the command-line). IMHO the REPL is a 
> grossly under-rated tool, and is a very good means towards 
> trial-and-error, and learning by example. Highly recommended! 
> 
> 
> > for i in range(len(row1)): 
> 
> This construction is very much a "code smell" for thinking that it is 
> not "pythonic". (and perhaps the motivation for this post) 
> 
> In Python (compared with many other languages) the "for" loop should 
> actually be pronounced "for-each". In other words when we pair the 
> code-construct with a list (for example): 
> 
> for each item in the list the computer should perform some suite of 
> commands. 
> 
> (the "suite" is everything 'inside' the for-each-loop - NB my 
> 'Python-betters' will quickly point-out that this feature is not limited 
> to Python-lists, but will work with any :iterable" - ref: 
> https://docs.python.org/3/tutorial/controlflow.html#for-statements) 
> 
> 
> Thus: 
> 
> > for item in headings: print( item ) 
> ... 
> V0 
> V1 
> V2 
> V3 
> 
> 
> The problem is that when working with matrices/matrixes, a math 
> background equips one with the idea of indices/indexes, eg the 
> ubiquitous subscript-i. Accordingly, when reading 'math' where a formula 
> uses the upper-case Greek "sigma" character, remember that it means "for 
> all" or "for each"! 
> 
> So, if Python doesn't use indexing or "pointers", how do we deal with 
> the problem? 
> 
> Unfortunately, at first glance, the pythonic approach may seem 
> more-complicated or even somewhat convoluted, but once the concepts 
> (and/or the Python idioms) are learned, it is quite manageable (and 
> applicable to many more applications than matrices/matrixes!)...
> > if row1[i] == key1: 
> > for j in range(len(row2)): 
> > if row2[j] == key2: 
> > res = df.iloc[:,j] 
> > print(res) # 6 
> > 
> > Is there any way to use built-in function for a more efficient code?
> This is where your idea bears fruit! 
> 
> There is a Python "built-in function": zip(), which will 'join' lists. 
> NB do not become confused between zip() and zip archive/compressed files! 
> 
> Most of the time reference book and web-page examples show zip() being 
> used to zip-together two lists into a single data-construct (which is an 
> iterable(!)). However, zip() will actually zip-together multiple (more 
> than two) "iterables". As the manual says: 
> 
> ?zip() returns an iterator of tuples, where the i-th tuple contains the 
> i-th element from each of the argument iterables.? 
> 
> Ah, so that's where the math-idea of subscript-i went! It has become 
> 'hidden' in Python's workings - or putting that another way: Python 
> looks after the subscripting for us (and given that 'out by one' errors 
> in pointers is a major source of coding-error in other languages, 
> thank-you very much Python!) 
> 
> First re-state the source-data as Python lists, (per above) - except 
> that I recommend the names be better-chosen to be more meaningful (to 
> your application)! 
> 
> 
> Now, (in the REPL) try using zip(): 
> 
> >>> zip( headings, row1, row2, results ) 
> <zip object at 0x7f655cca6bc0> 
> 
> Does that seem a very good illustration? Not really, but re-read the 
> quotation from the manual (above) where it says that zip returns an 
> iterator. If we want to see the values an iterator will produce, then 
> turn it into an iterable data-structure, eg: 
> 
> >>> list( zip( headings, row1, row2, results ) ) 
> [('V0', 4, 6, 2), ('V1', 1, 4, 3), ('V2', 1, 5, 6), ('V3', 1, 2, 7)] 
> 
> or, to see things more clearly, let me re-type it as: 
> 
> [ 
> ('V0', 4, 6, 2), 
> ('V1', 1, 4, 3), 
> ('V2', 1, 5, 6), 
> ('V3', 1, 2, 7) 
> ] 
> 
> 
> What we now see is actually a "transpose" of the original 'matrix' 
> presented in the post/question! 
> 
> (NB Python will perform this layout for us - read about the pprint library) 
> 
> 
> Another method which can also be employed (and which will illustrate the 
> loop required to code the eventual-solution(!)) is that Python's next() 
> will extract the first row of the transpose: 
> 
> >>> row = next( zip( headings, row1, row2, results ) ) 
> >>> row 
> ('V0', 4, 6, 2) 
> 
> 
> This is all-well-and-good, but that result is a tuple of four items 
> (corresponding to one column in the way the source-data was explained). 
> 
> If we need to consider the four individual data-items, that can be 
> improved using a Python feature called "tuple unpacking". Instead of the 
> above delivering a tuple which is then assigned to "row", the tuple can 
> be assigned to four "identifiers", eg 
> 
> >>> heading, row1_item, row2_item, result= next( zip( headings, row1, 
> row2, results ) ) 
> 
> (apologies about email word-wrapping - this is a single line of Python-code) 
> 
> 
> Which, to prove the case, could be printed: 
> 
> >>> heading, row1_item, row2_item, result 
> ('V0', 4, 6, 2) 
> 
> 
> (ref: 
> https://docs.python.org/3/tutorial/datastructures.html?highlight=tuple%20unpacking#tuples-and-sequences) 
> 
> 
> Thus, if we repeatedly ask for the next() row from the zip-ped 
> transpose, eventually it will respond with the row starting 'V2' - which 
> is the desired-result, ie the row containing the 1, the 5, and the 6 - 
> and if you follow-through using the REPL, will be clearly visible. 
> 
> 
> Finally, 'all' that is required, is a for-each-loop which will iterate 
> across/down the zip object, one tuple (row of the transpose) at a time, 
> AND perform the "tuple-unpacking" all in one command, with an 
> if-statement to detect the correct row/column: 
> 
> >>> for *tuple-unpacking* in *zip() etc*: 
> ... if row1_item == *what?* and row2_item == *what?* 
> ... print( *which* and *which identifier* ) 
> ... 
> V2 6 
> 
> Yes, three lines. It's as easy as that! 
> (when you know how) 
> 
> Worse: when you become more expert, you'll be able to compress all of 
> that down into a single-line solution - but it won't be as "readable" as 
> is this! 
> 
> 
> NB this question has a 'question-smell' of 'homework', so I'll not 
> complete the code for you - this is something *you* asked to learn and 
> the best way to learn is by 'doing' (not by 'reading'). 
> 
> However, please respond with your solution, or any further question 
> (with the next version of the code so-far, per this first-post - which 
> we appreciate!) 
> 
> Regardless, you asked 'the right question' (curiosity is the key to 
> learning) and in the right way/manner. Well done! 
> 
> 
> NBB the above code-outline does not consider the situation where the 
> search fails/the keys are not found! 
> 
> 
> For further information, please review: 
> https://docs.python.org/3/library/functions.html?highlight=zip#zip 
> 
> Also, further to the above discussion of combining lists and loops: 
> https://docs.python.org/3/tutorial/datastructures.html?highlight=zip#looping-techniques 
> 
> and with a similar application (to this post): 
> https://docs.python.org/3/faq/programming.html?highlight=zip#how-can-i-sort-one-list-by-values-from-another-list 
> 
> -- 
> Regards, 

You may also transpose your dataset. Then the index will become your column name and the column name become your index:
To read your dataset:

import pandas as pd
import io

DN = """
V0,V1,V2,V3
4,1,1,1
6,4,5,2
2,3,6,7
"""
df = pd.read_csv(io.StringIO(DN))

Transpose it:

dft = df.T

Find all the index with your condition:
 
idt = (dft[0] == 1) & (dft[1] == 5)

Print the columns that satisfy your condition:

print(dft[idt])

As you see, without explicit loop.

From hjp-python at hjp.at  Sat Jan  8 16:06:31 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sat, 8 Jan 2022 22:06:31 +0100
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
Message-ID: <20220108210631.uodonbypheirvl2d@hjp.at>

On 2021-12-26 20:40:03 +0400, Abdur-Rahmaan Janhangeer wrote:
> I have started a newspaper (not newsletter) focused
> on interesting reads on Python mailing lists. Don't tag
> on the fact that holiday seasons are the worst times for
> launch according to marketing folks, I started this to note
> down interesting mails. This might also be a great way to
> bring mailing list gems to a wider readership. So, here's
> the url https://pyherald.com/

Something like [LWN](https://lwn.net/) for Python? Neat.

Two suggestions: 

1. Could you add an RSS or Atom feed?

2. CSS ?word-break: break-all? seems like a really weird choice for
   English language text. If you really want to break words across
   lines, use ?hyphens: auto?.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220108/06c54f2c/attachment.sig>

From avigross at verizon.net  Sat Jan  8 17:00:51 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 8 Jan 2022 22:00:51 +0000 (UTC)
Subject: Extracting dataframe column with multiple conditions on row values
In-Reply-To: <6bc2ce03-75e3-48f7-8d7c-402345c6cb1cn@googlegroups.com>
References: <92704ABE-B6C4-4F7C-BF40-1883C61A3964.ref@hxcore.ol>
 <7530730e-b793-3075-2ec5-ee2e3160e0bb@DancesWithMice.info>
 <92704ABE-B6C4-4F7C-BF40-1883C61A3964@hxcore.ol>
 <mailman.125.1641604882.3079.python-list@python.org>
 <6bc2ce03-75e3-48f7-8d7c-402345c6cb1cn@googlegroups.com>
Message-ID: <310031172.1839437.1641679251128@mail.yahoo.com>


I have to wonder if when something looks like HOMEWORK, if it should be answered in detail, let alone using methods beyond what is expected in class.
The goal of this particular project seems to be to find one (or perhaps more) columns in some data structure like a dataframe that match two conditions (containing a copy of two numbers in one or more places) and then KNOW what column it was in. The reason I say that is because the next fairly nonsensical request is to then explicitly return what that column has in the row called 2, meaning the third row.
Perhaps stated another way: "what it the item in row/address 2 of the column that somewhere contains two additional specified contents called key1 and key2"?
My guess is that if the instructor wanted this to be solved using methods being taught, then loops may well be a way to go. Python and numpy/pandas make it often easier to do things with columns rather than in rows across them, albeit many things allow you to specify an axis. So, yes, transposing is a way to go that transforms the problem in a way easier to solve without thinking deeply. Some other languages allow relatively easy access in both directions of horizontally versus vertically. And this may be an example where solving it as a list of lists may also be easier.?
Is the solution at the bottom a solution? Before I check, I want to see if I understand the required functionality and ask if it is completely and unambiguously specified.?
For completeness, the question being asked may need to deal with a uniqueness issue. Is it possible multiple columns match the request and thus more than one answer is required to be returned? Is the row called 2 allowed to participate in the match or must it be excluded and the question becomes to find one (or more) columns that contain key1 somewhere else than row 2 and key2 (which may have to be different than key1 or not) somewhere else and THEN provide the corresponding entry from row 2 and that (or those) column(s)?
So in looking at the solution offered, what exactly was this supposed to do when dft is the transpose?
idt?= (dft[0] == 1) & (dft[1] == 5)
Was the code (way below in this message) tried out or just written for us to ponder? I tried it. I got an answer of:? ?? 0 1 2
 ? ?V2  1  5  6
That is not my understanding?of what was requested. Row 2 (shown transposed as a column) is being shown as a whole. The request was for item "2" which would be just 6. Something more like this:
print(dft[idt][2])

But the code makes no sense to me.? seems to explicitly test the first column (0) to see if it contains a 1 and then the second column (1) to see if it contains a 5. Not sure who cares about this hard-wired query as this is not my understanding of the question. You want any of the original three rows (now transposed)? tested to see if it contains BOTH.?
I may have read the requirements wrong or it may not be explained well. Until I am sure what is being asked and whether there is a good reason someone wants a different solution, I see no reason to provide yet another solution.But just for fund, assuming dft contains the transpose of the original data, will this work?
first = dft[dft.values == key1 ]second = first[first.values == key2 ]print(second[2])
I get a 6 as an answer and suppose it could be done in one more complex?expression if needed! LOL!
-----Original Message-----
From: Edmondo Giovannozzi <edmondo.giovannozzi at gmail.com>
To: python-list at python.org
Sent: Sat, Jan 8, 2022 8:00 am
Subject: Re: Extracting dataframe column with multiple conditions on row values

Il giorno sabato 8 gennaio 2022 alle 02:21:40 UTC+1 dn ha scritto:
> Salaam Mahmood,
> On 08/01/2022 12.07, Mahmood Naderan via Python-list wrote: 
> > I have a csv file like this 
> > V0,V1,V2,V3 
> > 4,1,1,1 
> > 6,4,5,2 
> > 2,3,6,7 
> > 
> > And I want to search two rows for a match and find the column. For 
> > example, I want to search row[0] for 1 and row[1] for 5. The corresponding 
> > column is V2 (which is the third column). Then I want to return the value 
> > at row[2] and the found column. The result should be 6 then.
> Not quite: isn't the "found column" also required?
> > I can manually extract the specified rows (with index 0 and 1 which are 
> > fixed) and manually iterate over them like arrays to find a match. Then I
> Perhaps this idea has been influenced by a similar solution in another 
> programming language. May I suggest that the better-answer you seek lies 
> in using Python idioms (as well as Python's tools)...
> > key1 = 1 
> > key2 = 5
> Fine, so far - excepting that this 'problem' is likely to be a small 
> part of some larger system. Accordingly, consider writing it as a 
> function. In which case, these two "keys" will become 
> function-parameters (and the two 'results' become return-values).
> > row1 = df.iloc[0] # row=[4,1,1,1] 
> > row2 = df.iloc[1] # row=[6,4,5,2]
> This is likely not native-Python. Let's create lists for 'everything', 
> just-because: 
> 
> >>> headings = [ "V0","V1","V2","V3" ] 
> >>> row1 = [4,1,1,1] 
> >>> row2 = [6,4,5,2] 
> >>> results = [ 2,3,6,7 ] 
> 
> 
> Note how I'm using the Python REPL (in a "terminal", type "python" (as 
> appropriate to your OpSys) at the command-line). IMHO the REPL is a 
> grossly under-rated tool, and is a very good means towards 
> trial-and-error, and learning by example. Highly recommended! 
> 
> 
> > for i in range(len(row1)): 
> 
> This construction is very much a "code smell" for thinking that it is 
> not "pythonic". (and perhaps the motivation for this post) 
> 
> In Python (compared with many other languages) the "for" loop should 
> actually be pronounced "for-each". In other words when we pair the 
> code-construct with a list (for example): 
> 
> for each item in the list the computer should perform some suite of 
> commands. 
> 
> (the "suite" is everything 'inside' the for-each-loop - NB my 
> 'Python-betters' will quickly point-out that this feature is not limited 
> to Python-lists, but will work with any :iterable" - ref: 
> https://docs.python.org/3/tutorial/controlflow.html#for-statements) 
> 
> 
> Thus: 
> 
> > for item in headings: print( item ) 
> ... 
> V0 
> V1 
> V2 
> V3 
> 
> 
> The problem is that when working with matrices/matrixes, a math 
> background equips one with the idea of indices/indexes, eg the 
> ubiquitous subscript-i. Accordingly, when reading 'math' where a formula 
> uses the upper-case Greek "sigma" character, remember that it means "for 
> all" or "for each"! 
> 
> So, if Python doesn't use indexing or "pointers", how do we deal with 
> the problem? 
> 
> Unfortunately, at first glance, the pythonic approach may seem 
> more-complicated or even somewhat convoluted, but once the concepts 
> (and/or the Python idioms) are learned, it is quite manageable (and 
> applicable to many more applications than matrices/matrixes!)...
> > if row1[i] == key1: 
> > for j in range(len(row2)): 
> > if row2[j] == key2: 
> > res = df.iloc[:,j] 
> > print(res) # 6 
> > 
> > Is there any way to use built-in function for a more efficient code?
> This is where your idea bears fruit! 
> 
> There is a Python "built-in function": zip(), which will 'join' lists. 
> NB do not become confused between zip() and zip archive/compressed files! 
> 
> Most of the time reference book and web-page examples show zip() being 
> used to zip-together two lists into a single data-construct (which is an 
> iterable(!)). However, zip() will actually zip-together multiple (more 
> than two) "iterables". As the manual says: 
> 
> ?zip() returns an iterator of tuples, where the i-th tuple contains the 
> i-th element from each of the argument iterables.? 
> 
> Ah, so that's where the math-idea of subscript-i went! It has become 
> 'hidden' in Python's workings - or putting that another way: Python 
> looks after the subscripting for us (and given that 'out by one' errors 
> in pointers is a major source of coding-error in other languages, 
> thank-you very much Python!) 
> 
> First re-state the source-data as Python lists, (per above) - except 
> that I recommend the names be better-chosen to be more meaningful (to 
> your application)! 
> 
> 
> Now, (in the REPL) try using zip(): 
> 
> >>> zip( headings, row1, row2, results ) 
> <zip object at 0x7f655cca6bc0> 
> 
> Does that seem a very good illustration? Not really, but re-read the 
> quotation from the manual (above) where it says that zip returns an 
> iterator. If we want to see the values an iterator will produce, then 
> turn it into an iterable data-structure, eg: 
> 
> >>> list( zip( headings, row1, row2, results ) ) 
> [('V0', 4, 6, 2), ('V1', 1, 4, 3), ('V2', 1, 5, 6), ('V3', 1, 2, 7)] 
> 
> or, to see things more clearly, let me re-type it as: 
> 
> [ 
> ('V0', 4, 6, 2), 
> ('V1', 1, 4, 3), 
> ('V2', 1, 5, 6), 
> ('V3', 1, 2, 7) 
> ] 
> 
> 
> What we now see is actually a "transpose" of the original 'matrix' 
> presented in the post/question! 
> 
> (NB Python will perform this layout for us - read about the pprint library) 
> 
> 
> Another method which can also be employed (and which will illustrate the 
> loop required to code the eventual-solution(!)) is that Python's next() 
> will extract the first row of the transpose: 
> 
> >>> row = next( zip( headings, row1, row2, results ) ) 
> >>> row 
> ('V0', 4, 6, 2) 
> 
> 
> This is all-well-and-good, but that result is a tuple of four items 
> (corresponding to one column in the way the source-data was explained). 
> 
> If we need to consider the four individual data-items, that can be 
> improved using a Python feature called "tuple unpacking". Instead of the 
> above delivering a tuple which is then assigned to "row", the tuple can 
> be assigned to four "identifiers", eg 
> 
> >>> heading, row1_item, row2_item, result= next( zip( headings, row1, 
> row2, results ) ) 
> 
> (apologies about email word-wrapping - this is a single line of Python-code) 
> 
> 
> Which, to prove the case, could be printed: 
> 
> >>> heading, row1_item, row2_item, result 
> ('V0', 4, 6, 2) 
> 
> 
> (ref: 
> https://docs.python.org/3/tutorial/datastructures.html?highlight=tuple%20unpacking#tuples-and-sequences) 
> 
> 
> Thus, if we repeatedly ask for the next() row from the zip-ped 
> transpose, eventually it will respond with the row starting 'V2' - which 
> is the desired-result, ie the row containing the 1, the 5, and the 6 - 
> and if you follow-through using the REPL, will be clearly visible. 
> 
> 
> Finally, 'all' that is required, is a for-each-loop which will iterate 
> across/down the zip object, one tuple (row of the transpose) at a time, 
> AND perform the "tuple-unpacking" all in one command, with an 
> if-statement to detect the correct row/column: 
> 
> >>> for *tuple-unpacking* in *zip() etc*: 
> ... if row1_item == *what?* and row2_item == *what?* 
> ... print( *which* and *which identifier* ) 
> ... 
> V2 6 
> 
> Yes, three lines. It's as easy as that! 
> (when you know how) 
> 
> Worse: when you become more expert, you'll be able to compress all of 
> that down into a single-line solution - but it won't be as "readable" as 
> is this! 
> 
> 
> NB this question has a 'question-smell' of 'homework', so I'll not 
> complete the code for you - this is something *you* asked to learn and 
> the best way to learn is by 'doing' (not by 'reading'). 
> 
> However, please respond with your solution, or any further question 
> (with the next version of the code so-far, per this first-post - which 
> we appreciate!) 
> 
> Regardless, you asked 'the right question' (curiosity is the key to 
> learning) and in the right way/manner. Well done! 
> 
> 
> NBB the above code-outline does not consider the situation where the 
> search fails/the keys are not found! 
> 
> 
> For further information, please review: 
> https://docs.python.org/3/library/functions.html?highlight=zip#zip 
> 
> Also, further to the above discussion of combining lists and loops: 
> https://docs.python.org/3/tutorial/datastructures.html?highlight=zip#looping-techniques 
> 
> and with a similar application (to this post): 
> https://docs.python.org/3/faq/programming.html?highlight=zip#how-can-i-sort-one-list-by-values-from-another-list 
> 
> -- 
> Regards, 

You may also transpose your dataset. Then the index will become your column name and the column name become your index:
To read your dataset:

import pandas as pd
import io

DN = """
V0,V1,V2,V3
4,1,1,1
6,4,5,2
2,3,6,7
"""
df = pd.read_csv(io.StringIO(DN))

Transpose it:

dft = df.T

Find all the index with your condition:
 
idt = (dft[0] == 1) & (dft[1] == 5)

Print the columns that satisfy your condition:

print(dft[idt])

As you see, without explicit loop.
-- 
https://mail.python.org/mailman/listinfo/python-list

From arj.python at gmail.com  Sun Jan  9 01:28:36 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sun, 9 Jan 2022 10:28:36 +0400
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <20220108210631.uodonbypheirvl2d@hjp.at>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
 <20220108210631.uodonbypheirvl2d@hjp.at>
Message-ID: <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>

Well yes XD though LWN covers Py topics well when it wants


1. Yes sure, did not expect RSS interest
2. Excuse my blunder, will do!

On Sun, 9 Jan 2022, 01:15 Peter J. Holzer, <hjp-python at hjp.at> wrote:

> On 2021-12-26 20:40:03 +0400, Abdur-Rahmaan Janhangeer wrote:
> > I have started a newspaper (not newsletter) focused
> > on interesting reads on Python mailing lists. Don't tag
> > on the fact that holiday seasons are the worst times for
> > launch according to marketing folks, I started this to note
> > down interesting mails. This might also be a great way to
> > bring mailing list gems to a wider readership. So, here's
> > the url https://pyherald.com/
>
> Something like [LWN](https://lwn.net/) for Python? Neat.
>
> Two suggestions:
>
> 1. Could you add an RSS or Atom feed?
>
> 2. CSS ?word-break: break-all? seems like a really weird choice for
>    English language text. If you really want to break words across
>    lines, use ?hyphens: auto?.
>
>         hp
>
> --
>    _  | Peter J. Holzer    | Story must make more sense than reality.
> |_|_) |                    |
> | |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |       challenge!"
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From pbryan at anode.ca  Sun Jan  9 02:32:53 2022
From: pbryan at anode.ca (Paul Bryan)
Date: Sat, 08 Jan 2022 23:32:53 -0800
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
 <20220108210631.uodonbypheirvl2d@hjp.at>
 <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
Message-ID: <07a0b6eb37707305269a1e0c65ac8e2308eb814b.camel@anode.ca>

+1 to RSS.

On Sun, 2022-01-09 at 10:28 +0400, Abdur-Rahmaan Janhangeer wrote:
> Well yes XD though LWN covers Py topics well when it wants
> 
> 
> 1. Yes sure, did not expect RSS interest
> 2. Excuse my blunder, will do!
> 
> On Sun, 9 Jan 2022, 01:15 Peter J. Holzer, <hjp-python at hjp.at> wrote:
> 
> > On 2021-12-26 20:40:03 +0400, Abdur-Rahmaan Janhangeer wrote:
> > > I have started a newspaper (not newsletter) focused
> > > on interesting reads on Python mailing lists. Don't tag
> > > on the fact that holiday seasons are the worst times for
> > > launch according to marketing folks, I started this to note
> > > down interesting mails. This might also be a great way to
> > > bring mailing list gems to a wider readership. So, here's
> > > the url https://pyherald.com/
> > 
> > Something like [LWN](https://lwn.net/) for Python? Neat.
> > 
> > Two suggestions:
> > 
> > 1. Could you add an RSS or Atom feed?
> > 
> > 2. CSS ?word-break: break-all? seems like a really weird choice for
> > ?? English language text. If you really want to break words across
> > ?? lines, use ?hyphens: auto?.
> > 
> > ??????? hp
> > 
> > --
> > ?? _? | Peter J. Holzer??? | Story must make more sense than
> > reality.
> > > _|_) |??????????????????? |
> > > > ? | hjp at hjp.at???????? |??? -- Charles Stross, "Creative
> > > > writing
> > __/?? | http://www.hjp.at/?|?????? challenge!"
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> > 


From narshad.380 at gmail.com  Sun Jan  9 02:04:20 2022
From: narshad.380 at gmail.com (NArshad)
Date: Sat, 8 Jan 2022 23:04:20 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <sr7on3$1ji0$1@gioia.aioe.org>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
Message-ID: <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>

On Friday, 7 January 2022 at 02:59:17 UTC+5, alister wrote:
> On Thu, 6 Jan 2022 10:55:30 -0800 (PST), NArshad wrote: 
> 
> > All this is going to be in python?s flask and HTML only 
> > 
> > 1. First, I have to check in the Excel sheet or table whether the book 
> > user has entered is present in the book bank or not. 
> >
> Excel is the wrong application for storing this data - it should be in a 
> database
> > 2. If a book is present and the quantity of the required book is greater 
> > than 0 (COPIES_LEFT column in excel file) and if the user wants the 
> > book, it will be assigned to the user which he will take from the book 
> > bank physically. When COPIES_LEFT will is less than or equal to 0 the 
> > message will be ?Book finished or not present?.
> Again Excel is not the correct application for processing this data
> > 
> > 3. The quantity of the book in the Excel file will be reduced by 1 in 
> > the COPIES_LEFT column and the name of the borrower or user will be 
> > entered/added in the Excel file table or sheet already made and the 
> > column name is BORROWER?S NAME. 
> >
> Database!
> > 4. The borrower?s or user name can be more than one so they will be 
> > separated with a comma in the Excel file BORROWER?S NAME column. 
> >
> Database
> > 
> > - All functions mentioned above are to be deployed on the website 
> > pythonhow.com so make according to 
> > https://pythonhow.com/python-tutorial/flask/web-development-with-python- 
> and-flask/ 
> > 
> > - Do you know any other websites to deploy a python web application?? 
> > 
> > - No time to switch from Excel to anywhere else. Please do not make any 
> > changes to the Excel file. 
> > 
> > - Tutorials and repositories of the web for such problems are also 
> > required. The same is required for python (flask, Django...) also.
> Sorry did not spot that this was a homework assignment 
> data should still be imported into a DB (a trivial task) It can be 
> exported back to a compatible format just as easily if hard copy output is 
> required 
> 
> 
> 
> 
> -- 
> "Rembrandt's first name was Beauregard, which is why he never used it." 
> -- Dave Barry



As written no time to switch from excel to anywhere else but if a certain database is required to make changes in Excel?s cell values then which database to use (example Access or what) and after the right selection of the database, how to import data to a database and then export back to a compatible format that is to Excel cells?.  

I mean how to do this written below:

?data should still be imported into a DB (a trivial task) it can be exported back to a compatible format just as easily if hard copy output is required?

The reason I have written:

?What to write on GitHub or on google search to find the necessary code to start with? ?

I will also be requiring a code to start with just as most people do. The same is the case with tutorials.  

This is not homework and I have been checking openpyxl for the last one or two months to find what is required by me when I found nothing of what is required by me then posted on this google group. 



From alister.ware at ntlworld.com  Sun Jan  9 05:55:47 2022
From: alister.ware at ntlworld.com (alister)
Date: Sun, 9 Jan 2022 10:55:47 -0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
Message-ID: <sreevj$at0$4@gioia.aioe.org>

On Sat, 8 Jan 2022 23:04:20 -0800 (PST), NArshad wrote:

> On Friday, 7 January 2022 at 02:59:17 UTC+5, alister wrote:
>> On Thu, 6 Jan 2022 10:55:30 -0800 (PST), NArshad wrote:
>> 
>> > All this is going to be in python?s flask and HTML only
>> > 
>> > 1. First, I have to check in the Excel sheet or table whether the
>> > book user has entered is present in the book bank or not.
>> >
>> Excel is the wrong application for storing this data - it should be in
>> a database
>> > 2. If a book is present and the quantity of the required book is
>> > greater than 0 (COPIES_LEFT column in excel file) and if the user
>> > wants the book, it will be assigned to the user which he will take
>> > from the book bank physically. When COPIES_LEFT will is less than or
>> > equal to 0 the message will be ?Book finished or not present?.
>> Again Excel is not the correct application for processing this data
>> > 
>> > 3. The quantity of the book in the Excel file will be reduced by 1 in
>> > the COPIES_LEFT column and the name of the borrower or user will be
>> > entered/added in the Excel file table or sheet already made and the
>> > column name is BORROWER?S NAME.
>> >
>> Database!
>> > 4. The borrower?s or user name can be more than one so they will be
>> > separated with a comma in the Excel file BORROWER?S NAME column.
>> >
>> Database
>> > 
>> > - All functions mentioned above are to be deployed on the website
>> > pythonhow.com so make according to
>> > https://pythonhow.com/python-tutorial/flask/web-development-with-
python-
>> and-flask/
>> > 
>> > - Do you know any other websites to deploy a python web application??
>> > 
>> > - No time to switch from Excel to anywhere else. Please do not make
>> > any changes to the Excel file.
>> > 
>> > - Tutorials and repositories of the web for such problems are also
>> > required. The same is required for python (flask, Django...) also.
>> Sorry did not spot that this was a homework assignment data should
>> still be imported into a DB (a trivial task) It can be exported back to
>> a compatible format just as easily if hard copy output is required
>> 
>> 
>> 
>> 
>> --
>> "Rembrandt's first name was Beauregard, which is why he never used it."
>> -- Dave Barry
> 
> 
> 
> As written no time to switch from excel to anywhere else but if a
> certain database is required to make changes in Excel?s cell values then
> which database to use (example Access or what) and after the right
> selection of the database, how to import data to a database and then
> export back to a compatible format that is to Excel cells?.
> 
> I mean how to do this written below:
> 
> ?data should still be imported into a DB (a trivial task) it can be
> exported back to a compatible format just as easily if hard copy output
> is required?
> 
> The reason I have written:
> 
> ?What to write on GitHub or on google search to find the necessary code
> to start with? ?
> 
> I will also be requiring a code to start with just as most people do.
> The same is the case with tutorials.
> 
> This is not homework and I have been checking openpyxl for the last one
> or two months to find what is required by me when I found nothing of
> what is required by me then posted on this google group.

1st save the data from excel as a csv file

you will find the csv module makes these much easier to deal with, even if 
you do not import into a db (although even then I would import into sql-
lite just for the benefits of the search algorithms)




-- 
Sometime in 1993 NANCY SINATRA will lead a BLOODLESS COUP on GUAM!!

From narshad.380 at gmail.com  Sun Jan  9 11:53:58 2022
From: narshad.380 at gmail.com (NArshad)
Date: Sun, 9 Jan 2022 08:53:58 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <sreevj$at0$4@gioia.aioe.org>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <sreevj$at0$4@gioia.aioe.org>
Message-ID: <3e14159b-4411-466b-805e-0a802c1a3750n@googlegroups.com>

-Whose time??  
My time
I do not have not time to switch to a database but if necessary I can use the database to make changes in Excel column entries. 

-No changes??

I cannot change the column names.


-?maybe the point is that at any time one has to be able to re-export to the original excel format?  (And why??  Do we actually need "reporting": then we can generate a PDF rather, otherwise we can just send out a CSV... etc.)  Is that it??

PDF is not required and I am not doing any reporting at all. 


-?loading the data from the CSV file in memory into a pandas dataframe, working with that, and eventually saving back to the CSV file might be just enough... as long as there is never more than one user concurrently using the system, or you need also implement some concurrency management (you need that with a database, too, anyway): but that's tutorial number 2, and already quite less trivial, since to begin with it requires a more thorough analysis?

No need to save the CSV file once the work is done CSV file will be exited without saving. If the above is going to work then I will be requiring something to see how to do all this that?s why written tutorial. For a more thorough analysis I have written once again what I want to do. 

Only the four functions that I have written below I have to make and that?s it. All the input will be entered by the user on a website:
1. First, I have to check in the Excel sheet or table whether the book user has entered is present in the book bank or not.
 
2. If a book is present and the quantity of the required book is greater than 0 (COPIES_LEFT column in excel file) and if the user wants the book, it will be assigned to the user which he will take from the book bank physically. When COPIES_LEFT is less than or equal to 0 the message will be ?Book finished or not present?.
 
3. The quantity of the book in the Excel file will be reduced by 1 in the COPIES_LEFT column and the name of the borrower or user will be entered/added in the Excel file table or sheet already made and the column name is BORROWER?S NAME.
 
4. The borrower?s or user name can be more than one so they will be separated with a comma in the Excel file BORROWER?S NAME column.


-1st save the data from excel as a CSV file

you will find the CSV module makes these much easier to deal with, even if
you do not import into a DB (although even then I would import into SQL-
lite just for the benefits of the search algorithms)

If in order to do the required task it is necessary to use SQLite and without using SQLite I cannot do what I want to do then I think I have to go for SQLite otherwise CSV looks to me as fine. To do everything using CSV I will be requiring a tutorial to see how to do all this. 

Anything not clear?

From edmondo.giovannozzi at gmail.com  Sun Jan  9 13:29:26 2022
From: edmondo.giovannozzi at gmail.com (Edmondo Giovannozzi)
Date: Sun, 9 Jan 2022 10:29:26 -0800 (PST)
Subject: Extracting dataframe column with multiple conditions on row values
In-Reply-To: <mailman.131.1641679255.3079.python-list@python.org>
References: <92704ABE-B6C4-4F7C-BF40-1883C61A3964.ref@hxcore.ol>
 <7530730e-b793-3075-2ec5-ee2e3160e0bb@DancesWithMice.info>
 <92704ABE-B6C4-4F7C-BF40-1883C61A3964@hxcore.ol>
 <mailman.125.1641604882.3079.python-list@python.org>
 <310031172.1839437.1641679251128@mail.yahoo.com>
 <6bc2ce03-75e3-48f7-8d7c-402345c6cb1cn@googlegroups.com>
 <mailman.131.1641679255.3079.python-list@python.org>
Message-ID: <d4510975-dde4-489b-98cc-470973c9dab1n@googlegroups.com>

Il giorno sabato 8 gennaio 2022 alle 23:01:13 UTC+1 Avi Gross ha scritto:
> I have to wonder if when something looks like HOMEWORK, if it should be answered in detail, let alone using methods beyond what is expected in class. 
> The goal of this particular project seems to be to find one (or perhaps more) columns in some data structure like a dataframe that match two conditions (containing a copy of two numbers in one or more places) and then KNOW what column it was in. The reason I say that is because the next fairly nonsensical request is to then explicitly return what that column has in the row called 2, meaning the third row. 
> Perhaps stated another way: "what it the item in row/address 2 of the column that somewhere contains two additional specified contents called key1 and key2"  
> My guess is that if the instructor wanted this to be solved using methods being taught, then loops may well be a way to go. Python and numpy/pandas make it often easier to do things with columns rather than in rows across them, albeit many things allow you to specify an axis. So, yes, transposing is a way to go that transforms the problem in a way easier to solve without thinking deeply. Some other languages allow relatively easy access in both directions of horizontally versus vertically. And this may be an example where solving it as a list of lists may also be easier.  
> Is the solution at the bottom a solution? Before I check, I want to see if I understand the required functionality and ask if it is completely and unambiguously specified.  
> For completeness, the question being asked may need to deal with a uniqueness issue. Is it possible multiple columns match the request and thus more than one answer is required to be returned? Is the row called 2 allowed to participate in the match or must it be excluded and the question becomes to find one (or more) columns that contain key1 somewhere else than row 2 and key2 (which may have to be different than key1 or not) somewhere else and THEN provide the corresponding entry from row 2 and that (or those) column(s)? 
> So in looking at the solution offered, what exactly was this supposed to do when dft is the transpose?
> idt = (dft[0] == 1) & (dft[1] == 5)
> Was the code (way below in this message) tried out or just written for us to ponder? I tried it. I got an answer of:     0 1 2 
>    V2 1 5 6 
> That is not my understanding of what was requested. Row 2 (shown transposed as a column) is being shown as a whole. The request was for item "2" which would be just 6. Something more like this: 
> print(dft[idt][2]) 
> 
> But the code makes no sense to me.  seems to explicitly test the first column (0) to see if it contains a 1 and then the second column (1) to see if it contains a 5. Not sure who cares about this hard-wired query as this is not my understanding of the question. You want any of the original three rows (now transposed)  tested to see if it contains BOTH.  
> I may have read the requirements wrong or it may not be explained well. Until I am sure what is being asked and whether there is a good reason someone wants a different solution, I see no reason to provide yet another solution.But just for fund, assuming dft contains the transpose of the original data, will this work? 
> first = dft[dft.values == key1 ]second = first[first.values == key2 ]print(second[2]) 
> I get a 6 as an answer and suppose it could be done in one more complex expression if needed! LOL!
> -----Original Message----- 
> From: Edmondo Giovannozzi <edmondo.g... at gmail.com> 
> To: pytho... at python.org 
> Sent: Sat, Jan 8, 2022 8:00 am 
> Subject: Re: Extracting dataframe column with multiple conditions on row values 
> 
> Il giorno sabato 8 gennaio 2022 alle 02:21:40 UTC+1 dn ha scritto: 
> > Salaam Mahmood, 
> > On 08/01/2022 12.07, Mahmood Naderan via Python-list wrote: 
> > > I have a csv file like this 
> > > V0,V1,V2,V3 
> > > 4,1,1,1 
> > > 6,4,5,2 
> > > 2,3,6,7 
> > > 
> > > And I want to search two rows for a match and find the column. For 
> > > example, I want to search row[0] for 1 and row[1] for 5. The corresponding 
> > > column is V2 (which is the third column). Then I want to return the value 
> > > at row[2] and the found column. The result should be 6 then. 
> > Not quite: isn't the "found column" also required? 
> > > I can manually extract the specified rows (with index 0 and 1 which are 
> > > fixed) and manually iterate over them like arrays to find a match. Then I 
> > Perhaps this idea has been influenced by a similar solution in another 
> > programming language. May I suggest that the better-answer you seek lies 
> > in using Python idioms (as well as Python's tools)... 
> > > key1 = 1 
> > > key2 = 5 
> > Fine, so far - excepting that this 'problem' is likely to be a small 
> > part of some larger system. Accordingly, consider writing it as a 
> > function. In which case, these two "keys" will become 
> > function-parameters (and the two 'results' become return-values). 
> > > row1 = df.iloc[0] # row=[4,1,1,1] 
> > > row2 = df.iloc[1] # row=[6,4,5,2] 
> > This is likely not native-Python. Let's create lists for 'everything', 
> > just-because: 
> > 
> > >>> headings = [ "V0","V1","V2","V3" ] 
> > >>> row1 = [4,1,1,1] 
> > >>> row2 = [6,4,5,2] 
> > >>> results = [ 2,3,6,7 ] 
> > 
> > 
> > Note how I'm using the Python REPL (in a "terminal", type "python" (as 
> > appropriate to your OpSys) at the command-line). IMHO the REPL is a 
> > grossly under-rated tool, and is a very good means towards 
> > trial-and-error, and learning by example. Highly recommended! 
> > 
> > 
> > > for i in range(len(row1)): 
> > 
> > This construction is very much a "code smell" for thinking that it is 
> > not "pythonic". (and perhaps the motivation for this post) 
> > 
> > In Python (compared with many other languages) the "for" loop should 
> > actually be pronounced "for-each". In other words when we pair the 
> > code-construct with a list (for example): 
> > 
> > for each item in the list the computer should perform some suite of 
> > commands. 
> > 
> > (the "suite" is everything 'inside' the for-each-loop - NB my 
> > 'Python-betters' will quickly point-out that this feature is not limited 
> > to Python-lists, but will work with any :iterable" - ref: 
> > https://docs.python.org/3/tutorial/controlflow.html#for-statements) 
> > 
> > 
> > Thus: 
> > 
> > > for item in headings: print( item ) 
> > ... 
> > V0 
> > V1 
> > V2 
> > V3 
> > 
> > 
> > The problem is that when working with matrices/matrixes, a math 
> > background equips one with the idea of indices/indexes, eg the 
> > ubiquitous subscript-i. Accordingly, when reading 'math' where a formula 
> > uses the upper-case Greek "sigma" character, remember that it means "for 
> > all" or "for each"! 
> > 
> > So, if Python doesn't use indexing or "pointers", how do we deal with 
> > the problem? 
> > 
> > Unfortunately, at first glance, the pythonic approach may seem 
> > more-complicated or even somewhat convoluted, but once the concepts 
> > (and/or the Python idioms) are learned, it is quite manageable (and 
> > applicable to many more applications than matrices/matrixes!)... 
> > > if row1[i] == key1: 
> > > for j in range(len(row2)): 
> > > if row2[j] == key2: 
> > > res = df.iloc[:,j] 
> > > print(res) # 6 
> > > 
> > > Is there any way to use built-in function for a more efficient code? 
> > This is where your idea bears fruit! 
> > 
> > There is a Python "built-in function": zip(), which will 'join' lists. 
> > NB do not become confused between zip() and zip archive/compressed files! 
> > 
> > Most of the time reference book and web-page examples show zip() being 
> > used to zip-together two lists into a single data-construct (which is an 
> > iterable(!)). However, zip() will actually zip-together multiple (more 
> > than two) "iterables". As the manual says: 
> > 
> > ?zip() returns an iterator of tuples, where the i-th tuple contains the 
> > i-th element from each of the argument iterables.? 
> > 
> > Ah, so that's where the math-idea of subscript-i went! It has become 
> > 'hidden' in Python's workings - or putting that another way: Python 
> > looks after the subscripting for us (and given that 'out by one' errors 
> > in pointers is a major source of coding-error in other languages, 
> > thank-you very much Python!) 
> > 
> > First re-state the source-data as Python lists, (per above) - except 
> > that I recommend the names be better-chosen to be more meaningful (to 
> > your application)! 
> > 
> > 
> > Now, (in the REPL) try using zip(): 
> > 
> > >>> zip( headings, row1, row2, results ) 
> > <zip object at 0x7f655cca6bc0> 
> > 
> > Does that seem a very good illustration? Not really, but re-read the 
> > quotation from the manual (above) where it says that zip returns an 
> > iterator. If we want to see the values an iterator will produce, then 
> > turn it into an iterable data-structure, eg: 
> > 
> > >>> list( zip( headings, row1, row2, results ) ) 
> > [('V0', 4, 6, 2), ('V1', 1, 4, 3), ('V2', 1, 5, 6), ('V3', 1, 2, 7)] 
> > 
> > or, to see things more clearly, let me re-type it as: 
> > 
> > [ 
> > ('V0', 4, 6, 2), 
> > ('V1', 1, 4, 3), 
> > ('V2', 1, 5, 6), 
> > ('V3', 1, 2, 7) 
> > ] 
> > 
> > 
> > What we now see is actually a "transpose" of the original 'matrix' 
> > presented in the post/question! 
> > 
> > (NB Python will perform this layout for us - read about the pprint library) 
> > 
> > 
> > Another method which can also be employed (and which will illustrate the 
> > loop required to code the eventual-solution(!)) is that Python's next() 
> > will extract the first row of the transpose: 
> > 
> > >>> row = next( zip( headings, row1, row2, results ) ) 
> > >>> row 
> > ('V0', 4, 6, 2) 
> > 
> > 
> > This is all-well-and-good, but that result is a tuple of four items 
> > (corresponding to one column in the way the source-data was explained). 
> > 
> > If we need to consider the four individual data-items, that can be 
> > improved using a Python feature called "tuple unpacking". Instead of the 
> > above delivering a tuple which is then assigned to "row", the tuple can 
> > be assigned to four "identifiers", eg 
> > 
> > >>> heading, row1_item, row2_item, result= next( zip( headings, row1, 
> > row2, results ) ) 
> > 
> > (apologies about email word-wrapping - this is a single line of Python-code) 
> > 
> > 
> > Which, to prove the case, could be printed: 
> > 
> > >>> heading, row1_item, row2_item, result 
> > ('V0', 4, 6, 2) 
> > 
> > 
> > (ref: 
> > https://docs.python.org/3/tutorial/datastructures.html?highlight=tuple%20unpacking#tuples-and-sequences) 
> > 
> > 
> > Thus, if we repeatedly ask for the next() row from the zip-ped 
> > transpose, eventually it will respond with the row starting 'V2' - which 
> > is the desired-result, ie the row containing the 1, the 5, and the 6 - 
> > and if you follow-through using the REPL, will be clearly visible. 
> > 
> > 
> > Finally, 'all' that is required, is a for-each-loop which will iterate 
> > across/down the zip object, one tuple (row of the transpose) at a time, 
> > AND perform the "tuple-unpacking" all in one command, with an 
> > if-statement to detect the correct row/column: 
> > 
> > >>> for *tuple-unpacking* in *zip() etc*: 
> > ... if row1_item == *what?* and row2_item == *what?* 
> > ... print( *which* and *which identifier* ) 
> > ... 
> > V2 6 
> > 
> > Yes, three lines. It's as easy as that! 
> > (when you know how) 
> > 
> > Worse: when you become more expert, you'll be able to compress all of 
> > that down into a single-line solution - but it won't be as "readable" as 
> > is this! 
> > 
> > 
> > NB this question has a 'question-smell' of 'homework', so I'll not 
> > complete the code for you - this is something *you* asked to learn and 
> > the best way to learn is by 'doing' (not by 'reading'). 
> > 
> > However, please respond with your solution, or any further question 
> > (with the next version of the code so-far, per this first-post - which 
> > we appreciate!) 
> > 
> > Regardless, you asked 'the right question' (curiosity is the key to 
> > learning) and in the right way/manner. Well done! 
> > 
> > 
> > NBB the above code-outline does not consider the situation where the 
> > search fails/the keys are not found! 
> > 
> > 
> > For further information, please review: 
> > https://docs.python.org/3/library/functions.html?highlight=zip#zip 
> > 
> > Also, further to the above discussion of combining lists and loops: 
> > https://docs.python.org/3/tutorial/datastructures.html?highlight=zip#looping-techniques 
> > 
> > and with a similar application (to this post): 
> > https://docs.python.org/3/faq/programming.html?highlight=zip#how-can-i-sort-one-list-by-values-from-another-list 
> > 
> > -- 
> > Regards, 
> 
> You may also transpose your dataset. Then the index will become your column name and the column name become your index: 
> To read your dataset: 
> 
> import pandas as pd 
> import io 
> 
> DN = """ 
> V0,V1,V2,V3 
> 4,1,1,1 
> 6,4,5,2 
> 2,3,6,7 
> """ 
> df = pd.read_csv(io.StringIO(DN)) 
> 
> Transpose it: 
> 
> dft = df.T 
> 
> Find all the index with your condition: 
> 
> idt = (dft[0] == 1) & (dft[1] == 5) 
> 
> Print the columns that satisfy your condition: 
> 
> print(dft[idt]) 
> 
> As you see, without explicit loop.
> -- 
> https://mail.python.org/mailman/listinfo/python-list

I was just showing that one can transpose the dataframe and use logical indexing, as the OP was asking for a fast and efficient solution. It is just a hint and not a complete solution. And, of course, one have to put everything inside a function.
 
And yes, I tested what I have written.

There could be other problems, of course, if some columns are not numeric, but all this details depend on the problem at hands.
But, I may not have understood completely the problem presented by the OP.

Cheers, :-)


From alister.ware at ntlworld.com  Sun Jan  9 14:02:56 2022
From: alister.ware at ntlworld.com (alister)
Date: Sun, 9 Jan 2022 19:02:56 -0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <sreevj$at0$4@gioia.aioe.org>
 <3e14159b-4411-466b-805e-0a802c1a3750n@googlegroups.com>
Message-ID: <srfbh0$gva$2@gioia.aioe.org>

On Sun, 9 Jan 2022 08:53:58 -0800 (PST), NArshad wrote:

> -Whose time??
> My time I do not have not time to switch to a database but if necessary
> I can use the database to make changes in Excel column entries.
> 
> -No changes??
> 
> I cannot change the column names.
> 
> 
> -?maybe the point is that at any time one has to be able to re-export to
> the original excel format?  (And why??  Do we actually need "reporting":
> then we can generate a PDF rather, otherwise we can just send out a
> CSV... etc.)  Is that it??
> 
> PDF is not required and I am not doing any reporting at all.
> 
> 
> -?loading the data from the CSV file in memory into a pandas dataframe,
> working with that, and eventually saving back to the CSV file might be
> just enough... as long as there is never more than one user concurrently
> using the system, or you need also implement some concurrency management
> (you need that with a database, too, anyway): but that's tutorial number
> 2, and already quite less trivial, since to begin with it requires a
> more thorough analysis?
> 
> No need to save the CSV file once the work is done CSV file will be
> exited without saving. If the above is going to work then I will be
> requiring something to see how to do all this that?s why written
> tutorial. For a more thorough analysis I have written once again what I
> want to do.
> 
> Only the four functions that I have written below I have to make and
> that?s it. All the input will be entered by the user on a website:
> 1. First, I have to check in the Excel sheet or table whether the book
> user has entered is present in the book bank or not.
>  
> 2. If a book is present and the quantity of the required book is greater
> than 0 (COPIES_LEFT column in excel file) and if the user wants the
> book, it will be assigned to the user which he will take from the book
> bank physically. When COPIES_LEFT is less than or equal to 0 the message
> will be ?Book finished or not present?.
>  
> 3. The quantity of the book in the Excel file will be reduced by 1 in
> the COPIES_LEFT column and the name of the borrower or user will be
> entered/added in the Excel file table or sheet already made and the
> column name is BORROWER?S NAME.
>  
> 4. The borrower?s or user name can be more than one so they will be
> separated with a comma in the Excel file BORROWER?S NAME column.
> 
> 
> -1st save the data from excel as a CSV file
> 
> you will find the CSV module makes these much easier to deal with, even
> if you do not import into a DB (although even then I would import into
> SQL-
> lite just for the benefits of the search algorithms)
> 
> If in order to do the required task it is necessary to use SQLite and
> without using SQLite I cannot do what I want to do then I think I have
> to go for SQLite otherwise CSV looks to me as fine. To do everything
> using CSV I will be requiring a tutorial to see how to do all this.
> 
> Anything not clear?

Sql-lite is not essential to the task CSV would be sufficient, but you may 
find that taking the time to learn it pays off later (even if you wait 
untill after this project)




-- 
Trying to establish voice contact ... please ____yell into 
keyboard.

From python at mrabarnett.plus.com  Sun Jan  9 16:02:20 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Sun, 9 Jan 2022 21:02:20 +0000
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
Message-ID: <7c49e7a1-2a7b-fc5f-b93b-e264dc68b909@mrabarnett.plus.com>

On 2022-01-09 07:04, NArshad wrote:
> On Friday, 7 January 2022 at 02:59:17 UTC+5, alister wrote:
>> On Thu, 6 Jan 2022 10:55:30 -0800 (PST), NArshad wrote: 
>> 
>> > All this is going to be in python?s flask and HTML only 
>> > 
>> > 1. First, I have to check in the Excel sheet or table whether the book 
>> > user has entered is present in the book bank or not. 
>> >
>> Excel is the wrong application for storing this data - it should be in a 
>> database
>> > 2. If a book is present and the quantity of the required book is greater 
>> > than 0 (COPIES_LEFT column in excel file) and if the user wants the 
>> > book, it will be assigned to the user which he will take from the book 
>> > bank physically. When COPIES_LEFT will is less than or equal to 0 the 
>> > message will be ?Book finished or not present?.
>> Again Excel is not the correct application for processing this data
>> > 
>> > 3. The quantity of the book in the Excel file will be reduced by 1 in 
>> > the COPIES_LEFT column and the name of the borrower or user will be 
>> > entered/added in the Excel file table or sheet already made and the 
>> > column name is BORROWER?S NAME. 
>> >
>> Database!
>> > 4. The borrower?s or user name can be more than one so they will be 
>> > separated with a comma in the Excel file BORROWER?S NAME column. 
>> >
>> Database
>> > 
>> > - All functions mentioned above are to be deployed on the website 
>> > pythonhow.com so make according to 
>> > https://pythonhow.com/python-tutorial/flask/web-development-with-python- 
>> and-flask/ 
>> > 
>> > - Do you know any other websites to deploy a python web application?? 
>> > 
>> > - No time to switch from Excel to anywhere else. Please do not make any 
>> > changes to the Excel file. 
>> > 
>> > - Tutorials and repositories of the web for such problems are also 
>> > required. The same is required for python (flask, Django...) also.
>> Sorry did not spot that this was a homework assignment 
>> data should still be imported into a DB (a trivial task) It can be 
>> exported back to a compatible format just as easily if hard copy output is 
>> required 
>> 
> 
> As written no time to switch from excel to anywhere else but if a certain database is required to make changes in Excel?s cell values then which database to use (example Access or what) and after the right selection of the database, how to import data to a database and then export back to a compatible format that is to Excel cells?.
> 
> I mean how to do this written below:
> 
> ?data should still be imported into a DB (a trivial task) it can be exported back to a compatible format just as easily if hard copy output is required?
> 
> The reason I have written:
> 
> ?What to write on GitHub or on google search to find the necessary code to start with? ?
> 
> I will also be requiring a code to start with just as most people do. The same is the case with tutorials.
> 
> This is not homework and I have been checking openpyxl for the last one or two months to find what is required by me when I found nothing of what is required by me then posted on this google group.
> 
> 
Using openpyxl is pretty straightforward:


from openpyxl import load_workbook
wb = load_workbook(spreadsheet_path)
sheet = wb.active

# Reading the values in cells:
print('Cell A1 contains', sheet['A1'].value)
print('Cell A2 contains', sheet['A2'].value)
print('Cell B1 contains', sheet['B1'].value)

# Alternatively:
print('Cell A1 contains', sheet.cell(1, 1).value)
print('Cell A2 contains', sheet.cell(1, 2).value)
print('Cell B1 contains', sheet.cell(2, 1).value)

# Changing the value in a cell:
sheet.cell(1, 20).value = 'TEST'

# A value of None means that the cell is empty.

wb.save(spreadsheet_path)
wb.close()

From hjp-python at hjp.at  Sun Jan  9 17:25:24 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Sun, 9 Jan 2022 23:25:24 +0100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <ts5mtgddlvtqflgfq150hhrbgjevqhogek@4ax.com>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <sreevj$at0$4@gioia.aioe.org>
 <3e14159b-4411-466b-805e-0a802c1a3750n@googlegroups.com>
 <ts5mtgddlvtqflgfq150hhrbgjevqhogek@4ax.com>
Message-ID: <20220109222524.xwxtohbrh6yw7224@hjp.at>

On 2022-01-09 13:08:51 -0500, Dennis Lee Bieber wrote:
> there are multiple ways to have Python access Excel -- from simple
> read and write modules xlrd/xlwt, (which is out-dated; xlrd does not
> work with .xlsx format, only the older .xls format).

This is not true. xlrd has supported .xlsx since at least 2014 (when I
started using it). For new projects I would recommend openpyxl though,
which is much more feature-complete.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220109/391c9d0a/attachment.sig>

From avigross at verizon.net  Sun Jan  9 17:29:02 2022
From: avigross at verizon.net (Avi Gross)
Date: Sun, 9 Jan 2022 22:29:02 +0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <ts5mtgddlvtqflgfq150hhrbgjevqhogek@4ax.com>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <sreevj$at0$4@gioia.aioe.org>
 <3e14159b-4411-466b-805e-0a802c1a3750n@googlegroups.com>
 <ts5mtgddlvtqflgfq150hhrbgjevqhogek@4ax.com>
Message-ID: <325439547.1954657.1641767342096@mail.yahoo.com>

Is this thread even close to being on track?
It is not really relevant to argue yet on whether to use EXCEL directly or a data.base. Many ways can be used to solve a problem and if the EXCEL sheet will never be updated manually or by some other program, it is sort of moot as you can ONE TIME transfer the existing data to any form you want and from then on work with that.
The higher level requirement of the project is not clear.
Is there a single application that wakes up and processes requests and eventually exits? If this is true, then there can be a discussion on how to do it, and of course the user can hire someone to do all the work if they are not willing to spend the time and learn. Getting it all done for free by volunteers is not something many are willing to do.
If multiple processes can dip in and out of the data, often concurrently, that is a whole new ball of wax. Others ahve mentioned that. It does not matter where the data is as in all cases you need ways to guarantee the data will not get messed up
But consider the simple case. Your program starts and runs ALONE. It initializes and reads in the data needed and stores it IN MEMORY. It might be in one or more pandas DataFrame structures, for example.
Now it processes requests and all the work in searching and updating data is done in memory. You can of course save a snapshot, or whatever logging, along the way in case of disaster. There are all kinds of ways to handle your needs such as searching for a record/row/whatever that contains the book you want, checking if it has been lent out, marking it appropriately and so on. The next request continues to operate on the in-memory data and your program will notice if someone else tries to take out a book no longer marked as available and so on. You can add all kinds of functionality to this loop that processes requests. Again, you design what? requests look like and how they are seen by your process and what replies it gives.
When done, or when you feel like it, you can save your data wherever you want. Here is an example of how to write to EXCEL:?https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_excel.html
Of course, many of us would suggest writing to a .CSV file that EXCEL can easily open and even update.
Now if you are not trusting, and want to save everything after each transaction and maybe even read it back in, fine. You may also want to not overwrite the previous file and that may mean renaming the old file or whatever makes you happy. If you have a lot of data and serious activity, as many libraries have, it could be a bit silly to read in a gigabyte of data all the time and it may indeed be a better idea to have a database back end that you send queries to and updates that typically are short and quick.
I am not sure why you are even talking about changing column names as normal ones can generally be the same. What some may have suggested is calculating additional columns from others just in Memory.
As with any tool, you need to learn enough to be dangerous. just copying snippets from the web may have drawbacks.
It sounds like one source of confusion is that your EXCEL data takes some shortcuts including lists of comma-separated values with multiple names, or something similar. That is not necessarily a barrier but may require more programming as searching for a specific user embedded within a larger amount of text may be harder than parsing the text into some other data structures that can more easily be searched individually. If you cannot change what you start with, that may be lots of extra work.
And it is not python alone what might require work but other languages like R too.?
Perhaps I missed a better description of the design.

-----Original Message-----
From: Dennis Lee Bieber <wlfraed at ix.netcom.com>
To: python-list at python.org
Sent: Sun, Jan 9, 2022 1:08 pm
Subject: Re: What to write or search on github to get the code for what is written below:

On Sun, 9 Jan 2022 08:53:58 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>
>I cannot change the column names.
>
??? But you can change the values under those columns? And that implies...
>
>

>No need to save the CSV file once the work is done CSV file will be exited without saving. If the above is going to work then I will be requiring something to see how to do all this that?s why written tutorial. For a more thorough analysis I have written once again what I want to do. 
>

??? You probably won't find "A tutorial"... You might find some examples
but you will have to figure out how to use the contents. For example, there
are multiple ways to have Python access Excel -- from simple read and write
modules xlrd/xlwt, (which is out-dated; xlrd does not work with .xlsx
format, only the older .xls format). Please review:
http://www.python-excel.org/ There is also the, on Windows, the potential
to use the pythonwin extensions OR ctypes module to make low-level calls
directly to the Excel DLLs (the pythonwin calls would look similar to how
Visual BASIC or C# would access Excel, and also VBA and VBS).

>Only the four functions that I have written below I have to make and that?s it. All the input will be entered by the user on a website:
>1. First, I have to check in the Excel sheet or table whether the book user has entered is present in the book bank or not.
>
??? Where does this requirement come from? Who is providing this "Excel
sheet or table"?

??? I saw nothing in your originally linked flask tutorial referencing
Excel or a book check-out system (I didn't even see much for interaction on
that tutorial -- you'll need to handle input forms for a web site of this
nature, which may get into tracking user sessions, cookie management, etc.)
The PythonHow site also does not, from what I read, host your finished
example -- they give instructions on creating a hosting account on another
service..

??? If this entire project is something you've dreamed up, then I'd suggest
dropping the Excel requirement, and going directly to a database system. Of
course, that will require, at a minimum, learning the basics of SQL and
then mapping that to the dialect used by the RDBM selected -- for
simplicity I'd suggest SQLite3. Python includes a module for that. 

??? SQLite3 is a file-server engine -- your application directly links to
the engine library and has access to the database file; versus a
client-server engine where your application sends request to the database
engine, and the engine handles the access to the database. If there is only
one instance of the Flask application the only thing you have to handle in
SQLite3 is transaction control -- rather than also being concerned about
multiple applications all opening the same database. With client-server
(MySQL/MariaDB, SQL Server, PostgreSQL...) you have to manage the server
itself (create user account for the application, etc.).

??? Learning SQL alone does not ensure a decent database design -- you
should also learn at least the first three Codd Normal Forms to decompose
your data into relations.

>2. If a book is present and the quantity of the required book is greater than 0 (COPIES_LEFT column in excel file) and if the user wants the book, it will be assigned to the user which he will take from the book bank physically. When COPIES_LEFT is less than or equal to 0 the message will be ?Book finished or not present?.
> 
>3. The quantity of the book in the Excel file will be reduced by 1 in the COPIES_LEFT column and the name of the borrower or user will be entered/added in the Excel file table or sheet already made and the column name is BORROWER?S NAME.

??? Which means you need to update this Excel file. That could present race
conditions if multiple users try to remove the same book (or even different
books). Excel does not have transaction control, so either you keep ALL the
data in memory (or copy the entire contents to a database system on
start-up AND copy the database back to Excel on application shut-down, you
run the risk of data corruption [and even the copy-in/copy-out could fail
if the application crashes at some stage]).

??? A database using transaction control should, at the least, mean on a
crash, the uncommitted transactions never took place (may require some time
on restart for the engine to clean up garbage workspace records).
Transaction control should also prevent the application from having two
sessions trying to update the same records at once -- the second session to
attempt an update will get an error, so you code needs to have a way to
delay/retry the update (possibly advising that user of new count value).

??? Note: many of the Python DB-API modules default to auto-commit for each
SQL statement executed. You'll need to ensure you've turned that off so
that multiple statement transactions can be performed. Read the
documentation for the module in use:
https://docs.python.org/3/library/sqlite3.html

https://www.google.com/search?q=sqlite+python+tutorial+pdf


> 
>4. The borrower?s or user name can be more than one so they will be separated with a comma in the Excel file BORROWER?S NAME column.

??? This is one of those spots where database normalization is pretty much
mandated. A normalized database table does not have "columns" that hold
multiple values. Instead one uses a separate table where only one value per
row is stored, but that row has an identifier (known as a foreign key) that
identifies the row in the other table carrying the rest of the information.

INVENTORY (*ID*, book details in multiple columns, Copies-on-Hand)

CHECKEDOUT (*ID*, _Inventory-ID_, name)

??? *ID*??? ??? primary (unique) key for the relation
??? _Inventory-ID_??? foreign key pointing to inventory record



-- 
??? Wulfraed? ? ? ? ? ? ? ? Dennis Lee Bieber? ? ? ? AF6VN
??? wlfraed at ix.netcom.com? ? http://wlfraed.microdiversity.freeddns.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

From cs at cskk.id.au  Sun Jan  9 18:15:27 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Mon, 10 Jan 2022 10:15:27 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <ts5mtgddlvtqflgfq150hhrbgjevqhogek@4ax.com>
References: <ts5mtgddlvtqflgfq150hhrbgjevqhogek@4ax.com>
Message-ID: <Ydtsj36Iosq0ClwG@cskk.homeip.net>

On 09Jan2022 13:08, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>>Only the four functions that I have written below I have to make and 
>>that?s it. All the input will be entered by the user on a website:
>>1. First, I have to check in the Excel sheet or table whether the book user has entered is present in the book bank or not.
>>
>	Where does this requirement come from? Who is providing this "Excel
>sheet or table"?

As a spectator, I have the impression that NArshad has to produce a 
little web interface to update records kept in a preexisting business 
system.

I've a friend who earns her $s doing data work for various corporates.  
Anecdotally, _many_ of these use Excel as their storage and work 
backend. Awful, but an existing situation. I infer that NArshad has to 
deal with such a situation.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From liedtke at punkt.de  Mon Jan 10 05:22:59 2022
From: liedtke at punkt.de (Lars Liedtke)
Date: Mon, 10 Jan 2022 11:22:59 +0100
Subject: Gunicorn - HTTP and HTTPS in the same instance?
In-Reply-To: <CANc-5UxwNO9u_QdwZZn2YROTUXiA4x9MnqHvBPu6Qt5ePKTLMg@mail.gmail.com>
References: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>
 <20220108161727.HRP0X993@iva4-b3ebd202b141.qloud-c.yandex.net>
 <CANc-5UxwNO9u_QdwZZn2YROTUXiA4x9MnqHvBPu6Qt5ePKTLMg@mail.gmail.com>
Message-ID: <fa29782e-35b6-faa2-a5a4-fdabeb9170d4@punkt.de>

I think this is more a thing of apporach. Nginx is quite simple to 
install and a config doing nothing else than redirecting https to https 
and proxying requests to a service (whichever tat is, in your case 
gunicorn) can become a nobrainer. That is what it became for me. 
Additionally the config for only this functionality are less than 10. So 
it's may seem complicated at first, but is way less oncxe you got used 
to it.

Cheers

Lars

Am 08.01.22 um 17:25 schrieb Skip Montanaro:
> Thanks all. I was hoping to get away without something more
> sophisticated like NGINX. This is just a piddly little archive of an
> old mailing list running on a single-core Ubuntu VM somewhere on the
> East Coast. Speed is not a real requirement. Load balancing seemed
> like overkill to me. Still, I guess if it has to be, then it has to
> be.
>
> Skip

-- 
punkt.de GmbH
Lars Liedtke
.infrastructure

Kaiserallee 13a	
76133 Karlsruhe

Tel. +49 721 9109 500
https://infrastructure.punkt.de
info at punkt.de

AG Mannheim 108285
Gesch?ftsf?hrer: J?rgen Egeling, Daniel Lienert, Fabian Stein


From m.wilson at btinternet.com  Mon Jan 10 10:27:37 2022
From: m.wilson at btinternet.com (Mandy and Michael Wilson)
Date: Mon, 10 Jan 2022 15:27:37 +0000 (GMT)
Subject: Inkscape
Message-ID: <15a0ca4d.4e93f.17e449a1a85.Webtop.96@btinternet.com>



hello Python


I wonder if you can help me out please.  I have recently added an 
extension into Inkscape called Axidraw which should enable me to 
hatchfill text, unforunately I am unable to use this facility as when I 
open a canvas in Inkscape and go to the axidraw extension I receive a 
message saying that I require Python 3.6 or greater.  I have downloaded 
and installed Python 3.10.1 but I am still receiving the same message. 
Can you please tell me what I am doing wrong ?  I look forward to 
hearing from you.



regards


Mandy Wilson

m. 07739 263 234








From liedtke at punkt.de  Mon Jan 10 05:28:56 2022
From: liedtke at punkt.de (Lars Liedtke)
Date: Mon, 10 Jan 2022 11:28:56 +0100
Subject: Gunicorn - HTTP and HTTPS in the same instance?
In-Reply-To: <CANc-5UxwNO9u_QdwZZn2YROTUXiA4x9MnqHvBPu6Qt5ePKTLMg@mail.gmail.com>
References: <CANc-5UyF1be7=rNi-u_RcCT3-=eBu=LMT4C3wQ2n4bHbfpfMig@mail.gmail.com>
 <20220108161727.HRP0X993@iva4-b3ebd202b141.qloud-c.yandex.net>
 <CANc-5UxwNO9u_QdwZZn2YROTUXiA4x9MnqHvBPu6Qt5ePKTLMg@mail.gmail.com>
Message-ID: <17a3be87-ab5c-84ee-a081-5330862795b0@punkt.de>

server {
 ???listen [::]:80;
 ???listen 80;


 ???server_name api.familie-liedtke.net;

 ???location / {
 ???????return 301 https://$host$request_uri;
 ???}

 ???include /usr/local/etc/nginx/include/letsencrypt.conf;
}

server {
 ???listen 443 ssl http2;
 ???listen [::]:443 ssl http2;

 ???server_name api.familie-liedtke.net;

 ???ssl_certificate 
/usr/local/etc/ssl/certs/api.familie-liedtke.net/fullchain.pem;
 ???ssl_certificate_key 
/usr/local/etc/ssl/certs/api.familie-liedtke.net/privkey.pem;

 ???root /var/www/api;

 ???location /weather/{
 ???????access_log /var/log/nginx/weather-access.log;
 ???????include uwsgi_params;
 ???????rewrite ^/weather(/.*)$ $1 break;
 ???????uwsgi_pass unix:///tmp/uwsgi.sock;
 ???}

 ???include /usr/local/etc/nginx/include/ssl.conf;
}

This is my config for a flask app on uwsgi. granted it is a bit more 
than the 10 lines I said. But there is not much sophistication to it.

Am 08.01.22 um 17:25 schrieb Skip Montanaro:
> Thanks all. I was hoping to get away without something more
> sophisticated like NGINX. This is just a piddly little archive of an
> old mailing list running on a single-core Ubuntu VM somewhere on the
> East Coast. Speed is not a real requirement. Load balancing seemed
> like overkill to me. Still, I guess if it has to be, then it has to
> be.
>
> Skip

-- 
punkt.de GmbH
Lars Liedtke
.infrastructure

Kaiserallee 13a	
76133 Karlsruhe

Tel. +49 721 9109 500
https://infrastructure.punkt.de
info at punkt.de

AG Mannheim 108285
Gesch?ftsf?hrer: J?rgen Egeling, Daniel Lienert, Fabian Stein

From narshad.380 at gmail.com  Mon Jan 10 11:39:19 2022
From: narshad.380 at gmail.com (NArshad)
Date: Mon, 10 Jan 2022 08:39:19 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <mailman.134.1641762330.3079.python-list@python.org>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <7c49e7a1-2a7b-fc5f-b93b-e264dc68b909@mrabarnett.plus.com>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <mailman.134.1641762330.3079.python-list@python.org>
Message-ID: <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>


Using openpyxl is pretty straightforward:


from openpyxl import load_workbook
wb = load_workbook(spreadsheet_path)
sheet = wb.active

# Reading the values in cells:
print('Cell A1 contains', sheet['A1'].value)
print('Cell A2 contains', sheet['A2'].value)
print('Cell B1 contains', sheet['B1'].value)

# Alternatively:
print('Cell A1 contains', sheet.cell(1, 1).value)
print('Cell A2 contains', sheet.cell(1, 2).value)
print('Cell B1 contains', sheet.cell(2, 1).value)

"""
The cell numbers (A1 or A2 or A3 or A4??? ) are not fixed they can be any. I don't know what the cell number is going to be that's what the problem is.

The user is going to enter the book name in an HTML form present on a website then it will be checked whether the book user has entered is present or not in the Excel file. If the book is present in the book bank and the user requires that book then one will be issued to the user and the total number of books will be reduced by 1 (one) and the user or borrower?s name will be entered in the Excel?s table row in which the book name is present separated by a comma by other borrower names. The borrower's name can be more than one because more than one copies of the book are there as these are the books that are taught in schools. 
"""
 
# Changing the value in a cell:
sheet.cell(1, 20).value = 'TEST'

# A value of None means that the cell is empty.

wb.save(spreadsheet_path)
wb.close()

From torriem at gmail.com  Mon Jan 10 14:11:36 2022
From: torriem at gmail.com (Michael Torrie)
Date: Mon, 10 Jan 2022 12:11:36 -0700
Subject: Inkscape
In-Reply-To: <15a0ca4d.4e93f.17e449a1a85.Webtop.96@btinternet.com>
References: <15a0ca4d.4e93f.17e449a1a85.Webtop.96@btinternet.com>
Message-ID: <c36bb52d-aed5-e739-9d6d-564aaec679dd@gmail.com>

On 1/10/22 8:27 AM, Mandy and Michael Wilson via Python-list wrote:
> I wonder if you can help me out please.  I have recently added an 
> extension into Inkscape called Axidraw which should enable me to 
> hatchfill text, unforunately I am unable to use this facility as when I 
> open a canvas in Inkscape and go to the axidraw extension I receive a 
> message saying that I require Python 3.6 or greater.  I have downloaded 
> and installed Python 3.10.1 but I am still receiving the same message. 
> Can you please tell me what I am doing wrong ?  I look forward to 
> hearing from you.

Most likely you'll need to ask the folks on the Inkscape forums or
mailing list.  Very few of us have any working understanding of how
Python is used within Inkscape.

What operating system are you using?  If you're running Windows, then
installing Python 3.10 will have no effect on Inkscape because inkscape
does not know about any other installed versions of Python.  It's built
with and ships with a specific version of Python inside of Inkscape.  So
it seems to me like you need to install the latest version of Inkscape,
which should come with the latest supported version of Python inside of it.

From nattaylor at gmail.com  Mon Jan 10 14:10:05 2022
From: nattaylor at gmail.com (Nat Taylor)
Date: Mon, 10 Jan 2022 14:10:05 -0500
Subject: symlinks with python3 http.server.CGIHTTPRequestHandler
Message-ID: <CA+6dARZJYahGpUVzyC49CiQwf_Tz983c3vLMY+MddKSB_qJSSg@mail.gmail.com>

Is it possible to get http.server.CGIHTTPRequestHandler to run a symlink-ed
script?

In the example below, GET /cgi-bin/test.py results in a 404 because it is a
symlink.

% mkdir -p test/cgi-bin
% cd test
% vi test.py
% chmod +x test.py
% ln -s test.py cgi-bin
% cp test.py cgi-bin/test2.py
% chmod +x cgi-bin/test2.py
% python3 -m http.server --cgi 8090
Serving HTTP on :: port 8090 (http://[::]:8090/) ...
::1 - - [06/Jan/2022 09:21:48] code 404, message No such CGI script
('/cgi-bin/test.py')
::1 - - [06/Jan/2022 09:21:48] "GET /cgi-bin/test.py HTTP/1.1" 404 -
::1 - - [06/Jan/2022 09:21:59] "GET /cgi-bin/test2.py HTTP/1.1" 200 -

From jcasale at activenetwerx.com  Mon Jan 10 13:43:20 2022
From: jcasale at activenetwerx.com (Joseph L. Casale)
Date: Mon, 10 Jan 2022 18:43:20 +0000
Subject: Script profiling details
Message-ID: <62b6034aed664df6a713ef7818ea6e4e@activenetwerx.com>

I am trying to track down a slow script startup time. I have executed the
script using `python -m cProfile -o profile /path/script.py` and read through
the results, but the largest culprit only shows various built-ins.

I expected this given the implementation, but I was hoping to get some
finer details so I can track down the specific module or at least the specific
file so I have a place to start reviewing code for optimizations.

Is there something I can use to analyze the existing profile output or to generate
it with more feedback?

Thanks,
jlc

From python at mrabarnett.plus.com  Mon Jan 10 14:21:17 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 10 Jan 2022 19:21:17 +0000
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <7c49e7a1-2a7b-fc5f-b93b-e264dc68b909@mrabarnett.plus.com>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <mailman.134.1641762330.3079.python-list@python.org>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
Message-ID: <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>

On 2022-01-10 16:39, NArshad wrote:
> 
> Using openpyxl is pretty straightforward:
> 
> 
> from openpyxl import load_workbook
> wb = load_workbook(spreadsheet_path)
> sheet = wb.active
> 
> # Reading the values in cells:
> print('Cell A1 contains', sheet['A1'].value)
> print('Cell A2 contains', sheet['A2'].value)
> print('Cell B1 contains', sheet['B1'].value)
> 
> # Alternatively:
> print('Cell A1 contains', sheet.cell(1, 1).value)
> print('Cell A2 contains', sheet.cell(1, 2).value)
> print('Cell B1 contains', sheet.cell(2, 1).value)
> 
> """
> The cell numbers (A1 or A2 or A3 or A4??? ) are not fixed they can be any. I don't know what the cell number is going to be that's what the problem is.
> 
> The user is going to enter the book name in an HTML form present on a website then it will be checked whether the book user has entered is present or not in the Excel file. If the book is present in the book bank and the user requires that book then one will be issued to the user and the total number of books will be reduced by 1 (one) and the user or borrower?s name will be entered in the Excel?s table row in which the book name is present separated by a comma by other borrower names. The borrower's name can be more than one because more than one copies of the book are there as these are the books that are taught in schools.
> """
>   
[snip]
How are the relevant cells identified in the spreadsheet?

It's often the case that the cells on the first row contain text as 
column labels. If that's what you have in your spreadsheet, then read 
the cells on the first row for the column labels and put them in a dict 
to map from column label to column number.

From barry at barrys-emacs.org  Mon Jan 10 17:35:54 2022
From: barry at barrys-emacs.org (Barry)
Date: Mon, 10 Jan 2022 22:35:54 +0000
Subject: Script profiling details
In-Reply-To: <62b6034aed664df6a713ef7818ea6e4e@activenetwerx.com>
References: <62b6034aed664df6a713ef7818ea6e4e@activenetwerx.com>
Message-ID: <CDA36A7A-A8EF-41DE-BEE7-91598E0CE39F@barrys-emacs.org>

S

> On 10 Jan 2022, at 19:29, Joseph L. Casale <jcasale at activenetwerx.com> wrote:
> 
> ?I am trying to track down a slow script startup time. I have executed the
> script using `python -m cProfile -o profile /path/script.py` and read through
> the results, but the largest culprit only shows various built-ins.
> 
> I expected this given the implementation, but I was hoping to get some
> finer details so I can track down the specific module or at least the specific
> file so I have a place to start reviewing code for optimizations.
> 
> Is there something I can use to analyze the existing profile output or to generate
> it with more feedback?

Read the profile analyses docs/code and report using options that show you which callers are using the builtins.

If you need details reply and I will look at the scripts I use that produce a number of report types,

Barry
> 
> Thanks,
> jlc
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From Marco.Sulla.Python at gmail.com  Mon Jan 10 18:25:57 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Tue, 11 Jan 2022 00:25:57 +0100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CAPTjJmqJ+nZfO-T6bkCbSERVko2TbQP5g11y_oB0+ZgUx06cfw@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
 <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
 <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>
 <CAPTjJmqJ+nZfO-T6bkCbSERVko2TbQP5g11y_oB0+ZgUx06cfw@mail.gmail.com>
Message-ID: <CABbU2U_d+Sj8naMwz8jWPbc2+2=z1EmoSRYwjnAeA9Dceg1F=g@mail.gmail.com>

On Wed, 5 Jan 2022 at 23:02, Chris Angelico <rosuav at gmail.com> wrote:
>
> On Thu, Jan 6, 2022 at 8:01 AM Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
> >
> > On Wed, 5 Jan 2022 at 14:16, Chris Angelico <rosuav at gmail.com> wrote:
> > > That's an entirely invisible optimization, but it's more than just
> > > "frozenset is faster than set". It's that a frozenset or tuple can be
> > > stored as a function's constants, which is a massive difference.
> >
> > Can you explain this?
>
> Play around with dis.dis and timeit.

? I don't understand. You're talking about function constants. What
are they? I can't dig deep into something if I can't know what it is.
Maybe are you talking about function default values for parameters?

> > > Function positional arguments aren't interchangeable, so it makes
> > > sense to have them as a tuple.
> >
> > You are wrong, since kwarg is a dict. Indeed I proposed to use
> > frozendict for kwargs, and Guido said that it's a pity that this will
> > break a lot of existing Python code :D, since the fact that args is
> > _immutable_ and kwargs not always bothered him.
>
> Excuse me? I mentioned kwargs in the part that you removed from the
> quote, and the part you're quoting explicitly says "positional
> arguments".

Ok, I quote also the other part:

> (Function *keyword* arguments, on the other hand, are different; as
> long as the mapping from keys to values is maintained, you can remove
> some of them and pass the rest on, without fundamentally changing
> their meaning.)

First of all, I repeat, Guido said (more or less) that in a perfect
world, kwargs are immutable. Or maybe I did not understand what he
said, maybe he said that in a perfect world also args are mutable. But
I suppose it's more probable the first hypothesis :D

Secondly, you can also get the args from a function, transform it in a
list, change something and pass it unpacked to another function. You
will not change the meaning of the tuple, since, well, you copied it
in another mutable object. The original object is untouched.

I perfectly agree that, in the majority of cases, returning an
immutable vs a mutable are a matter of... sense? Meaning? Ok, I
perfectly agree. But IMHO there are many cases in which immutable
objects are used for a matter of speed, and I bet that args is one of
them.

> > Anyway, I'm starting to think that neither set nor frozenset are good
> > for dict items:
> >
> > (venv_3_10) marco at buzz:~$ python
> > Python 3.10.0 (heads/3.10-dirty:f6e8b80d20, Nov 18 2021, 19:16:18)
> > [GCC 10.1.1 20200718] on linux
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>> a = {1: 2}
> > >>> b = {3: []}
> > >>> a | b
> > {1: 2, 3: []}
> > >>> a.items() | b.items()
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> > TypeError: unhashable type: 'list'
> > >>>
>
> Well yes. Only dict keys can be considered to be set-like.

This is not true. It's at least from Python 3.6, and I think also
before, that almost the full Set API was added to both keys and items
view.
Items indeed are a sort of set, in a mathematical sense, since any
pair (key, value) is unique, even if value is mutable.

> I don't
> know WHAT you think you're trying to do here, but if you ever thought
> of set operations on dict values, you may want to completely rethink
> what you're doing.

set ops on values? Never said that :) I said that currently you can
operate on item views with set operators. This is a fact.

I also said that, since py sets accept only hashable objects, maybe
another ad-hoc object should be used for the result of the items
operations.
But maybe the change isn't worth the additional trouble. Indeed I
didn't know about the new set methods and operations on dict views
until I explored dictobject.c

> Performance is not an automatic result of immutability. That simply
> isn't how it works.

Of course. You can use a proxy and slow down almost everything much
more. Or you can simply create a version of the mutable object with
fewer methods, as more or less frozenset is. I checked the
implementation, no fast iteration is implemented. I do not understand
why in `for x in {1, 2, 3}` the set is substituted by a frozenset.

From rosuav at gmail.com  Mon Jan 10 19:03:11 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 11 Jan 2022 11:03:11 +1100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U_d+Sj8naMwz8jWPbc2+2=z1EmoSRYwjnAeA9Dceg1F=g@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
 <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
 <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>
 <CAPTjJmqJ+nZfO-T6bkCbSERVko2TbQP5g11y_oB0+ZgUx06cfw@mail.gmail.com>
 <CABbU2U_d+Sj8naMwz8jWPbc2+2=z1EmoSRYwjnAeA9Dceg1F=g@mail.gmail.com>
Message-ID: <CAPTjJmqXv8C5ZbQKG21nbZsGGifk6J3yj1m1OH0C-OUKuM9gBQ@mail.gmail.com>

On Tue, Jan 11, 2022 at 10:26 AM Marco Sulla
<Marco.Sulla.Python at gmail.com> wrote:
>
> On Wed, 5 Jan 2022 at 23:02, Chris Angelico <rosuav at gmail.com> wrote:
> >
> > On Thu, Jan 6, 2022 at 8:01 AM Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
> > >
> > > On Wed, 5 Jan 2022 at 14:16, Chris Angelico <rosuav at gmail.com> wrote:
> > > > That's an entirely invisible optimization, but it's more than just
> > > > "frozenset is faster than set". It's that a frozenset or tuple can be
> > > > stored as a function's constants, which is a massive difference.
> > >
> > > Can you explain this?
> >
> > Play around with dis.dis and timeit.
>
> ? I don't understand. You're talking about function constants. What
> are they? I can't dig deep into something if I can't know what it is.
> Maybe are you talking about function default values for parameters?

No, I'm talking about constants. Every function has them.

> Of course. You can use a proxy and slow down almost everything much
> more. Or you can simply create a version of the mutable object with
> fewer methods, as more or less frozenset is. I checked the
> implementation, no fast iteration is implemented. I do not understand
> why in `for x in {1, 2, 3}` the set is substituted by a frozenset.

Constants. Like I said, play around with dis.dis, and explore what's
already happening. A set can't be a constant, a frozenset can be.
Constants are way faster than building from scratch.

Explore. Play around. I'm not going to try to explain everything in detail.

If you're delving into the details of the C implementation of the
dictionary, I would have expected you'd already be familiar with the
way that functions behave.

ChrisA

From kvratkin at yandex.ru  Tue Jan 11 02:22:50 2022
From: kvratkin at yandex.ru (Kirill Ratkin)
Date: Tue, 11 Jan 2022 10:22:50 +0300
Subject: Script profiling details
In-Reply-To: <62b6034aed664df6a713ef7818ea6e4e@activenetwerx.com>
References: <62b6034aed664df6a713ef7818ea6e4e@activenetwerx.com>
Message-ID: <7231641885014@mail.yandex.ru>

   Hi Joseph,
   **
   Did you try scalene profiler?
   Recently I solved a similar problem and scalene really helped me.
   **
   It creates a well-formed HTML report, and you can optionally
   enable/disable report about
   a code outside your project, for example - standard library.
   **
   Here it is - [1]https://pypi.org/project/scalene/
   **
   10.01.2022, 22:28, "Joseph L. Casale" <jcasale at activenetwerx.com>:

     I am trying to track down a slow script startup time. I have executed
     the
     script using `python -m cProfile -o profile /path/script.py` and read
     through
     the results, but the largest culprit only shows various built-ins.

     I expected this given the implementation, but I was hoping to get some
     finer details so I can track down the specific module or at least the
     specific
     file so I have a place to start reviewing code for optimizations.

     Is there something I can use to analyze the existing profile output or
     to generate
     it with more feedback?

     Thanks,
     jlc

     --
     [2]https://mail.python.org/mailman/listinfo/python-list

References

   Visible links
   1. https://pypi.org/project/scalene/
   2. https://mail.python.org/mailman/listinfo/python-list

From kvratkin at yandex.ru  Tue Jan 11 06:43:51 2022
From: kvratkin at yandex.ru (Kirill Ratkin)
Date: Tue, 11 Jan 2022 14:43:51 +0300
Subject: symlinks with python3 http.server.CGIHTTPRequestHandler
In-Reply-To: <CA+6dARZJYahGpUVzyC49CiQwf_Tz983c3vLMY+MddKSB_qJSSg@mail.gmail.com>
References: <CA+6dARZJYahGpUVzyC49CiQwf_Tz983c3vLMY+MddKSB_qJSSg@mail.gmail.com>
Message-ID: <3c23587a-84a8-e6c2-7005-779a39d11687@yandex.ru>

Hi

Maybe you have some restrictions on file system level. Some selinux for 
example.


I try similar steps on my local 'linux mint' and ... linked script is 
called my http.server without errors.


Here is my 'tree':

??? cgi-bin
??? ??? test.py -> ../orig.py
??? orig.py

All files are executable and test.py is link to orig.py which is in one 
directory level up:

$ ls -Rla
.:
total 16
drwxrwxr-x? 3 kirill kirill 4096 Jan 11 14:04 .
drwxrwxrwt 23 root?? root?? 4096 Jan 11 14:11 ..
drwxrwxr-x? 2 kirill kirill 4096 Jan 11 14:03 cgi-bin
-rwxrwxr-x? 1 kirill kirill?? 31 Jan 11 14:04 orig.py

./cgi-bin:
total 8
drwxrwxr-x 2 kirill kirill 4096 Jan 11 14:03 .
drwxrwxr-x 3 kirill kirill 4096 Jan 11 14:04 ..
lrwxrwxrwx 1 kirill kirill?? 10 Jan 11 14:03 test.py -> ../orig.py

I do 'curl' request:

curl -v http://0.0.0.0:8000/cgi-bin/test.py

And http.server get log:

$ python3 -m http.server --cgi 8000
Serving HTTP on 0.0.0.0 port 8000 (http://0.0.0.0:8000/) ...
127.0.0.1 - - [11/Jan/2022 14:40:19] "GET /cgi-bin/test.py HTTP/1.1" 200 -


On 1/10/22 22:10, Nat Taylor wrote:
> python3 -m http.server --cgi 8090

From rosuav at gmail.com  Tue Jan 11 06:59:04 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 11 Jan 2022 22:59:04 +1100
Subject: symlinks with python3 http.server.CGIHTTPRequestHandler
In-Reply-To: <CA+6dARZJYahGpUVzyC49CiQwf_Tz983c3vLMY+MddKSB_qJSSg@mail.gmail.com>
References: <CA+6dARZJYahGpUVzyC49CiQwf_Tz983c3vLMY+MddKSB_qJSSg@mail.gmail.com>
Message-ID: <CAPTjJmqqf5SkWRxc5cOFOBYK5iD09MOVG5LNmuDOL8+-y5tOaQ@mail.gmail.com>

On Tue, Jan 11, 2022 at 6:17 AM Nat Taylor <nattaylor at gmail.com> wrote:
>
> Is it possible to get http.server.CGIHTTPRequestHandler to run a symlink-ed
> script?
>
> In the example below, GET /cgi-bin/test.py results in a 404 because it is a
> symlink.
>
> % mkdir -p test/cgi-bin
> % cd test
> % vi test.py
> % chmod +x test.py
> % ln -s test.py cgi-bin

I just noticed something. This might not produce the effect you want;
a symlink aims at a specific named target, and if you use a relative
path, it's relative to the directory containing the symlink, not the
directory you were in when you created it. What happens if, instead,
you do this:

% cd cgi-bin
% ln -s ../test.py .

? Conversely, can you, with whichever setup you're looking at, read
the source code for the script by catting it from cgi-bin? It might be
that the link is actually unreadable or pointing to the wrong place.

ChrisA

From dieter at handshake.de  Tue Jan 11 12:14:12 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Tue, 11 Jan 2022 18:14:12 +0100
Subject: Script profiling details
In-Reply-To: <62b6034aed664df6a713ef7818ea6e4e@activenetwerx.com>
References: <62b6034aed664df6a713ef7818ea6e4e@activenetwerx.com>
Message-ID: <25053.47844.354597.86444@ixdm.fritz.box>

Joseph L. Casale wrote at 2022-1-10 18:43 +0000:
> ...
>I expected this given the implementation, but I was hoping to get some
>finer details so I can track down the specific module or at least the specific
>file so I have a place to start reviewing code for optimizations.
>
>Is there something I can use to analyze the existing profile output or to generate
>it with more feedback?

You might try `py-spy`.

From Marco.Sulla.Python at gmail.com  Tue Jan 11 13:49:20 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Tue, 11 Jan 2022 19:49:20 +0100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CAPTjJmqXv8C5ZbQKG21nbZsGGifk6J3yj1m1OH0C-OUKuM9gBQ@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
 <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
 <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>
 <CAPTjJmqJ+nZfO-T6bkCbSERVko2TbQP5g11y_oB0+ZgUx06cfw@mail.gmail.com>
 <CABbU2U_d+Sj8naMwz8jWPbc2+2=z1EmoSRYwjnAeA9Dceg1F=g@mail.gmail.com>
 <CAPTjJmqXv8C5ZbQKG21nbZsGGifk6J3yj1m1OH0C-OUKuM9gBQ@mail.gmail.com>
Message-ID: <CABbU2U-wosnDQhN4eMZZfvypJ=hMf7u9sfvVRDjtS8J8e7X11w@mail.gmail.com>

Ok... so I suppose, since you're inviting me to use dis and look at the
bytecode, that are you talking about constants in assembly, so const in C?
Sorry for the confusion, I'm not so skilled in C and I know nearly nothing
about assembly. Furthermore I never look at the bytecode of any language
before, so I simply didn't understand you.

I think this is what you mean:

>>> dis.dis("for _ in {1, 2}: pass")
  1           0 SETUP_LOOP              12 (to 14)
              2 LOAD_CONST               3 (frozenset({1, 2}))
              4 GET_ITER
        >>    6 FOR_ITER                 4 (to 12)
              8 STORE_NAME               0 (_)
             10 JUMP_ABSOLUTE            6
        >>   12 POP_BLOCK
        >>   14 LOAD_CONST               2 (None)
             16 RETURN_VALUE
>>> a = {1, 2}
>>> dis.dis("for _ in a: pass")
  1           0 SETUP_LOOP              12 (to 14)
              2 LOAD_NAME                0 (a)
              4 GET_ITER
        >>    6 FOR_ITER                 4 (to 12)
              8 STORE_NAME               1 (_)
             10 JUMP_ABSOLUTE            6
        >>   12 POP_BLOCK
        >>   14 LOAD_CONST               0 (None)
             16 RETURN_VALUE


On Tue, 11 Jan 2022 at 01:05, Chris Angelico <rosuav at gmail.com> wrote:

> On Tue, Jan 11, 2022 at 10:26 AM Marco Sulla
> <Marco.Sulla.Python at gmail.com> wrote:
> >
> > On Wed, 5 Jan 2022 at 23:02, Chris Angelico <rosuav at gmail.com> wrote:
> > >
> > > On Thu, Jan 6, 2022 at 8:01 AM Marco Sulla <
> Marco.Sulla.Python at gmail.com> wrote:
> > > >
> > > > On Wed, 5 Jan 2022 at 14:16, Chris Angelico <rosuav at gmail.com>
> wrote:
> > > > > That's an entirely invisible optimization, but it's more than just
> > > > > "frozenset is faster than set". It's that a frozenset or tuple can
> be
> > > > > stored as a function's constants, which is a massive difference.
> > > >
> > > > Can you explain this?
> > >
> > > Play around with dis.dis and timeit.
> >
> > ? I don't understand. You're talking about function constants. What
> > are they? I can't dig deep into something if I can't know what it is.
> > Maybe are you talking about function default values for parameters?
>
> No, I'm talking about constants. Every function has them.
>
> > Of course. You can use a proxy and slow down almost everything much
> > more. Or you can simply create a version of the mutable object with
> > fewer methods, as more or less frozenset is. I checked the
> > implementation, no fast iteration is implemented. I do not understand
> > why in `for x in {1, 2, 3}` the set is substituted by a frozenset.
>
> Constants. Like I said, play around with dis.dis, and explore what's
> already happening. A set can't be a constant, a frozenset can be.
> Constants are way faster than building from scratch.
>
> Explore. Play around. I'm not going to try to explain everything in detail.
>
> If you're delving into the details of the C implementation of the
> dictionary, I would have expected you'd already be familiar with the
> way that functions behave.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From rosuav at gmail.com  Tue Jan 11 14:44:11 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 12 Jan 2022 06:44:11 +1100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U-wosnDQhN4eMZZfvypJ=hMf7u9sfvVRDjtS8J8e7X11w@mail.gmail.com>
References: <CABbU2U9EBev-MWb+5j=uUZJaYAtKHw+raAviOFpd50d9ojsA2w@mail.gmail.com>
 <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
 <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
 <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>
 <CAPTjJmqJ+nZfO-T6bkCbSERVko2TbQP5g11y_oB0+ZgUx06cfw@mail.gmail.com>
 <CABbU2U_d+Sj8naMwz8jWPbc2+2=z1EmoSRYwjnAeA9Dceg1F=g@mail.gmail.com>
 <CAPTjJmqXv8C5ZbQKG21nbZsGGifk6J3yj1m1OH0C-OUKuM9gBQ@mail.gmail.com>
 <CABbU2U-wosnDQhN4eMZZfvypJ=hMf7u9sfvVRDjtS8J8e7X11w@mail.gmail.com>
Message-ID: <CAPTjJmrrwhOHZ3SznBsAhdrCR3YNDaB6D5Bmy-NyB+sKmUkA-w@mail.gmail.com>

On Wed, Jan 12, 2022 at 5:49 AM Marco Sulla
<Marco.Sulla.Python at gmail.com> wrote:
>
> Ok... so I suppose, since you're inviting me to use dis and look at the bytecode, that are you talking about constants in assembly, so const in C? Sorry for the confusion, I'm not so skilled in C and I know nearly nothing about assembly. Furthermore I never look at the bytecode of any language before, so I simply didn't understand you.
>

No, I'm talking about constants in Python.

> I think this is what you mean:
>
> >>> dis.dis("for _ in {1, 2}: pass")
>   1           0 SETUP_LOOP              12 (to 14)
>               2 LOAD_CONST               3 (frozenset({1, 2}))

This is a constant.

>               4 GET_ITER
>         >>    6 FOR_ITER                 4 (to 12)
>               8 STORE_NAME               0 (_)
>              10 JUMP_ABSOLUTE            6
>         >>   12 POP_BLOCK
>         >>   14 LOAD_CONST               2 (None)

This is a constant.

>              16 RETURN_VALUE
> >>> a = {1, 2}
> >>> dis.dis("for _ in a: pass")
>   1           0 SETUP_LOOP              12 (to 14)
>               2 LOAD_NAME                0 (a)
>               4 GET_ITER
>         >>    6 FOR_ITER                 4 (to 12)
>               8 STORE_NAME               1 (_)
>              10 JUMP_ABSOLUTE            6
>         >>   12 POP_BLOCK
>         >>   14 LOAD_CONST               0 (None)

This is a constant.

>              16 RETURN_VALUE
>

Try the same thing with other code and see whether you can see a
difference. In other words, *play around with dis.dis*.

If you're trying to hack on the internals of the CPython dictionary
implementation and do not understand how Python bytecode is executed,
you are doomed to make many MANY errors of judgment about performance.

ChrisA

From arj.python at gmail.com  Tue Jan 11 15:35:46 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Wed, 12 Jan 2022 00:35:46 +0400
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <07a0b6eb37707305269a1e0c65ac8e2308eb814b.camel@anode.ca>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
 <20220108210631.uodonbypheirvl2d@hjp.at>
 <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
 <07a0b6eb37707305269a1e0c65ac8e2308eb814b.camel@anode.ca>
Message-ID: <CADrxXXnfUvQbge--Ngp6hMJpp_RVcGWmyAO9N7zbfARQw7ARJA@mail.gmail.com>

Added RSS:

2.0 unless later versions have some advantages:

https://pyherald.com/rss.xml

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

>

From pbryan at anode.ca  Tue Jan 11 15:38:44 2022
From: pbryan at anode.ca (Paul Bryan)
Date: Tue, 11 Jan 2022 12:38:44 -0800
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <CADrxXXnfUvQbge--Ngp6hMJpp_RVcGWmyAO9N7zbfARQw7ARJA@mail.gmail.com>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
 <20220108210631.uodonbypheirvl2d@hjp.at>
 <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
 <07a0b6eb37707305269a1e0c65ac8e2308eb814b.camel@anode.ca>
 <CADrxXXnfUvQbge--Ngp6hMJpp_RVcGWmyAO9N7zbfARQw7ARJA@mail.gmail.com>
Message-ID: <1ef4904596383a720a6e4fbdd28969cde5ba2602.camel@anode.ca>

Subscribed. ??

On Wed, 2022-01-12 at 00:35 +0400, Abdur-Rahmaan Janhangeer wrote:
> Added RSS:
> 
> 2.0 unless later versions have some advantages:
> 
> https://pyherald.com/rss.xml
> 
> Kind Regards,
> 
> Abdur-Rahmaan Janhangeer
> about | blog?
> github
> Mauritius
> 


From hjp-python at hjp.at  Tue Jan 11 15:57:36 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Tue, 11 Jan 2022 21:57:36 +0100
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <1ef4904596383a720a6e4fbdd28969cde5ba2602.camel@anode.ca>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
 <20220108210631.uodonbypheirvl2d@hjp.at>
 <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
 <07a0b6eb37707305269a1e0c65ac8e2308eb814b.camel@anode.ca>
 <CADrxXXnfUvQbge--Ngp6hMJpp_RVcGWmyAO9N7zbfARQw7ARJA@mail.gmail.com>
 <1ef4904596383a720a6e4fbdd28969cde5ba2602.camel@anode.ca>
Message-ID: <20220111205736.wf7ezf7njpxcx47d@hjp.at>

On 2022-01-11 12:38:44 -0800, Paul Bryan wrote:
> Subscribed. ??

Same.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220111/964e308a/attachment.sig>

From hjp-python at hjp.at  Tue Jan 11 16:07:18 2022
From: hjp-python at hjp.at (Peter J. Holzer)
Date: Tue, 11 Jan 2022 22:07:18 +0100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <CABbU2U-wosnDQhN4eMZZfvypJ=hMf7u9sfvVRDjtS8J8e7X11w@mail.gmail.com>
References: <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
 <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
 <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>
 <CAPTjJmqJ+nZfO-T6bkCbSERVko2TbQP5g11y_oB0+ZgUx06cfw@mail.gmail.com>
 <CABbU2U_d+Sj8naMwz8jWPbc2+2=z1EmoSRYwjnAeA9Dceg1F=g@mail.gmail.com>
 <CAPTjJmqXv8C5ZbQKG21nbZsGGifk6J3yj1m1OH0C-OUKuM9gBQ@mail.gmail.com>
 <CABbU2U-wosnDQhN4eMZZfvypJ=hMf7u9sfvVRDjtS8J8e7X11w@mail.gmail.com>
Message-ID: <20220111210718.r5vvjobf4i4ufjzu@hjp.at>

On 2022-01-11 19:49:20 +0100, Marco Sulla wrote:
> I think this is what you mean:
> 
> >>> dis.dis("for _ in {1, 2}: pass")
>   1           0 SETUP_LOOP              12 (to 14)
>               2 LOAD_CONST               3 (frozenset({1, 2}))
>               4 GET_ITER
>         >>    6 FOR_ITER                 4 (to 12)
>               8 STORE_NAME               0 (_)
>              10 JUMP_ABSOLUTE            6
>         >>   12 POP_BLOCK
>         >>   14 LOAD_CONST               2 (None)
>              16 RETURN_VALUE
> >>> a = {1, 2}
> >>> dis.dis("for _ in a: pass")
>   1           0 SETUP_LOOP              12 (to 14)
>               2 LOAD_NAME                0 (a)
>               4 GET_ITER
>         >>    6 FOR_ITER                 4 (to 12)
>               8 STORE_NAME               1 (_)
>              10 JUMP_ABSOLUTE            6
>         >>   12 POP_BLOCK
>         >>   14 LOAD_CONST               0 (None)
>              16 RETURN_VALUE

I think you have omitted the part that Chris was hinting at.

>>> dis.dis("a = {1, 2};\nfor _ in a: pass")
  1           0 LOAD_CONST               0 (1)
              2 LOAD_CONST               1 (2)
              4 BUILD_SET                2
              6 STORE_NAME               0 (a)

  2           8 LOAD_NAME                0 (a)
             10 GET_ITER
        >>   12 FOR_ITER                 4 (to 18)
             14 STORE_NAME               1 (_)
             16 JUMP_ABSOLUTE           12
        >>   18 LOAD_CONST               2 (None)
             20 RETURN_VALUE

Now compare

              2 LOAD_CONST               3 (frozenset({1, 2}))

with

  1           0 LOAD_CONST               0 (1)
              2 LOAD_CONST               1 (2)
              4 BUILD_SET                2

and you see the difference between using a frozenset as a constant and
building a set at runtime.

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://mail.python.org/pipermail/python-list/attachments/20220111/3254e652/attachment.sig>

From spluque at gmail.com  Tue Jan 11 17:09:57 2022
From: spluque at gmail.com (Sebastian Luque)
Date: Tue, 11 Jan 2022 16:09:57 -0600
Subject: sharing data across Examples docstrings
Message-ID: <87v8ypgc0a.fsf@phoca.sebkatja.org>

Hello,

I am searching for a mechanism for sharing data across Examples sections
in docstrings within a class.  For instance:

class Foo:

    def foo(self):
        """Method foo title

        The example generating data below may be much more laborious.

        Examples
        --------
        >>> x = list("abc")  # may be very long and tedious to generate

        """
        pass

    def bar(self):
        """Method bar title

        Examples
        --------
        >>> # do something else with x from foo Example

        """
        pass


Thanks,
-- 
Seb

From rosuav at gmail.com  Tue Jan 11 17:17:03 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 12 Jan 2022 09:17:03 +1100
Subject: sharing data across Examples docstrings
In-Reply-To: <87v8ypgc0a.fsf@phoca.sebkatja.org>
References: <87v8ypgc0a.fsf@phoca.sebkatja.org>
Message-ID: <CAPTjJmrws0tGHWoQVrPPZtTjzD1XLW0Q8W7f+47rf49eO8cgQA@mail.gmail.com>

On Wed, Jan 12, 2022 at 9:11 AM Sebastian Luque <spluque at gmail.com> wrote:
>
> Hello,
>
> I am searching for a mechanism for sharing data across Examples sections
> in docstrings within a class.  For instance:

This seems like trying to cram too much information into the
docstring, but oh well... do what you will.

I'd recommend a decorator. The easiest way would probably be to have a
placeholder of some sort in the actual docstring, and in the
decorator, you replace __doc__ with the modified form.

To do what you're asking for, your decorator would either need to go
through every method in the class and mutate its docstring (in which
case you'd decorate the class), or mutate the docstring of exactly one
function (in which case you'd decorate that method).

ChrisA

From cs at cskk.id.au  Tue Jan 11 17:28:16 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Wed, 12 Jan 2022 09:28:16 +1100
Subject: sharing data across Examples docstrings
In-Reply-To: <87v8ypgc0a.fsf@phoca.sebkatja.org>
References: <87v8ypgc0a.fsf@phoca.sebkatja.org>
Message-ID: <Yd4EgI5rHHDmc/zf@cskk.homeip.net>

On 11Jan2022 16:09, Sebastian Luque <spluque at gmail.com> wrote:
>I am searching for a mechanism for sharing data across Examples 
>sections
>in docstrings within a class.  For instance:
>
>class Foo:
>
>    def foo(self):
>        """Method foo title
>
>        The example generating data below may be much more laborious.
>
>        Examples
>        --------
>        >>> x = list("abc")  # may be very long and tedious to generate
>
>        """
>        pass
>
>    def bar(self):
>        """Method bar title
>
>        Examples
>        --------
>        >>> # do something else with x from foo Example
>
>        """
>        pass

Personally I'd be inclined to put long identical examples in the class 
docstring instead of the method, but that may not be appropriate.

I've got an @fmtdoc decorator I use mostly for embedding "constants" in 
docstrings, in my cs.deco module: https://pypi.org/project/cs.deco/, 
thus:

    from cs.deco import fmtdoc

It turns your docstring into a formatted string. You could embed a 
repeated example that way:

    EXAMPLE = r'''
        long example here
    '''


    @fmtdoc
    def foo(...):
        ''' Blah blah.

            Example:
            {EXAMPLE}
        '''

Note - @fmtdoc is only suitable for limited things - use with 
discretion.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From spluque at gmail.com  Tue Jan 11 20:23:22 2022
From: spluque at gmail.com (Sebastian Luque)
Date: Tue, 11 Jan 2022 19:23:22 -0600
Subject: sharing data across Examples docstrings
In-Reply-To: <Yd4EgI5rHHDmc/zf@cskk.homeip.net> (Cameron Simpson's message of
 "Wed, 12 Jan 2022 09:28:16 +1100")
References: <87v8ypgc0a.fsf@phoca.sebkatja.org>
 <Yd4EgI5rHHDmc/zf@cskk.homeip.net>
Message-ID: <87mtk1g31x.fsf@phoca.sebkatja.org>

On Wed, 12 Jan 2022 09:28:16 +1100,
Cameron Simpson <cs at cskk.id.au> wrote:

[...]

> Personally I'd be inclined to put long identical examples in the class
> docstring instead of the method, but that may not be appropriate.

Good point, and perhaps it's best to put a comprehensive example in the
class docstring, rather than scatter it across the methods' docstrings.
The situation is one in which the methods are typically (but not always)
intended to be used as part of a pipeline of operations; e.g. Foo.foo()
would almost always be used before Foo.bar().

Thanks,
-- 
Seb

From jcasale at activenetwerx.com  Tue Jan 11 22:10:10 2022
From: jcasale at activenetwerx.com (Joseph L. Casale)
Date: Wed, 12 Jan 2022 03:10:10 +0000
Subject: Script profiling details
In-Reply-To: <25053.47844.354597.86444@ixdm.fritz.box>
References: <62b6034aed664df6a713ef7818ea6e4e@activenetwerx.com>
 <25053.47844.354597.86444@ixdm.fritz.box>
Message-ID: <7cf437a0a7ac40b1972ae57dff9a3b38@activenetwerx.com>

> You might try `py-spy`.

That worked well, I started trying to get more data from the profile
output with the stats module but didn't quite get there. 

Thank you everyone,
jlc

From robin at reportlab.com  Wed Jan 12 05:22:22 2022
From: robin at reportlab.com (Robin Becker)
Date: Wed, 12 Jan 2022 10:22:22 +0000
Subject: preserving entities with lxml
Message-ID: <35c7bce2-947d-018d-78ab-eefbdadeac9c@everest.reportlab.co.uk>

I have a puzzle over how lxml & entities should be 'preserved' code below illustrates. To preserve I change & --> &amp; 
in the source and add resolve_entities=False to the parser definition. The escaping means we only have one kind of 
entity &amp; which means lxml will preserve it. For whatever reason lxml won't preserve character entities eg &#33;.

The simple parse from string and conversion tostring shows that the parsing at least took notice of it.

However, I want to create a tuple tree so have to use tree.text, tree.getchildren() and tree.tail for access.

When I use those I expected to have to undo the escaping to get back the original entities, but it seems they are 
already done.

Good for me, but if the tree knows how it was created (tostring shows that) why is it ignored with attribute access?

if __name__=='__main__':
     from lxml import etree as ET
     #initial xml
     xml = b'<a attr="&mysym; &lt; &amp; &gt; &#33;">aaaaa &mysym; &lt; &amp; &gt; &#33; AAAAA</a>'
     #escaped xml
     xxml = xml.replace(b'&',b'&amp;')

     myparser = ET.XMLParser(resolve_entities=False)
     tree = ET.fromstring(xxml,parser=myparser)

     #use tostring
     print(f'using tostring\n{xxml=!r}\n{ET.tostring(tree)=!r}\n')

     #now access the items using text & children & text
     print(f'using attributes\n{tree.text=!r}\n{tree.getchildren()=!r}\n{tree.tail=!r}')

when run I see this

$ python tmp/tlp.py
using tostring
xxml=b'<a attr="&amp;mysym; &amp;lt; &amp;amp; &amp;gt; &amp;#33;">aaaaa &amp;mysym; &amp;lt; &amp;amp; &amp;gt; 
&amp;#33; AAAAA</a>'
ET.tostring(tree)=b'<a attr="&amp;mysym; &amp;lt; &amp;amp; &amp;gt; &amp;#33;">aaaaa &amp;mysym; &amp;lt; &amp;amp; 
&amp;gt; &amp;#33; AAAAA</a>'

using attributes
tree.text='aaaaa &mysym; &lt; &amp; &gt; &#33; AAAAA'
tree.getchildren()=[]
tree.tail=None
-- 
Robin Becker

From dieter at handshake.de  Wed Jan 12 15:49:23 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Wed, 12 Jan 2022 21:49:23 +0100
Subject: preserving entities with lxml
In-Reply-To: <35c7bce2-947d-018d-78ab-eefbdadeac9c@everest.reportlab.co.uk>
References: <35c7bce2-947d-018d-78ab-eefbdadeac9c@everest.reportlab.co.uk>
Message-ID: <25055.16083.125267.882016@ixdm.fritz.box>

Robin Becker wrote at 2022-1-12 10:22 +0000:
>I have a puzzle over how lxml & entities should be 'preserved' code below illustrates. To preserve I change & --> &amp;
>in the source and add resolve_entities=False to the parser definition. The escaping means we only have one kind of
>entity &amp; which means lxml will preserve it. For whatever reason lxml won't preserve character entities eg &#33;.
>
>The simple parse from string and conversion tostring shows that the parsing at least took notice of it.
>
>However, I want to create a tuple tree so have to use tree.text, tree.getchildren() and tree.tail for access.
>
>When I use those I expected to have to undo the escaping to get back the original entities, but it seems they are
>already done.
>
>Good for me, but if the tree knows how it was created (tostring shows that) why is it ignored with attribute access?
>
>if __name__=='__main__':
>     from lxml import etree as ET
>     #initial xml
>     xml = b'<a attr="&mysym; &lt; &amp; &gt; &#33;">aaaaa &mysym; &lt; &amp; &gt; &#33; AAAAA</a>'
>     #escaped xml
>     xxml = xml.replace(b'&',b'&amp;')
>
>     myparser = ET.XMLParser(resolve_entities=False)
>     tree = ET.fromstring(xxml,parser=myparser)
>
>     #use tostring
>     print(f'using tostring\n{xxml=!r}\n{ET.tostring(tree)=!r}\n')
>
>     #now access the items using text & children & text
>     print(f'using attributes\n{tree.text=!r}\n{tree.getchildren()=!r}\n{tree.tail=!r}')
>
>when run I see this
>
>$ python tmp/tlp.py
>using tostring
>xxml=b'<a attr="&amp;mysym; &amp;lt; &amp;amp; &amp;gt; &amp;#33;">aaaaa &amp;mysym; &amp;lt; &amp;amp; &amp;gt;
>&amp;#33; AAAAA</a>'
>ET.tostring(tree)=b'<a attr="&amp;mysym; &amp;lt; &amp;amp; &amp;gt; &amp;#33;">aaaaa &amp;mysym; &amp;lt; &amp;amp;
>&amp;gt; &amp;#33; AAAAA</a>'
>
>using attributes
>tree.text='aaaaa &mysym; &lt; &amp; &gt; &#33; AAAAA'
>tree.getchildren()=[]
>tree.tail=None

Apparently, the `resolve_entities=False` was not effective: otherwise,
your tree content should have more structure (especially some
entity reference children).

`&#<value>` is not an entity reference but a character reference.
It may rightfully be treated differently from entity references.

From samuelmarks at gmail.com  Tue Jan 11 00:30:39 2022
From: samuelmarks at gmail.com (samue...@gmail.com)
Date: Mon, 10 Jan 2022 21:30:39 -0800 (PST)
Subject: ast.parse, ast.dump, but with comment preservation?
In-Reply-To: <mailman.7.1639652192.1567.python-list@python.org>
References: <07e8e6c6-1dcb-f25e-1a7c-30b10db3d818@bourneuf.net>
 <f36486c8-2380-4eaa-becf-504a0be8c882n@googlegroups.com>
 <mailman.7.1639652192.1567.python-list@python.org>
Message-ID: <9eeae185-fe56-426f-95c6-62f8ac4e9922n@googlegroups.com>

On Thursday, December 16, 2021 at 5:56:51 AM UTC-5, lucas wrote:
> Hi ! 
> 
> Maybe RedBaron may help you ? 
> 
> https://github.com/PyCQA/redbaron 
> 
> IIRC, it aims to conserve the exact same representation of the source 
> code, including comments and empty lines. 
> 
> --lucas
> On 16/12/2021 04:37, samue... at gmail.com wrote: 
> > I wrote a little open-source tool to expose internal constructs in OpenAPI. Along the way, I added related functionality to: 
> > - Generate/update a function prototype to/from a class 
> > - JSON schema 
> > - Automatically add type annotations to all function arguments, class attributes, declarations, and assignments 
> > 
> > alongside a bunch of other features. All implemented using just the builtin modules (plus astor on Python < 3.9; and optionally black). 
> > 
> > Now I'm almost at the point where I can run it?without issue?against, e.g., the entire TensorFlow codebase. Unfortunately this is causing huge `diff`s because the comments aren't preserved (and there are some whitespace issues? but I should be able to resolve the latter). 
> > 
> > Is the only viable solution available to rewrite around redbaron | libcst? - I don't need to parse the comments just dump them out unedited whence they're found? 
> > 
> > Thanks for any suggestions 
> > 
> > PS: Library is https://github.com/SamuelMarks/cdd-python (might relicense with CC0? anyway too early for others to use; wait for the 0.1.0 release ;])

Ended up writing my own CST and added it to that library of mine (link above).

My target is adding/removing/changing of: docstrings, function return types, function arguments, and Assign/AnnAssign. All but the last are now implemented.

I was careful not to replace code elsewhere in my codebase, so everything except my new CST code (in its own files) stays, and everything else works exclusively with the builtin `ast` module as before.

From wlfraed at ix.netcom.com  Tue Jan 11 10:06:12 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Tue, 11 Jan 2022 10:06:12 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <sr7on3$1ji0$1@gioia.aioe.org>
 <7c49e7a1-2a7b-fc5f-b93b-e264dc68b909@mrabarnett.plus.com>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
Message-ID: <os6rtgpoit9ogr2mo2477f797iq0bf5mid@4ax.com>


	*** Apologies for the repost. Since Gmane made the list a read-only
group, I finally broke down and reinstated Giganews comp.lang.python.
Unfortunately I'd missed that this came back with X-NoArchive active and
Google doesn't even let such messages show up for a day -- so the OP hasn't
seen any of my responses.

	As a courtesy, I will NOT be reposting the other four responses I've
made over the last few days. {If I do, it will be as a single consolidated
response} ***

On Mon, 10 Jan 2022 22:31:00 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>-?How are the relevant cells identified in the spreadsheet??
>The column headings are:
>BOOK_NAME
>BOOK_AUTHOR
>BOOK_ISBN
>TOTAL_COPIES
>COPIES_LEFT
>BORROWER?S_NAME
>ISSUE_DATE
>RETURN_DATE
>

	So... Besides "BORROWER'S_NAME" you also have a pair of dates you have
to track in parallel, and which should also need to be updated whenever you
change the borrower field. Furthermore, if you plan to separate those with
commas, you'll need to escape any embedded commas or you'll find that names
like "John Doe, Jr" will mess up the correspondence as you'd treat that as
two names on reading the borrower field. Also you need to be aware of the
limits for Excel text cells -- while you could stuff 32kB of text into a
cell, Excel itself will only display the first 1024 characters. That might
be sufficient if the average name is around 31 characters (32 with your
comma separator) as it would allow 32 names to be entered and still display
in Excel itself. Oh, and to track multiple dates in a cell, you'll have to
convert from date to text when writing the cell, and from text back to date
when reading the cell -- since you can't comma separate multiple dates.

	Total_Copies - Copies_Left should be equal to the number of names (and
dates). In short, this is a very messy structure to be maintaining. If not
using an RDBM, at the very least borrower/issue date/return date should be
moved to a separate sheet which also has "Book ID" (the row number in the
first sheet with the book). That way you'd have one record per borrower,
and can easily add new records at the bottom of the sheet (might need to
use a "Book ID" of "0" to indicate a deleted record (when a borrower
returns the book) so you can reuse the slot, since you'd need some way to
identify the end of the data -- most likely by a blank record..

>-?If that's what you have in your spreadsheet, then read the cells on the first row for the column labels and put them in a dict to map from column label to column number.?
>
>This written above I do not understand how to code. 

	Have you gone through the Python Tutorial? Dictionaries are one of
Python's basic data structures. https://docs.python.org/3/tutorial/

	You are unlikely to find anything near to your application on-line --
pretty much anyone doing something like a library check-out system will be
using a relational database rather than spread sheets. At worst, they may
have a spread sheet import operation to do initial population of the
database, though even that might be using SQL operations (Windows supports
Excel files as an ODBC data source). See:
https://docs.microsoft.com/en-us/cpp/data/odbc/data-source-managing-connections-odbc?view=msvc-170
They are unlikely to be dong any exports to Excel -- that's the realm of
report logic. According to
https://support.sas.com/documentation/onlinedoc/dfdmstudio/2.5/dmpdmsug/Content/dfDMStd_T_Excel_ODBC.html
"""
Note: You cannot use a DSN to write output to Excel format. You can,
however, use a Text File Output node in a data job to write output in CSV
format. You can then import the CSV file into Excel.
"""
	A Java-biased (old Java -- the interface to ODBC has been removed from
current Java) example that doesn't seem to need "named ranges" is
https://querysurge.zendesk.com/hc/en-us/articles/205766136-Writing-SQL-Queries-against-Excel-files-using-ODBC-connection-Deprecated-Excel-SQL-

	Or...
https://www.red-gate.com/simple-talk/databases/sql-server/database-administration-sql-server/getting-data-between-excel-and-sql-server-using-odbc/
(which also indicates that it is possible to update the file via ODBC...
But note the constraints regarding having 64-bit vs 32-bit drivers).
Obviously you'll need to translate the PowerShell syntax into Python's ODBC
DB-API interface (which is a bit archaic as I recall -- does not match
current DP-API specifications).



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From samuelmarks at gmail.com  Tue Jan 11 11:25:02 2022
From: samuelmarks at gmail.com (samue...@gmail.com)
Date: Tue, 11 Jan 2022 08:25:02 -0800 (PST)
Subject: ast.parse, ast.dump, but with comment preservation?
In-Reply-To: <mailman.7.1639652192.1567.python-list@python.org>
References: <07e8e6c6-1dcb-f25e-1a7c-30b10db3d818@bourneuf.net>
 <f36486c8-2380-4eaa-becf-504a0be8c882n@googlegroups.com>
 <mailman.7.1639652192.1567.python-list@python.org>
Message-ID: <e356bcdb-3907-4177-bb95-9f7c085ce72an@googlegroups.com>

> > PS: Library is https://github.com/SamuelMarks/cdd-python (might relicense with CC0? anyway too early for others to use; wait for the 0.1.0 release ;])
Ended up writing my own CST and added it to that library of mine (link above).

My target is adding/removing/changing of: docstrings, function return types, function arguments, and Assign/AnnAssign. All but the last are now implemented.

I was careful not to replace code elsewhere in my codebase, so everything except my new CST code (in its own files) stays, and everything else works exclusively with the builtin `ast` module as before.

From wlfraed at ix.netcom.com  Tue Jan 11 11:39:23 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Tue, 11 Jan 2022 11:39:23 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
Message-ID: <qnbrtgtavtg67ce1mervn7cosrd2db6nar@4ax.com>


	***	Going back to the post in the thread as I've other concerns (and
have turned off the old X-NoArchive setting	***

On Thu, 6 Jan 2022 10:55:30 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>All this is going to be in python?s flask and HTML only
>
>1. First, I have to check in the Excel sheet or table whether the book user has entered is present in the book bank or not.
>
>2. If a book is present and the quantity of the required book is greater than 0 (COPIES_LEFT column in excel file) and if the user wants the book, it will be assigned to the user which he will take from the book bank physically. When COPIES_LEFT will is less than or equal to 0 the message will be ?Book finished or not present?.
>
>3. The quantity of the book in the Excel file will be reduced by 1 in the COPIES_LEFT column and the name of the borrower or user will be entered/added in the Excel file table or sheet already made and the column name is BORROWER?S NAME.
>
>4. The borrower?s or user name can be more than one so they will be separated with a comma in the Excel file BORROWER?S NAME column.
>
>
>- All functions mentioned above are to be deployed on the website pythonhow.com so make according to https://pythonhow.com/python-tutorial/flask/web-development-with-python-and-flask/
>
>- Do you know any other websites to deploy a python web application??

	There are likely plenty -- How much do you want to pay? How much
support do you need? How much traffic do you expect.

	Note that the PythonHOW tutorial is suggesting creating a student/hobby
account on Heroku (free, and fairly limited). Heroku provides Linux
containers, and for Python you can only make use of add-ons that can be
installed using PIP. As Linux, none of the Windows specific modules will be
available (no Excel ODBC, no use of pythonwin extensions calling directly
into the Excel DLLs).

	Who is going to be using the Excel file? and how are they going to get
to it? Your Heroku container does not run Excel, and I'm not even sure how
you would get it to the Heroku container (possibly it can be done as part
of the Python application upload). 

	I don't even know if SQLite3 is viable -- as I recall, Linux Python
installs rely upon system installed SQLite3 libraries, not installed via
PIP. Heroku pushes PostgreSQL for data storage. It may cost to add others
(MariaDB/MySQL).


https://elements.heroku.com/buildpacks/heroku/heroku-buildpack-python

>
>- No time to switch from Excel to anywhere else. Please do not make any changes to the Excel file.
>

	Again, if you are deploying to something like Heroku for the
application -- the Excel file will have to be deployed also, and no one
except your application will be able to see it there. Under this situation,
there is no reason/excuse to keep the data in the very inefficient format
you've defined in the most recent message. Import into some supported
database and normalize the data to make updates easier.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From narshad.380 at gmail.com  Tue Jan 11 01:31:00 2022
From: narshad.380 at gmail.com (NArshad)
Date: Mon, 10 Jan 2022 22:31:00 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <mailman.144.1641842486.3079.python-list@python.org>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <7c49e7a1-2a7b-fc5f-b93b-e264dc68b909@mrabarnett.plus.com>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
Message-ID: <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>

-?How are the relevant cells identified in the spreadsheet??
The column headings are:
BOOK_NAME
BOOK_AUTHOR
BOOK_ISBN
TOTAL_COPIES
COPIES_LEFT
BORROWER?S_NAME
ISSUE_DATE
RETURN_DATE


-?It's often the case that the cells on the first row contain text as column labels.? 

These I have written above.


-?If that's what you have in your spreadsheet, then read the cells on the first row for the column labels and put them in a dict to map from column label to column number.?

This written above I do not understand how to code. 

From python at mrabarnett.plus.com  Wed Jan 12 16:39:19 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 12 Jan 2022 21:39:19 +0000
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <7c49e7a1-2a7b-fc5f-b93b-e264dc68b909@mrabarnett.plus.com>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
Message-ID: <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>

On 2022-01-11 06:31, NArshad wrote:
> -?How are the relevant cells identified in the spreadsheet??
> The column headings are:
> BOOK_NAME
> BOOK_AUTHOR
> BOOK_ISBN
> TOTAL_COPIES
> COPIES_LEFT
> BORROWER?S_NAME
> ISSUE_DATE
> RETURN_DATE
> 
> 
> -?It's often the case that the cells on the first row contain text as column labels.?
> 
> These I have written above.
> 
> 
> -?If that's what you have in your spreadsheet, then read the cells on the first row for the column labels and put them in a dict to map from column label to column number.?
> 
> This written above I do not understand how to code.

Well, you know how to read the contents of a cell, and how to put items 
into a dict (the key will be the cell contents and the value will be the 
column number).

The column numbers will go from 1 to sheet.last_column, although some of 
them might be empty (their value will be None), which you can remove 
from the dict afterwards.

From robin at reportlab.com  Thu Jan 13 04:13:43 2022
From: robin at reportlab.com (Robin Becker)
Date: Thu, 13 Jan 2022 09:13:43 +0000
Subject: preserving entities with lxml
In-Reply-To: <25055.16083.125267.882016@ixdm.fritz.box>
References: <35c7bce2-947d-018d-78ab-eefbdadeac9c@everest.reportlab.co.uk>
 <25055.16083.125267.882016@ixdm.fritz.box>
Message-ID: <f92683e8-8909-44a3-4318-786adb0ab565@everest.reportlab.co.uk>

On 12/01/2022 20:49, Dieter Maurer wrote:
.......
>>
>> when run I see this
>>
>> $ python tmp/tlp.py
>> using tostring
>> xxml=b'<a attr="&amp;mysym; &amp;lt; &amp;amp; &amp;gt; &amp;#33;">aaaaa &amp;mysym; &amp;lt; &amp;amp; &amp;gt;
>> &amp;#33; AAAAA</a>'
>> ET.tostring(tree)=b'<a attr="&amp;mysym; &amp;lt; &amp;amp; &amp;gt; &amp;#33;">aaaaa &amp;mysym; &amp;lt; &amp;amp;
>> &amp;gt; &amp;#33; AAAAA</a>'
>>
>> using attributes
>> tree.text='aaaaa &mysym; &lt; &amp; &gt; &#33; AAAAA'
>> tree.getchildren()=[]
>> tree.tail=None
> 
> Apparently, the `resolve_entities=False` was not effective: otherwise,
> your tree content should have more structure (especially some
> entity reference children).
> 
except that the tree knows not to expand the entities using ET.tostring so in some circumstances resolve_entities=False 
does work.

I expected that the tree would contain the parsed (unexpanded) values, but referencing the actual tree.text/tail/attrib 
doesn't give the expected results. There's no criticism here, it makes my life a bit easier. If I had wanted the 
unexpanded values in the attrib/text/tail it would be more of a problem.


> `&#<value>` is not an entity reference but a character reference.
> It may rightfully be treated differently from entity references.
I understand the difference, but lxml (and perhaps libxml2) doesn't provide a way to turn off character reference 
expansion. This makes using lxml for source transformation a bit harder since the original text is not preserved.


From dieter at handshake.de  Thu Jan 13 04:29:49 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Thu, 13 Jan 2022 10:29:49 +0100
Subject: preserving entities with lxml
In-Reply-To: <f92683e8-8909-44a3-4318-786adb0ab565@everest.reportlab.co.uk>
References: <35c7bce2-947d-018d-78ab-eefbdadeac9c@everest.reportlab.co.uk>
 <25055.16083.125267.882016@ixdm.fritz.box>
 <f92683e8-8909-44a3-4318-786adb0ab565@everest.reportlab.co.uk>
Message-ID: <25055.61709.862129.992096@ixdm.fritz.box>

Robin Becker wrote at 2022-1-13 09:13 +0000:
>On 12/01/2022 20:49, Dieter Maurer wrote:
> ...
>> Apparently, the `resolve_entities=False` was not effective: otherwise,
>> your tree content should have more structure (especially some
>> entity reference children).
>>
>except that the tree knows not to expand the entities using ET.tostring so in some circumstances resolve_entities=False
>does work.

I think this is a misunderstanding: `tostring` will represent the text character `&` as `&amp;`.

From robin at reportlab.com  Thu Jan 13 04:58:09 2022
From: robin at reportlab.com (Robin Becker)
Date: Thu, 13 Jan 2022 09:58:09 +0000
Subject: preserving entities with lxml
In-Reply-To: <25055.61709.862129.992096@ixdm.fritz.box>
References: <35c7bce2-947d-018d-78ab-eefbdadeac9c@everest.reportlab.co.uk>
 <25055.16083.125267.882016@ixdm.fritz.box>
 <f92683e8-8909-44a3-4318-786adb0ab565@everest.reportlab.co.uk>
 <25055.61709.862129.992096@ixdm.fritz.box>
Message-ID: <1042fe01-f8f7-c803-5402-e6fd4aacdc5f@everest.reportlab.co.uk>

On 13/01/2022 09:29, Dieter Maurer wrote:
> Robin Becker wrote at 2022-1-13 09:13 +0000:
>> On 12/01/2022 20:49, Dieter Maurer wrote:
>> ...
>>> Apparently, the `resolve_entities=False` was not effective: otherwise,
>>> your tree content should have more structure (especially some
>>> entity reference children).
>>>
>> except that the tree knows not to expand the entities using ET.tostring so in some circumstances resolve_entities=False
>> does work.
> 
> I think this is a misunderstanding: `tostring` will represent the text character `&` as `&amp;`.

aaahhhh,

thanks I see now. So tostring is actually restoring some of the entities which on input are normally expanded. If that 
means resolve_entities=False does not work at all then I guess there's no need to use it at all. The initial transform

& --> &amp;

does what I need as it is reversed on output of the tree fragments.

Wonder what resolve_entities is actually used for then? All the docs seem to say

> resolve_entities - replace entities by their text value (default: True)

I assumed False would mean that they would pass through the parse
--
Robin Becker


From as at sci.fi  Thu Jan 13 05:08:33 2022
From: as at sci.fi (Anssi Saari)
Date: Thu, 13 Jan 2022 12:08:33 +0200
Subject: Scope confusion in Python REPL
Message-ID: <sm0y23kkkwu.fsf@lakka.kapsi.fi>


I ran into what seems odd scoping to me when playing with some matching
examples for 3.10.

I kinda thought that if I do from foo import * and from bar import * in
the Python REPL, I'd get everything from foo and bar in the main
scope. Or whatever the scope is at the prompt.

And yet, if I define function foo in module foo and function bar in
module bar and import as above, I can't call function bar from function
foo. But if I define functions foo and bar at the prompt, then I can. So
what's the difference in scoping here?

From rosuav at gmail.com  Thu Jan 13 07:27:26 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 13 Jan 2022 23:27:26 +1100
Subject: Scope confusion in Python REPL
In-Reply-To: <sm0y23kkkwu.fsf@lakka.kapsi.fi>
References: <sm0y23kkkwu.fsf@lakka.kapsi.fi>
Message-ID: <CAPTjJmoFQuexLN1dyXwBqrb5RSUKZu3jFZtRPJ3sKvRKib1uPQ@mail.gmail.com>

On Thu, Jan 13, 2022 at 9:43 PM Anssi Saari <as at sci.fi> wrote:
>
>
> I ran into what seems odd scoping to me when playing with some matching
> examples for 3.10.
>
> I kinda thought that if I do from foo import * and from bar import * in
> the Python REPL, I'd get everything from foo and bar in the main
> scope. Or whatever the scope is at the prompt.

That is correct.

> And yet, if I define function foo in module foo and function bar in
> module bar and import as above, I can't call function bar from function
> foo. But if I define functions foo and bar at the prompt, then I can. So
> what's the difference in scoping here?

That's because, *inside function bar*, you don't have foo in scope.
Both functions are available at the REPL's module scope, but they both
exist in their own contexts.

When you define a function, it remembers its context. Inside that
function, you can refer to anything else in the function, or in any
surrounding function, or in the module. That's how closures work:

def make_adder(a):
    def adder(b):
        return a + b
    return adder

Whenever you call make_adder, it will give you back a brand new
function, which will remember the value of 'a'.

The same is true of module-level names. If I have a function like this:

a = 42
def adder(b):
    return a + b

and I import that function into some other module, it'll use 'a' from
its own module globals, not the module you're calling it from.

When you import something, all you're doing is getting a local
reference to it; "from foo import make_adder" is basically like saying
"import foo; make_adder = foo.make_adder". The function itself is
still the same, and it still remembers its original context.

Defining two functions at the prompt is like putting them in the same
file; they're able to refer to each other directly. What you may want
to do is have one module import the other one, which would then allow
them to refer to each other.

Hope that helps!

ChrisA

From as at sci.fi  Thu Jan 13 09:35:29 2022
From: as at sci.fi (Anssi Saari)
Date: Thu, 13 Jan 2022 16:35:29 +0200
Subject: Scope confusion in Python REPL
In-Reply-To: <CAPTjJmoFQuexLN1dyXwBqrb5RSUKZu3jFZtRPJ3sKvRKib1uPQ@mail.gmail.com>
 (Chris Angelico's message of "Thu, 13 Jan 2022 23:27:26 +1100")
References: <sm0y23kkkwu.fsf@lakka.kapsi.fi>
 <CAPTjJmoFQuexLN1dyXwBqrb5RSUKZu3jFZtRPJ3sKvRKib1uPQ@mail.gmail.com>
Message-ID: <sm0tue71z66.fsf@lakka.kapsi.fi>

Chris Angelico <rosuav at gmail.com> writes:

> When you import something, all you're doing is getting a local
> reference to it; "from foo import make_adder" is basically like saying
> "import foo; make_adder = foo.make_adder". The function itself is
> still the same, and it still remembers its original context.

Thanks, this clears it up.

The pattern matching examples that I was looking at when I got confused
are at
https://mathspp.com/blog/pydonts/pattern-matching-tutorial-for-pythonic-code
in case anyone is interested. Fairly neat stuff in my opinion.

From narshad.380 at gmail.com  Thu Jan 13 13:44:01 2022
From: narshad.380 at gmail.com (NArshad)
Date: Thu, 13 Jan 2022 10:44:01 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <mailman.165.1642023749.3079.python-list@python.org>
References: <199c23c7-de58-44ae-a216-760c8f36c506n@googlegroups.com>
 <sr7on3$1ji0$1@gioia.aioe.org>
 <7c49e7a1-2a7b-fc5f-b93b-e264dc68b909@mrabarnett.plus.com>
 <4513f4cf-5a7d-4ba8-a4cb-72a75f89b2d9n@googlegroups.com>
 <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
Message-ID: <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>


- ?if you are deploying to something like Heroku for the application -- the Excel file will have to be deployed also, and no one except your application will be able to see it there. Under this situation, there is no reason/excuse to keep the data in the very inefficient format you've defined in the most recent message. Import into some supported database and normalize the data to make updates easier.?

Ok with deploying the Excel file also. Next time I will switch to another database.



-?Well, you know how to read the contents of a cell, and how to put items into a dict (the key will be the cell contents and the value will be the column number).?

No, I don?t know. I tried but still the same.



-?The column numbers will go from 1 to sheet.last_column, although some of them might be empty (their value will be None), which you can remove from the dict afterward.?

No need to do with the whole spreadsheet only the first column is enough

From wlfraed at ix.netcom.com  Thu Jan 13 15:22:50 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 13 Jan 2022 15:22:50 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
Message-ID: <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>

On Thu, 13 Jan 2022 10:44:01 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

	Please arrange to use some client that does proper quote attribution.
It gets difficult to read these when you have snippets from multiple posts
with no attribution of who wrote the snippet, and when it was posted, etc.

>
>- ?if you are deploying to something like Heroku for the application -- the Excel file will have to be deployed also, and no one except your application will be able to see it there. Under this situation, there is no reason/excuse to keep the data in the very inefficient format you've defined in the most recent message. Import
into some supported database and normalize the data to make updates easier.?
>
>Ok with deploying the Excel file also. Next time I will switch to another database.
>

	There is still that lingering question of who can see the data. Once
deployed, the only way to access that Excel file will be either by the
owner of the Heroku instance (you -- using the deployment process; FTP? I
presume if you can upload, you'll be able to download) or via your
application. 

	As you've described this system, the only thing your application will
do is record "check-outs" by tracking available copies of books, and the
name (and dates?) of the person doing the check-out. There is no reporting
function -- not even a print-out for the user of which books they've just
reserved, so how does anyone actually verify they are to receive a physical
book after reserving it. Is there any logic to prevent someone from
reserving multiple copies of a book... You'll also need logic to ensure
that multiple borrowers connecting to the application at the same time
can't both reserve the same copy of any book, or that one borrower doesn't
block others from reserving other books at the same time. Spreadsheets are
not "multiple user" files, and if each session (borrower connecting to the
application) results in the application trying to open the spreadsheet
you'll have conflicts.


>
>
>-?Well, you know how to read the contents of a cell, and how to put items into a dict (the key will be the cell contents and the value will be the column number).?
>
>No, I don?t know. I tried but still the same.
>

	Tried what?

	All throughout this thread you have never shown us a sample of code
you've written. When asking for help about a Python feature, It is common
courtesy to provide a minimal complete example (ie, something that readers
can cut&paste into their environment to test) that reproduces whatever
problem one is having.

	Other posters have provided snippets of how to read from an Excel file.

	If you don't know how to put stuff into a dictionary ("dict") structure
you need to step back from this application and study a Python tutorial as
dicts, lists, tuples are core data structures available in the language
(whereas in my ancient days, we'd have had to learn how to code a
dictionary from scratch -- look up "hashed head multiply linked list"
someday <G>)



>
>
>-?The column numbers will go from 1 to sheet.last_column, although some of them might be empty (their value will be None), which you can remove from the dict afterward.?
>
>No need to do with the whole spreadsheet only the first column is enough

	Really? Per your earlier post

-=-=-=-
On Mon, 10 Jan 2022 22:31:00 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>The column headings are:
>BOOK_NAME
>BOOK_AUTHOR
>BOOK_ISBN
>TOTAL_COPIES
>COPIES_LEFT
>BORROWER?S_NAME
>ISSUE_DATE
>RETURN_DATE
>
-=-=-=-

... the first column is just "book names"... Don't you need to access the
various copy count columns along with borrower names (and dates)?

	The purpose of the dictionary is to map your column headings to the
actual spreadsheet access (either "A".."<whatever>" letters, or numeric
1..<whatever> indices). That way, later in the code, instead of having to
keep track of cryptic column IDs ("I need the 'copies_left' column -- is
that column 5 or 6?") you'd use, say, column_dict["COPIES_LEFT"] where the
dictionary has something like
			{	..., "COPIES_LEFT" : 5, 	...	}



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From wlfraed at ix.netcom.com  Thu Jan 13 15:48:35 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 13 Jan 2022 15:48:35 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
Message-ID: <1o21ugp5ji1k2ng3qia48bmmk03b7u1tce@4ax.com>

On Thu, 13 Jan 2022 15:22:50 -0500, Dennis Lee Bieber
<wlfraed at ix.netcom.com> declaimed the following:

	Talking to myself in public again... Bad habit...

>	As you've described this system, the only thing your application will
>do is record "check-outs" by tracking available copies of books, and the
>name (and dates?) of the person doing the check-out. There is no reporting
>function -- not even a print-out for the user of which books they've just
>reserved, so how does anyone actually verify they are to receive a physical
>book after reserving it. Is there any logic to prevent someone from
>reserving multiple copies of a book... You'll also need logic to ensure
>that multiple borrowers connecting to the application at the same time
>can't both reserve the same copy of any book, or that one borrower doesn't
>block others from reserving other books at the same time. Spreadsheets are
>not "multiple user" files, and if each session (borrower connecting to the
>application) results in the application trying to open the spreadsheet
>you'll have conflicts.
>

	Perhaps
https://data-flair.training/blogs/library-management-system-python-project/
can provide some clues -- though it is tkinter based, not a web
application... (Though they don't seem to be tracking issue/due dates <G>).

	Or, for a web application (using Django, not Flask) see
https://techvidvan.com/tutorials/python-library-management-system/

	If you need more examples to purloin, rather than developing your own
implementation, just hit Google...
https://www.google.com/search?q=python+library+reservation+system which is
how I found the above two examples.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From rosuav at gmail.com  Thu Jan 13 16:24:43 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 14 Jan 2022 08:24:43 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <1o21ugp5ji1k2ng3qia48bmmk03b7u1tce@4ax.com>
References: <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <1o21ugp5ji1k2ng3qia48bmmk03b7u1tce@4ax.com>
Message-ID: <CAPTjJmqAryB2grD3UuuyheSZwh39D8-p8fDAhK4baf6J04P+PQ@mail.gmail.com>

On Fri, Jan 14, 2022 at 7:56 AM Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>
> On Thu, 13 Jan 2022 15:22:50 -0500, Dennis Lee Bieber
> <wlfraed at ix.netcom.com> declaimed the following:
>
>         Talking to myself in public again... Bad habit...

Not as bad as singing choruses in public, which - or so I'm told, by a
mad girl in opera - is quite, quite mad.

ChrisA

From wlfraed at ix.netcom.com  Thu Jan 13 16:37:09 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Thu, 13 Jan 2022 16:37:09 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <1o21ugp5ji1k2ng3qia48bmmk03b7u1tce@4ax.com>
 <CAPTjJmqAryB2grD3UuuyheSZwh39D8-p8fDAhK4baf6J04P+PQ@mail.gmail.com>
 <mailman.172.1642109096.3079.python-list@python.org>
Message-ID: <po61ugl19lcdjfepsgv4mgor3c8n8bic6c@4ax.com>

On Fri, 14 Jan 2022 08:24:43 +1100, Chris Angelico <rosuav at gmail.com>
declaimed the following:

>On Fri, Jan 14, 2022 at 7:56 AM Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>>
>> On Thu, 13 Jan 2022 15:22:50 -0500, Dennis Lee Bieber
>> <wlfraed at ix.netcom.com> declaimed the following:
>>
>>         Talking to myself in public again... Bad habit...
>
>Not as bad as singing choruses in public, which - or so I'm told, by a
>mad girl in opera - is quite, quite mad.
>

	I usually put "eighth notes" (emulated as o/~) around the "talking to
myself in public"...
https://www.google.com/search?q=talking+to+myself+in+public+lyrics


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From avigross at verizon.net  Thu Jan 13 18:08:19 2022
From: avigross at verizon.net (Avi Gross)
Date: Thu, 13 Jan 2022 23:08:19 +0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
References: <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
Message-ID: <18305827.21103.1642115299530@mail.yahoo.com>


I am not replying to anything below so I have removed it.
So I need to remind people of the topic and how it has wandered.
Someone has data in a not particularly great format in an EXCEL spreadsheet. They want to somehow use an external language like Python to manipulate the contents from outside and keep the results inside the same or perhaps another EXCEL file.
Now EXCEL has had programming available for a long time within the spreadsheet and perhaps some of that was once used within the actual spreadsheet and stored as macros or whatever using something like VBA. Or perhaps someone used to update by hand.
So the suggestions to copy the data ONCE into something else (a CSV, database, whatever) that can be used externally are reasonable but seem to be resisted. I would guess some of the "missing" functionality others wonder about such as making reports, may still be available within other sheets or parts of the EXCEL .xlsx or other format file.
An interesting third option that has not been mentioned would be to port the data into another application like Google Sheets which has a fairly similar spreadsheet format with very different internals but many ways for a program to read and adjust the contents such as Google Apps Script that can also do things like send out emails and so on.
Instead, someone suggested Python which indeed, with lots of work, can open just about ANY nonsensical file and diddle around and rewrite it, but WHY? Unless you find the right modules and use them carefully, it seems the hard way to go. Sure, you can in theory do it but the amount of learning and work involved seems excessive? for what you get. And when you are done with what was requested here, you probably need to add other features that will again be painful. If you were hiring someone with experience, you might be paying more and more and end up with a package hard for anyone else to understand, let alone maintain and expand.
An intermediate solution seems to be to copy the darn data from EXCEL and modifies it to be easy to use and make changes? but make available functionality that saves the data when needed back into an EXCEL file, but most of the time manipulates it the easy way.?
As such, if the request continues to be to do it the hard way, i may be among those who ignores this thread any further.

From PythonList at DancesWithMice.info  Thu Jan 13 18:23:09 2022
From: PythonList at DancesWithMice.info (dn)
Date: Fri, 14 Jan 2022 12:23:09 +1300
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <1o21ugp5ji1k2ng3qia48bmmk03b7u1tce@4ax.com>
References: <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <1o21ugp5ji1k2ng3qia48bmmk03b7u1tce@4ax.com>
Message-ID: <8d5cab5b-8380-f09f-4b52-e1bb8d44c523@DancesWithMice.info>

On 14/01/2022 09.48, Dennis Lee Bieber wrote:
> On Thu, 13 Jan 2022 15:22:50 -0500, Dennis Lee Bieber
> <wlfraed at ix.netcom.com> declaimed the following:
> 
> 	Talking to myself in public again... Bad habit...

Recommend that you not start any arguments then
- they will be unwinnable!

-- 
Regards,
=dn

From greg.ewing at canterbury.ac.nz  Thu Jan 13 23:19:41 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 14 Jan 2022 17:19:41 +1300
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <mailman.174.1642116207.3079.python-list@python.org>
References: <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <1o21ugp5ji1k2ng3qia48bmmk03b7u1tce@4ax.com>
 <8d5cab5b-8380-f09f-4b52-e1bb8d44c523@DancesWithMice.info>
 <mailman.174.1642116207.3079.python-list@python.org>
Message-ID: <07819408-b28c-4311-4f7d-f28ff4461057@canterbury.ac.nz>

On 14/01/22 12:23 pm, dn wrote:
> On 14/01/2022 09.48, Dennis Lee Bieber wrote:
>> On Thu, 13 Jan 2022 15:22:50 -0500, Dennis Lee Bieber
>> <wlfraed at ix.netcom.com> declaimed the following:
>>
>> 	Talking to myself in public again... Bad habit...
> 
> Recommend that you not start any arguments then
> - they will be unwinnable!

But at least they'll be free of charge!

-- 
Greg


From wlfraed at ix.netcom.com  Fri Jan 14 11:34:29 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Fri, 14 Jan 2022 11:34:29 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
Message-ID: <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>

On Thu, 13 Jan 2022 23:08:19 +0000 (UTC), Avi Gross <avigross at verizon.net>
declaimed the following:

>So the suggestions to copy the data ONCE into something else (a CSV, database, whatever) that can be used externally are reasonable but seem to be resisted. I would guess some of the "missing" functionality others wonder about such as making reports, may still be available within other sheets or parts of the EXCEL .xlsx or other format file.

	Ignoring the data store itself, the OP has also stated (if poorly) that
this application is supposed to be hosted on Heroku (I suspect this is an
assumption they made, as that minimal "Hello World" Flask tutorial they are
using gives instructions for a limited free/hobbyist Heroku instance). If
this application is really for some client (the book repository staff) one
would think they'd have specified the web hosting system in advance.

	That (web hosting) compounds the problem of the lack of reporting...
The defined application is essentially "write-only" -- in that the only
operation available is to reserve/check-out book(s) based upon availability
count. There is no means provided by which these reservations are passed on
to whoever is responsible for physically handing out books, nor to track
who has received their copy. Are any/all "repository" employees supposed to
access the spreadsheet? How? Do they all have the account/password to the
Heroku (or other web hosting service) in order to FTP the data down for
local usage (how many such copies will there be, and how are they
synchronized).

	Or is the CONOPS that only the repository employees access the
application, on behalf of the person making the reservation/check-out
request -- and hence the application is merely recording information of who
has physically received a book (in which case the overhead of tracking
availability seems superfluous, as the staff would see that the table(s) of
books does not have the title, so it is unavailable for handing over).



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From jgossage at gmail.com  Fri Jan 14 12:40:52 2022
From: jgossage at gmail.com (Jonathan Gossage)
Date: Fri, 14 Jan 2022 12:40:52 -0500
Subject: Using PIP in Python 3.10 on Windows 10
Message-ID: <CAApdmf3KYWFGuqfRjH=WxQBAtvdgKSV6zCxw=RtRaZrnDdTU6A@mail.gmail.com>

I have installed Python 3.10.1 on Windows 10 using the recommended Windows
Installer. When I try to access PIP from the command line, I get the
following result, even though Python itself is accessible.


C:\Users\jgoss>python
Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()

C:\Users\jgoss>pip install pip --upgrade
Fatal error in launcher: Unable to create process using '"C:\Program
Files\Python310\python.exe"  "C:\Program Files\Python310\Scripts\pip.exe"
install pip --upgrade': The system cannot find the file specified.
During the installation, I chose to install Python in a non-default
location and to set the Environment variables.
 The result of this attempt is shown below:


C:\Users\jgoss>python
Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()

C:\Users\jgoss>pip install pip --upgrade
Fatal error in launcher: Unable to create process using '"C:\Program
Files\Python310\python.exe"  "C:\Program Files\Python310\Scripts\pip.exe"
install pip --upgrade': The system cannot find the file specified.

It looks as if the launcher is expecting to find Python installed at
c:\Program Files\Python3.10 whereas it has actually been installed at
D:\Users\jgoss\AppData\local\python\python3.10. It seems that the launcher
has not been updated to the latest installation location for python and
that it also needs to handle a non-default install location. The same
problem occurs if I take the install option to install to the default
location.
Is there any workaround as PIP is essential to my environment?

-- 
Jonathan Gossage

From mats at wichmann.us  Fri Jan 14 14:08:25 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Fri, 14 Jan 2022 12:08:25 -0700
Subject: Using PIP in Python 3.10 on Windows 10
In-Reply-To: <CAApdmf3KYWFGuqfRjH=WxQBAtvdgKSV6zCxw=RtRaZrnDdTU6A@mail.gmail.com>
References: <CAApdmf3KYWFGuqfRjH=WxQBAtvdgKSV6zCxw=RtRaZrnDdTU6A@mail.gmail.com>
Message-ID: <8b5cd678-22be-4230-fec2-3a4806155bed@wichmann.us>


On 1/14/22 10:40, Jonathan Gossage wrote:
> I have installed Python 3.10.1 on Windows 10 using the recommended Windows
> Installer. When I try to access PIP from the command line, I get the
> following result, even though Python itself is accessible.

By this do you mean the python.org installer or the Microsoft Store
installer - they're similar but have some differences.

> 
> 
> C:\Users\jgoss>python
> Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
> bit (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> quit()
> 
> C:\Users\jgoss>pip install pip --upgrade
> Fatal error in launcher: Unable to create process using '"C:\Program
> Files\Python310\python.exe"  "C:\Program Files\Python310\Scripts\pip.exe"
> install pip --upgrade': The system cannot find the file specified.
> During the installation, I chose to install Python in a non-default
> location and to set the Environment variables.
>  The result of this attempt is shown below:
> 
> 
> C:\Users\jgoss>python
> Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
> bit (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> quit()
> 
> C:\Users\jgoss>pip install pip --upgrade
> Fatal error in launcher: Unable to create process using '"C:\Program
> Files\Python310\python.exe"  "C:\Program Files\Python310\Scripts\pip.exe"
> install pip --upgrade': The system cannot find the file specified.
> 
> It looks as if the launcher is expecting to find Python installed at
> c:\Program Files\Python3.10 whereas it has actually been installed at
> D:\Users\jgoss\AppData\local\python\python3.10.

earlier you said "non-standard location" but that's not non-standard at
all, that's actually the standard place for the python.org installer to
put it if you requested a single user install (as opposed to a
system-wide one).

>  It seems that the launcher
> has not been updated to the latest installation location for python and
> that it also needs to handle a non-default install location. The same
> problem occurs if I take the install option to install to the default
> location.
> Is there any workaround as PIP is essential to my environment?
> 

If you can find Python, whether it's called "python" or "py" in your
case, then you should be able to reach pip with "python -m pip" or "py
-m pip".  If you're using the python.org version, you probably want to
use "py" to launch Python.

It looks like you're stumbling over the little stub Microsoft leaves
around which isn't Python, but rather, if you call it, issues a prompt
to install from the Microsoft Store.  That might be why the sequnce
being issued doesn't work. That part of the issue might be solvable by
fiddling the order of your PATH environment vars.

From python at mrabarnett.plus.com  Fri Jan 14 14:12:24 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Fri, 14 Jan 2022 19:12:24 +0000
Subject: Using PIP in Python 3.10 on Windows 10
In-Reply-To: <CAApdmf3KYWFGuqfRjH=WxQBAtvdgKSV6zCxw=RtRaZrnDdTU6A@mail.gmail.com>
References: <CAApdmf3KYWFGuqfRjH=WxQBAtvdgKSV6zCxw=RtRaZrnDdTU6A@mail.gmail.com>
Message-ID: <b510ce5c-c757-10b4-a4b3-5d547cd64714@mrabarnett.plus.com>

On 2022-01-14 17:40, Jonathan Gossage wrote:
> I have installed Python 3.10.1 on Windows 10 using the recommended Windows
> Installer. When I try to access PIP from the command line, I get the
> following result, even though Python itself is accessible.
> 
> 
> C:\Users\jgoss>python
> Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
> bit (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> quit()
> 
> C:\Users\jgoss>pip install pip --upgrade
> Fatal error in launcher: Unable to create process using '"C:\Program
> Files\Python310\python.exe"  "C:\Program Files\Python310\Scripts\pip.exe"
> install pip --upgrade': The system cannot find the file specified.
> During the installation, I chose to install Python in a non-default
> location and to set the Environment variables.
>   The result of this attempt is shown below:
> 
> 
> C:\Users\jgoss>python
> Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
> bit (AMD64)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> quit()
> 
> C:\Users\jgoss>pip install pip --upgrade
> Fatal error in launcher: Unable to create process using '"C:\Program
> Files\Python310\python.exe"  "C:\Program Files\Python310\Scripts\pip.exe"
> install pip --upgrade': The system cannot find the file specified.
> 
> It looks as if the launcher is expecting to find Python installed at
> c:\Program Files\Python3.10 whereas it has actually been installed at
> D:\Users\jgoss\AppData\local\python\python3.10. It seems that the launcher
> has not been updated to the latest installation location for python and
> that it also needs to handle a non-default install location. The same
> problem occurs if I take the install option to install to the default
> location.
> Is there any workaround as PIP is essential to my environment?
> 
These days it's recommended that you use the Python Launcher and the pip 
module:

py -m pip install pip --upgrade

From eryksun at gmail.com  Fri Jan 14 15:44:42 2022
From: eryksun at gmail.com (Eryk Sun)
Date: Fri, 14 Jan 2022 14:44:42 -0600
Subject: Using PIP in Python 3.10 on Windows 10
In-Reply-To: <8b5cd678-22be-4230-fec2-3a4806155bed@wichmann.us>
References: <CAApdmf3KYWFGuqfRjH=WxQBAtvdgKSV6zCxw=RtRaZrnDdTU6A@mail.gmail.com>
 <8b5cd678-22be-4230-fec2-3a4806155bed@wichmann.us>
Message-ID: <CACL+1atSMg4fZ8TAh2bJ_2=f6-WfGwfRanZxtMz+6J+3aY5qWw@mail.gmail.com>

On 1/14/22, Mats Wichmann <mats at wichmann.us> wrote:
> On 1/14/22 10:40, Jonathan Gossage wrote:
>
> By this do you mean the python.org installer or the Microsoft Store
> installer - they're similar but have some differences.

Jonathan is not using the store app. The store app is installed to the
system in "%ProgramFiles%\WindowsApps" and "%ProgramData%\Packages",
and in each user's "%LocalAppData%\Packages". The user's enabled app
aliases are created in "%LocalAppData%\Microsoft\WindowsApps".

>> It looks as if the launcher is expecting to find Python installed at
>> c:\Program Files\Python3.10 whereas it has actually been installed at
>> D:\Users\jgoss\AppData\local\python\python3.10.
>
> earlier you said "non-standard location" but that's not non-standard at
> all, that's actually the standard place for the python.org installer to
> put it if you requested a single user install (as opposed to a
> system-wide one).

Jonathan did not install to a standard location. The standard location
for a per-user installation is
"%LocalAppData%\Programs\Python\PythonXY[-32]". If "%LocalAppdata%" is
"D:\Users\jgoss\AppData\local", then the standard location for 64-bit
3.10 would be "D:\Users\jgoss\AppData\local\Programs\Python\Python310".

> If you can find Python, whether it's called "python" or "py" in your
> case, then you should be able to reach pip with "python -m pip" or "py
> -m pip".  If you're using the python.org version, you probably want to
> use "py" to launch Python.

Upgrading pip in Windows always requires running the module with `-m
pip`. By default the py launcher is installed, in which case pip 3.10
can be upgraded with the following command:

    py -3.10 -m pip install --upgrade pip

If Python is installed for all users, the above command may require
elevating to get administrator access. Afterwards you can use the
"pip.exe" launcher, but first check `where.exe pip` to make sure the
first one found is the right one. If not, modify your per-user and/or
system PATH to fix it.

From lukasz at langa.pl  Fri Jan 14 18:15:49 2022
From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=)
Date: Sat, 15 Jan 2022 00:15:49 +0100
Subject: [RELEASE] Python 3.10.2, 3.9.10, and 3.11.0a4 are now available
Message-ID: <81010491-85C5-4D3F-941A-B94742BA14A9@langa.pl>

Hi there,
before we begin the usual round of release notes, please do note that the three new versions of Python released today do not contain Windows installers yet. This is temporary, due to a more complex than expected code signing certificate renewal.
We?ve held the releases all week while the situation is getting resolved but the urgency of 3.10.2 in particular made us release without the Windows installers after all. We apologize for the inconvenience and are doing everything we can to put the Windows installer in place as soon as possible.

We?re rooting for both Ee Durbin and Steve Dower who are helping us resolve this. Thanks for your hard work! Hopefully, by this time next week, this will only be a footnote in release management history.

The releases you?re looking at were all cursed in some way. What a way to start 2022! Besides the certificate hold up, Python 3.10.2 is an expedited release (you?ll want to upgrade, read below!), Python 3.11.0a4 had almost 20 (sic, twenty!) release blockers before being finally green, and Python 3.9.10 was made from a new M1 Mac on macOS Monterey which made the usually boring process quite a ride. We?re hoping 2022 won?t be this intense all year!


 <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-available/13146#python-3102-1>Python 3.10.2

Get it here: https://www.python.org/downloads/release/python-3102/ <https://www.python.org/downloads/release/python-3102/>  <https://www.blogger.com/#>
This is a special bugfix release ahead of schedule to address a memory leak that was happening on certain function calls when using Cython <https://github.com/cython/cython>. The memory leak consisted of a small constant amount of bytes in certain function calls from Cython code. Although in most cases this was not very noticeable, it was very impactful for long-running applications and certain usage patterns. Check bpo-46347 <https://bugs.python.org/issue46347> for more information.

Upgrading existing Python 3.10 installations is highly recommended. Even though this is an expedited release, it still contains over 100 other bug fixes. See the change log <https://docs.python.org/release/3.10.2/whatsnew/changelog.html> for details.

The next Python 3.10 maintenance release will be 3.10.3, currently scheduled for 2022-04-04.


 <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-available/13146#python-3910-2>Python 3.9.10

Get it here: https://www.python.org/downloads/release/python-3910/  <https://www.blogger.com/#>
Python 3.9.10 is the ninth maintenance release of the legacy 3.9 series. Note: Python 3.10 is now the latest feature release series of Python 3.

Python 3.9 micro-releases enter double digits! There?s been 130 commits since 3.9.9 which is a higher number of fixes for this stage of the life cycle compared to 3.8. See the changelog <https://docs.python.org/release/3.9.10/whatsnew/changelog.html> for details on what changed.

As a reminder, on macOS, the default installer is now the new universal2 variant. It?s compatible with Mac OS X 10.9 and newer, including macOS 11 Big Sur and macOS 12 Monterey. Python installed with this variant will work natively on Apple Silicon processors.

The next Python 3.9 maintenance release will be 3.9.11, currently scheduled for Pi Day '22 (2022-03-14).


 <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-available/13146#python-3110a4-3>Python 3.11.0a4

Get it here: https://www.python.org/downloads/release/python-3110a4/  <https://www.blogger.com/#>
Python 3.11 is still in development. This release, 3.11.0a4, is the fourth of seven planned alpha releases.

Alpha releases are intended to make it easier to test the current state of new features and bug fixes by the community, as well as to test the release process.

During the alpha phase, features may be added up until the start of the beta phase (2022-05-06) and, if necessary, may be modified or deleted up until the release candidate phase (2022-08-01). Please keep in mind that this is a preview release and its use is not recommended for production environments.

Many new features for Python 3.11 are still being planned and written. Among the new major new features and changes so far:

PEP 657 <https://www.python.org/dev/peps/pep-0657/> ? Include Fine-Grained Error Locations in Tracebacks
PEP 654 <https://www.python.org/dev/peps/pep-0654/> ? Exception Groups and except*
The Faster CPython Project <https://github.com/faster-cpython> is already yielding some exciting results: this version of CPython 3.11 is ~ 19% faster on the geometric mean of the PyPerformance benchmarks <>, compared to 3.10.0.
(Hey, fellow core developer, if a feature you find important is missing from this list, let Pablo know <mailto:pablogsal at python.org>.)
The next pre-release of Python 3.11 will be 3.11.0a5, currently scheduled for Wednesday, 2022-02-02.


 <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-available/13146#python-36-is-pining-for-the-fjords-4>Python 3.6 is pining for the fjords

Python 3.6 is no more. It?s an ex-Python. It has ceased to be. On December 23rd 2021 is has reached its end-of-life phase <https://www.python.org/dev/peps/pep-0494/> after five successful years.

It?s been the first truly popular Python 3 release, introducing f-strings to the world and making big improvements to both asyncio (async generators!) and typing (variable annotations!).

We?d like to congratulate Ned Deily @nad <https://discuss.python.org/u/nad> on successfully driving the 3.6 series to completion as Release Manager. He?s not fully retired yet, as 3.7, which he is also managing, is still receiving security patches until June 2023.


 <https://discuss.python.org/t/python-3-10-2-3-9-10-and-3-11-0a4-are-now-available/13146#we-hope-you-enjoy-the-new-releases-5>We hope you enjoy the new releases

Your friendly release team,
Pablo Galindo Salgado @pablogsal <https://discuss.python.org/u/pablogsal>
?ukasz Langa @ambv <https://discuss.python.org/u/ambv>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://mail.python.org/pipermail/python-list/attachments/20220115/af0b1500/attachment.sig>

From bob.griffin76 at gmail.com  Fri Jan 14 20:12:03 2022
From: bob.griffin76 at gmail.com (Bob Griffin)
Date: Fri, 14 Jan 2022 20:12:03 -0500
Subject: keep getting a syntax error on the very first program I am running
Message-ID: <C9C2B9D2-841C-427A-BD61-7D06CCD9B4C5@hxcore.ol>

   I am running this program and keep getting this error.  Is this normal?



   Invalid syntax.  Perhaps you forgot a comma?



   Also the t in tags is highlighted.



   I even tried different versions of Python also.



   Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
   bit (AMD64)] on win32

   Type "help", "copyright", "credits" or "license()" for more information.

   print("Hello World")

   Hello World

   input("\n\nPress the enter key to exit.")







   Sent from [1]Mail for Windows



References

   Visible links
   1. https://go.microsoft.com/fwlink/?LinkId=550986

From python at mrabarnett.plus.com  Fri Jan 14 20:47:19 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Sat, 15 Jan 2022 01:47:19 +0000
Subject: keep getting a syntax error on the very first program I am running
In-Reply-To: <C9C2B9D2-841C-427A-BD61-7D06CCD9B4C5@hxcore.ol>
References: <C9C2B9D2-841C-427A-BD61-7D06CCD9B4C5@hxcore.ol>
Message-ID: <17eb4ac1-796b-bb0e-0134-c9c11656d6ed@mrabarnett.plus.com>

On 2022-01-15 01:12, Bob Griffin wrote:
>     I am running this program and keep getting this error.  Is this normal?
> 
> 
> 
>     Invalid syntax.  Perhaps you forgot a comma?
> 
> 
> 
>     Also the t in tags is highlighted.
> 
> 
> 
>     I even tried different versions of Python also.
> 
> 
> 
>     Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
>     bit (AMD64)] on win32
> 
>     Type "help", "copyright", "credits" or "license()" for more information.
> 
>     print("Hello World")
> 
>     Hello World
> 
>     input("\n\nPress the enter key to exit.")
> 
I get an error only if I use the entirety of it as a program. Is that 
what you're doing, perhaps?

It's not actually a program, but the output from an interactive session.

You start IDLE (I'm assuming that what you're using) and it prints:

     Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC 
v.1929 64
     bit (AMD64)] on win32

and then shows a prompt.

At the prompt you type:

     print("Hello World")

and it print the response:

     Hello World

and so on.

From jon+usenet at unequivocal.eu  Fri Jan 14 20:45:58 2022
From: jon+usenet at unequivocal.eu (Jon Ribbens)
Date: Sat, 15 Jan 2022 01:45:58 -0000 (UTC)
Subject: keep getting a syntax error on the very first program I am running
References: <C9C2B9D2-841C-427A-BD61-7D06CCD9B4C5@hxcore.ol>
 <mailman.180.1642209213.3079.python-list@python.org>
Message-ID: <slrnsu49qm.3sc.jon+usenet@raven.unequivocal.eu>

On 2022-01-15, Bob Griffin <bob.griffin76 at gmail.com> wrote:
>    I am running this program and keep getting this error.  Is this normal?
>
>    Invalid syntax.  Perhaps you forgot a comma?
>
>    Also the t in tags is highlighted.
>
>    I even tried different versions of Python also.
>
>    Python 3.10.1 (tags/v3.10.1:2cd268a, Dec  6 2021, 19:10:37) [MSC v.1929 64
>    bit (AMD64)] on win32
>
>    Type "help", "copyright", "credits" or "license()" for more information.
>
>    print("Hello World")
>
>    Hello World
>
>    input("\n\nPress the enter key to exit.")

You're only supposed to enter the lines:

    print("Hello World")

and

    input("\n\nPress the enter key to exit.")

The other lines are showing you the output you should see.

From martinp.dipaola at gmail.com  Fri Jan 14 21:53:38 2022
From: martinp.dipaola at gmail.com (Martin Di Paola)
Date: Sat, 15 Jan 2022 02:53:38 +0000
Subject: sharing data across Examples docstrings
In-Reply-To: <87v8ypgc0a.fsf@phoca.sebkatja.org>
References: <87v8ypgc0a.fsf@phoca.sebkatja.org>
Message-ID: <20220115025338.rei6ynyansqtmaxy@gmail.com>

Hello,

I understand that you want to share data across examples (docstrings) 
because you are running doctest to validate them (and test).

The doctest implementation evaluates each docstring separately without 
sharing the context so the short answer is "no".

This is a limitation of doctest but it is not the only testing engine 
that you can use.

You could use "byexample" ( https://byexamples.github.io ) which it 
shares the context by default.

byexample has more features and fixes other caveats of doctests, but 
don't take me too serious, I have a natural bias because I'm its author.

If you want to go with byexample, you may want to try its "doctest 
compatibility mode" first so you don't have to rewrite any test.
( https://byexamples.github.io/byexample/recipes/python-doctest )

Let me know if it is useful for you.

Thanks,
Martin.

On Tue, Jan 11, 2022 at 04:09:57PM -0600, Sebastian Luque wrote:
>Hello,
>
>I am searching for a mechanism for sharing data across Examples sections
>in docstrings within a class.  For instance:
>
>class Foo:
>
>    def foo(self):
>        """Method foo title
>
>        The example generating data below may be much more laborious.
>
>        Examples
>        --------
>        >>> x = list("abc")  # may be very long and tedious to generate
>
>        """
>        pass
>
>    def bar(self):
>        """Method bar title
>
>        Examples
>        --------
>        >>> # do something else with x from foo Example
>
>        """
>        pass
>
>
>Thanks,
>-- 
>Seb
>-- 
>https://mail.python.org/mailman/listinfo/python-list

From Marco.Sulla.Python at gmail.com  Sat Jan 15 10:49:26 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sat, 15 Jan 2022 16:49:26 +0100
Subject: Pickle segfaults with custom type
In-Reply-To: <CABbU2U_1juvW8fye8K+U31tWgNX9myN+eDqqM0g7GtJG7HP-ng@mail.gmail.com>
References: <CABbU2U_1juvW8fye8K+U31tWgNX9myN+eDqqM0g7GtJG7HP-ng@mail.gmail.com>
Message-ID: <CABbU2U-0a9X5jH8J6ag5SeTtnVjFvqHL3BAseZszjF3xVFTMGA@mail.gmail.com>

Found. I simply forgot:


    if (PyType_Ready(&PyFrozenDictIterKey_Type) < 0) {
        goto fail;
    }

in the frozendict_exec function for the module.

On Fri, 7 Jan 2022 at 20:27, Marco Sulla <Marco.Sulla.Python at gmail.com>
wrote:

> I have a custom implementation of dict using a C extension. All works but
> the pickling of views and iter types. Python segfaults if I try to pickle
> them.
>
> For example, I have:
>
>
> static PyTypeObject PyFrozenDictIterKey_Type = {
>     PyVarObject_HEAD_INIT(NULL, 0)
>     "frozendict.keyiterator",                   /* tp_name */
>     sizeof(dictiterobject),                     /* tp_basicsize */
>     0,                                          /* tp_itemsize */
>     /* methods */
>     (destructor)dictiter_dealloc,               /* tp_dealloc */
>     0,                                          /* tp_vectorcall_offset */
>     0,                                          /* tp_getattr */
>     0,                                          /* tp_setattr */
>     0,                                          /* tp_as_async */
>     0,                                          /* tp_repr */
>     0,                                          /* tp_as_number */
>     0,                                          /* tp_as_sequence */
>     0,                                          /* tp_as_mapping */
>     PyObject_HashNotImplemented,                /* tp_hash */
>     0,                                          /* tp_call */
>     0,                                          /* tp_str */
>     PyObject_GenericGetAttr,                    /* tp_getattro */
>     0,                                          /* tp_setattro */
>     0,                                          /* tp_as_buffer */
>     Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,    /* tp_flags */
>     0,                                          /* tp_doc */
>     (traverseproc)dictiter_traverse,            /* tp_traverse */
>     0,                                          /* tp_clear */
>     0,                                          /* tp_richcompare */
>     0,                                          /* tp_weaklistoffset */
>     PyObject_SelfIter,                          /* tp_iter */
>     (iternextfunc)frozendictiter_iternextkey,   /* tp_iternext */
>     dictiter_methods,                           /* tp_methods */
>     0,
> };
>
> This is the backtrace I get with gdb:
>
> #0  PyObject_Hash (v=0x7f043ce15540 <PyFrozenDictIterKey_Type>) at
> ../cpython_3_10/Objects/object.c:788
> #1  0x000000000048611c in PyDict_GetItemWithError (op=0x7f043e1f4900,
> key=key at entry=0x7f043ce15540 <PyFrozenDictIterKey_Type>)
>     at ../cpython_3_10/Objects/dictobject.c:1520
> #2  0x00007f043ce227f6 in save (self=self at entry=0x7f043d8507d0,
> obj=obj at entry=0x7f043e1fb0b0, pers_save=pers_save at entry=0)
>     at /home/marco/sources/cpython_3_10/Modules/_pickle.c:4381
> #3  0x00007f043ce2534d in dump (self=self at entry=0x7f043d8507d0,
> obj=obj at entry=0x7f043e1fb0b0) at
> /home/marco/sources/cpython_3_10/Modules/_pickle.c:4515
> #4  0x00007f043ce2567f in _pickle_dumps_impl (module=<optimized out>,
> buffer_callback=<optimized out>, fix_imports=<optimized out>,
> protocol=<optimized out>,
>     obj=0x7f043e1fb0b0) at
> /home/marco/sources/cpython_3_10/Modules/_pickle.c:1203
> #5  _pickle_dumps (module=<optimized out>, args=<optimized out>,
> nargs=<optimized out>, kwnames=<optimized out>)
>     at /home/marco/sources/cpython_3_10/Modules/clinic/_pickle.c.h:619
>
> and so on. The problematic part is in the second frame. Indeed the code of
> _pickle.c here is:
>
>
>         reduce_func = PyDict_GetItemWithError(st->dispatch_table,
>                                               (PyObject *)type);
>
> The problem is that type is NULL. It tries to get the attribute tp_hash
> and it segfaults.
>
> I tried to change the header of the type to:
>
> PyVarObject_HEAD_INIT(&PyType_Type, 0)
>
> This way it works but, as known, it does not compile on Windows.
>
> The strange fact is that pickling the main type works, even if the type is
> NULL, as suggested for a custom type. This is the main type:
>
> PyTypeObject PyFrozenDict_Type = {
>     PyVarObject_HEAD_INIT(NULL, 0)
>     "frozendict." FROZENDICT_CLASS_NAME,        /* tp_name */
>     sizeof(PyFrozenDictObject),                 /* tp_basicsize */
>     0,                                          /* tp_itemsize */
>     (destructor)dict_dealloc,                   /* tp_dealloc */
>     0,                                          /* tp_vectorcall_offset */
>     0,                                          /* tp_getattr */
>     0,                                          /* tp_setattr */
>     0,                                          /* tp_as_async */
>     (reprfunc)frozendict_repr,                  /* tp_repr */
>     &frozendict_as_number,                      /* tp_as_number */
>     &dict_as_sequence,                          /* tp_as_sequence */
>     &frozendict_as_mapping,                     /* tp_as_mapping */
>     (hashfunc)frozendict_hash,                  /* tp_hash */
>     0,                                          /* tp_call */
>     0,                                          /* tp_str */
>     PyObject_GenericGetAttr,                    /* tp_getattro */
>     0,                                          /* tp_setattro */
>     0,                                          /* tp_as_buffer */
>     Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC
>         | Py_TPFLAGS_BASETYPE
>         | _Py_TPFLAGS_MATCH_SELF
>         | Py_TPFLAGS_MAPPING,                   /* tp_flags */
>     frozendict_doc,                             /* tp_doc */
>     dict_traverse,                              /* tp_traverse */
>     0,                                          /* tp_clear */
>     dict_richcompare,                           /* tp_richcompare */
>     0,                                          /* tp_weaklistoffset */
>     (getiterfunc)frozendict_iter,               /* tp_iter */
>     0,                                          /* tp_iternext */
>     frozendict_mapp_methods,                    /* tp_methods */
>     0,                                          /* tp_members */
>     0,                                          /* tp_getset */
>     0,                                          /* tp_base */
>     0,                                          /* tp_dict */
>     0,                                          /* tp_descr_get */
>     0,                                          /* tp_descr_set */
>     0,                                          /* tp_dictoffset */
>     0,                                          /* tp_init */
>     PyType_GenericAlloc,                        /* tp_alloc */
>     frozendict_new,                             /* tp_new */
>     PyObject_GC_Del,                            /* tp_free */
>     .tp_vectorcall = frozendict_vectorcall,
> };
>

From narshad.380 at gmail.com  Sat Jan 15 05:38:34 2022
From: narshad.380 at gmail.com (NArshad)
Date: Sat, 15 Jan 2022 02:38:34 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
References: <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
Message-ID: <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>

What Dennis Lee is saying I will see to it later.

Right now what mrabarrnett is saying is of use. 



Why does the code written below is not giving any output?

    xls = ExcelFile('ABC.xlsx')
    df = xls.parse(xls.sheet_names[0], index_col=1)
    x=df.to_dict()
    print (x)

Only the contents of the first column and the column number is required in the dictionary

From mats at wichmann.us  Sat Jan 15 15:05:00 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Sat, 15 Jan 2022 13:05:00 -0700
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <18305827.21103.1642115299530@mail.yahoo.com>
References: <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
Message-ID: <103472bc-36cc-899e-3711-7f177039f3a4@wichmann.us>

On 1/13/22 16:08, Avi Gross via Python-list wrote:
> 
> I am not replying to anything below so I have removed it.

> Instead, someone suggested Python which indeed, with lots of work, can open just about ANY nonsensical file and diddle around and rewrite it, but WHY? 

well, the topic *was* raised on a PYTHON list...



From nt_mahmood at yahoo.com  Sat Jan 15 15:56:22 2022
From: nt_mahmood at yahoo.com (Mahmood Naderan)
Date: Sat, 15 Jan 2022 20:56:22 +0000 (UTC)
Subject: Writing a string with comma in one column of CSV file
References: <852369471.347508.1642280182000.ref@mail.yahoo.com>
Message-ID: <852369471.347508.1642280182000@mail.yahoo.com>

Hi,
I use the following line to write some information to a CSV file which is comma delimited.

f = open(output_file, 'w', newline='')
wr = csv.writer(f)
...
f.write(str(n) + "," + str(key) + "\n" )


Problem is that key is a string which may contain ',' and this causes the final CSV file to have more than 2 columns, while I want to write the whole key as a single column.

I know that wr.writerow([key]) writes the entire key in one column, but I would like to do the same with write(). Any idea to fix that?


Regards,
Mahmood

From PythonList at DancesWithMice.info  Sat Jan 15 16:11:08 2022
From: PythonList at DancesWithMice.info (dn)
Date: Sun, 16 Jan 2022 10:11:08 +1300
Subject: Writing a string with comma in one column of CSV file
In-Reply-To: <852369471.347508.1642280182000@mail.yahoo.com>
References: <852369471.347508.1642280182000.ref@mail.yahoo.com>
 <852369471.347508.1642280182000@mail.yahoo.com>
Message-ID: <81a7b4da-aa34-988c-5a75-77dd7d0e8f9d@DancesWithMice.info>

On 16/01/2022 09.56, Mahmood Naderan via Python-list wrote:
> Hi,
> I use the following line to write some information to a CSV file which is comma delimited.
> 
> f = open(output_file, 'w', newline='')
> wr = csv.writer(f)
> ...
> f.write(str(n) + "," + str(key) + "\n" )
> 
> 
> Problem is that key is a string which may contain ',' and this causes the final CSV file to have more than 2 columns, while I want to write the whole key as a single column.
> 
> I know that wr.writerow([key]) writes the entire key in one column, but I would like to do the same with write(). Any idea to fix that?


This is 'CSV' rather than 'Python'. The solution is to "escape" the
string. Ref:
https://stackoverflow.com/questions/769621/dealing-with-commas-in-a-csv-file

-- 
Regards,
=dn

From avigross at verizon.net  Sat Jan 15 16:24:40 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 15 Jan 2022 21:24:40 +0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <103472bc-36cc-899e-3711-7f177039f3a4@wichmann.us>
References: <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <103472bc-36cc-899e-3711-7f177039f3a4@wichmann.us>
Message-ID: <138589658.350582.1642281880407@mail.yahoo.com>

Mats,
Yes, this is a Python mailing list and I welcome people interested in doing something in Python who need a little help or advice but have some idea of what they are doing and present us with enough info more than "something does not work."

Yes, the topic was raised on a Python list but I did not get the impression the person asking necessarily knew a lot about Python or maybe other languages. I may well be wrong. They seem to have been handed something to work on and maybe are searching for some way to do it. You can do what was asked in plenty of languages, some easier and some harder. But some are better set to do things in a browser/server model than others.
One impression is they were suggested to use a web platform of sorts and from there went to Python as compatible for the server side. But the overall tone suggests to me that they want to be given a quick solution for a problem they do not really understand, let alone the tools they want to use. And I do not perceive much willingness to take the time to do much learning. My willingness to help and my time to do so, are limited and I said so. Others, feel free. The reality is many here want to be hired for large and complex jobs.
If someone had suggested to them to use?JavaScript?written inside a web page to read the file and do searches, and forgot to tell them that writing the file back onto some server might not be trivial, I wonder if they would have gone in another direction.
Again, these are my impressions and why I wondered if there was a GOOD reason they asked about Python. My experience in the past includes seeing posts by people who sent messages to multiple groups I was on for multiple languages and asked how to do something. They just were hoping someone would post complete code and they could then move on without learning?anything. Maybe I am a tad jaundiced and prefer honest requests from people willing to listen to what feedback we provide and show some code for us to look at and perhaps help improve.

-----Original Message-----
From: Mats Wichmann <mats at wichmann.us>
To: python-list at python.org
Sent: Sat, Jan 15, 2022 3:05 pm
Subject: Re: What to write or search on github to get the code for what is written below:

On 1/13/22 16:08, Avi Gross via Python-list wrote:
> 
> I am not replying to anything below so I have removed it.

> Instead, someone suggested Python which indeed, with lots of work, can open just about ANY nonsensical file and diddle around and rewrite it, but WHY? 

well, the topic *was* raised on a PYTHON list...


-- 
https://mail.python.org/mailman/listinfo/python-list

From alister.ware at ntlworld.com  Sat Jan 15 16:00:02 2022
From: alister.ware at ntlworld.com (alister)
Date: Sat, 15 Jan 2022 21:00:02 -0000 (UTC)
Subject: Writing a string with comma in one column of CSV file
References: <852369471.347508.1642280182000.ref@mail.yahoo.com>
 <852369471.347508.1642280182000@mail.yahoo.com>
 <mailman.185.1642280186.3079.python-list@python.org>
Message-ID: <srvcki$1g41$2@gioia.aioe.org>

On Sat, 15 Jan 2022 20:56:22 +0000 (UTC), Mahmood Naderan wrote:

> Hi,
> I use the following line to write some information to a CSV file which
> is comma delimited.
> 
> f = open(output_file, 'w', newline='')
> wr = csv.writer(f)
> ...
> f.write(str(n) + "," + str(key) + "\n" )
> 
> 
> Problem is that key is a string which may contain ',' and this causes
> the final CSV file to have more than 2 columns, while I want to write
> the whole key as a single column.
> 
> I know that wr.writerow([key]) writes the entire key in one column, but
> I would like to do the same with write(). Any idea to fix that?
> 
> 
> Regards,
> Mahmood

you need to quote the data
the easies way to ensure this is to inculde to QUOTE_ALL option when 
opening the file

wr = csv.writer(output, quoting=csv.QUOTE_ALL)



-- 
Chocolate chip.

From nt_mahmood at yahoo.com  Sat Jan 15 16:37:28 2022
From: nt_mahmood at yahoo.com (Mahmood Naderan)
Date: Sat, 15 Jan 2022 21:37:28 +0000 (UTC)
Subject: Writing a string with comma in one column of CSV file
In-Reply-To: <srvcki$1g41$2@gioia.aioe.org>
References: <852369471.347508.1642280182000.ref@mail.yahoo.com>
 <852369471.347508.1642280182000@mail.yahoo.com>
 <mailman.185.1642280186.3079.python-list@python.org>
 <srvcki$1g41$2@gioia.aioe.org>
Message-ID: <1938212965.351412.1642282648518@mail.yahoo.com>

Right. I was also able to put all columns in a string and then use writerow().
Thanks.



Regards,
Mahmood






On Saturday, January 15, 2022, 10:33:08 PM GMT+1, alister via Python-list <python-list at python.org> wrote: 





On Sat, 15 Jan 2022 20:56:22 +0000 (UTC), Mahmood Naderan wrote:

> Hi,
> I use the following line to write some information to a CSV file which
> is comma delimited.
> 
> f = open(output_file, 'w', newline='')
> wr = csv.writer(f)
> ...
> f.write(str(n) + "," + str(key) + "\n" )
> 
> 
> Problem is that key is a string which may contain ',' and this causes
> the final CSV file to have more than 2 columns, while I want to write
> the whole key as a single column.
> 
> I know that wr.writerow([key]) writes the entire key in one column, but
> I would like to do the same with write(). Any idea to fix that?
> 
> 
> Regards,
> Mahmood

you need to quote the data
the easies way to ensure this is to inculde to QUOTE_ALL option when 
opening the file

wr = csv.writer(output, quoting=csv.QUOTE_ALL)



-- 
Chocolate chip.
-- 
https://mail.python.org/mailman/listinfo/python-list

From cs at cskk.id.au  Sat Jan 15 16:37:57 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Sun, 16 Jan 2022 08:37:57 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
References: <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
Message-ID: <YeM+tTb0mlGYQXXk@cskk.homeip.net>

On 15Jan2022 02:38, NArshad <narshad.380 at gmail.com> wrote:
>Why does the code written below is not giving any output?
>
>    xls = ExcelFile('ABC.xlsx')
>    df = xls.parse(xls.sheet_names[0], index_col=1)
>    x=df.to_dict()
>    print (x)
>
>Only the contents of the first column and the column number is required 
>in the dictionary

The code above _must_ produce some output, even if it is very short, or 
an error traceback. Is "x" an empty dictionary? Are you getting an 
exception? What is in ABC.xlsx, at least in part? What modules are you 
importing in order to get the name "ExcelFile"? I'm guessing pandas, but 
that is from a web search.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From avigross at verizon.net  Sat Jan 15 16:40:40 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 15 Jan 2022 21:40:40 +0000 (UTC)
Subject: Writing a string with comma in one column of CSV file
In-Reply-To: <852369471.347508.1642280182000@mail.yahoo.com>
References: <852369471.347508.1642280182000.ref@mail.yahoo.com>
 <852369471.347508.1642280182000@mail.yahoo.com>
Message-ID: <1787720018.346094.1642282840439@mail.yahoo.com>

Mahmood,
Ask yourself WHY you want to do what you are doing. Are you using the power and features of the language or trying to do it step by step in the way that earlier languages often made you do it?
Yes, there are ways to include commas in fields of a CSV file and they can lead to complications you can completely avoid by NOT using a CSV file. Use something where commas are not a field separator and something else that is not likely to be in your data (or when found is removed or perhaps replaced with something like a space) and can thus be used.
Many files use a TAB or other symbols to delimit data and may be saved with names like a .TSV file or others. Python can trivially read in such files often simply by telling it to read something like a CSV but specifying the field separator is a tab or some other character.
But I have another dumb question. Why are you writing your CSV file by hand and a line at a time? I mean there is nothing wrong with that but many people have programs where they make an object like a DataFrame and manipulate that to have all the data they need and assuming we call the structure df, and they are using the pandas module, they can write the entire thing out like this:

df.to_csv('new_file.sv', sep='\t', index=False)

The pandas package fairly easily allows you to load in all the data you need rom an external source, search in what you have, make changes or additions, and write it out to many kinds of files.
Just a thought. If you like your way, fine, I see another reply suggesting how to hide the commas but that can be a problem if humans read and edit the results in the external file and do not follow through.

-----Original Message-----
From: Mahmood Naderan via Python-list <python-list at python.org>
To: DL Neil via Python-list <python-list at python.org>
Sent: Sat, Jan 15, 2022 3:56 pm
Subject: Writing a string with comma in one column of CSV file

Hi,
I use the following line to write some information to a CSV file which is comma delimited.

f = open(output_file, 'w', newline='')
wr = csv.writer(f)
...
f.write(str(n) + "," + str(key) + "\n" )


Problem is that key is a string which may contain ',' and this causes the final CSV file to have more than 2 columns, while I want to write the whole key as a single column.

I know that wr.writerow([key]) writes the entire key in one column, but I would like to do the same with write(). Any idea to fix that?


Regards,
Mahmood
-- 
https://mail.python.org/mailman/listinfo/python-list

From Marco.Sulla.Python at gmail.com  Sun Jan 16 09:29:31 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sun, 16 Jan 2022 15:29:31 +0100
Subject: Doc or example about conda custom build?
Message-ID: <CABbU2U8zJPydJa95KkNwKNCcU9Csj8TTE5j=OXraauvByWTV9A@mail.gmail.com>

Sorry for being maybe a little OT. I tried to get help from other Conda
users, from chat and from the mailing list without success.

I would add a custom build on my conda package. Is there somewhere a doc or
an example about it?

(Specifically, I want to pass a custom parameter to the setup.py that lets
me package only the pure py version of the code.)

From Marco.Sulla.Python at gmail.com  Sun Jan 16 09:32:20 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Sun, 16 Jan 2022 15:32:20 +0100
Subject: Why operations between dict views return a set and not a
 frozenset?
In-Reply-To: <20220111210718.r5vvjobf4i4ufjzu@hjp.at>
References: <CAPTjJmoRm26RiPsfhSph_Zwj=gJFyDN47u3kcvxfQPSM+fTf1g@mail.gmail.com>
 <CABbU2U9Pazu_yjdm8veY+VnsGLpq9axSEXmF-eJrTV3ZjHax3g@mail.gmail.com>
 <CAPTjJmoW0gV0dp9W=m=taB5vFZ4yy+DwAj9H9OUH+qXDkzhmmw@mail.gmail.com>
 <CABbU2U_EPGTgGkhiMLxg86o9-NyxLXc9jGLBE-qf8jnG4fM-Sw@mail.gmail.com>
 <CAPTjJmq=_Fo9ArCu-r0wZB-70WL0AjOqWttGqb_TLT62HBPnfQ@mail.gmail.com>
 <CABbU2U_cV2m=QoNG+6w2RvRR05pLuoEyYFk8GbLx6P2zqPQWgg@mail.gmail.com>
 <CAPTjJmqJ+nZfO-T6bkCbSERVko2TbQP5g11y_oB0+ZgUx06cfw@mail.gmail.com>
 <CABbU2U_d+Sj8naMwz8jWPbc2+2=z1EmoSRYwjnAeA9Dceg1F=g@mail.gmail.com>
 <CAPTjJmqXv8C5ZbQKG21nbZsGGifk6J3yj1m1OH0C-OUKuM9gBQ@mail.gmail.com>
 <CABbU2U-wosnDQhN4eMZZfvypJ=hMf7u9sfvVRDjtS8J8e7X11w@mail.gmail.com>
 <20220111210718.r5vvjobf4i4ufjzu@hjp.at>
Message-ID: <CABbU2U-3VKFZ2TY2c1N-+YjT=1beCP_nG4Hyk=gvG-TbL0FkLg@mail.gmail.com>

Thank you a lot for letting me understand :)

On Tue, 11 Jan 2022 at 22:09, Peter J. Holzer <hjp-python at hjp.at> wrote:

> On 2022-01-11 19:49:20 +0100, Marco Sulla wrote:
> > I think this is what you mean:
> >
> > >>> dis.dis("for _ in {1, 2}: pass")
> >   1           0 SETUP_LOOP              12 (to 14)
> >               2 LOAD_CONST               3 (frozenset({1, 2}))
> >               4 GET_ITER
> >         >>    6 FOR_ITER                 4 (to 12)
> >               8 STORE_NAME               0 (_)
> >              10 JUMP_ABSOLUTE            6
> >         >>   12 POP_BLOCK
> >         >>   14 LOAD_CONST               2 (None)
> >              16 RETURN_VALUE
> > >>> a = {1, 2}
> > >>> dis.dis("for _ in a: pass")
> >   1           0 SETUP_LOOP              12 (to 14)
> >               2 LOAD_NAME                0 (a)
> >               4 GET_ITER
> >         >>    6 FOR_ITER                 4 (to 12)
> >               8 STORE_NAME               1 (_)
> >              10 JUMP_ABSOLUTE            6
> >         >>   12 POP_BLOCK
> >         >>   14 LOAD_CONST               0 (None)
> >              16 RETURN_VALUE
>
> I think you have omitted the part that Chris was hinting at.
>
> >>> dis.dis("a = {1, 2};\nfor _ in a: pass")
>   1           0 LOAD_CONST               0 (1)
>               2 LOAD_CONST               1 (2)
>               4 BUILD_SET                2
>               6 STORE_NAME               0 (a)
>
>   2           8 LOAD_NAME                0 (a)
>              10 GET_ITER
>         >>   12 FOR_ITER                 4 (to 18)
>              14 STORE_NAME               1 (_)
>              16 JUMP_ABSOLUTE           12
>         >>   18 LOAD_CONST               2 (None)
>              20 RETURN_VALUE
>
> Now compare
>
>               2 LOAD_CONST               3 (frozenset({1, 2}))
>
> with
>
>   1           0 LOAD_CONST               0 (1)
>               2 LOAD_CONST               1 (2)
>               4 BUILD_SET                2
>
> and you see the difference between using a frozenset as a constant and
> building a set at runtime.
>
>         hp
>
> --
>    _  | Peter J. Holzer    | Story must make more sense than reality.
> |_|_) |                    |
> | |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
> __/   | http://www.hjp.at/ |       challenge!"
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From mats at wichmann.us  Sun Jan 16 10:18:29 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Sun, 16 Jan 2022 08:18:29 -0700
Subject: Writing a string with comma in one column of CSV file
In-Reply-To: <852369471.347508.1642280182000@mail.yahoo.com>
References: <852369471.347508.1642280182000.ref@mail.yahoo.com>
 <852369471.347508.1642280182000@mail.yahoo.com>
Message-ID: <75332dd6-622a-1ff0-aae7-8b1ddb14e893@wichmann.us>

On 1/15/22 13:56, Mahmood Naderan via Python-list wrote:
> Hi,
> I use the following line to write some information to a CSV file which is comma delimited.
> 
> f = open(output_file, 'w', newline='')
> wr = csv.writer(f)
> ...
> f.write(str(n) + "," + str(key) + "\n" )
> 
> 
> Problem is that key is a string which may contain ',' and this causes the final CSV file to have more than 2 columns, while I want to write the whole key as a single column.

One of the reasons csv is a horrible data interchange format.

If you must... the convention for Excel, which is usually the reason
people are using csv, is you can enclose the entire comma-containing
field in "quote marks" (afaik it must be double-quote).  The convention
for other consumers of csv may or may not accept this - if it's not for
Excel you'll need to check. One gotcha - the opening quote must appear
immediately after the preceding comma separator, you can't pad with
spaces. That is:

"this","should","be","okay, I think"

and not:

"this",  "will",  "not",  "work",  "correctly, I fear"


(the single-word row entries don't have to be in quotes, of course)

If you use the Python csv module it should take care of this - you can
specify the separator sequence and the quote sequence, and it knows to
quote a field correctly if it contains the separator.  Highly recommended.


From mats at wichmann.us  Sun Jan 16 10:42:26 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Sun, 16 Jan 2022 08:42:26 -0700
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <138589658.350582.1642281880407@mail.yahoo.com>
References: <mailman.134.1641762330.3079.python-list@python.org>
 <702eda61-96b8-7c89-f209-0cdb0f6888a7@mrabarnett.plus.com>
 <c7aa1c51-6280-4243-9cdd-7889c56117cfn@googlegroups.com>
 <mailman.144.1641842486.3079.python-list@python.org>
 <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <103472bc-36cc-899e-3711-7f177039f3a4@wichmann.us>
 <138589658.350582.1642281880407@mail.yahoo.com>
Message-ID: <ebe5e791-9dd9-f887-f9f5-b187e850633e@wichmann.us>

On 1/15/22 14:24, Avi Gross via Python-list wrote:
> Mats,
> Yes, this is a Python mailing list and I welcome people interested in doing something in Python who need a little help or advice but have some idea of what they are doing and present us with enough info more than "something does not work."
> 
> Yes, the topic was raised on a Python list but I did not get the impression the person asking necessarily knew a lot about Python or maybe other languages. I may well be wrong. They seem to have been handed something to work on and maybe are searching for some way to do it. You can do what was asked in plenty of languages, some easier and some harder. But some are better set to do things in a browser/server model than others.

I only meant, in response to "someone suggested" in the quoted snip
below, since it was asked here, someone suggesting Python was a pretty
expected outcome...


> -----Original Message-----
> From: Mats Wichmann <mats at wichmann.us>
> To: python-list at python.org
> Sent: Sat, Jan 15, 2022 3:05 pm
> Subject: Re: What to write or search on github to get the code for what is written below:
> 
> On 1/13/22 16:08, Avi Gross via Python-list wrote:
>>
>> I am not replying to anything below so I have removed it.
> 
>> Instead, someone suggested Python which indeed, with lots of work, can open just about ANY nonsensical file and diddle around and rewrite it, but WHY? 
> 
> well, the topic *was* raised on a PYTHON list...
> 
> 


From wlfraed at ix.netcom.com  Sun Jan 16 11:51:42 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sun, 16 Jan 2022 11:51:42 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
Message-ID: <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>

On Sat, 15 Jan 2022 02:38:34 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

	A bit of an improvement -- actual code...

	But... still not a minimal /runnable/ example...
>
>Why does the code written below is not giving any output?
>
>    xls = ExcelFile('ABC.xlsx')

	Where is "ExcelFile" defined? Best match I could find after spending
time with Google is the pandas package. Pandas is likely overkill for this
situation -- being optimized for analysis of numerical tabular data. In
particular, it expects each column to be of one data type, not mixed data
types, in order to apply aggregate functions (min, max, mean, std.dev.,
etc.) to the columns.

>    df = xls.parse(xls.sheet_names[0], index_col=1)

	Based upon documentation, pandas.ExcelFile.parse() internally uses the
openpyxl package for files of type .xlsx -- for this application openpyxl
is probably all that is needed.

	Documentation for .parse() (actually the underlying .read_excel()
operation) indicates that the desired sheet can be specified by name OR BY
POSITION -- with the default value being "0" [first sheet]. It is somewhat
perverse to be retrieving a list sheet names from the file, only to then
pass the first name back into the function which has to match the name up
against the list to determine the position... Unless the file is using
multiple sheets for data -- in which case you need some logic to select
something other than the first sheet -- you could leave that argument off
and let it default.

	index_col=1		says to use the SECOND COLUMN of the sheet to provide
row-labels for the "dataframe" (unless documentation states otherwise, the
first item of a collection is "0", not "1")

>    x=df.to_dict()

	What do you really expect from this conversion? With no arguments it
will produce a nested dictionary of the form

		{	"column1name"	:	{	"row1name"	: r1-c1-value	,
									"row2name" : r2-c1-value		},
			"column2name"	:	{	"row1name" : r1-c2-value,
									"row2name" : r2-c2-value		}	}

>    print (x)
>
>Only the contents of the first column and the column number is required in the dictionary

	You won't get that with .to_dict() -- it uses the column names for the
major grouping, and then uses the row names (which are already garbage as
you told pandas to use the second column for row names) to select the VALUE
stored in that row for that column. Also, while Python implementation of
dictionaries has changed over the years, the theoretical basis (associative
mapping, hash table) does not define an order for the keys of the
dictionary. Unless you are certain of how a dictionary handles
insert/delete of keys, you can not assume the first key out was the first
key that went in.

	It would also be helpful to have a snippet of the spreadsheet -- say
the first five or six rows, along with the first five or six columns;
exported as a CSV file (binary attachments get stripped from this forum,
but a CSV is text and can be pasted in -- 5x5 should be small enough that
those trying to assist can then extract it from the post, clean up any line
wrappings, and import into a blank spreadsheet.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From greg.ewing at canterbury.ac.nz  Sun Jan 16 17:38:05 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 17 Jan 2022 11:38:05 +1300
Subject: Writing a string with comma in one column of CSV file
In-Reply-To: <mailman.193.1642346316.3079.python-list@python.org>
References: <852369471.347508.1642280182000.ref@mail.yahoo.com>
 <852369471.347508.1642280182000@mail.yahoo.com>
 <75332dd6-622a-1ff0-aae7-8b1ddb14e893@wichmann.us>
 <mailman.193.1642346316.3079.python-list@python.org>
Message-ID: <j4jl2gF2ttkU1@mid.individual.net>

On 17/01/22 4:18 am, Mats Wichmann wrote:
> the convention for Excel, which is usually the reason
> people are using csv, is you can enclose the entire comma-containing
> field in "quote marks" (afaik it must be double-quote).

And to include a double quote in a field, quote the field and
double the double-quote.

Another quirk is that Excel allows newlines in a quoted field,
which causes problems for naive parsers that split the input
into lines first and then analyse it into fields. (A particular
non-Python one I use professionally is guilty of this...)

-- 
Greg

From shishaozhong at gmail.com  Mon Jan 17 04:31:31 2022
From: shishaozhong at gmail.com (Shaozhong SHI)
Date: Mon, 17 Jan 2022 09:31:31 +0000
Subject: How to set environmental variables for Python
Message-ID: <CA+i5Jwac2H0cso9F1ABYbm5dTok=nXxHAkZcKjdQE80YfXsT-Q@mail.gmail.com>

I got quite a few version of Python on my machine.

How do I set environmental variables for Python 3.6.1 to work?

Regards,

David

From PythonList at DancesWithMice.info  Mon Jan 17 05:55:48 2022
From: PythonList at DancesWithMice.info (dn)
Date: Mon, 17 Jan 2022 23:55:48 +1300
Subject: How to set environmental variables for Python
In-Reply-To: <CA+i5Jwac2H0cso9F1ABYbm5dTok=nXxHAkZcKjdQE80YfXsT-Q@mail.gmail.com>
References: <CA+i5Jwac2H0cso9F1ABYbm5dTok=nXxHAkZcKjdQE80YfXsT-Q@mail.gmail.com>
Message-ID: <051b2f4c-068b-f807-d163-c0595486587c@DancesWithMice.info>

On 17/01/2022 22.31, Shaozhong SHI wrote:
> I got quite a few version of Python on my machine.
> 
> How do I set environmental variables for Python 3.6.1 to work?


Set from Python, or set in the OpSys?

https://docs.python.org/3/library/os.html?highlight=environment%20variable

MS-Win: https://docs.python.org/3/using/windows.html#setting-envvars
-- 
Regards,
=dn

From shishaozhong at gmail.com  Mon Jan 17 06:36:24 2022
From: shishaozhong at gmail.com (Shaozhong SHI)
Date: Mon, 17 Jan 2022 11:36:24 +0000
Subject: How to set environmental variables for Python
In-Reply-To: <051b2f4c-068b-f807-d163-c0595486587c@DancesWithMice.info>
References: <CA+i5Jwac2H0cso9F1ABYbm5dTok=nXxHAkZcKjdQE80YfXsT-Q@mail.gmail.com>
 <051b2f4c-068b-f807-d163-c0595486587c@DancesWithMice.info>
Message-ID: <CA+i5JwZr35PfzmYw4pE97CRY2BuLT8eFd5jRrGo5azN_ZcwxDg@mail.gmail.com>

Set Operation System but not disturbing existing setting.  Only to add at
the command line.

Regards,

David

On Mon, 17 Jan 2022 at 10:57, dn via Python-list <python-list at python.org>
wrote:

> On 17/01/2022 22.31, Shaozhong SHI wrote:
> > I got quite a few version of Python on my machine.
> >
> > How do I set environmental variables for Python 3.6.1 to work?
>
>
> Set from Python, or set in the OpSys?
>
> https://docs.python.org/3/library/os.html?highlight=environment%20variable
>
> MS-Win: https://docs.python.org/3/using/windows.html#setting-envvars
> --
> Regards,
> =dn
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From sinamobasheri at outlook.com  Mon Jan 17 10:37:08 2022
From: sinamobasheri at outlook.com (Sina Mobasheri)
Date: Mon, 17 Jan 2022 15:37:08 +0000
Subject: About Python Compressed Archive or Binaries
Message-ID: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>

Java offers download JDK as Compressed Archive or NodeJS offers download Node as Binaries both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working


I'm aware that Python also have something called Embedded Zip for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that the user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offering


My question is why is Python haven't an option for downloading as Compressed Archive?
Is it's due to a technical issue? political reasons? or is it just simply because nobody has thought of doing it?


From narshad.380 at gmail.com  Mon Jan 17 04:55:15 2022
From: narshad.380 at gmail.com (NArshad)
Date: Mon, 17 Jan 2022 01:55:15 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
References: <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
Message-ID: <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>

Avi Gross:

-?They just were hoping someone would post complete code and they could then move on without learning anything.?

This is due to the time factor

From rosuav at gmail.com  Mon Jan 17 10:47:38 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 18 Jan 2022 02:47:38 +1100
Subject: About Python Compressed Archive or Binaries
In-Reply-To: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
References: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
Message-ID: <CAPTjJmrtjP2k6D3HyaO3=t6TytUN0d5489Jj7QL4sze1zvqggg@mail.gmail.com>

On Tue, Jan 18, 2022 at 2:42 AM Sina Mobasheri
<sinamobasheri at outlook.com> wrote:
>
> Java offers download JDK as Compressed Archive or NodeJS offers download Node as Binaries both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working
>
>
> I'm aware that Python also have something called Embedded Zip for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that the user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offering
>
>
> My question is why is Python haven't an option for downloading as Compressed Archive?
> Is it's due to a technical issue? political reasons? or is it just simply because nobody has thought of doing it?
>

https://docs.python.org/3/library/zipapp.html

:)

ChrisA

From rosuav at gmail.com  Mon Jan 17 10:49:12 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 18 Jan 2022 02:49:12 +1100
Subject: About Python Compressed Archive or Binaries
In-Reply-To: <CAPTjJmrtjP2k6D3HyaO3=t6TytUN0d5489Jj7QL4sze1zvqggg@mail.gmail.com>
References: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CAPTjJmrtjP2k6D3HyaO3=t6TytUN0d5489Jj7QL4sze1zvqggg@mail.gmail.com>
Message-ID: <CAPTjJmpYO5Oc-1onQ7MogCZSxrnhM4Q55AuYgeQDSvHzgjuLFQ@mail.gmail.com>

On Tue, Jan 18, 2022 at 2:47 AM Chris Angelico <rosuav at gmail.com> wrote:
>
> On Tue, Jan 18, 2022 at 2:42 AM Sina Mobasheri
> <sinamobasheri at outlook.com> wrote:
> >
> > Java offers download JDK as Compressed Archive or NodeJS offers download Node as Binaries both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working
> >
> >
> > I'm aware that Python also have something called Embedded Zip for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that the user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offering
> >
> >
> > My question is why is Python haven't an option for downloading as Compressed Archive?
> > Is it's due to a technical issue? political reasons? or is it just simply because nobody has thought of doing it?
> >

Premature send, oops.

That's already possible: https://docs.python.org/3/library/zipapp.html :)

Alternatively, if you mean the CPython binary itself, then that is
available in a number of formats, so I'm not sure what you're asking
about in that case.

ChrisA

From rosuav at gmail.com  Mon Jan 17 10:51:22 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 18 Jan 2022 02:51:22 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
References: <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
Message-ID: <CAPTjJmq8Wbtp3TNiZU2VGOQ+GQXq9p-jdLh81nLEAw+1bZOBvQ@mail.gmail.com>

On Tue, Jan 18, 2022 at 2:46 AM NArshad <narshad.380 at gmail.com> wrote:
>
> Avi Gross:
>
> -?They just were hoping someone would post complete code and they could then move on without learning anything.?
>
> This is due to the time factor

Then pay someone to write it. There are plenty of contractors out
there. You can't expect this list to be a free consulting programmer
for you.

I strongly encourage everyone to stop helping the OP until there's
some code to help with.

(I'm not surprised, given that the original request was basically
"what should I search for to magically find what I want".)

ChrisA

From sinamobasheri at outlook.com  Mon Jan 17 10:54:33 2022
From: sinamobasheri at outlook.com (Sina Mobasheri)
Date: Mon, 17 Jan 2022 15:54:33 +0000
Subject: Why There Is No Python Compressed Archive or Binaries ?
Message-ID: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>

Java offers download JDK as Compressed Archive<https://www.oracle.com/java/technologies/downloads/#jdk17-linux> or NodeJS offers download Node as Binaries<https://nodejs.org/en/download/current/> both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working

I'm aware that Python also have something called Embedded Zip<https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip> for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offers

My question is why Python hasn't option for downloading as Compressed Archive ?

From grant.b.edwards at gmail.com  Mon Jan 17 11:07:07 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 17 Jan 2022 08:07:07 -0800 (PST)
Subject: Why There Is No Python Compressed Archive or Binaries ?
References: =?utf-8?q?=3CAS8PR06MB759290041865856ECF07F317AD579=40AS8PR06MB7?=
 =?utf-8?q?592=2Eeurprd06=2Eprod=2Eoutlook=2Ecom=3E?=
Message-ID: <61e5942b.1c69fb81.75787.499a@mx.google.com>

On 2022-01-17, Sina Mobasheri <sinamobasheri at outlook.com> wrote:
> Java offers download JDK...
> [...]
> My question is why Python hasn't option for downloading as
> Compressed Archive ?

Isn't that what the installers are?

--
Grant

From wlfraed at ix.netcom.com  Mon Jan 17 11:41:36 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Mon, 17 Jan 2022 11:41:36 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <CAPTjJmq8Wbtp3TNiZU2VGOQ+GQXq9p-jdLh81nLEAw+1bZOBvQ@mail.gmail.com>
 <mailman.201.1642434694.3079.python-list@python.org>
Message-ID: <3h5buglmrdjk73ql7h44pk0gptq9jvn61b@4ax.com>

On Tue, 18 Jan 2022 02:51:22 +1100, Chris Angelico <rosuav at gmail.com>
declaimed the following:

>
>I strongly encourage everyone to stop helping the OP until there's
>some code to help with.
>
	Well, we did squeeze about four lines of code from the OP -- though
still incomplete (no import statements so we have to research what 3rd
party module the OP might be invoking -- and they seem to have totally
skipped direct usage of an module for reading Excel format files in favor
of loading the massive pandas package [with its need for those direct
reading modules along with SciPy/NumPy -- does it also load matplotlib?]).

>(I'm not surprised, given that the original request was basically
>"what should I search for to magically find what I want".)

	To which the closest viable response I've extracted is to Google

			python library management system

(presuming Python is a firm requirement). Of course all of those have the
problems

		1A		They use an RDBM (most examples seem to be MySQL based,
though SQLite3 might show up in some)
		1B		No sign of Excel spreadsheets anywhere

		2A		They handle adding/removing books from the inventory
(permanent actions), along with
		2B		Check-out/Check-in of books from the inventory (OP seems to
only focus on Check-out/Reservation)
		2C		They may even track who is allowed to run the application
(staff only... for all functions or just some, etc.)

		3		Every copy of a book is an individual record in the
database -- if there are 30 copies of a book, there are 30 records for it,
and each copy can only have one check-out at a time; no "decrement" of a
"number available".

		4		Some are local/desktop applications using Qt or similar GUI
framework, others might be web-based but likely not Flask


	{And, from my personal viewpoint, they may be using SQLAlchemy... I
have the O'Reilly text for that, and find it still makes no sense to me...
I can code direct SQL queries faster than trying to figure out the
equivalent functionality in SQLAlchemy... And if the OP has a fixation on
spreadsheet views of the data, SQLAlchemy will really be a mystery.

}


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From wlfraed at ix.netcom.com  Mon Jan 17 11:46:19 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Mon, 17 Jan 2022 11:46:19 -0500
Subject: Why There Is No Python Compressed Archive or Binaries ?
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <61e5942b.1c69fb81.75787.499a@mx.google.com>
 <mailman.203.1642435630.3079.python-list@python.org>
Message-ID: <s37bug919j861964c20migg2k43clcq8iu@4ax.com>

On Mon, 17 Jan 2022 08:07:07 -0800 (PST), Grant Edwards
<grant.b.edwards at gmail.com> declaimed the following:

>On 2022-01-17, Sina Mobasheri <sinamobasheri at outlook.com> wrote:
>> Java offers download JDK...
>> [...]
>> My question is why Python hasn't option for downloading as
>> Compressed Archive ?
>
>Isn't that what the installers are?

	I get the impression they are looking for a zero-install option... IE:
unpack onto a USB flash drive, take to another computer, and just be able
to run from the flash drive without needing to fiddle with PATH, registry,
etc. A way to run Python on shared (library) computers that may not have it
installed, and for which users have no ability to install applications.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From grant.b.edwards at gmail.com  Mon Jan 17 12:22:24 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 17 Jan 2022 09:22:24 -0800 (PST)
Subject: Why There Is No Python Compressed Archive or Binaries ?
References: =?utf-8?q?=3CAS8PR06MB759290041865856ECF07F317AD579=40AS8PR06MB7?=
 =?utf-8?q?592=2Eeurprd06=2Eprod=2Eoutlook=2Ecom=3E?=
 <61e5942b.1c69fb81.75787.499a@mx.google.com>
 <mailman.203.1642435630.3079.python-list@python.org>
 <s37bug919j861964c20migg2k43clcq8iu@4ax.com>
Message-ID: <61e5a5d0.1c69fb81.f44bf.4fd7@mx.google.com>

On 2022-01-17, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> On Mon, 17 Jan 2022 08:07:07 -0800 (PST), Grant Edwards
><grant.b.edwards at gmail.com> declaimed the following:
>
>>On 2022-01-17, Sina Mobasheri <sinamobasheri at outlook.com> wrote:
>>> Java offers download JDK...
>>> [...]
>>> My question is why Python hasn't option for downloading as
>>> Compressed Archive ?
>>
>>Isn't that what the installers are?
>
> 	I get the impression they are looking for a zero-install option... IE:
> unpack onto a USB flash drive, take to another computer, and just be able
> to run from the flash drive without needing to fiddle with PATH, registry,
> etc.

OK, I see.

> A way to run Python on shared (library) computers that may not have it
> installed, and for which users have no ability to install applications.

I guess I don't know enough about Windows to comment. I've packaged
apps for Windows that include the Python interpreter and libs, but
they still need to be "installed".

--
Grant



From avigross at verizon.net  Mon Jan 17 12:22:08 2022
From: avigross at verizon.net (Avi Gross)
Date: Mon, 17 Jan 2022 17:22:08 +0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
References: <d116acea-57c3-36e5-6504-c35a12ea7374@mrabarnett.plus.com>
 <b17e162c-58c4-4f55-bc8a-032753ced64an@googlegroups.com>
 <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
Message-ID: <1680987126.600079.1642440128127@mail.yahoo.com>


I can appreciate people under time pressure wanting the job DONE first and maybe learning more after. So, yes, it makes perfect sense to delegate the task to others with expertise or ask for advice.
This forum may mean many things to many people but for me, it is a place to offer guidance and small amounts of help as a voluntary activity. Doing a major project for free is not necessarily why I am here. I prefer being hired and compensated, LOL!
But I share the frustration when someone dumps an ill-defined project on you without realizing what they handed you is an old application that cannot trivially be shifted into something new without serious effort and preparation.
The feedback here has included suggestions that we were shown only a part of the task needed and that it would be a good idea to not even try to keep your data in some EXCEL format if it was used heavily in ways perhaps never intended. Obviously, feeling urgency to do something quickly will make it hard to accept our suggestions, but nonetheless, a request here, as has been pointed out, should anticipate a python-oriented answer. Asking us here to make a set of macros for EXCEL to do this task would be inappropriate. Some are suggesting that a solution that keeps reading and writing to an EXCEL file for each transaction is also not necessarily a great design.
I think your answer makes me amend what I wrote. I grant you may be interested in learning but due to the press of time, you want a working set of software first.
Good luck with that.?

-----Original Message-----
From: NArshad <narshad.380 at gmail.com>
To: python-list at python.org
Sent: Mon, Jan 17, 2022 4:55 am
Subject: Re: What to write or search on github to get the code for what is written below:

Avi Gross:

-?They just were hoping someone would post complete code and they could then move on without learning anything.?

This is due to the time factor
-- 
https://mail.python.org/mailman/listinfo/python-list

From kvratkin at yandex.ru  Mon Jan 17 12:49:44 2022
From: kvratkin at yandex.ru (Kirill Ratkin)
Date: Mon, 17 Jan 2022 20:49:44 +0300
Subject: About Python Compressed Archive or Binaries
In-Reply-To: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
References: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
Message-ID: <dfdb9924-ae2c-c62a-8902-585adc9ab571@yandex.ru>

Hi,

Yes, this is good question for Windows users.

Of course, you can download installer exe-file -> do installation -> 
pack directory with python interpreter to zip (for example, or 7z) -> 
copy archive file to another place/computer and unpack.

But it will not work out of box because original installer modifies 
resources in MZ file and path to stdlib is there.

So just to use interpreter from that archive file you need to open 
python.exe file and fix it (in any binary/hex editor).

This simple procedure but ... anyway ... I agree with Sina.

It would be nice to have just zip file with python interpreter (not 
executable installer), unpack it anywhere, add path? to this 'anywhere' 
to PATH, and use it.

Java/DotNet/Go have this option. But python - not.

And question is - why?


On 1/17/22 18:37, Sina Mobasheri wrote:
> Java offers download JDK as Compressed Archive or NodeJS offers download Node as Binaries both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working
>
>
> I'm aware that Python also have something called Embedded Zip for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that the user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offering
>
>
> My question is why is Python haven't an option for downloading as Compressed Archive?
> Is it's due to a technical issue? political reasons? or is it just simply because nobody has thought of doing it?
>

From cspealma at redhat.com  Mon Jan 17 13:49:13 2022
From: cspealma at redhat.com (Calvin Spealman)
Date: Mon, 17 Jan 2022 13:49:13 -0500
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
Message-ID: <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>

I maintain a small project that provides this, a drop-in Python runtime you
can ship without installation called Feet. Get it? It makes Python run.

https://github.com/ironfroggy/feet

On Mon, Jan 17, 2022 at 11:16 AM Sina Mobasheri <sinamobasheri at outlook.com>
wrote:

> Java offers download JDK as Compressed Archive<
> https://www.oracle.com/java/technologies/downloads/#jdk17-linux> or
> NodeJS offers download Node as Binaries<
> https://nodejs.org/en/download/current/> both give us a compressed file
> for Linux and windows that we can just unzipped it and put in a custom
> directory and set some environment variables and start working
>
> I'm aware that Python also have something called Embedded Zip<
> https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip>
> for Windows and nothing like that for Linux as far as I know, and I think
> this Embedded Zip is not something that user wants to work with that
> directly it's for embedding in a C++ application, so it's not the same as
> options that Java and NodeJS offers
>
> My question is why Python hasn't option for downloading as Compressed
> Archive ?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>

-- 

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.spealman at redhat.com  M: +1.336.210.5107
[image: https://red.ht/sig] <https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

From sinamobasheri at outlook.com  Mon Jan 17 14:07:56 2022
From: sinamobasheri at outlook.com (Sina Mobasheri)
Date: Mon, 17 Jan 2022 19:07:56 +0000
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
Message-ID: <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>

It's cool project definitely something that I'm personally interested about, but I talking about compressed archive of cpython that we can simply unzipped and starting developing an app, not running  an app that already developed... ?
________________________________
From: Calvin Spealman <cspealma at redhat.com>
Sent: Monday, January 17, 2022 10:19:13 PM
To: Sina Mobasheri <sinamobasheri at outlook.com>
Cc: python-list at python.org <python-list at python.org>
Subject: Re: Why There Is No Python Compressed Archive or Binaries ?

I maintain a small project that provides this, a drop-in Python runtime you can ship without installation called Feet. Get it? It makes Python run.

https://github.com/ironfroggy/feet

On Mon, Jan 17, 2022 at 11:16 AM Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>> wrote:
Java offers download JDK as Compressed Archive<https://www.oracle.com/java/technologies/downloads/#jdk17-linux> or NodeJS offers download Node as Binaries<https://nodejs.org/en/download/current/> both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working

I'm aware that Python also have something called Embedded Zip<https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip> for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offers

My question is why Python hasn't option for downloading as Compressed Archive ?
--
https://mail.python.org/mailman/listinfo/python-list



--

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.spealman at redhat.com<mailto:calvin.spealman at redhat.com>  M: +1.336.210.5107<tel:+1.336.210.5107>

[https://red.ht/sig]<https://red.ht/sig>
TRIED. TESTED. TRUSTED.<https://redhat.com/trusted>

From sinamobasheri at outlook.com  Mon Jan 17 14:11:01 2022
From: sinamobasheri at outlook.com (Sina Mobasheri)
Date: Mon, 17 Jan 2022 19:11:01 +0000
Subject: About Python Compressed Archive or Binaries
In-Reply-To: <dfdb9924-ae2c-c62a-8902-585adc9ab571@yandex.ru>
References: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <dfdb9924-ae2c-c62a-8902-585adc9ab571@yandex.ru>
Message-ID: <AS8PR06MB7592DCBC894DA9C692EA9B34AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>

That's exactly what I mean thank you ???
________________________________
From: Python-list <python-list-bounces+sinamobasheri=outlook.com at python.org> on behalf of Kirill Ratkin <kvratkin at yandex.ru>
Sent: Monday, January 17, 2022 9:19:44 PM
To: python-list at python.org <python-list at python.org>
Subject: Re: About Python Compressed Archive or Binaries

Hi,

Yes, this is good question for Windows users.

Of course, you can download installer exe-file -> do installation ->
pack directory with python interpreter to zip (for example, or 7z) ->
copy archive file to another place/computer and unpack.

But it will not work out of box because original installer modifies
resources in MZ file and path to stdlib is there.

So just to use interpreter from that archive file you need to open
python.exe file and fix it (in any binary/hex editor).

This simple procedure but ... anyway ... I agree with Sina.

It would be nice to have just zip file with python interpreter (not
executable installer), unpack it anywhere, add path  to this 'anywhere'
to PATH, and use it.

Java/DotNet/Go have this option. But python - not.

And question is - why?


On 1/17/22 18:37, Sina Mobasheri wrote:
> Java offers download JDK as Compressed Archive or NodeJS offers download Node as Binaries both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working
>
>
> I'm aware that Python also have something called Embedded Zip for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that the user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offering
>
>
> My question is why is Python haven't an option for downloading as Compressed Archive?
> Is it's due to a technical issue? political reasons? or is it just simply because nobody has thought of doing it?
>
--
https://mail.python.org/mailman/listinfo/python-list

From cspealma at redhat.com  Mon Jan 17 14:22:52 2022
From: cspealma at redhat.com (Calvin Spealman)
Date: Mon, 17 Jan 2022 14:22:52 -0500
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
 <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
Message-ID: <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com>

Well, on its own, I'd say the reason we don't have such a download is that
it wouldn't be very useful.

On Mon, Jan 17, 2022 at 2:08 PM Sina Mobasheri <sinamobasheri at outlook.com>
wrote:

> It's cool project definitely something that I'm personally interested
> about, but I talking about compressed archive of cpython that we can simply
> unzipped and starting developing an app, not running  an app that already
> developed... ?
> ------------------------------
> *From:* Calvin Spealman <cspealma at redhat.com>
> *Sent:* Monday, January 17, 2022 10:19:13 PM
> *To:* Sina Mobasheri <sinamobasheri at outlook.com>
> *Cc:* python-list at python.org <python-list at python.org>
> *Subject:* Re: Why There Is No Python Compressed Archive or Binaries ?
>
> I maintain a small project that provides this, a drop-in Python runtime
> you can ship without installation called Feet. Get it? It makes Python run.
>
> https://github.com/ironfroggy/feet
>
> On Mon, Jan 17, 2022 at 11:16 AM Sina Mobasheri <sinamobasheri at outlook.com>
> wrote:
>
> Java offers download JDK as Compressed Archive<
> https://www.oracle.com/java/technologies/downloads/#jdk17-linux> or
> NodeJS offers download Node as Binaries<
> https://nodejs.org/en/download/current/> both give us a compressed file
> for Linux and windows that we can just unzipped it and put in a custom
> directory and set some environment variables and start working
>
> I'm aware that Python also have something called Embedded Zip<
> https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip>
> for Windows and nothing like that for Linux as far as I know, and I think
> this Embedded Zip is not something that user wants to work with that
> directly it's for embedding in a C++ application, so it's not the same as
> options that Java and NodeJS offers
>
> My question is why Python hasn't option for downloading as Compressed
> Archive ?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
>
> --
>
> CALVIN SPEALMAN
>
> SENIOR QUALITY ENGINEER
>
> calvin.spealman at redhat.com  M: +1.336.210.5107
> [image: https://red.ht/sig] <https://red.ht/sig>
> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>


-- 

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.spealman at redhat.com  M: +1.336.210.5107
[image: https://red.ht/sig] <https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

From sinamobasheri at outlook.com  Mon Jan 17 14:53:50 2022
From: sinamobasheri at outlook.com (Sina Mobasheri)
Date: Mon, 17 Jan 2022 19:53:50 +0000
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
 <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com>
Message-ID: <AS8PR06MB7592F3EBB0AE19DF68FD5B56AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>

Consider scenario that I want run python 3.10 in CentOS 8, I think last python version in CentOS repository is 3.6, if I use epel I can get 3.8 so ..., I think (correct me if I'm wrong ??) the only way that I can run python 3.10 is to compile it manually, which is need to know what dependencies python needs for compilation ... (different distribution different packages, which packages for what, you can see that it is intimidating for beginners like me)
It's useful to just use wget <python.org/ftp/python/3.10.1/compressed. tar.xz>, unzipped, ser path and ta-da you have cpython 3.10 in CentOS 8

This is Linux specific use case that I can think of, I'm sure there are plenty for windows, consider scenario that I wrote script for scraping some site and entered in some excel worksheet I can simply ship cpython with my script to clients machine and there is no need that client install cpython by himself...

Sorry about my bad grammar
________________________________
From: Calvin Spealman <cspealma at redhat.com>
Sent: Monday, January 17, 2022, 22:53
To: Sina Mobasheri
Cc: python-list at python.org
Subject: Re: Why There Is No Python Compressed Archive or Binaries ?

Well, on its own, I'd say the reason we don't have such a download is that it wouldn't be very useful.

On Mon, Jan 17, 2022 at 2:08 PM Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>> wrote:
It's cool project definitely something that I'm personally interested about, but I talking about compressed archive of cpython that we can simply unzipped and starting developing an app, not running  an app that already developed... ?
________________________________
From: Calvin Spealman <cspealma at redhat.com<mailto:cspealma at redhat.com>>
Sent: Monday, January 17, 2022 10:19:13 PM
To: Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>>
Cc: python-list at python.org<mailto:python-list at python.org> <python-list at python.org<mailto:python-list at python.org>>
Subject: Re: Why There Is No Python Compressed Archive or Binaries ?

I maintain a small project that provides this, a drop-in Python runtime you can ship without installation called Feet. Get it? It makes Python run.

https://github.com/ironfroggy/feet

On Mon, Jan 17, 2022 at 11:16 AM Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>> wrote:
Java offers download JDK as Compressed Archive<https://www.oracle.com/java/technologies/downloads/#jdk17-linux> or NodeJS offers download Node as Binaries<https://nodejs.org/en/download/current/> both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working

I'm aware that Python also have something called Embedded Zip<https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip> for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offers

My question is why Python hasn't option for downloading as Compressed Archive ?
--
https://mail.python.org/mailman/listinfo/python-list



--

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.spealman at redhat.com<mailto:calvin.spealman at redhat.com>  M: +1.336.210.5107<tel:+1.336.210.5107>

[https://red.ht/sig]<https://red.ht/sig>
TRIED. TESTED. TRUSTED.<https://redhat.com/trusted>


--

CALVIN SPEALMAN

SENIOR QUALITY ENGINEER

calvin.spealman at redhat.com<mailto:calvin.spealman at redhat.com>  M: +1.336.210.5107<tel:+1.336.210.5107>

[https://red.ht/sig]<https://red.ht/sig>
TRIED. TESTED. TRUSTED.<https://redhat.com/trusted>


From antoon.pardon at vub.be  Mon Jan 17 15:19:36 2022
From: antoon.pardon at vub.be (Antoon Pardon)
Date: Mon, 17 Jan 2022 21:19:36 +0100
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <AS8PR06MB7592F3EBB0AE19DF68FD5B56AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
 <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com>
 <AS8PR06MB7592F3EBB0AE19DF68FD5B56AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
Message-ID: <756e2ab2-e368-5316-1939-6ae4831f7ae9@vub.be>

You could try miniconda.

Op 17/01/2022 om 20:53 schreef Sina Mobasheri:
> Consider scenario that I want run python 3.10 in CentOS 8, I think last python version in CentOS repository is 3.6, if I use epel I can get 3.8 so ..., I think (correct me if I'm wrong ??) the only way that I can run python 3.10 is to compile it manually, which is need to know what dependencies python needs for compilation ... (different distribution different packages, which packages for what, you can see that it is intimidating for beginners like me)
> It's useful to just use wget <python.org/ftp/python/3.10.1/compressed. tar.xz>, unzipped, ser path and ta-da you have cpython 3.10 in CentOS 8
>
> This is Linux specific use case that I can think of, I'm sure there are plenty for windows, consider scenario that I wrote script for scraping some site and entered in some excel worksheet I can simply ship cpython with my script to clients machine and there is no need that client install cpython by himself...
>
> Sorry about my bad grammar
> ________________________________
> From: Calvin Spealman <cspealma at redhat.com>
> Sent: Monday, January 17, 2022, 22:53
> To: Sina Mobasheri
> Cc: python-list at python.org
> Subject: Re: Why There Is No Python Compressed Archive or Binaries ?
>
> Well, on its own, I'd say the reason we don't have such a download is that it wouldn't be very useful.
>
> On Mon, Jan 17, 2022 at 2:08 PM Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>> wrote:
> It's cool project definitely something that I'm personally interested about, but I talking about compressed archive of cpython that we can simply unzipped and starting developing an app, not running  an app that already developed... ?
> ________________________________
> From: Calvin Spealman <cspealma at redhat.com<mailto:cspealma at redhat.com>>
> Sent: Monday, January 17, 2022 10:19:13 PM
> To: Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>>
> Cc: python-list at python.org<mailto:python-list at python.org> <python-list at python.org<mailto:python-list at python.org>>
> Subject: Re: Why There Is No Python Compressed Archive or Binaries ?
>
> I maintain a small project that provides this, a drop-in Python runtime you can ship without installation called Feet. Get it? It makes Python run.
>
> https://github.com/ironfroggy/feet
>
> On Mon, Jan 17, 2022 at 11:16 AM Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>> wrote:
> Java offers download JDK as Compressed Archive<https://www.oracle.com/java/technologies/downloads/#jdk17-linux> or NodeJS offers download Node as Binaries<https://nodejs.org/en/download/current/> both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working
>
> I'm aware that Python also have something called Embedded Zip<https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip> for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offers
>
> My question is why Python hasn't option for downloading as Compressed Archive ?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
>
> --
>
> CALVIN SPEALMAN
>
> SENIOR QUALITY ENGINEER
>
> calvin.spealman at redhat.com<mailto:calvin.spealman at redhat.com>  M: +1.336.210.5107<tel:+1.336.210.5107>
>
> [https://red.ht/sig]<https://red.ht/sig>
> TRIED. TESTED. TRUSTED.<https://redhat.com/trusted>
>
>
> --
>
> CALVIN SPEALMAN
>
> SENIOR QUALITY ENGINEER
>
> calvin.spealman at redhat.com<mailto:calvin.spealman at redhat.com>  M: +1.336.210.5107<tel:+1.336.210.5107>
>
> [https://red.ht/sig]<https://red.ht/sig>
> TRIED. TESTED. TRUSTED.<https://redhat.com/trusted>
>


From sinamobasheri at outlook.com  Mon Jan 17 15:35:05 2022
From: sinamobasheri at outlook.com (Sina Mobasheri)
Date: Mon, 17 Jan 2022 20:35:05 +0000
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <756e2ab2-e368-5316-1939-6ae4831f7ae9@vub.be>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
 <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com>
 <AS8PR06MB7592F3EBB0AE19DF68FD5B56AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <756e2ab2-e368-5316-1939-6ae4831f7ae9@vub.be>
Message-ID: <AS8PR06MB75921A2918A6F2B2C86BC830AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>

Yes? sure, actually I can continue working and developing with python without this feature no problem but it's something that I like and I'm just curious about it, about why Python doesn't implement this kind of installation (most languages do, Java, NodeJS, Deno, PHP, Go, DotNet) is it because some limitations (technically or politically) or maybe in future they would do that... ?

________________________________
From: Python-list <python-list-bounces+sinamobasheri=outlook.com at python.org> on behalf of Antoon Pardon <antoon.pardon at vub.be>
Sent: Monday, January 17, 2022, 23:51
To: python-list at python.org
Subject: Re: Why There Is No Python Compressed Archive or Binaries ?

You could try miniconda.

Op 17/01/2022 om 20:53 schreef Sina Mobasheri:
> Consider scenario that I want run python 3.10 in CentOS 8, I think last python version in CentOS repository is 3.6, if I use epel I can get 3.8 so ..., I think (correct me if I'm wrong ??) the only way that I can run python 3.10 is to compile it manually, which is need to know what dependencies python needs for compilation ... (different distribution different packages, which packages for what, you can see that it is intimidating for beginners like me)
> It's useful to just use wget <python.org/ftp/python/3.10.1/compressed. tar.xz>, unzipped, ser path and ta-da you have cpython 3.10 in CentOS 8
>
> This is Linux specific use case that I can think of, I'm sure there are plenty for windows, consider scenario that I wrote script for scraping some site and entered in some excel worksheet I can simply ship cpython with my script to clients machine and there is no need that client install cpython by himself...
>
> Sorry about my bad grammar
> ________________________________
> From: Calvin Spealman <cspealma at redhat.com>
> Sent: Monday, January 17, 2022, 22:53
> To: Sina Mobasheri
> Cc: python-list at python.org
> Subject: Re: Why There Is No Python Compressed Archive or Binaries ?
>
> Well, on its own, I'd say the reason we don't have such a download is that it wouldn't be very useful.
>
> On Mon, Jan 17, 2022 at 2:08 PM Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>> wrote:
> It's cool project definitely something that I'm personally interested about, but I talking about compressed archive of cpython that we can simply unzipped and starting developing an app, not running  an app that already developed... ?
> ________________________________
> From: Calvin Spealman <cspealma at redhat.com<mailto:cspealma at redhat.com>>
> Sent: Monday, January 17, 2022 10:19:13 PM
> To: Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>>
> Cc: python-list at python.org<mailto:python-list at python.org> <python-list at python.org<mailto:python-list at python.org>>
> Subject: Re: Why There Is No Python Compressed Archive or Binaries ?
>
> I maintain a small project that provides this, a drop-in Python runtime you can ship without installation called Feet. Get it? It makes Python run.
>
> https://github.com/ironfroggy/feet
>
> On Mon, Jan 17, 2022 at 11:16 AM Sina Mobasheri <sinamobasheri at outlook.com<mailto:sinamobasheri at outlook.com>> wrote:
> Java offers download JDK as Compressed Archive<https://www.oracle.com/java/technologies/downloads/#jdk17-linux> or NodeJS offers download Node as Binaries<https://nodejs.org/en/download/current/> both give us a compressed file for Linux and windows that we can just unzipped it and put in a custom directory and set some environment variables and start working
>
> I'm aware that Python also have something called Embedded Zip<https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip> for Windows and nothing like that for Linux as far as I know, and I think this Embedded Zip is not something that user wants to work with that directly it's for embedding in a C++ application, so it's not the same as options that Java and NodeJS offers
>
> My question is why Python hasn't option for downloading as Compressed Archive ?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>
>
> --
>
> CALVIN SPEALMAN
>
> SENIOR QUALITY ENGINEER
>
> calvin.spealman at redhat.com<mailto:calvin.spealman at redhat.com>  M: +1.336.210.5107<tel:+1.336.210.5107>
>
> [https://red.ht/sig]<https://red.ht/sig>
> TRIED. TESTED. TRUSTED.<https://redhat.com/trusted>
>
>
> --
>
> CALVIN SPEALMAN
>
> SENIOR QUALITY ENGINEER
>
> calvin.spealman at redhat.com<mailto:calvin.spealman at redhat.com>  M: +1.336.210.5107<tel:+1.336.210.5107>
>
> [https://red.ht/sig]<https://red.ht/sig>
> TRIED. TESTED. TRUSTED.<https://redhat.com/trusted>
>

--
https://mail.python.org/mailman/listinfo/python-list


From wlfraed at ix.netcom.com  Mon Jan 17 15:39:25 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Mon, 17 Jan 2022 15:39:25 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
Message-ID: <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>

On Mon, 17 Jan 2022 01:55:15 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:


>This is due to the time factor

	And that opens another can of worms... Before this "assignment" was
given to you, surely someone made some sort of estimate of how long it
would take to produce, taking into account your (apparent) lack of
familiarity with any of the technology required.

	Python, Flask, and web-hosting (on Heroku) all appear to be a result of
stumbling onto a "Hello World" tutorial that does no persist ant data
storage and relies upon a free/hobbyist level account on the hosting
provider -- an obvious flag that this is not a well thought out project for
some actual customer/employer (it might qualify as an internship assignment
at some school looking to automate the reservation/check-out of text
books). The lack of understanding in how Python dictionaries are
manipulated doesn't leave one impressed either...

	In one respect, given the limited functionality stated, one gets the
impression of a class GROUP assignment, in which the individual functions
were divvied up to each group member to work, with pretty much no analysis
of suitable data storage, etc. (You only gave some sort of CONOPS for
"users" to "reserve" books; no CONOPS for unreserving books, an implication
that after reserving books the user will go to some desk [in library, in
store {unlikely if there is a return date for books}, some warehouse] and
receive a copy of the reserved book -- but no CONOPS for how this deck
validates that the user has reserved any particular book [no report printed
by user of reservations made during a session, no search function for desk
staff, etc.).

	The Excel requirement comes across as "this is how we've been doing it,
and we have a lot of data in the file we don't want to lose". In other
words -- one should have defined a migration path to replace the Excel
spreadsheet with something suited to a multi-user and/or web-based
application. That, or someone in the group has squashed any discussion on
data storage to force their biased viewpoint. (I've been on the fringe of
one such: in a department with over 80 VAX FORTRAN-77 programmers [some of
us quite skilled in working the internals of the language] and some 20-30
PDP Macro-11 programmers, the Macro-11 group was tasked with porting their
application to VAX systems. One person held out for the port to be done in
VAX Pascal "because colleges were teaching Pascal" at the time [TurboPascal
in all likelihood] totally ignoring the massive F-77 expertise available;
he also is rumored to have threatened to leave if Pascal was not chosen and
management caved-in. Even though VAX Pascal was much different from
TurboPascal [no development environment, though it did support separate
compilation and linking], it did not have advanced math
operations/functions -- just the ones found in Jensen&Wirth, try doing
orbital computations with a subset of trig functions -- requiring
programmers to import the F-77 runtime library for the missing functions. A
few years later the manager confessed that submitting to Pascal was a
mistake -- it didn't take long for the VAX Pascal project to be ported to
HPUX workstations using C and X-window system).


	I also note that during the course of this thread, the spreadsheet
appears to have gone from a somewhat logical "one-row per title"

TITLE	|	AUTHORs	|	INVENTORY		|	AVAILABLE	...
A Book		Someguy		10					9			..
Book 2		Who Dat		100					100			...
Popular		Madonna		30					1			...

to the illogical "one-column per title"

TITLE		A Book			Book 2			Popular
AUTHORs	Someguy		Who Dat		Madonna
INVENTORY	10				100				30
AVAILABLE	9				100				1
...

Consider that Excel only supports some 16K columns, but 1M rows, and
expanding each column to show, say a 200 character title (there are some
long book titles in the world) results in only two or three books being
visible at a time... And there is no "side scroll" page key. In contrast
the first layout would only need one column to be expanded, the rest can be
narrower, and one can use page-up/page-down to scroll through a lot of
books at a time. Oh, and while a cell can hold some 4K text -- Excel itself
can only display the first 255 of those. That's going to put a limit on how
many names you can fit in the reserved-by column (about 10 25char names)
and easily find while scrolling. More than that and you have use "Find"
operations from the menu.

Heroku-Specific	note: a recent web-page I encountered searching for
information for a different question indicates that Heroku does not support
SQLite3 and, by extension, ANY file-based dynamic data storage (so, no
Excel files either). An Heroku Python "dyno" [virtualized work container]
is initialized using a requirements file -- that file basically lists all
the 3rd party Python modules that need to be installed using "pip". If the
"dyno" is stopped (application shut down to upload modifications, or maybe
just to remove an application that has been idle for some hours) it, and
any data files, are just deleted -- when someone tries to connect to the
app again, a new "dyno" is spun up and loaded from scratch. One is supposed
to use one of the client server databases provided by Heroku for persist
ant storage (since those are not inside "dyno" container -- the app makes a
network connection to the database server.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From barry at barrys-emacs.org  Mon Jan 17 15:59:14 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Mon, 17 Jan 2022 20:59:14 +0000
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <AS8PR06MB7592F3EBB0AE19DF68FD5B56AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
 <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com>
 <AS8PR06MB7592F3EBB0AE19DF68FD5B56AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
Message-ID: <0A3DF885-DF64-4D7A-A33C-83BE6C315415@barrys-emacs.org>



> On 17 Jan 2022, at 19:53, Sina Mobasheri <sinamobasheri at outlook.com> wrote:
> 
> Consider scenario that I want run python 3.10 in CentOS 8, I think last python version in CentOS repository is 3.6, if I use epel I can get 3.8 so ..., I think (correct me if I'm wrong ??) the only way that I can run python 3.10 is to compile it manually, which is need to know what dependencies python needs for compilation ... (different distribution different packages, which packages for what, you can see that it is intimidating for beginners like me)
> It's useful to just use wget <python.org/ftp/python/3.10.1/compressed. tar.xz>, unzipped, ser path and ta-da you have cpython 3.10 in CentOS 8

Are there features in python 3.10 that are missing from 3.6 that you need?
If not then the easy thing to do is use 3.6.

You could build a docker containers you based on fedora 35 that has the python 3.10
and the python libs that you need installed and use that.

You would use podman and its tools to build and run the container on Centos 8.

For my work I'm planning to use 3.6 on Centos 8 as that is one thing I can avoid packaging
and maintaining myself.

Barry


From grant.b.edwards at gmail.com  Mon Jan 17 16:13:05 2022
From: grant.b.edwards at gmail.com (Grant Edwards)
Date: Mon, 17 Jan 2022 13:13:05 -0800 (PST)
Subject: Why There Is No Python Compressed Archive or Binaries ?
References: =?utf-8?q?=3CAS8PR06MB759290041865856ECF07F317AD579=40AS8PR06MB7?=
 =?utf-8?q?592=2Eeurprd06=2Eprod=2Eoutlook=2Ecom=3E?=
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com> 
 =?utf-8?q?=3CAS8PR06MB75922996F15074A6D438B687AD579=40AS8PR06MB7592=2Eeurpr?=
 =?utf-8?q?d06=2Eprod=2Eoutlook=2Ecom=3E?=
 <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com> 
 =?utf-8?q?=3CAS8PR06MB7592F3EBB0AE19DF68FD5B56AD579=40AS8PR06MB7592=2Eeurpr?=
 =?utf-8?q?d06=2Eprod=2Eoutlook=2Ecom=3E?=
 <756e2ab2-e368-5316-1939-6ae4831f7ae9@vub.be> =?utf-8?q?=3CAS8PR06MB75921A2?=
 =?utf-8?q?918A6F2B2C86BC830AD579=40AS8PR06MB7592=2Eeurprd06=2Eprod=2Eoutloo?=
 =?utf-8?q?k=2Ecom=3E?=
Message-ID: <61e5dbe1.1c69fb81.a34b3.8478@mx.google.com>

On 2022-01-17, Sina Mobasheri <sinamobasheri at outlook.com> wrote:

> Yes sure, actually I can continue working and developing with python
> without this feature no problem but it's something that I like and
> I'm just curious about it, about why Python doesn't implement this
> kind of installation

You talk about "Python" implementing something. Python is a language.

If what you want hasn't been implmented, it's because there haven't
been any _people_ who have wanted it enough to do it. I spent 90
seconds googling and found that what you wanted has been implemented a
couple times for Windows. There was "Portable Python," which appears
to have been abandonded.

There's also WinPython <http://winpython.github.io/> which seems to be
active.  AFAICT, you just unzip it and run it (nothing needs to be
"installed"). It says you can even move that directory to another
machine and run it there if you want.

Both of those were for Windows.

It's probably never been done for Linux because Linux distros pretty
much all come with Python already installed by default, and it's
usually trivial to install alternative versions as well (and keep them
all updated) via whatever package manager the Distro uses.

--
Grant

From cs at cskk.id.au  Mon Jan 17 16:37:56 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 18 Jan 2022 08:37:56 +1100
Subject: How to set environmental variables for Python
In-Reply-To: <CA+i5JwZr35PfzmYw4pE97CRY2BuLT8eFd5jRrGo5azN_ZcwxDg@mail.gmail.com>
References: <CA+i5JwZr35PfzmYw4pE97CRY2BuLT8eFd5jRrGo5azN_ZcwxDg@mail.gmail.com>
Message-ID: <YeXhtO9nywHdRl6h@cskk.homeip.net>

On 17Jan2022 11:36, Shaozhong SHI <shishaozhong at gmail.com> wrote:
>Set Operation System but not disturbing existing setting.  Only to add at
>the command line.

If you mean: "set on the command line so that I run some script using 
Python 3.6.1", usually you would just invoke the specific Python 3.6.1 
executable.

You can do that directly, or modify $PATH (UNIX, %path% on Windows?) to 
find that executable first when looking for the "python" (or "python3" 
or "py") command, or use a virtual environment.

The first approach (direct execution) might look like this:

    [~]fleet2*> /usr/local/bin/python3.10
    Python 3.10.0 (v3.10.0:b494f5935c, Oct  4 2021, 14:59:20) [Clang 
    12.0.5 (clang-1205.0.22.11)] on darwin
    Type "help", "copyright", "credits" or "license" for more 
    information.
    >>>

That's on my local Mac, "[~]fleet2*>" is my prompt, and there's a Python 
3.10 installed as /usr/local/bin/python3.10.

The second approach might look like this:

    env PATH=/Library/Frameworks/Python.framework/Versions/3.10/bin:$PATH python

That places the Python 3.10 "bin" directory in my $PATH _ahead_ of all 
other paths, so that "python" is found there first, thus running the 
desired python version:

    [~]fleet2*> env PATH=/Library/Frameworks/Python.framework/Versions/3.10/bin:$PATH python3
    Python 3.10.0 (v3.10.0:b494f5935c, Oct  4 2021, 14:59:20) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>>

That particular long path is an artifact of how Python is installed on 
my Mac. Adjust for your platform.

The third approach is to use a virtual environment, a common approach 
for python development. A virtual environment is a little install 
directory based on a particular python version, where you can install a 
custom set of third party modules. You make one like this:

    /usr/local/bin/python3.10 -m venv venv

That uses the "venv" module from python 3.10 to create a new virtual 
environment in the directory "venv" (in the current directory).

From that point onward you want the virtual env "bin" directory in your 
PATH:

    export PATH=$PWD/venv/bin:$PATH

and thereon, _in that shell_, "python3" will run the python from the 
virtual environment (which uses the python3.10 you used to create the 
venv) and "pip3" will install modules into that virtual environment, not 
disturbing other setups.

Virtualenvs come with an "activate" script whose purpose it to set up 
your current shell to use the environment; they essentially do the 
"export" above and also fiddle your prompt to remind you that you're 
using a particular environment. You don't need to use that - technically 
it is enough to directly invoke the python3 executable from the 
environment. Fiddling $PATH lets other things find that "python3" by 
default.

Cheers,
Cameron Simpson <cs at cskk.id.au>


From rosuav at gmail.com  Mon Jan 17 17:07:27 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 18 Jan 2022 09:07:27 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
References: <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
Message-ID: <CAPTjJmqnzBwP4diBKQZ_OPWDSvze71x-K24dxbO4UD+kB6YzAw@mail.gmail.com>

On Tue, Jan 18, 2022 at 7:44 AM Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> Heroku-Specific note: a recent web-page I encountered searching for
> information for a different question indicates that Heroku does not support
> SQLite3 and, by extension, ANY file-based dynamic data storage (so, no
> Excel files either). An Heroku Python "dyno" [virtualized work container]
> is initialized using a requirements file -- that file basically lists all
> the 3rd party Python modules that need to be installed using "pip". If the
> "dyno" is stopped (application shut down to upload modifications, or maybe
> just to remove an application that has been idle for some hours) it, and
> any data files, are just deleted -- when someone tries to connect to the
> app again, a new "dyno" is spun up and loaded from scratch. One is supposed
> to use one of the client server databases provided by Heroku for persist
> ant storage (since those are not inside "dyno" container -- the app makes a
> network connection to the database server.

This is correct. I've used Heroku for a number of projects, since it's
easy to deploy there and can be done for zero dollars. If you know how
to use PostgreSQL from Python, it's almost trivially easy to deploy
that to Heroku (just need to arrange your credentials, nothing more).
In contrast, manipulating external files is tricky; in fact, if you
want to do something simple like "upload an image to use as your
avatar", it's probably easier to store that in Postrges than on the
file system.

Part of the reason for the "wipe everything on shutdown" policy is
that it makes Heroku apps extremely easy to scale. You want one
process? Done. Want two dynos so you can serve more clients at once?
Easy. But for that to work, you have to make absolutely sure that
everything is stored externally to the dyno itself.

Like everything else, it requires discipline, a set of rules that you
follow. (And it's a lot easier than some - for instance, the
discipline required for a live-code-update system is a lot harder.)
But while it isn't particularly difficult, it does require some
comprehension. Some time investment. Some actual effort. And that's
what the OP might be running into difficulties.

(For what it's worth, we used Heroku at my last teaching job; it's a
great way to ensure that every student is able to deploy projects, and
since it doesn't cost anything to have a bunch of projects up there,
it doesn't hurt to deploy apps that don't have any "real-world use".
Originally we taught them to use MongoDB, which requires external
hosting, but switched that out for PostgreSQL, which is ever so much
better to work with.)

ChrisA

From kvratkin at yandex.ru  Tue Jan 18 01:31:26 2022
From: kvratkin at yandex.ru (Kirill Ratkin)
Date: Tue, 18 Jan 2022 09:31:26 +0300
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <61e5dbe1.1c69fb81.a34b3.8478@mx.google.com>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
 <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com>
 <AS8PR06MB7592F3EBB0AE19DF68FD5B56AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <756e2ab2-e368-5316-1939-6ae4831f7ae9@vub.be>
 <AS8PR06MB75921A2918A6F2B2C86BC830AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <61e5dbe1.1c69fb81.a34b3.8478@mx.google.com>
Message-ID: <b65922e2-fe4b-c212-fba8-e31841d77d71@yandex.ru>

Hi Grant

Hmmm...? definitly you are right in particular solution.

But Ok, let me show example.


I often use Go in parallel with Python and sometimes I switch between 
Windows/Linux also. On both systems I just download Go toolset as 
tarball/zip file and unpack in place where I like.

The point here is Go toolset officially distributed as tarball/zip for 
all supported operating system. This is not PortableGo or WinGo or some 
anther third party Go distribution. This is one of supported way do get 
Go toolset.

On Windows I put toolset on separate drive 'D:\Go' and on linux - 
$HOME/.local/go

And I set up several environment variables (doing 'setx' on Windows and 
edit .profile on linux): GOPATH, GOROOT, GOCACHE, ... and modify my PATH.

Now I can build any Go project. I don't care about which Go compiler was 
set on OS (Linux or Windows) before. I just unpack tarball/zip in place 
where I have permissions and use it.


I take Go just for example. In same way you can unpack and use Java SDK 
and DotNet SDK. All these toolsets have option (provided by vendor) to 
be downloaded as compressed file.


You are absolutely right. It's easy to google and find something like 
winpython. But I'm sure there are reasons why www.python.org doesn't 
provide this.


On 1/18/22 00:13, Grant Edwards wrote:
> On 2022-01-17, Sina Mobasheri <sinamobasheri at outlook.com> wrote:
>
>> Yes sure, actually I can continue working and developing with python
>> without this feature no problem but it's something that I like and
>> I'm just curious about it, about why Python doesn't implement this
>> kind of installation
> You talk about "Python" implementing something. Python is a language.
>
> If what you want hasn't been implmented, it's because there haven't
> been any _people_ who have wanted it enough to do it. I spent 90
> seconds googling and found that what you wanted has been implemented a
> couple times for Windows. There was "Portable Python," which appears
> to have been abandonded.
>
> There's also WinPython <http://winpython.github.io/> which seems to be
> active.  AFAICT, you just unzip it and run it (nothing needs to be
> "installed"). It says you can even move that directory to another
> machine and run it there if you want.
>
> Both of those were for Windows.
>
> It's probably never been done for Linux because Linux distros pretty
> much all come with Python already installed by default, and it's
> usually trivial to install alternative versions as well (and keep them
> all updated) via whatever package manager the Distro uses.
>
> --
> Grant

From eryksun at gmail.com  Tue Jan 18 08:48:30 2022
From: eryksun at gmail.com (Eryk Sun)
Date: Tue, 18 Jan 2022 07:48:30 -0600
Subject: About Python Compressed Archive or Binaries
In-Reply-To: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
References: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
Message-ID: <CACL+1auCff-mYRfuZ5WOm3hHHDH+fvkqv2qbTRXizfQmpQDd=Q@mail.gmail.com>

On 1/17/22, Sina Mobasheri <sinamobasheri at outlook.com> wrote:
>
> I'm aware that Python also have something called Embedded Zip for Windows
> and nothing like that for Linux as far as I know, and I think this Embedded
> Zip is not something that the user wants to work with that directly it's for
> embedding in a C++ application, so it's not the same as options that Java
> and NodeJS offering

Use the NuGet packages:

https://docs.python.org/3/using/windows.html#the-nuget-org-packages

These are zipped archives of Python 3.x.y that can be unpacked and
used anywhere you want.

From christian at python.org  Tue Jan 18 09:58:13 2022
From: christian at python.org (Christian Heimes)
Date: Tue, 18 Jan 2022 15:58:13 +0100
Subject: About Python Compressed Archive or Binaries
In-Reply-To: <dfdb9924-ae2c-c62a-8902-585adc9ab571@yandex.ru>
References: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <dfdb9924-ae2c-c62a-8902-585adc9ab571@yandex.ru>
Message-ID: <261efe71-5460-18a2-7ca9-177333c98d66@python.org>

On 17/01/2022 18.49, Kirill Ratkin wrote:
> It would be nice to have just zip file with python interpreter (not 
> executable installer), unpack it anywhere, add path? to this 'anywhere' 
> to PATH, and use it.
> 
> Java/DotNet/Go have this option. But python - not.
> 
> And question is - why?

Java is developed by Oracle (revenue 49 billion USD)
.NET is developed by Microsoft (revenue 161 billion USD)
Go is developed by Google (revenue of parent company is 182 billion USD)
Python is developed by a bunch of (mostly unpoaid) volunteers under the 
umbrella of the PSF (revenue 3.1 million USD).

Do you see a pattern? :)

A distributable binary format for CPython is high on our list. Brett 
Cannon, Nathaniel Smith, and I have been discussion a proposal recently. 
I landed a bunch of changes in 3.11-dev that are both useful for binary 
distribution and for Web Assembly builds. Because all work is done in 
our free time, there is only little progress.

Christian

From mats at wichmann.us  Tue Jan 18 11:22:26 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Tue, 18 Jan 2022 09:22:26 -0700
Subject: Why There Is No Python Compressed Archive or Binaries ?
In-Reply-To: <b65922e2-fe4b-c212-fba8-e31841d77d71@yandex.ru>
References: <AS8PR06MB759290041865856ECF07F317AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz6na+F4w5hW7PxJLKVtn3EWYXRtYnNTZQV+stXnenF7=A@mail.gmail.com>
 <AS8PR06MB75922996F15074A6D438B687AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <CACo5Rz7ofdxAfpoj9jkbFjOfGDgyuByX+P3dUEetXHTZKJVwsw@mail.gmail.com>
 <AS8PR06MB7592F3EBB0AE19DF68FD5B56AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <756e2ab2-e368-5316-1939-6ae4831f7ae9@vub.be>
 <AS8PR06MB75921A2918A6F2B2C86BC830AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <61e5dbe1.1c69fb81.a34b3.8478@mx.google.com>
 <b65922e2-fe4b-c212-fba8-e31841d77d71@yandex.ru>
Message-ID: <f99d3a37-7d4a-ee9f-3a42-c7c5af239354@wichmann.us>

On 1/17/22 23:31, Kirill Ratkin wrote:
> Hi Grant
> 
> Hmmm...? definitly you are right in particular solution.
> 
> But Ok, let me show example.
> 
> 
> I often use Go in parallel with Python and sometimes I switch between
> Windows/Linux also. On both systems I just download Go toolset as
> tarball/zip file and unpack in place where I like.
> 
> The point here is Go toolset officially distributed as tarball/zip for
> all supported operating system. This is not PortableGo or WinGo or some
> anther third party Go distribution. This is one of supported way do get
> Go toolset.
> 
> On Windows I put toolset on separate drive 'D:\Go' and on linux -
> $HOME/.local/go
> 
> And I set up several environment variables (doing 'setx' on Windows and
> edit .profile on linux): GOPATH, GOROOT, GOCACHE, ... and modify my PATH.
> 
> Now I can build any Go project. I don't care about which Go compiler was
> set on OS (Linux or Windows) before. I just unpack tarball/zip in place
> where I have permissions and use it.


The part of this that is Set up Environment Variables and PATH can be
handled by Python's virtualenvs.  You create one starting from a
possibly local Python layout, or from a system one, and the virtualenv
handles all the "fiddling" so while that env is active, just "python"
and "pip" work for that environment.

A nice way to manage this is through a project called pyenv.  There is
now a Windows port of this so it can work there as well (I've never used
it on WIndows, personally).

So while there may not currently be a zip/tar archive you can unpack and
go, I can get started quite easily on a new version.  Let's say I
decidded I needed to test something on Python 3.7 but my main Python is
already 3.10:

pyenv install -l | grep "3\.7"
... list of available versions that contain 3.7

pyenv install 3.7.12   # sets up 3.7.12 in pyenv's local versions tree
pyenv virtualenv 3.7.12 venv-3712  # create virtualenv using 3.7.12 as base
pyenv activate venv-3712

and now I'm running inside a virtualenv using 3.7.12, which I can now
provision for the work I want to do...


From sinamobasheri at outlook.com  Tue Jan 18 15:19:30 2022
From: sinamobasheri at outlook.com (Sina Mobasheri)
Date: Tue, 18 Jan 2022 20:19:30 +0000
Subject: About Python Compressed Archive or Binaries
In-Reply-To: <261efe71-5460-18a2-7ca9-177333c98d66@python.org>
References: <AS8PR06MB7592DE921AD954FD71909931AD579@AS8PR06MB7592.eurprd06.prod.outlook.com>
 <dfdb9924-ae2c-c62a-8902-585adc9ab571@yandex.ru>
 <261efe71-5460-18a2-7ca9-177333c98d66@python.org>
Message-ID: <AS8PR06MB759217F7F5E186536D322D2AAD589@AS8PR06MB7592.eurprd06.prod.outlook.com>

Thanks I really appreciate that, its time consuming task and take lots of hard work for sure, I was thinking it's technical issue for python that there isn't binary format for it, so I'm happy to hear that we will have that option in the future ???
________________________________
From: Python-list <python-list-bounces+sinamobasheri=outlook.com at python.org> on behalf of Christian Heimes <christian at python.org>
Sent: Tuesday, January 18, 2022, 18:29
To: python-list at python.org
Subject: Re: About Python Compressed Archive or Binaries

On 17/01/2022 18.49, Kirill Ratkin wrote:
> It would be nice to have just zip file with python interpreter (not
> executable installer), unpack it anywhere, add path  to this 'anywhere'
> to PATH, and use it.
>
> Java/DotNet/Go have this option. But python - not.
>
> And question is - why?

Java is developed by Oracle (revenue 49 billion USD)
.NET is developed by Microsoft (revenue 161 billion USD)
Go is developed by Google (revenue of parent company is 182 billion USD)
Python is developed by a bunch of (mostly unpoaid) volunteers under the
umbrella of the PSF (revenue 3.1 million USD).

Do you see a pattern? :)

A distributable binary format for CPython is high on our list. Brett
Cannon, Nathaniel Smith, and I have been discussion a proposal recently.
I landed a bunch of changes in 3.11-dev that are both useful for binary
distribution and for Web Assembly builds. Because all work is done in
our free time, there is only little progress.

Christian
--
https://mail.python.org/mailman/listinfo/python-list


From renda.saptoe at yahoo.com  Tue Jan 18 10:15:43 2022
From: renda.saptoe at yahoo.com (Renda Saptoe)
Date: Tue, 18 Jan 2022 15:15:43 +0000 (UTC)
Subject: Trouble downloading Python
References: <1624759430.794324.1642518943442.ref@mail.yahoo.com>
Message-ID: <1624759430.794324.1642518943442@mail.yahoo.com>

Good day,
I am experiencing issues trying to download Python. I would please need some assistance to help download the progam to my laptop.

Kind regards
Renda



From narshad.380 at gmail.com  Tue Jan 18 10:37:07 2022
From: narshad.380 at gmail.com (NArshad)
Date: Tue, 18 Jan 2022 07:37:07 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <mailman.217.1642457260.3079.python-list@python.org>
References: <mailman.165.1642023749.3079.python-list@python.org>
 <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <CAPTjJmqnzBwP4diBKQZ_OPWDSvze71x-K24dxbO4UD+kB6YzAw@mail.gmail.com>
 <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
Message-ID: <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>

Avi Gross:

What does the website "https://mail.python.org/mailman/listinfo/python-list" do? 

Can I use this for the discussions which I require?

From wlfraed at ix.netcom.com  Tue Jan 18 14:44:06 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Tue, 18 Jan 2022 14:44:06 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <CAPTjJmqnzBwP4diBKQZ_OPWDSvze71x-K24dxbO4UD+kB6YzAw@mail.gmail.com>
 <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
Message-ID: <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>

On Tue, 18 Jan 2022 07:37:07 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>Avi Gross:
>

	Not Avi Gross, but that is partly because you replied to Chris
Angelico, who was replying to my post replying to an earlier one of
yours...

>What does the website "https://mail.python.org/mailman/listinfo/python-list" do? 
>
>Can I use this for the discussions which I require?

	It is most likely the same forum...

	The Python mailing list (which does get spam filtered, unlike the
Usenet newsgroup, so doesn't see as much junk injected via Google) is
gatewayed with Usenet comp.lang.python. comp.lang.python is what Google
gateways for its Python group. Anything posted on the Google Python group
is seen by comp.lang.python, and from there gets seen by the Python mailing
list (and things go the other way also). The only difference is message
management -- email vs news reader vs whatever Google's interface of the
week inflicts, and how much spam comes through. For me, mailing lists take
too much management to set up filters to file mailing list traffic to a
special mail box (and then if I need other filters, to ensure they apply
before or after the mail box filing) whereas a decent news reader
automatically files messages by their group.









	And you really need to find a client that follows (or properly use the
one you have) email/netnews /threading/ and attribution conventions. Avi
Gross's post was on a different path (there was a three-way split from your
earlier post).

	If you are going to be explicitly asking questions of a person, that
post should be a follow-up (reply) to the nearest relevant post made by
that person -- not just tacked onto the end of the posts that came in
during the day with all content removed and ad hoc comments inserted.



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From wlfraed at ix.netcom.com  Tue Jan 18 16:17:56 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Tue, 18 Jan 2022 16:17:56 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <461278e6-8684-4d63-a053-45f47ede4840n@googlegroups.com>
 <fnv0ugt1noot6kr883fl3e32a9nnq51a65@4ax.com>
 <18305827.21103.1642115299530@mail.yahoo.com>
 <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
Message-ID: <ed6eug9htni0ridn20shh6qmjctpv80qo1@4ax.com>


	o/~		talking to myself in public	o/~

On Mon, 17 Jan 2022 15:39:25 -0500, Dennis Lee Bieber
<wlfraed at ix.netcom.com> declaimed the following:


>	In one respect, given the limited functionality stated, one gets the
>impression of a class GROUP assignment, in which the individual functions
>were divvied up to each group member to work, with pretty much no analysis
>of suitable data storage, etc. (You only gave some sort of CONOPS for
>"users" to "reserve" books; no CONOPS for unreserving books, an implication
>that after reserving books the user will go to some desk [in library, in
>store {unlikely if there is a return date for books}, some warehouse] and
>receive a copy of the reserved book -- but no CONOPS for how this deck
>validates that the user has reserved any particular book [no report printed
>by user of reservations made during a session, no search function for desk
>staff, etc.).

	I'm in a fey mood today... The OP (and the rest of you reading) will
likely just skip this message... It's a rambling free-association harangue
on a possible RDBM schema -- which makes a few assumptions (described
following) and looks wildly different from the OP's vague data...

	Inferred from various posts is that this
		*	NOT a library (which may have at most 5 copies of popular
books)
		*	NOT a book store (which would not have check-out/check-in
dates)

	The best use case I can see is for something like a high school in
which textbooks are issued to students at the start of the course, and
collected at the end. For a small high school, say 600 students, there
might be 150 taking, say 3rd year Trigonometry, split among 6 session of 25
students each day. This justifies not tracking every copy of a book by full
title, et al.

	I'm going to take that as the overall system CONOPS.

	Needed data: book title, book author(s), ISBN and/or other call number
(some texts may be old enough to predate ISBN), # copies, # available,
dates for check-out and return, student name. Probably more as I go down...

	Schema format notation:

		relation(attribute list)
			special constraints description

where 
	*attribute* is a non-null, unique index, primary key (preferably
autonumber)
	_attribute_ is a non-null, duplicates allowed index, foreign key (in
the form relation-attribute)
	/attribute/ is an optional/alternate indexed attribute

Publisher(*ID*, name)

Book(*ID*, ISBN, alt-call, title, _Publisher-ID_, copyright-date)
	constraint(ISBN not null OR alt-call not null)

Author(*ID*, /last-name/, first-name)
	constraint(last-name not null)

Book-Author(*ID*, _Book-ID_, _Author-ID_)
	{this links authors to books, and allows for books to have more than
one author}

Copy(*ID*, _Book-ID_, copy-number, status, condition)
	constraint(status in "A", "O", "R", status not null, default "A")
	constraint(copy-number not null, [Book-ID copy-number] is unique)
		{A = available, O = checked out, R = reserved but not yet checked
out}
		{condition contains notes on known damage, etc. for this copy}

Student(*ID*, /last-name/, first-name)
	constraint(last-name not null)
		{I expect this to expand with student ID number, home address,
etc.)

CheckOut(*ID*, _Copy-ID_, _Student-ID_, reservation-date, checkout-date,
due-date, active)
	constraint([Copy-ID Student-ID] is unique)
	constraint(active not null boolean default true)
		{can automate cancelling reservations after n-days, and can also be
used to produce a list of past-due}
		{active is used to allow for history of check outs, set to false
when book is returned (and hence made Available again)

	SEVEN relations (aka "tables") in an RDBM. While each book /title/
appears only once, it does take a small record to identify each copy of the
book and track availability of the copy. Reserving a book requires
transaction to select the first copy with status = "A" for that title,
update status to "R", insert a checkout record with the currently selected
student and current date for reservation date. When actually given the
book, a transaction to find the active checkout record for the student, for
that title is done, the checkout and due dates are set, the copy record is
updated for status = "O". When returned, a transaction is done to locate
the checkout record, update active to false, update Copy record to set
status to "A", and optionally update the condition field to indicate
damages inflicted by the student while it was checked out. If a reserve
operation fails (another session snagged the same copy record and updated
it) one just repeats the transaction trying for the next available copy
record -- only if there are no copies with status "A" do you have to abort
and tell the student they can't have the book.

	There is no "count" of copies, no "count" of available -- these can be
computed based upon SQL cnt() functions; untested

	select b.title as Title, 
		cnt(cpy1.copy_number) as Total, 
		cnt(cpy2.copy_number) as Available
	from Book as b 
	inner join Copy as cpy1 
		on b.ID = cpy1.Book-ID
	left join Copy as cpy2
		on cpy1.ID = cpy2.ID and cpy2.status = "A"
	group by b.title
	order by b.title;


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From wlfraed at ix.netcom.com  Tue Jan 18 16:49:36 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Tue, 18 Jan 2022 16:49:36 -0500
Subject: Trouble downloading Python
References: <1624759430.794324.1642518943442.ref@mail.yahoo.com>
 <1624759430.794324.1642518943442@mail.yahoo.com>
 <mailman.223.1642539026.3079.python-list@python.org>
Message-ID: <pbceugdvmfshdpv6e3o823qai6ui1lfbav@4ax.com>

On Tue, 18 Jan 2022 15:15:43 +0000 (UTC), Renda Saptoe
<renda.saptoe at yahoo.com> declaimed the following:

>Good day,
>I am experiencing issues trying to download Python. I would please need some assistance to help download the progam to my laptop.

	Insufficient information (you've given the equivalent of "my arm hurts"
without stating which arm, where it hurts, and the nature of the pain; a
big difference between "it was caught in a door jamb" and "a chainsaw cut
it off").

	Where are you downloading from (the URL might be of use), which version
(if there are more than one version at that URL). WHAT OS! For Linux it is
often easiest to just use the distribution package manager to
install/upgrade Python.

	What happens when you run the downloaded file? How are you trying to
run Python?

	What happens if you open up a command shell and enter just Python (or,
for safety python3, since you shouldn't even be looking at python 2.x but
some Linux distributions still use 2.x for there own uses)

-=-=-=-
Microsoft Windows [Version 10.0.19041.1415]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Wulfraed>python3
Python ActivePython 3.8.2 (ActiveState Software Inc.) based on
 on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

wulfraed at ElusiveUnicorn:~$ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
-=-=-=-

	Python is not a development environment. It is a compiler/interpreter
for files containing Python source code, normally run from a command shell.
Many installations do include a Tkinter-based program called IDLE which can
be run if one really needs an IDE -- though there are many third-party IDEs
for Python which may be better.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From avigross at verizon.net  Tue Jan 18 23:17:28 2022
From: avigross at verizon.net (Avi Gross)
Date: Wed, 19 Jan 2022 04:17:28 +0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
References: <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <CAPTjJmqnzBwP4diBKQZ_OPWDSvze71x-K24dxbO4UD+kB6YzAw@mail.gmail.com>
 <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
Message-ID: <1845217957.793613.1642565848222@mail.yahoo.com>

I do not manage any python lists or have any say in how they run so I have no idea why I am being asked by name below, as Dennis pointed out.
So I won't reply on whatever I am being asked, but want to point out that many forums may be asked questions and some people on the forum will not respond or will not accept a user that bombards with too many questions or requests for more detailed answers and especially when not given enough but appropriate information.
Not everything in life is free. Python as a free language is but the expertise in ways to solve specific problems using Python, let alone EXCEL, that you want, often is not.
Simple requests like how to read in data from a format like .CSV or a tab in .XLSX files can be easily answered, of course. But correcting what sounds like a horrible data storage without a redesign is often not of much interest to others.
I think this group has already spent way too much time on whatever this issue is and provided lots of useful advice which apparently does not get taken. So don't pull me in again. I have moved on.


-----Original Message-----
From: Dennis Lee Bieber <wlfraed at ix.netcom.com>
To: python-list at python.org
Sent: Tue, Jan 18, 2022 2:44 pm
Subject: Re: What to write or search on github to get the code for what is written below:

On Tue, 18 Jan 2022 07:37:07 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>Avi Gross:
>

??? Not Avi Gross, but that is partly because you replied to Chris
Angelico, who was replying to my post replying to an earlier one of
yours...

>What does the website "https://mail.python.org/mailman/listinfo/python-list" do? 
>
>Can I use this for the discussions which I require?

??? It is most likely the same forum...

??? The Python mailing list (which does get spam filtered, unlike the
Usenet newsgroup, so doesn't see as much junk injected via Google) is
gatewayed with Usenet comp.lang.python. comp.lang.python is what Google
gateways for its Python group. Anything posted on the Google Python group
is seen by comp.lang.python, and from there gets seen by the Python mailing
list (and things go the other way also). The only difference is message
management -- email vs news reader vs whatever Google's interface of the
week inflicts, and how much spam comes through. For me, mailing lists take
too much management to set up filters to file mailing list traffic to a
special mail box (and then if I need other filters, to ensure they apply
before or after the mail box filing) whereas a decent news reader
automatically files messages by their group.









??? And you really need to find a client that follows (or properly use the
one you have) email/netnews /threading/ and attribution conventions. Avi
Gross's post was on a different path (there was a three-way split from your
earlier post).

??? If you are going to be explicitly asking questions of a person, that
post should be a follow-up (reply) to the nearest relevant post made by
that person -- not just tacked onto the end of the posts that came in
during the day with all content removed and ad hoc comments inserted.



-- 
??? Wulfraed? ? ? ? ? ? ? ? Dennis Lee Bieber? ? ? ? AF6VN
??? wlfraed at ix.netcom.com? ? http://wlfraed.microdiversity.freeddns.org/
-- 
https://mail.python.org/mailman/listinfo/python-list

From tony.flury at btinternet.com  Wed Jan 19 05:57:30 2022
From: tony.flury at btinternet.com (Tony Flury)
Date: Wed, 19 Jan 2022 10:57:30 +0000
Subject: Puzzling behaviour of Py_IncRef
Message-ID: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>

I am writing a C extension module for an AVL tree, and I am trying to 
ensure reference counting is done correctly. I was having a problem with 
the reference counting so I worked up this little POC of the problem, 
and I hope someone can explain this.

Extension function :

    static PyObject *_Node_test_ref_count(PyObject *self)
    {
     ??? printf("\nIncrementing ref count for self - just for the hell
    of it\n");
     ??? printf("\n before self has a ref count of %ld\n", Py_REFCNT(self));
     ??? Py_INCREF(self);
     ??? printf("\n after self has a ref count of %ld\n", Py_REFCNT(self));
     ??? fflush(stdout);
     ??? return self;
    }

As you can see this function purely increments the reference count of 
the instance.

/Note: I understand normally this would be the wrong this to do, but 
this is a POC of the issue, not live code. In the live code I am 
attaching a 2nd nodes to each other, and the live code therefore 
increments the ref-count for both objects - so even if the Python code 
deletes it's reference the reference count for the instance should still 
be 1 in order to ensure it doesn't get garbage collected./

This function is exposed as the test_ref method.

This is the test case :

 ??? def test_000_009_test_ref_count(self):
 ??????? node = _Node("Hello")
 ??????? self.assertEqual(sys.getrefcount(node), 2)
 ??????? node.test_ref()
 ??????? self.assertEqual(sys.getrefcount(node), 3)

The output of this test case is :

test_000_009_test_ref_count (__main__.TestNode) ...
Incrementing ref count for self - just for the hell of it

 ?before self has a ref count of 2

 ?after self has a ref count of 3
FAIL

======================================================================
FAIL: test_000_009_test_ref_count (__main__.TestNode)
----------------------------------------------------------------------
Traceback (most recent call last):
 ? File 
"/home/tony/Development/python/orderedtree/tests/test_orderedtree.py", 
line 62, in test_000_009_test_ref_count
 ??? self.assertEqual(sys.getrefcount(node), 3)
AssertionError: 2 != 3

So I understand why the first assert will be true - when the 
sys.getrefcount() function is called the ref count is incremented 
temporarily (as a borrowed reference), so there are now two references - 
the 'node' variable, and the borrowed reference in the function call.

We then call the 'test_ref' method, and again that call causes a 
borrowed reference (hence the ref count being 2 initially within the 
method). The 'test_ref' method increments the reference of the instance 
- as you can see from the output we now have a ref count of 3 - (that 
count is the 'node' variable in the test case, the borrowed reference 
due to the method call, and the artificial increment from the 'ref_test' 
method).

When the 'ref_test' method exits I would expect the ref count of the 
instance to now be 2 (one for the 'node' variable, and one as a result 
of the artificial increment increment').

I would therefore expect the 2nd assertEqual in the test case to 
succeed. - in this case the borrowed reference within sys.getfrefcount() 
should cause the count to be 3.

As you see though that 2nd assertEqual fails - suggesting that the 
refcount of 'node' is actually only 1 when the 'test_ref' method exits.

Can someone explain why the 'test_ref' method fails to change the 
refcount of the 'node' instance.


From rosuav at gmail.com  Wed Jan 19 06:09:45 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 19 Jan 2022 22:09:45 +1100
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
References: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
Message-ID: <CAPTjJmq==4CHdPU2GiXefPG+OUG-jA6LRqWdxDxkdzjdTccANw@mail.gmail.com>

On Wed, Jan 19, 2022 at 10:00 PM Tony Flury via Python-list
<python-list at python.org> wrote:
> Extension function :
>
>     static PyObject *_Node_test_ref_count(PyObject *self)
>     {
>          printf("\nIncrementing ref count for self - just for the hell
>     of it\n");
>          printf("\n before self has a ref count of %ld\n", Py_REFCNT(self));
>          Py_INCREF(self);
>          printf("\n after self has a ref count of %ld\n", Py_REFCNT(self));
>          fflush(stdout);

At this point, the refcount has indeed been increased.

>          return self;
>     }

And then you say "my return value is this object".

The normal thing to do is to add a reference to whatever you're
returning. For instance, Py_RETURN_NONE will incref None and then
return it.

So you're incrementing the refcount, then returning it without
incrementing the refcount. Your code is actually equivalent to "return
self".

In order to actually leak a reference, you'd need to incref it twice.

ChrisA

From tony.flury at btinternet.com  Wed Jan 19 09:22:10 2022
From: tony.flury at btinternet.com (Tony Flury)
Date: Wed, 19 Jan 2022 14:22:10 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <CAPTjJmq==4CHdPU2GiXefPG+OUG-jA6LRqWdxDxkdzjdTccANw@mail.gmail.com>
References: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
 <CAPTjJmq==4CHdPU2GiXefPG+OUG-jA6LRqWdxDxkdzjdTccANw@mail.gmail.com>
Message-ID: <e640d910-50e6-7485-db97-e1a7b62339e0@btinternet.com>


On 19/01/2022 11:09, Chris Angelico wrote:
> On Wed, Jan 19, 2022 at 10:00 PM Tony Flury via Python-list
> <python-list at python.org> wrote:
>> Extension function :
>>
>>      static PyObject *_Node_test_ref_count(PyObject *self)
>>      {
>>           printf("\nIncrementing ref count for self - just for the hell
>>      of it\n");
>>           printf("\n before self has a ref count of %ld\n", Py_REFCNT(self));
>>           Py_INCREF(self);
>>           printf("\n after self has a ref count of %ld\n", Py_REFCNT(self));
>>           fflush(stdout);
> At this point, the refcount has indeed been increased.
>
>>           return self;
>>      }
> And then you say "my return value is this object".
>
> The normal thing to do is to add a reference to whatever you're
> returning. For instance, Py_RETURN_NONE will incref None and then
> return it.
>
> So you're incrementing the refcount, then returning it without
> incrementing the refcount. Your code is actually equivalent to "return
> self".
>
> In order to actually leak a reference, you'd need to incref it twice.
>
> ChrisA


Chris - I am still puzzled - does? doing 'return self' automatically 
decrement the ref count of the object ?, and why is that the desired 
behaviour ? Effectively it results in a decrement of two, since at the 
exit of the function the ref count is only 1 (as witnessed by the 
subsequent call to assertEqual).

(I am not suggesting that it should be changed - I understand that would 
be a breaking change !).

You say I am returning it without incrementing, but I am explicitly 
incrementing it before the return.




From rosuav at gmail.com  Wed Jan 19 09:56:17 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 20 Jan 2022 01:56:17 +1100
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <e640d910-50e6-7485-db97-e1a7b62339e0@btinternet.com>
References: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
 <CAPTjJmq==4CHdPU2GiXefPG+OUG-jA6LRqWdxDxkdzjdTccANw@mail.gmail.com>
 <e640d910-50e6-7485-db97-e1a7b62339e0@btinternet.com>
Message-ID: <CAPTjJmqZfnw34DAwfXXc+-QcwkogChsF_sWDgi8XojvOTuNQ8A@mail.gmail.com>

On Thu, Jan 20, 2022 at 1:22 AM Tony Flury <tony.flury at btinternet.com> wrote:
>
>
> On 19/01/2022 11:09, Chris Angelico wrote:
> > On Wed, Jan 19, 2022 at 10:00 PM Tony Flury via Python-list
> > <python-list at python.org> wrote:
> >> Extension function :
> >>
> >>      static PyObject *_Node_test_ref_count(PyObject *self)
> >>      {
> >>           printf("\nIncrementing ref count for self - just for the hell
> >>      of it\n");
> >>           printf("\n before self has a ref count of %ld\n", Py_REFCNT(self));
> >>           Py_INCREF(self);
> >>           printf("\n after self has a ref count of %ld\n", Py_REFCNT(self));
> >>           fflush(stdout);
> > At this point, the refcount has indeed been increased.
> >
> >>           return self;
> >>      }
> > And then you say "my return value is this object".
> >
> > The normal thing to do is to add a reference to whatever you're
> > returning. For instance, Py_RETURN_NONE will incref None and then
> > return it.
> >
> > So you're incrementing the refcount, then returning it without
> > incrementing the refcount. Your code is actually equivalent to "return
> > self".
> >
> > In order to actually leak a reference, you'd need to incref it twice.
> >
> > ChrisA
>
>
> Chris - I am still puzzled - does  doing 'return self' automatically
> decrement the ref count of the object ?, and why is that the desired
> behaviour ? Effectively it results in a decrement of two, since at the
> exit of the function the ref count is only 1 (as witnessed by the
> subsequent call to assertEqual).

Imagine you have a function which creates a brand new object. What
should its reference count be just before you return it? What should
the refcount be after it's been given to the caller? There has to be a
reference at all times, but only one, so that it can be properly
garbage collected; the act of returning it from a function has to
steal away a reference.

> (I am not suggesting that it should be changed - I understand that would
> be a breaking change !).
>
> You say I am returning it without incrementing, but I am explicitly
> incrementing it before the return.
>

Yes, you are incrementing it - but only once. The act of returning
something that you already have a reference to has to increment the
reference count and then return the thing. Picture this Python code:

_THING = object()
def get_thing():
    return _THING

When you call get_thing(), it needs to give you back a new reference
to that object. Written in C, that would have to increment the
refcount, then return the thing. (If you disassemble that code in
CPython, what you'll see is that it does a LOAD_GLOBAL, which does the
reference count incrementing, and then it returns what's on the top of
the stack. In C, you simply return the pointer, so you manually
increment the refcount first.)

In order to increment the reference count and keep it higher, you need
to (1) increment the reference count, then (2) return the object,
which consists of incrementing the reference count and returning the
pointer.

ChrisA

From barry at barrys-emacs.org  Wed Jan 19 17:10:26 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Wed, 19 Jan 2022 22:10:26 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
References: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
Message-ID: <61D4CCFA-BD35-4D5B-8911-3A6D4089ABF5@barrys-emacs.org>



> On 19 Jan 2022, at 10:57, Tony Flury via Python-list <python-list at python.org> wrote:
> 
> I am writing a C extension module for an AVL tree, and I am trying to ensure reference counting is done correctly. I was having a problem with the reference counting so I worked up this little POC of the problem, and I hope someone can explain this.

The ref counting in the C API not easy to get right. Sometimes you must inc some times you must not inc.
There are a lot of libraries that take that problem away from you.

For example I maintain PyCXX that is a C++ library that allows your drive the C API without needed to worry about ref counting.
There are lots of other libraries as well that aim to do the same thing.

http://cxx.sourceforge.net/ <http://cxx.sourceforge.net/>

Barry




> 
> Extension function :
> 
>   static PyObject *_Node_test_ref_count(PyObject *self)
>   {
>        printf("\nIncrementing ref count for self - just for the hell
>   of it\n");
>        printf("\n before self has a ref count of %ld\n", Py_REFCNT(self));
>        Py_INCREF(self);
>        printf("\n after self has a ref count of %ld\n", Py_REFCNT(self));
>        fflush(stdout);
>        return self;
>   }
> 
> As you can see this function purely increments the reference count of the instance.
> 
> /Note: I understand normally this would be the wrong this to do, but this is a POC of the issue, not live code. In the live code I am attaching a 2nd nodes to each other, and the live code therefore increments the ref-count for both objects - so even if the Python code deletes it's reference the reference count for the instance should still be 1 in order to ensure it doesn't get garbage collected./
> 
> This function is exposed as the test_ref method.
> 
> This is the test case :
> 
>     def test_000_009_test_ref_count(self):
>         node = _Node("Hello")
>         self.assertEqual(sys.getrefcount(node), 2)
>         node.test_ref()
>         self.assertEqual(sys.getrefcount(node), 3)
> 
> The output of this test case is :
> 
> test_000_009_test_ref_count (__main__.TestNode) ...
> Incrementing ref count for self - just for the hell of it
> 
>  before self has a ref count of 2
> 
>  after self has a ref count of 3
> FAIL
> 
> ======================================================================
> FAIL: test_000_009_test_ref_count (__main__.TestNode)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/home/tony/Development/python/orderedtree/tests/test_orderedtree.py", line 62, in test_000_009_test_ref_count
>     self.assertEqual(sys.getrefcount(node), 3)
> AssertionError: 2 != 3
> 
> So I understand why the first assert will be true - when the sys.getrefcount() function is called the ref count is incremented temporarily (as a borrowed reference), so there are now two references - the 'node' variable, and the borrowed reference in the function call.
> 
> We then call the 'test_ref' method, and again that call causes a borrowed reference (hence the ref count being 2 initially within the method). The 'test_ref' method increments the reference of the instance - as you can see from the output we now have a ref count of 3 - (that count is the 'node' variable in the test case, the borrowed reference due to the method call, and the artificial increment from the 'ref_test' method).
> 
> When the 'ref_test' method exits I would expect the ref count of the instance to now be 2 (one for the 'node' variable, and one as a result of the artificial increment increment').
> 
> I would therefore expect the 2nd assertEqual in the test case to succeed. - in this case the borrowed reference within sys.getfrefcount() should cause the count to be 3.
> 
> As you see though that 2nd assertEqual fails - suggesting that the refcount of 'node' is actually only 1 when the 'test_ref' method exits.
> 
> Can someone explain why the 'test_ref' method fails to change the refcount of the 'node' instance.
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From bjlockie021 at gmail.com  Wed Jan 19 22:16:19 2022
From: bjlockie021 at gmail.com (James Smith)
Date: Wed, 19 Jan 2022 19:16:19 -0800 (PST)
Subject: empty stdout (subprocess.run)
Message-ID: <3fa8e083-a305-4123-94a7-d145f69ca3een@googlegroups.com>

I'm trying to run a shell command but the stdout is empty:

import subprocess

torrentno=8
cmd="/usr/bin/transmission-remote --torrent %s --info", str(torrentno)
res=subprocess.run(cmd, shell=True, check=True, universal_newlines=True, capture_output=True)
print(res)

CompletedProcess(args=('/usr/bin/transmission-remote --torrent %s --info', '1'), returncode=0, stdout='', stderr='')

From wlfraed at ix.netcom.com  Wed Jan 19 23:08:34 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Wed, 19 Jan 2022 23:08:34 -0500
Subject: empty stdout (subprocess.run)
References: <3fa8e083-a305-4123-94a7-d145f69ca3een@googlegroups.com>
Message-ID: <ecnhug5oi5dunhpseintb7mja4d7hl3pdj@4ax.com>

On Wed, 19 Jan 2022 19:16:19 -0800 (PST), James Smith
<bjlockie021 at gmail.com> declaimed the following:

>I'm trying to run a shell command but the stdout is empty:
>
>import subprocess
>
>torrentno=8
>cmd="/usr/bin/transmission-remote --torrent %s --info", str(torrentno)

	Don't you need to provide for that %s? Perhaps

cmd="/usr/bin/transmission-remote --torrent %s --info" % torrentno

As coded, you are passing a TUPLE of two strings as cmd (fyi: the %s will
take the string representation of whatever is the matching argument on the
right, so str() isn't needed -- and if you need a number for both --torrent
and --info you'll need another %s, and (torrentno, torrentno) on the right)

>res=subprocess.run(cmd, shell=True, check=True, universal_newlines=True, capture_output=True)
>print(res)
>
>CompletedProcess(args=('/usr/bin/transmission-remote --torrent %s --info', '1'), returncode=0, stdout='', stderr='')


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From cs at cskk.id.au  Wed Jan 19 23:13:59 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Thu, 20 Jan 2022 15:13:59 +1100
Subject: empty stdout (subprocess.run)
In-Reply-To: <3fa8e083-a305-4123-94a7-d145f69ca3een@googlegroups.com>
References: <3fa8e083-a305-4123-94a7-d145f69ca3een@googlegroups.com>
Message-ID: <Yejhh4yF9l588jj+@cskk.homeip.net>

On 19Jan2022 19:16, James Smith <bjlockie021 at gmail.com> wrote:
>I'm trying to run a shell command but the stdout is empty:
>
>import subprocess
>
>torrentno=8
>cmd="/usr/bin/transmission-remote --torrent %s --info", str(torrentno)
>res=subprocess.run(cmd, shell=True, check=True, universal_newlines=True, capture_output=True)
>print(res)
>
>CompletedProcess(args=('/usr/bin/transmission-remote --torrent %s --info', '1'), returncode=0, stdout='', stderr='')

If you're using shell=True (please don't) you want cmd to be a string.  
You have defined it as a 2-tuple. You can even see that in the 
CompletedProcess object you printed.

I think you want to % substitue torrentno into the format string.

But I recommend you use shell=False and make:

    cmd = ["/usr/bin/transmission-remote", "--torrent", str(torrentno), "--info"]

because assembling commands with % formatting is a recipe for injection 
attacks. Look up "Little Bobby Tables", which is an SQL example of what 
you're doing with the shell.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From arj.python at gmail.com  Thu Jan 20 04:22:51 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Thu, 20 Jan 2022 13:22:51 +0400
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
Message-ID: <CADrxXXm9Xw=w3ER63vbnZCpMXFCr1EZpM+tZRQi61rLkCRKv0g@mail.gmail.com>

Now I have a concern. For the RSS feed. Does your client filter by the
latest
item or pubDate? I published another edition yesterday but noticed that
the item was not the latest in the list.

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius


On Sun, Dec 26, 2021 at 8:40 PM Abdur-Rahmaan Janhangeer <
arj.python at gmail.com> wrote:

> Greetings lists,
>
> I have started a newspaper (not newsletter) focused
> on interesting reads on Python mailing lists. Don't tag
> on the fact that holiday seasons are the worst times for
> launch according to marketing folks, I started this to note
> down interesting mails. This might also be a great way to
> bring mailing list gems to a wider readership. So, here's
> the url https://pyherald.com/
>
> Kind Regards,
>
> Abdur-Rahmaan Janhangeer
> about <https://compileralchemy.github.io/> | blog
> <https://www.pythonkitchen.com>
> github <https://github.com/Abdur-RahmaanJ>
> Mauritius
>

From drsalists at gmail.com  Thu Jan 20 17:12:51 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Thu, 20 Jan 2022 14:12:51 -0800
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
 <20220108210631.uodonbypheirvl2d@hjp.at>
 <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
Message-ID: <CAGGBd_rr6oe-VeMVv0wy_T__onEo68zot9gMeANF7-rcSn5vzg@mail.gmail.com>

There's also Python Weekly:
https://www.pythonweekly.com/

On Sat, Jan 8, 2022 at 10:29 PM Abdur-Rahmaan Janhangeer <
arj.python at gmail.com> wrote:

> Well yes XD though LWN covers Py topics well when it wants
>
>
> 1. Yes sure, did not expect RSS interest
> 2. Excuse my blunder, will do!
>
>

> --
> https://mail.python.org/mailman/listinfo/python-list
>

From bjlockie021 at gmail.com  Wed Jan 19 23:18:59 2022
From: bjlockie021 at gmail.com (James Smith)
Date: Wed, 19 Jan 2022 20:18:59 -0800 (PST)
Subject: empty stdout (subprocess.run)
In-Reply-To: <ecnhug5oi5dunhpseintb7mja4d7hl3pdj@4ax.com>
References: <3fa8e083-a305-4123-94a7-d145f69ca3een@googlegroups.com>
 <ecnhug5oi5dunhpseintb7mja4d7hl3pdj@4ax.com>
Message-ID: <6e1d4c5f-afaa-4cc5-b105-7c1273f98f01n@googlegroups.com>

On Wednesday, January 19, 2022 at 11:08:58 PM UTC-5, Dennis Lee Bieber wrote:

> Don't you need to provide for that %s? Perhaps 
> 
> cmd="/usr/bin/transmission-remote --torrent %s --info" % torrentno 

That works, thanks.

From bjlockie021 at gmail.com  Wed Jan 19 23:26:01 2022
From: bjlockie021 at gmail.com (James Smith)
Date: Wed, 19 Jan 2022 20:26:01 -0800 (PST)
Subject: empty stdout (subprocess.run)
In-Reply-To: <mailman.230.1642652051.3079.python-list@python.org>
References: <Yejhh4yF9l588jj+@cskk.homeip.net>
 <3fa8e083-a305-4123-94a7-d145f69ca3een@googlegroups.com>
 <mailman.230.1642652051.3079.python-list@python.org>
Message-ID: <38e9997b-42d1-4676-a4eb-b8318573723en@googlegroups.com>

On Wednesday, January 19, 2022 at 11:14:28 PM UTC-5, cameron... at gmail.com wrote:

> But I recommend you use shell=False and make: 
> 
> cmd = ["/usr/bin/transmission-remote", "--torrent", str(torrentno), "--info"] 

I like that. :-)

From narshad.380 at gmail.com  Thu Jan 20 05:59:43 2022
From: narshad.380 at gmail.com (NArshad)
Date: Thu, 20 Jan 2022 02:59:43 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <mailman.224.1642565856.3079.python-list@python.org>
References: <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <CAPTjJmqnzBwP4diBKQZ_OPWDSvze71x-K24dxbO4UD+kB6YzAw@mail.gmail.com>
 <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
 <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
Message-ID: <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>


- I will try to follow all that you people are saying, but it will take time or next time. The chance of concurrency is very less. Kindly don't write big descriptions.

- The weblink Avi Gross has given is very useful.







From greg.ewing at canterbury.ac.nz  Thu Jan 20 17:33:19 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 21 Jan 2022 11:33:19 +1300
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <mailman.226.1642590598.3079.python-list@python.org>
References: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
 <CAPTjJmq==4CHdPU2GiXefPG+OUG-jA6LRqWdxDxkdzjdTccANw@mail.gmail.com>
 <mailman.226.1642590598.3079.python-list@python.org>
Message-ID: <j4u69iF3qpvU1@mid.individual.net>

On 20/01/22 12:09 am, Chris Angelico wrote:
> At this point, the refcount has indeed been increased.
> 
>>           return self;
>>      }
> 
> And then you say "my return value is this object".
> 
> So you're incrementing the refcount, then returning it without
> incrementing the refcount. Your code is actually equivalent to "return
> self".

Chris, you're not making any sense. This is C code, so there's no
way that "return x" can change the reference count of x.

 > The normal thing to do is to add a reference to whatever you're
 > returning. For instance, Py_RETURN_NONE will incref None and then
 > return it.
 >

The OP understands that this is not a normal thing to do. He's
trying to deliberately leak a reference for the purpose of diagnosing
a problem.

It would be interesting to see what the actual refcount is after
calling this function.

-- 
Greg

From rosuav at gmail.com  Thu Jan 20 18:12:19 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 21 Jan 2022 10:12:19 +1100
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <j4u69iF3qpvU1@mid.individual.net>
References: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
 <CAPTjJmq==4CHdPU2GiXefPG+OUG-jA6LRqWdxDxkdzjdTccANw@mail.gmail.com>
 <mailman.226.1642590598.3079.python-list@python.org>
 <j4u69iF3qpvU1@mid.individual.net>
Message-ID: <CAPTjJmoNk2gLNBXYhXNn=ryL9eFcRrP8wdW9bRvf3wSUmEJUjA@mail.gmail.com>

On Fri, 21 Jan 2022 at 10:10, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> On 20/01/22 12:09 am, Chris Angelico wrote:
> > At this point, the refcount has indeed been increased.
> >
> >>           return self;
> >>      }
> >
> > And then you say "my return value is this object".
> >
> > So you're incrementing the refcount, then returning it without
> > incrementing the refcount. Your code is actually equivalent to "return
> > self".
>
> Chris, you're not making any sense. This is C code, so there's no
> way that "return x" can change the reference count of x.

Yeah, I wasn't clear there. It was equivalent to *the Python code*
"return self". My apologies.

>  > The normal thing to do is to add a reference to whatever you're
>  > returning. For instance, Py_RETURN_NONE will incref None and then
>  > return it.
>  >
>
> The OP understands that this is not a normal thing to do. He's
> trying to deliberately leak a reference for the purpose of diagnosing
> a problem.
>
> It would be interesting to see what the actual refcount is after
> calling this function.
>

Yes, and that's why I was saying it would need a *second* incref.

ChrisA

From arj.python at gmail.com  Thu Jan 20 21:13:02 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Fri, 21 Jan 2022 06:13:02 +0400
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <CAGGBd_rr6oe-VeMVv0wy_T__onEo68zot9gMeANF7-rcSn5vzg@mail.gmail.com>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
 <20220108210631.uodonbypheirvl2d@hjp.at>
 <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
 <CAGGBd_rr6oe-VeMVv0wy_T__onEo68zot9gMeANF7-rcSn5vzg@mail.gmail.com>
Message-ID: <CADrxXXmmVRehfHJA8rPKCrXzoQtgyM5hO-mFy2fOb6TbucqVWg@mail.gmail.com>

Well, this one is a newspaper as opposed to being a newsletter.

Newsletters are a collection of links. See

AwesomePython
PythonWeekly
PyCoders Weekly

Newspapers also contain articles, the above don't.

As some mails pointed out above, this is like
the lwn of Python

I have been avoiding the topics and formats used by existing
newsletters by focusing on news among others.

But its funny to see one of them starting to cover news
by writing an external article on a website they own
then link it in the newsletter. It's what you call
competition i guess XD

The focus of this one is and remains mailing lists / discuss platforms.
A look at the archives and you can see publishing
interesting mails. Something you don't have with newsletters.

The main aim of it is getting myself updated with
discussions and pep status and sharing it with the world.
Something i feel i was not getting with existing resources.
I had cool tutos/articles, new libs, popular posts,
job openings, events but not a complete overview
of meta posts.

On Fri, 21 Jan 2022, 02:13 Dan Stromberg, <drsalists at gmail.com> wrote:

>
> There's also Python Weekly:
> https://www.pythonweekly.com/
>
> On Sat, Jan 8, 2022 at 10:29 PM Abdur-Rahmaan Janhangeer <
> arj.python at gmail.com> wrote:
>
>> Well yes XD though LWN covers Py topics well when it wants
>>
>>
>> 1. Yes sure, did not expect RSS interest
>> 2. Excuse my blunder, will do!
>>
>>
>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>
>

From arj.python at gmail.com  Sat Jan 22 00:04:38 2022
From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer)
Date: Sat, 22 Jan 2022 09:04:38 +0400
Subject: A Newspaper for Python Mailing Lists
In-Reply-To: <CADrxXXmmVRehfHJA8rPKCrXzoQtgyM5hO-mFy2fOb6TbucqVWg@mail.gmail.com>
References: <CADrxXXkyz3RQcvfqCqV5U=o7r8N2O+z6Y5SX3Yc-6YMhHcn9SA@mail.gmail.com>
 <20220108210631.uodonbypheirvl2d@hjp.at>
 <CADrxXXkxKYRKbznnpGcfM-HVB+axAzL0Av3XfFZtJi8ZuZrRjw@mail.gmail.com>
 <CAGGBd_rr6oe-VeMVv0wy_T__onEo68zot9gMeANF7-rcSn5vzg@mail.gmail.com>
 <CADrxXXmmVRehfHJA8rPKCrXzoQtgyM5hO-mFy2fOb6TbucqVWg@mail.gmail.com>
Message-ID: <CADrxXXkC939udHXwp33_cMCBMc8R+7S5KoadUG-se2RWBa+dXQ@mail.gmail.com>

On the news side i was wrong as newsletters have been covering it since
long on a monthly basis, but not as in depth as it should be but awesome
job though. Else i have been going through early archives but the quality
of postings are not very great. The mailing lists have made awesome
progress over the years! Now the topics are very serious, the conversations
contribute much. Keep it up!

Kind Regards,

Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-RahmaanJ>
Mauritius

From narshad.380 at gmail.com  Sat Jan 22 05:22:14 2022
From: narshad.380 at gmail.com (NArshad)
Date: Sat, 22 Jan 2022 02:22:14 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
References: <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <CAPTjJmqnzBwP4diBKQZ_OPWDSvze71x-K24dxbO4UD+kB6YzAw@mail.gmail.com>
 <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
 <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
Message-ID: <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>


The user is going to enter the book name as the input of an HTML form on a website and I have to check whether the book is present or not in the Excel table. openpyxl preferred pandas is also fine anything left. Case sensitivity is not required. I tried to find code or tutorial on google search for all this but no use that's why..................

This time the choice of HTML is right or not??

From boblatest at yahoo.com  Sat Jan 22 06:29:08 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 22 Jan 2022 11:29:08 GMT
Subject: "undefined symbol" in C extension module
Message-ID: <j52844Fr70sU1@mid.individual.net>

Hi guys,

I've written some CPython extension modules in the past without problems. Now
after moving to a new Archlinux box with Python3.10 installed, I can't build
them any more. Or rather, I can build them but not use them due to "undefined
symbols" during linking. Here's ldd's output when used on the "spam" example
library from the docs:

	linux-vdso.so.1 (0x00007ffe2454a000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007fb6b6eb9000)
	/usr/lib64/ld-linux-x86-64.so.2 (0x00007fb6b70a4000)
undefined symbol: PyObject_Init	(./build/lib.linux-x86_64-3.10/
    spam.cpython-310-x86_64-linux-gnu.so)

[and more of the same]

It seems that I need to install some library to make this work, but which one?

From wlfraed at ix.netcom.com  Sat Jan 22 12:45:06 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sat, 22 Jan 2022 12:45:06 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
 <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
Message-ID: <nadougdll1vfkaimep4lta7ep7mui64nke@4ax.com>

On Sat, 22 Jan 2022 02:22:14 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>
>The user is going to enter the book name as the input of an HTML form on a website and I have to check whether the book is present or not in the Excel table. openpyxl preferred pandas is also fine anything left. Case sensitivity is not required. I tried to find code or tutorial on google search for all this but no use that's why..................
>
>This time the choice of HTML is right or not??

	HTML is the core basis for any web-site presentation -- no HTML, no
web-page.

	Interaction with a displayed web-page is via... CGI (individual scripts
that process returned data -- somewhat slow as originally each invocation
requires a full server process startup and shutdown; advances were made to
try to keep a single process running for multiple invocations), AJAX
(Javascript running in the browser making internet requests and modifying
the "document object model" [DOM] in the browser to update the page without
hitting the server for a full update transfer).

	Flask, Django, and some others are packages to consolidate what had
been static pages and CGI into a framework that handles an entire "site" --
via templates for look&feel, session management (and cookies) so that
interaction can be tracked to users (HTTP is a fire&forget system -- every
URL sent to the server is treated as a completely separate request).

	Above those frameworks are things like Zope -- so-called content
management frameworks.


	And again... You will not find anything like you want... NOBODY is
going to write a web application using a spreadsheet as the primary data
storage. A spreadsheet, and custom transformation code, MIGHT be used to
initially populate a database. (M$ SQL Server Integration Services is a
whole system for defining import/transformation/clean-up "functions" for
data sources to data base). A spreadsheet might be available as a
report/extraction format from the database.

	Using a web server means you have to be able to handle (near)
simultaneous requests from multiple users and be able to keep those
interactions distinct. That is going to require you to implement some sort
of access control for a spreadsheet, since spreadsheets are single-user
entities (you might get away with shared reading as long as you never need
to update any field in the spreadsheet; as soon as you need to update, you
need to be able to lock records so only one session can update it, and you
need to have a way for other sessions to detect such changes and update
that sessions display for review). On possibility might be to write a
separate process wherein only that process opens the spreadsheet -- all the
web-page stuff will have to generate complete I/O requests to the
spreadsheet process, it will make the changes, and return whatever data is
applicable ("complete" meaning that, if you need to change three fields in
a record, all three commands are provided as ONE request to the spreadsheet
process, and it does all the changes in the request before it goes on to
read the next request). This still falls short of detecting overlapping
changes -- two users want to do something with the same record; they each
read the record, then send commands to change the same field. Which ever
one is received first should complete, and the other needs to be rejected
and resubmitted.



	Show us code you've written, and we can assist in debugging it. But you
couldn't afford to have any of us write the application for you! (To cover
insurance, taxes, etc. an independent contractor would probably charge you
$100+ an HOUR -- and the first hours look like they'd be spent getting
detailed requirements from you, discussing design problems [spreadsheet as
data storage], before even getting to any coding. That's real-world
software engineering.)

	If you are doing a web application, how are you going to host it? Who
is responsible for managing the web server? Domain name? Firewalls?
Certificates if you need HTTPS rather than plain insecure HTTP. 

	I have a Raspberry-Pi with Nginx serving static pages over insecure
HTTP as I've never applied for a certificate -- using a dynamic DNS
service. It is not suited for high-demand as it is behind my ISP router,
and my uplink rate is only a tenth of my downlink rate (which isn't the
fastest thing out there to begin with [Ugh -- Hope it's the weather -- my
downlink is down to 10Mbps, when nominal is closer to 14Mbps]). Someday I
may try creating a Flask application for it, just for learning.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From rosuav at gmail.com  Sat Jan 22 15:56:02 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 23 Jan 2022 07:56:02 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <nadougdll1vfkaimep4lta7ep7mui64nke@4ax.com>
References: <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
 <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <nadougdll1vfkaimep4lta7ep7mui64nke@4ax.com>
Message-ID: <CAPTjJmoYzAfJ+MffUS3qOb9+NYmfO8SFhOOBcHPbDeTsdBmc=g@mail.gmail.com>

On Sun, 23 Jan 2022 at 07:47, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>         If you are doing a web application, how are you going to host it? Who
> is responsible for managing the web server? Domain name? Firewalls?
> Certificates if you need HTTPS rather than plain insecure HTTP.
>
>         I have a Raspberry-Pi with Nginx serving static pages over insecure
> HTTP as I've never applied for a certificate -- using a dynamic DNS
> service. It is not suited for high-demand as it is behind my ISP router,
> and my uplink rate is only a tenth of my downlink rate (which isn't the
> fastest thing out there to begin with [Ugh -- Hope it's the weather -- my
> downlink is down to 10Mbps, when nominal is closer to 14Mbps]). Someday I
> may try creating a Flask application for it, just for learning.
>

Thanks to LetsEncrypt, certificates shouldn't be too hard for any
public-facing server. The Pi can renew and install its own
certificates in a 100% automated process, as long as you can continue
receiving port 80 traffic as well as the port 443 that the live server
would use.

ChrisA

From avigross at verizon.net  Sat Jan 22 16:27:11 2022
From: avigross at verizon.net (Avi Gross)
Date: Sat, 22 Jan 2022 21:27:11 +0000 (UTC)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
References: <mailman.173.1642115307.3079.python-list@python.org>
 <ue83ug9bm0gpgpbg15f0sl13ed911ka0fh@4ax.com>
 <8f3ea49e-5770-45af-94d0-6d21afb9b470n@googlegroups.com>
 <2eg8ugpr1cemf47bkvqpf7dkk5a802v8rg@4ax.com>
 <def4da39-545f-44dc-b2c1-1a462d15e023n@googlegroups.com>
 <CAPTjJmqnzBwP4diBKQZ_OPWDSvze71x-K24dxbO4UD+kB6YzAw@mail.gmail.com>
 <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
 <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
Message-ID: <1702885446.594273.1642886831321@mail.yahoo.com>


I keep wondering about the questions asked by NArshad here. His message can be read below mine, for context.
This is a place focused on using the Python language. The web page being in HTML is beyond irrelevant and in particular, web pages generally are in HTML even if only as a way to call other functionality.
What I gather is that the web page, irrelevant if in HTML or JAVA or JavaScript or a connection with a server running python or anything else, asks the user to type in a book name to search for. NOTHING is of any importance here except capturing the exact text and sending it, or perhaps a lower case version, back to your server that does the search.
Does that make sense? If not, stop reading as nothing else will.
So, back on the server side, you get a single string of text that might look like "Great Expectations" and you want to find out if that matches a book title exactly or perhaps partially. Again, agreed?
So your program needs to have data available. What if the names of all the books were written in a regular text file one line at a time? Yes, they aren't but WHAT IF?
You would open the file and perhaps read a line at a time and compare the lower-case version of that line to a lower case version of the requested book name. An exact match could mean you close the file and return that you found it, however that is done. No match means read the next line and repeat. If you get to the end of the file without finding it, you failed and close the file and return that result.
Of course some might do a bit more than convert to lower case such as removing multiple white space in both things being compared, or dropping other parts like commas of quotes as in changing "Mine,? ? ? didn't I? ? say!" to "mine didnt i say" so more things match as intended. Plenty you can do if you wish. Python has plenty of ways to do all that and more.
Now forget the plain text file and consider the following. Read the EXCEL file one unit/cell at a time in whatever column and row the data starts. This is doable, albeit not necessarily ideal and basically can be the same algorithm with a substitution in the part that gets the next line to getting the cell below it. Testing for end of file may be different too.
But what I think makes a bit more sense is to setup the server to have a rather long-term process that sits there and waits for requests. It starts by reading in ALL the data and perhaps modifying it once in ways like I suggest above. In python you would now have some data structure such as a list or set or even dictionary or some variation from numpy or pandas. It does not matter much. The data structure holding all books, or maybe all unique book names, would be there and ready and waiting.
Your program now sleeps until woken up and it is given the book name being searched for. It now simply needs to apply whatever transformations you want to the received name and then do one of many kinds of lookup to see if it is in the data structure representing the titles known to the library. Using a set or dictionary means no search as it hashes the result and rapidly finds out if it has previously been stored. Other methods like lists have choices between a brute force linear search if the data remains in the original order or letting python do the work by asking if it is "in" the list to more sophisticated methods like keeping it sorted and doing a binary search.
I am guessing that for your need, a relatively unsophisticated method may work fine and it can be changed if your project scales up to millions of books and multiple people searching at the same time.
And note you are now asking about a very limited thing. As others have pointed out, there can be much more complex requests such as checking if a copy of the book is free, and reserving it so others requesting it see as checked out and logging it back in again and more. And obviously more work is needed if you want to support more of a google-style search engine that matches books that contain only parts of your request.
But note what I am describing can be done by pretty much ANY language as long as it can open and read your data storage format. HTML all by itself can display a form with a text box and some kind of SUBMIT button and when you click on it call a CGI on a server and display?what it returns. For the simple scenario you mention, you do not need openpyxl. But my guess is you are making something with additional functionality based on earlier posts and thus need to take care that anything you do can be extended to add functionally, so a raw HTML solution may not easily meet other needs.?
I keep hearing you searching for a solution and you may find one but mostly you need to design your own solution that meets your needs and use the search more for small pieces of the design like "how to efficiently search using a set in python" ...
As has been said repeatedly, if you really need the result soon and really are not yet up to the job or have time to learn, then either tell them you can't do it or get someone hired. But note, in the latter case you need to stop wasting time and tell them in great detail what you want done and let them see the data and so on. Otherwise, you may pay for lots of hours or have people walk away.
AGross

-----Original Message-----
From: NArshad <narshad.380 at gmail.com>
To: python-list at python.org
Sent: Sat, Jan 22, 2022 5:22 am
Subject: Re: What to write or search on github to get the code for what is written below:


The user is going to enter the book name as the input of an HTML form on a website and I have to check whether the book is present or not in the Excel table. openpyxl preferred pandas is also fine anything left. Case sensitivity is not required. I tried to find code or tutorial on google search for all this but no use that's why..................

This time the choice of HTML is right or not??
-- 
https://mail.python.org/mailman/listinfo/python-list

From wlfraed at ix.netcom.com  Sat Jan 22 16:18:49 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sat, 22 Jan 2022 16:18:49 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
 <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
Message-ID: <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>

On Sat, 22 Jan 2022 02:22:14 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>
>The user is going to enter the book name as the input of an HTML form on a website and I have to check whether the book is present or not in the Excel table

	As written, the user will have to know the exact name of the book, as
it exists in the spreadsheet, and enter exactly that... 

	Is it going to be case sensitive?

		A Matter of Life and Death
is not the same as
		A Matter Of Life And Death

and partial entries won't match either

		A Matter of Life

	Do you intend to implement fuzzy logic for finding the target given
partial input? If you do, how do you plan to differentiate between two or
more books that match the logic? Do you strip punctuation?

	Most systems will, instead, display a list of all books (unless a
filter is specified -- and may require the user to start with a filter just
to avoid a massive scrolling listing), and let the user select from the
list (radio button, check box, item number). A filter would be <display all
books with all the words "matter", "life" in the name>.


	At the level your questions have been -- my biggest suggestion would be
to forget the web form and multi-user problems and write a text-based
console program that can be run from a command line shell. WHEN you get
that working you can consider keeping the logic that interfaces with the
data store, and replace the "presentation" stuff with HTML generation and
web-server integration (any web application will look quite different in
how the code is structured, since a console application has only one flow
-- start the app, make a selection, move to next phase of activity... A web
application has every action as a distinct connection and needs identifying
tokens [cookies] to let the logic know what was done previously)


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From cl at isbd.net  Sat Jan 22 16:26:37 2022
From: cl at isbd.net (Chris Green)
Date: Sat, 22 Jan 2022 21:26:37 +0000
Subject: How to test for maildir 'folder' in Python?
Message-ID: <dd9ubi-r545.ln1@esprimo.zbmc.eu>

I have a script that walks a quite deep tree of mail messages to find
and archive old messages.  I'm trying to convert it from mbox to
maildir (as I now store my mail in maildir format).

So I need to test whether a point I have reached in the hierarchy is a
maildir mailbox or not.  Using mbox format it's easy because 'folders'
are directories and mailboxes are files.  However with maildir the
'folders' have directories within them so the simple tree walking goes
down a level too far and finds 'folders' which aren't mailboxes called
'cur', 'new' and 'tmp'.

Is there any 'ready made' way in python to tell whether a directory is
a maildir mailbox?  If not I suppose I'll simply have to check if
there are 'cur', 'new' and 'tmp' directories within the directory
which may or may not be a maildir.

-- 
Chris Green
?

From rosuav at gmail.com  Sat Jan 22 17:17:38 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 23 Jan 2022 09:17:38 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
References: <kmgbugd8s3au66ebi7toqq3bv48rm9k4qd@4ax.com>
 <mailman.217.1642457260.3079.python-list@python.org>
 <4bcba6cc-45a4-4b66-97c6-4c4081e536a4n@googlegroups.com>
 <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
Message-ID: <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>

On Sun, 23 Jan 2022 at 09:15, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> A web
> application has every action as a distinct connection and needs identifying
> tokens [cookies] to let the logic know what was done previously
>

Usually. Fortunately, we have SOME features that can make life easier,
but in general, yes, web apps need to think in discrete actions with
minimal maintained state.

Absolutely agree with making a console app first. Though I rather
suspect the OP doesn't want to write any code at all.

ChrisA

From cs at cskk.id.au  Sat Jan 22 18:43:35 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Sun, 23 Jan 2022 10:43:35 +1100
Subject: How to test for maildir 'folder' in Python?
In-Reply-To: <dd9ubi-r545.ln1@esprimo.zbmc.eu>
References: <dd9ubi-r545.ln1@esprimo.zbmc.eu>
Message-ID: <YeyWp3wZUhE31sct@cskk.homeip.net>

On 22Jan2022 21:26, Chris Green <cl at isbd.net> wrote:
>So I need to test whether a point I have reached in the hierarchy is a
>maildir mailbox or not.  Using mbox format it's easy because 'folders'
>are directories and mailboxes are files.  However with maildir the
>'folders' have directories within them so the simple tree walking goes
>down a level too far and finds 'folders' which aren't mailboxes called
>'cur', 'new' and 'tmp'.
>
>Is there any 'ready made' way in python to tell whether a directory is
>a maildir mailbox?  If not I suppose I'll simply have to check if
>there are 'cur', 'new' and 'tmp' directories within the directory
>which may or may not be a maildir.

I was hoping that the provided mailbox/Maildir class had something, but 
apparently not.

So I suggest fetching my cs.mailutils module from PyPI and using its 
ismaildir(path) function, which does exactly what you describe.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From wlfraed at ix.netcom.com  Sat Jan 22 21:54:23 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sat, 22 Jan 2022 21:54:23 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
Message-ID: <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>

On Sun, 23 Jan 2022 09:17:38 +1100, Chris Angelico <rosuav at gmail.com>
declaimed the following:


>
>Absolutely agree with making a console app first. Though I rather
>suspect the OP doesn't want to write any code at all.
>

	Oh, it's gone beyond suspicion -- considering that, at just 10 lines
per day, over the last 16 days, they should have 160 lines of code (whether
it works or not) that could be presented for evaluation. What is considered
industry standard? 10 lines an hour? Assuming the OP isn't spending time on
requirements analysis and documentation. (My best example is something I
did back around 1990: 18 months consisting of ~600 lines F77, ~2000 lines
of C, and ~2500 lines of DECWindow UIL definition; along with learning both
DECWindow and GKS within it -- so, yes, the overall total is about 2 lines
per day, but I had to develop/document requirements, present them to the
customer, get approval, develop/document the design, present /that/ to the
customer, get approval, before even getting to the code and writing a user
manual for the system... I suspect easily half the time was spent just on
paperwork.)

	All we've been shown is four lines, and that wasn't complete enough to
run stand-alone (presuming we ever see a sample of the data to be
processed. There's no complete CONOPS on how this application is to be used
(the most complete we've seen is "if the book is found, decrement some
counter, add user's name to another field" -- and? no report, no one else
is going to look at this information?).

	On my part (having no transportation, stuck in a neck brace after
having rolled over my late Jeep, etc.) the alternative to wasting time here
would be to dismantle an AR-10 class rifle and install an adjustable target
(but not top-end match) trigger and, if really ambitious, install an
improved trigger in my Ruger MK-II pistol (I have the parts for both --
just need to set up space and time to do the work).


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From drsalists at gmail.com  Sat Jan 22 23:59:18 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Sat, 22 Jan 2022 20:59:18 -0800
Subject: "undefined symbol" in C extension module
In-Reply-To: <j52844Fr70sU1@mid.individual.net>
References: <j52844Fr70sU1@mid.individual.net>
Message-ID: <CAGGBd_p5JNjqvecDkb-2KSpvQAuDPwaNcGJBX7w_TXdHBAWMgg@mail.gmail.com>

Perhaps try:
https://stromberg.dnsalias.org/svn/find-sym/trunk

It tries to find symbols in C libraries.

In this case, I believe you'll find it in -lpythonx.ym


On Sat, Jan 22, 2022 at 12:43 PM Robert Latest via Python-list <
python-list at python.org> wrote:

> Hi guys,
>
> I've written some CPython extension modules in the past without problems.
> Now
> after moving to a new Archlinux box with Python3.10 installed, I can't
> build
> them any more. Or rather, I can build them but not use them due to
> "undefined
> symbols" during linking. Here's ldd's output when used on the "spam"
> example
> library from the docs:
>
>         linux-vdso.so.1 (0x00007ffe2454a000)
>         libc.so.6 => /usr/lib/libc.so.6 (0x00007fb6b6eb9000)
>         /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb6b70a4000)
> undefined symbol: PyObject_Init (./build/lib.linux-x86_64-3.10/
>     spam.cpython-310-x86_64-linux-gnu.so)
>
> [and more of the same]
>
> It seems that I need to install some library to make this work, but which
> one?
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From barry at barrys-emacs.org  Sun Jan 23 07:20:30 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun, 23 Jan 2022 12:20:30 +0000
Subject: How to test for maildir 'folder' in Python?
In-Reply-To: <dd9ubi-r545.ln1@esprimo.zbmc.eu>
References: <dd9ubi-r545.ln1@esprimo.zbmc.eu>
Message-ID: <C9ADD9DB-7C68-4E7E-AE4B-152922F24B1D@barrys-emacs.org>



> On 22 Jan 2022, at 21:26, Chris Green <cl at isbd.net> wrote:
> 
> I have a script that walks a quite deep tree of mail messages to find
> and archive old messages.  I'm trying to convert it from mbox to
> maildir (as I now store my mail in maildir format).
> 
> So I need to test whether a point I have reached in the hierarchy is a
> maildir mailbox or not.  Using mbox format it's easy because 'folders'
> are directories and mailboxes are files.  However with maildir the
> 'folders' have directories within them so the simple tree walking goes
> down a level too far and finds 'folders' which aren't mailboxes called
> 'cur', 'new' and 'tmp'.
> 
> Is there any 'ready made' way in python to tell whether a directory is
> a maildir mailbox?  If not I suppose I'll simply have to check if
> there are 'cur', 'new' and 'tmp' directories within the directory
> which may or may not be a maildir.

You do not need to walk the tree.

The structure is 
	Maildir/cur, new, tmp - The INBOX
	Maildir/<encoded-mail-folder>/cur, new, tmp

The encoding prefixed with a "." and uses a "." between folder name parts.

An example from my Maildir is ".Python.Users/" for a folder that client
shows as Python/Users

You can see this by doing:

	 ls -a ~/Maildir

Barry

		


> 
> -- 
> Chris Green
> ?
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From cl at isbd.net  Sun Jan 23 04:07:21 2022
From: cl at isbd.net (Chris Green)
Date: Sun, 23 Jan 2022 09:07:21 +0000
Subject: How to test for maildir 'folder' in Python?
References: <dd9ubi-r545.ln1@esprimo.zbmc.eu>
 <YeyWp3wZUhE31sct@cskk.homeip.net>
 <mailman.239.1642895026.3079.python-list@python.org>
Message-ID: <9fivbi-h0k7.ln1@esprimo.zbmc.eu>

Cameron Simpson <cs at cskk.id.au> wrote:
> On 22Jan2022 21:26, Chris Green <cl at isbd.net> wrote:
> >So I need to test whether a point I have reached in the hierarchy is a
> >maildir mailbox or not.  Using mbox format it's easy because 'folders'
> >are directories and mailboxes are files.  However with maildir the
> >'folders' have directories within them so the simple tree walking goes
> >down a level too far and finds 'folders' which aren't mailboxes called
> >'cur', 'new' and 'tmp'.
> >
> >Is there any 'ready made' way in python to tell whether a directory is
> >a maildir mailbox?  If not I suppose I'll simply have to check if
> >there are 'cur', 'new' and 'tmp' directories within the directory
> >which may or may not be a maildir.
> 
> I was hoping that the provided mailbox/Maildir class had something, but 
> apparently not.
> 
So was I! :-)  It seems strange that something isn't provided as it's
rather fundamental to handling maildir.


> So I suggest fetching my cs.mailutils module from PyPI and using its 
> ismaildir(path) function, which does exactly what you describe.
> 
Thanks Cameron.

-- 
Chris Green
?

From boblatest at yahoo.com  Sun Jan 23 07:43:29 2022
From: boblatest at yahoo.com (Robert Latest)
Date: 23 Jan 2022 12:43:29 GMT
Subject: "undefined symbol" in C extension module
References: <j52844Fr70sU1@mid.individual.net>
 <CAGGBd_p5JNjqvecDkb-2KSpvQAuDPwaNcGJBX7w_TXdHBAWMgg@mail.gmail.com>
 <mailman.240.1642913972.3079.python-list@python.org>
Message-ID: <j550rhFcr67U1@mid.individual.net>

Dan Stromberg wrote:
> Perhaps try:
> https://stromberg.dnsalias.org/svn/find-sym/trunk
>
> It tries to find symbols in C libraries.
>
> In this case, I believe you'll find it in -lpythonx.ym

Thanks! Found out that ldd produces many errors also with working python
libraries. Turns out I tried to revive a package from v2.7. Need to adapt
it to 3.x

From cl at isbd.net  Sun Jan 23 07:55:20 2022
From: cl at isbd.net (Chris Green)
Date: Sun, 23 Jan 2022 12:55:20 +0000
Subject: How to test for maildir 'folder' in Python?
References: <dd9ubi-r545.ln1@esprimo.zbmc.eu>
 <C9ADD9DB-7C68-4E7E-AE4B-152922F24B1D@barrys-emacs.org>
 <mailman.241.1642940440.3079.python-list@python.org>
Message-ID: <oqvvbi-mbf8.ln1@esprimo.zbmc.eu>

Barry Scott <barry at barrys-emacs.org> wrote:
> 
> 
> > On 22 Jan 2022, at 21:26, Chris Green <cl at isbd.net> wrote:
> > 
> > I have a script that walks a quite deep tree of mail messages to find
> > and archive old messages.  I'm trying to convert it from mbox to
> > maildir (as I now store my mail in maildir format).
> > 
> > So I need to test whether a point I have reached in the hierarchy is a
> > maildir mailbox or not.  Using mbox format it's easy because 'folders'
> > are directories and mailboxes are files.  However with maildir the
> > 'folders' have directories within them so the simple tree walking goes
> > down a level too far and finds 'folders' which aren't mailboxes called
> > 'cur', 'new' and 'tmp'.
> > 
> > Is there any 'ready made' way in python to tell whether a directory is
> > a maildir mailbox?  If not I suppose I'll simply have to check if
> > there are 'cur', 'new' and 'tmp' directories within the directory
> > which may or may not be a maildir.
> 
> You do not need to walk the tree.
> 
> The structure is 
>         Maildir/cur, new, tmp - The INBOX
>         Maildir/<encoded-mail-folder>/cur, new, tmp
> 
> The encoding prefixed with a "." and uses a "." between folder name parts.
> 
> An example from my Maildir is ".Python.Users/" for a folder that client
> shows as Python/Users
> 
> You can see this by doing:
> 
>          ls -a ~/Maildir
> 
You're talking about one specific (and IMHO not very good) way of
creating a maildir hierarchy.  Here's what I see on my system:-

    chris at esprimo$ cd mail
    /home/chris/mail
    chris at esprimo$ ls -a
    .  ..  Gm  In  Ju  Li  Tm  folder
    chris at esprimo$ ls -a folder
    .         apexLodge     entertainment  greville  houseHome  maxMum    reshapers  tractor
    ..        archive       family         hardware  internet   money     riding     travel
    Scans     boating       france         health    isbd       pcc       software   ukraa
    Shopping  buySellAdmin  friends        holidays  jrml       personal  telecoms   vehicles
    chris at esprimo$ ls -a folder/Shopping
    .          cabin       cycling      garden     maxine      pets         stationery
    ..         car         diy          giftsEtc   maxmum      photography  telecoms
    apexLodge  clothesEtc  electrical   groupon    medicine    riding       toolstation
    boat       computer    electronics  houseHome  motorcycle  screwfix
    books      cpc         food         jrml       music       sportsEtc

No dot prefixes and my maildir hierarchy is a *real* directory hierarchy so
all the other file management utilities on my system can be used easily.



-- 
Chris Green
?

From toby at tobiah.org  Sun Jan 23 10:34:26 2022
From: toby at tobiah.org (Tobiah)
Date: Sun, 23 Jan 2022 07:34:26 -0800
Subject: Pandas or Numpy
Message-ID: <ssjsi3$3h0$1@gioia.aioe.org>

I know very little about either.  I need to handle score input files
for Csound.  Each line is a list of floating point values where each
column has a particular meaning to the program.

I need to compose large (hundreds, thousands, maybe millions) lists
and be able to do math on, or possibly sort by various columns, among other
operations.  A common requirement would be to do the same math operation
on each value in a column, or redistribute the values according to an
exponential curve, etc.

One wrinkle is that the first column of a Csound score is actually a
single character.  I was thinking if the data types all had to be the
same, then I'd make a translation table or just use the ascii value
of the character, but if I could mix types that might be a smidge better.

It seems like both libraries are possible choices.  Would one
be the obvious choice for me?


Thanks!

From rosuav at gmail.com  Sun Jan 23 12:25:04 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 24 Jan 2022 04:25:04 +1100
Subject: Pandas or Numpy
In-Reply-To: <ssjsi3$3h0$1@gioia.aioe.org>
References: <ssjsi3$3h0$1@gioia.aioe.org>
Message-ID: <CAPTjJmq3Ky22JGq1a3Ndz5NL+4f=QNLbA173G-8dExgJB1rPnQ@mail.gmail.com>

On Mon, 24 Jan 2022 at 04:10, Tobiah <toby at tobiah.org> wrote:
>
> I know very little about either.  I need to handle score input files
> for Csound.  Each line is a list of floating point values where each
> column has a particular meaning to the program.
>
> I need to compose large (hundreds, thousands, maybe millions) lists
> and be able to do math on, or possibly sort by various columns, among other
> operations.  A common requirement would be to do the same math operation
> on each value in a column, or redistribute the values according to an
> exponential curve, etc.
>
> One wrinkle is that the first column of a Csound score is actually a
> single character.  I was thinking if the data types all had to be the
> same, then I'd make a translation table or just use the ascii value
> of the character, but if I could mix types that might be a smidge better.
>
> It seems like both libraries are possible choices.  Would one
> be the obvious choice for me?
>

I'm not an expert, but that sounds like a job for Pandas to me. It's
excellent at handling tabular data, and yes, it's fine with a mixture
of types. Everything else you've described should work fine (not sure
how to redistribute on an exponential curve, but I'm sure it's not
hard).

BTW, Pandas is built on top of Numpy, so it's kinda "both".

ChrisA

From dieter at handshake.de  Sun Jan 23 12:34:10 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Sun, 23 Jan 2022 18:34:10 +0100
Subject: "undefined symbol" in C extension module
In-Reply-To: <j52844Fr70sU1@mid.individual.net>
References: <j52844Fr70sU1@mid.individual.net>
Message-ID: <25069.37266.981190.862787@ixdm.fritz.box>

Robert Latest wrote at 2022-1-22 11:29 GMT:
>I've written some CPython extension modules in the past without problems. Now
>after moving to a new Archlinux box with Python3.10 installed, I can't build
>them any more. Or rather, I can build them but not use them due to "undefined
>symbols" during linking. Here's ldd's output when used on the "spam" example
>library from the docs:
>
>	linux-vdso.so.1 (0x00007ffe2454a000)
>	libc.so.6 => /usr/lib/libc.so.6 (0x00007fb6b6eb9000)
>	/usr/lib64/ld-linux-x86-64.so.2 (0x00007fb6b70a4000)
>undefined symbol: PyObject_Init	(./build/lib.linux-x86_64-3.10/
>    spam.cpython-310-x86_64-linux-gnu.so)

There are important C-API changes for Python 3.10 (and above).
You may need to modify your extensions such that they work with
recent Python versions.

From drsalists at gmail.com  Sun Jan 23 12:59:41 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Sun, 23 Jan 2022 09:59:41 -0800
Subject: mac app from a python script?
Message-ID: <CAGGBd_pQ9MWTPYPLBXzRDp32zFCh06d+9GQ47vxnPp6KFuqVMg@mail.gmail.com>

Hi folks.

I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
if you're curious.

It works the way I want on Linux, but on macOS I seem to have to start it
from the command line, like:
    hcm --gui
...because I don't know how to create a macOS "app" that goes under
/Applications.

I don't really care about having a single executable on macOS, and I don't
really care about creating a .dmg or .pkg file. I'd be perfectly happy just
running "make install" and putting a #!'d script under /Applications with
appropriate metadata - but if it's easier to do a single executable, .dmg
or .pkg, I'd be fine with that.

I've experimented with a few different options for this (months ago),
mostly py2app, but it doesn't appear to like gi.repository.Gtk much.

What's the most straightforward way of installing a Python script under
/Applications on macOS?  I'd -love- to find a way of doing something
analogous to Linux' desktop-file-install - that is, something that isn't
click-happy, but GUI's are acceptable too if they'll get the job done.

Thanks!

From juliushamilton100 at gmail.com  Sun Jan 23 13:05:06 2022
From: juliushamilton100 at gmail.com (Julius Hamilton)
Date: Sun, 23 Jan 2022 19:05:06 +0100
Subject: Pandas or Numpy
In-Reply-To: <CAPTjJmq3Ky22JGq1a3Ndz5NL+4f=QNLbA173G-8dExgJB1rPnQ@mail.gmail.com>
References: <ssjsi3$3h0$1@gioia.aioe.org>
 <CAPTjJmq3Ky22JGq1a3Ndz5NL+4f=QNLbA173G-8dExgJB1rPnQ@mail.gmail.com>
Message-ID: <CAEsMKX3qeN4GbSXzfFkYS+7RFWCyAjoypyrSJKKLis0u7b494A@mail.gmail.com>

Hey,


I don?t know but in case you don?t get other good answers, I?m pretty sure
Numpy is more of a mathematical library and Pandas is definitely for
handling spreadsheet data.


So maybe both.


Julius

On Sun 23. Jan 2022 at 18:28, Chris Angelico <rosuav at gmail.com> wrote:

> On Mon, 24 Jan 2022 at 04:10, Tobiah <toby at tobiah.org> wrote:
> >
> > I know very little about either.  I need to handle score input files
> > for Csound.  Each line is a list of floating point values where each
> > column has a particular meaning to the program.
> >
> > I need to compose large (hundreds, thousands, maybe millions) lists
> > and be able to do math on, or possibly sort by various columns, among
> other
> > operations.  A common requirement would be to do the same math operation
> > on each value in a column, or redistribute the values according to an
> > exponential curve, etc.
> >
> > One wrinkle is that the first column of a Csound score is actually a
> > single character.  I was thinking if the data types all had to be the
> > same, then I'd make a translation table or just use the ascii value
> > of the character, but if I could mix types that might be a smidge better.
> >
> > It seems like both libraries are possible choices.  Would one
> > be the obvious choice for me?
> >
>
> I'm not an expert, but that sounds like a job for Pandas to me. It's
> excellent at handling tabular data, and yes, it's fine with a mixture
> of types. Everything else you've described should work fine (not sure
> how to redistribute on an exponential curve, but I'm sure it's not
> hard).
>
> BTW, Pandas is built on top of Numpy, so it's kinda "both".
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>

From drsalists at gmail.com  Sun Jan 23 13:20:50 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Sun, 23 Jan 2022 10:20:50 -0800
Subject: "undefined symbol" in C extension module
In-Reply-To: <j550rhFcr67U1@mid.individual.net>
References: <j52844Fr70sU1@mid.individual.net>
 <CAGGBd_p5JNjqvecDkb-2KSpvQAuDPwaNcGJBX7w_TXdHBAWMgg@mail.gmail.com>
 <mailman.240.1642913972.3079.python-list@python.org>
 <j550rhFcr67U1@mid.individual.net>
Message-ID: <CAGGBd_pVp2YP-fb=BPTeG7bak6Hy_Yo7AJb2RbRWBSuQHiRstQ@mail.gmail.com>

On Sun, Jan 23, 2022 at 9:02 AM Robert Latest via Python-list <
python-list at python.org> wrote:

> Dan Stromberg wrote:
> > Perhaps try:
> > https://stromberg.dnsalias.org/svn/find-sym/trunk
> >
> > It tries to find symbols in C libraries.
> >
> > In this case, I believe you'll find it in -lpythonx.ym
>
> Thanks! Found out that ldd produces many errors also with working python
> libraries. Turns out I tried to revive a package from v2.7. Need to adapt
> it to 3.x
>

I wonder if you're missing a -Wl,-rpath -Wl,/path/to/library/dir
?

From avigross at verizon.net  Sun Jan 23 13:29:09 2022
From: avigross at verizon.net (Avi Gross)
Date: Sun, 23 Jan 2022 18:29:09 +0000 (UTC)
Subject: Pandas or Numpy
In-Reply-To: <CAEsMKX3qeN4GbSXzfFkYS+7RFWCyAjoypyrSJKKLis0u7b494A@mail.gmail.com>
References: <ssjsi3$3h0$1@gioia.aioe.org>
 <CAPTjJmq3Ky22JGq1a3Ndz5NL+4f=QNLbA173G-8dExgJB1rPnQ@mail.gmail.com>
 <CAEsMKX3qeN4GbSXzfFkYS+7RFWCyAjoypyrSJKKLis0u7b494A@mail.gmail.com>
Message-ID: <983499973.718802.1642962549481@mail.yahoo.com>

Definitely it sounds like you may use both. Quite a bit of what people do using DataFrame objects includes working on copies of individual columns, which often are numpy Series or the like and in the other direction, can be used to create or amend a pandas DataFrame. Plus, many operations used to select some subset of rows will use things like??a data structure holding the integer indexes you want or a boolean where True means take it and False means ignore it.?
Many real life applications just incorporate both numpy as np and pandas as pd and sometimes also use other Python functionality such as lists or matrices which unfortunately generally are just a list of lists.?
Python was built with a different philosophy than some languages like R in which much of what is in numpy and pandas is built in, in? other ways, and has been extended by packages. Python built mostly on flexible lists so the modules you are asking about do make faster and perhaps better versions. And, to be fair, python has lots of nifty features that R is largely missing and had to be added externally.?
Both used properly can do a nice job on the kind of things you want but with a warning. Your description suggests some of the data you will be using or making can get quite large. So make sure you look into the dtypes for parts of your data so you do not store small integers in full sized integers but in signed or unsigned bytes. Data with only two possible values, might be stored as boolean. And note many operations can be done in place, rather than creating a new object. If you are worried about space usage, or time spent on garbage collection, as in any programming language, there are recognized ideas about how you might tighten up your code using existing paradigms. I do admit some tricks have costs and it takes real testing to see if it even matters to try. But in general, much of numpy and pandas are already optimized in lots of compiled code? so using these rather than python lists and other data structures can already be a big plus.
Good luck.

-----Original Message-----
From: Julius Hamilton <juliushamilton100 at gmail.com>
To: Chris Angelico <rosuav at gmail.com>
Cc: python-list at python.org
Sent: Sun, Jan 23, 2022 1:05 pm
Subject: Re: Pandas or Numpy

Hey,


I don?t know but in case you don?t get other good answers, I?m pretty sure
Numpy is more of a mathematical library and Pandas is definitely for
handling spreadsheet data.


So maybe both.


Julius

On Sun 23. Jan 2022 at 18:28, Chris Angelico <rosuav at gmail.com> wrote:

> On Mon, 24 Jan 2022 at 04:10, Tobiah <toby at tobiah.org> wrote:
> >
> > I know very little about either.? I need to handle score input files
> > for Csound.? Each line is a list of floating point values where each
> > column has a particular meaning to the program.
> >
> > I need to compose large (hundreds, thousands, maybe millions) lists
> > and be able to do math on, or possibly sort by various columns, among
> other
> > operations.? A common requirement would be to do the same math operation
> > on each value in a column, or redistribute the values according to an
> > exponential curve, etc.
> >
> > One wrinkle is that the first column of a Csound score is actually a
> > single character.? I was thinking if the data types all had to be the
> > same, then I'd make a translation table or just use the ascii value
> > of the character, but if I could mix types that might be a smidge better.
> >
> > It seems like both libraries are possible choices.? Would one
> > be the obvious choice for me?
> >
>
> I'm not an expert, but that sounds like a job for Pandas to me. It's
> excellent at handling tabular data, and yes, it's fine with a mixture
> of types. Everything else you've described should work fine (not sure
> how to redistribute on an exponential curve, but I'm sure it's not
> hard).
>
> BTW, Pandas is built on top of Numpy, so it's kinda "both".
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

From barry at barrys-emacs.org  Sun Jan 23 16:33:56 2022
From: barry at barrys-emacs.org (Barry)
Date: Sun, 23 Jan 2022 21:33:56 +0000
Subject: How to test for maildir 'folder' in Python?
In-Reply-To: <oqvvbi-mbf8.ln1@esprimo.zbmc.eu>
References: <oqvvbi-mbf8.ln1@esprimo.zbmc.eu>
Message-ID: <A0329D2D-80C8-4B73-89D9-F64AEB1E545B@barrys-emacs.org>



> On 23 Jan 2022, at 17:08, Chris Green <cl at isbd.net> wrote:
> 
> ?Barry Scott <barry at barrys-emacs.org> wrote:
>> 
>> 
>>>> On 22 Jan 2022, at 21:26, Chris Green <cl at isbd.net> wrote:
>>> 
>>> I have a script that walks a quite deep tree of mail messages to find
>>> and archive old messages.  I'm trying to convert it from mbox to
>>> maildir (as I now store my mail in maildir format).
>>> 
>>> So I need to test whether a point I have reached in the hierarchy is a
>>> maildir mailbox or not.  Using mbox format it's easy because 'folders'
>>> are directories and mailboxes are files.  However with maildir the
>>> 'folders' have directories within them so the simple tree walking goes
>>> down a level too far and finds 'folders' which aren't mailboxes called
>>> 'cur', 'new' and 'tmp'.
>>> 
>>> Is there any 'ready made' way in python to tell whether a directory is
>>> a maildir mailbox?  If not I suppose I'll simply have to check if
>>> there are 'cur', 'new' and 'tmp' directories within the directory
>>> which may or may not be a maildir.
>> 
>> You do not need to walk the tree.
>> 
>> The structure is 
>>        Maildir/cur, new, tmp - The INBOX
>>        Maildir/<encoded-mail-folder>/cur, new, tmp
>> 
>> The encoding prefixed with a "." and uses a "." between folder name parts.
>> 
>> An example from my Maildir is ".Python.Users/" for a folder that client
>> shows as Python/Users
>> 
>> You can see this by doing:
>> 
>>         ls -a ~/Maildir
>> 
> You're talking about one specific (and IMHO not very good) way of
> creating a maildir hierarchy.  Here's what I see on my system:

Oh right! That is imposed by dovecot that I use for the imap service.

Barry

>    chris at esprimo$ cd mail
>    /home/chris/mail
>    chris at esprimo$ ls -a
>    .  ..  Gm  In  Ju  Li  Tm  folder
>    chris at esprimo$ ls -a folder
>    .         apexLodge     entertainment  greville  houseHome  maxMum    reshapers  tractor
>    ..        archive       family         hardware  internet   money     riding     travel
>    Scans     boating       france         health    isbd       pcc       software   ukraa
>    Shopping  buySellAdmin  friends        holidays  jrml       personal  telecoms   vehicles
>    chris at esprimo$ ls -a folder/Shopping
>    .          cabin       cycling      garden     maxine      pets         stationery
>    ..         car         diy          giftsEtc   maxmum      photography  telecoms
>    apexLodge  clothesEtc  electrical   groupon    medicine    riding       toolstation
>    boat       computer    electronics  houseHome  motorcycle  screwfix
>    books      cpc         food         jrml       music       sportsEtc
> 
> No dot prefixes and my maildir hierarchy is a *real* directory hierarchy so
> all the other file management utilities on my system can be used easily.
> 
> 
> 
> -- 
> Chris Green
> ?
> -- 
> https://mail.python.org/mailman/listinfo/python-list


From barry at barrys-emacs.org  Sun Jan 23 16:37:02 2022
From: barry at barrys-emacs.org (Barry)
Date: Sun, 23 Jan 2022 21:37:02 +0000
Subject: mac app from a python script?
In-Reply-To: <CAGGBd_pQ9MWTPYPLBXzRDp32zFCh06d+9GQ47vxnPp6KFuqVMg@mail.gmail.com>
References: <CAGGBd_pQ9MWTPYPLBXzRDp32zFCh06d+9GQ47vxnPp6KFuqVMg@mail.gmail.com>
Message-ID: <838B9F3E-E7C9-4AB9-80CC-700AA83A1FA5@barrys-emacs.org>



> On 23 Jan 2022, at 18:02, Dan Stromberg <drsalists at gmail.com> wrote:
> 
> ?Hi folks.
> 
> I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
> Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
> if you're curious.
> 
> It works the way I want on Linux, but on macOS I seem to have to start it
> from the command line, like:
>    hcm --gui
> ...because I don't know how to create a macOS "app" that goes under
> /Applications.
> 
> I don't really care about having a single executable on macOS, and I don't
> really care about creating a .dmg or .pkg file. I'd be perfectly happy just
> running "make install" and putting a #!'d script under /Applications with
> appropriate metadata - but if it's easier to do a single executable, .dmg
> or .pkg, I'd be fine with that.
> 
> I've experimented with a few different options for this (months ago),
> mostly py2app, but it doesn't appear to like gi.repository.Gtk much.
> 
> What's the most straightforward way of installing a Python script under
> /Applications on macOS?  I'd -love- to find a way of doing something
> analogous to Linux' desktop-file-install - that is, something that isn't
> click-happy, but GUI's are acceptable too if they'll get the job done.

I do not have experience with great, but you might try pyinstaller.
I use it to make a PyQt Mac app successfully.

It?s command line plus setup script.

Barry
> 
> Thanks!
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From barry at barrys-emacs.org  Sun Jan 23 16:49:36 2022
From: barry at barrys-emacs.org (Barry)
Date: Sun, 23 Jan 2022 21:49:36 +0000
Subject: mac app from a python script?
In-Reply-To: <838B9F3E-E7C9-4AB9-80CC-700AA83A1FA5@barrys-emacs.org>
References: <838B9F3E-E7C9-4AB9-80CC-700AA83A1FA5@barrys-emacs.org>
Message-ID: <678D8EDF-982E-4EC5-8503-EC51F9DA233B@barrys-emacs.org>



> On 23 Jan 2022, at 21:40, Barry <barry at barrys-emacs.org> wrote:
> 
> ?
> 
>> On 23 Jan 2022, at 18:02, Dan Stromberg <drsalists at gmail.com> wrote:
>> 
>> ?Hi folks.
>> 
>> I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
>> Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
>> if you're curious.
>> 
>> It works the way I want on Linux, but on macOS I seem to have to start it
>> from the command line, like:
>>   hcm --gui
>> ...because I don't know how to create a macOS "app" that goes under
>> /Applications.
>> 
>> I don't really care about having a single executable on macOS, and I don't
>> really care about creating a .dmg or .pkg file. I'd be perfectly happy just
>> running "make install" and putting a #!'d script under /Applications with
>> appropriate metadata - but if it's easier to do a single executable, .dmg
>> or .pkg, I'd be fine with that.
>> 
>> I've experimented with a few different options for this (months ago),
>> mostly py2app, but it doesn't appear to like gi.repository.Gtk much.
>> 
>> What's the most straightforward way of installing a Python script under
>> /Applications on macOS?  I'd -love- to find a way of doing something
>> analogous to Linux' desktop-file-install - that is, something that isn't
>> click-happy, but GUI's are acceptable too if they'll get the job done.
> 
> I do not have experience with great, but you might try pyinstaller.
> I use it to make a PyQt Mac app successfully.

Gtk spell correct to great?

> 
> It?s command line plus setup script.
> 
> Barry
>> 
>> Thanks!
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>> 


From cs at cskk.id.au  Sun Jan 23 21:28:43 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Mon, 24 Jan 2022 13:28:43 +1100
Subject: mac app from a python script?
In-Reply-To: <838B9F3E-E7C9-4AB9-80CC-700AA83A1FA5@barrys-emacs.org>
References: <838B9F3E-E7C9-4AB9-80CC-700AA83A1FA5@barrys-emacs.org>
Message-ID: <Ye4O21CwXTkniBhH@cskk.homeip.net>

On 23Jan2022 21:37, Barry Scott <barry at barrys-emacs.org> wrote:
>I do not have experience with great, but you might try pyinstaller.
>I use it to make a PyQt Mac app successfully.

And I have used py2app/py2applet to create a PyQt Mac app.

AIUI, they both do the same basic task: bundle a Python interpreter and 
all the implied libraries from your script (i.e. including PyQt) into a 
Mac App tree, with an executable in the right place to invoke your 
module using the shipped interpreter.

So they make a Thing.app directory. A .dmg is just a disc image people 
ship these things in, often merely containing the .app and a symlink or 
alias to /Applications for the user to drag the app to.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From wlfraed at ix.netcom.com  Sun Jan 23 12:21:26 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sun, 23 Jan 2022 12:21:26 -0500
Subject: Pandas or Numpy
References: <ssjsi3$3h0$1@gioia.aioe.org>
Message-ID: <ga2rugp1e4ibur3sa8b724fo1h78ttvv45@4ax.com>

On Sun, 23 Jan 2022 07:34:26 -0800, Tobiah <toby at tobiah.org> declaimed the
following:

	I'm going to do a little rearranging of your paragraphs, since most of
them are domain specific, whereas the last (original) paragraph actually
gets to a core...

	Caveat: I've not written anything making use of either package, so my
only basis for commenting is what I've read on web sites (like the pandas
documentation site)

>
>It seems like both libraries are possible choices.  Would one
>be the obvious choice for me?
>
	pandas USES numpy internally but expands on it...

https://www.geeksforgeeks.org/difference-between-pandas-vs-numpy/
"""
A numpy array is a grid of values (of the same type) that are indexed by a
tuple of positive integers,
"""
Pandas provide high performance, fast, easy to use data structures and data
analysis tools for manipulating numeric data and time series.
"""

	Pandas, I believe, might get closer to what one might find in
statistical packages (like R) in that it supports tables/data-frames in
which each column may be a different data type. I don't know if it actually
has the statistics concepts of "factors" (eg: a column containing
"male"/"female" is not really a text column but closer to an enumeration
type).



>I need to compose large (hundreds, thousands, maybe millions) lists
>and be able to do math on, or possibly sort by various columns, among other
>operations.  A common requirement would be to do the same math operation
>on each value in a column, or redistribute the values according to an
>exponential curve, etc.

	En-mass operations should be supported; not sure about the
"redistribute" -- if you can define a function that takes one input
parameter (the existing value) and returns the redistributed value, I'd
think it should be feasible.

>
>One wrinkle is that the first column of a Csound score is actually a
>single character.  I was thinking if the data types all had to be the
>same, then I'd make a translation table or just use the ascii value
>of the character, but if I could mix types that might be a smidge better.
>

	Based upon the comparison I linked, pandas should be applicable for
this. For pure numpy, you'd likely be better off maintaining a separate
list (though sorting will require some tricks to keep the numpy array in
sync with the character list).

	Note that the comparison warns that /indexing/ in pandas can be slow.
If your manipulation is always "apply operationX to columnY" it should be
okay -- but "apply operationX to the nth row of columnY", and repeat for
other rows, is going to be slow.



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From mats at python.org  Mon Jan 24 10:30:41 2022
From: mats at python.org (Mats Wichmann)
Date: Mon, 24 Jan 2022 08:30:41 -0700
Subject: mac app from a python script?
In-Reply-To: <CAGGBd_pQ9MWTPYPLBXzRDp32zFCh06d+9GQ47vxnPp6KFuqVMg@mail.gmail.com>
References: <CAGGBd_pQ9MWTPYPLBXzRDp32zFCh06d+9GQ47vxnPp6KFuqVMg@mail.gmail.com>
Message-ID: <ff0c5cef-7e4f-18d5-b478-821b91db1df8@python.org>

On 1/23/22 10:59, Dan Stromberg wrote:
> Hi folks.
> 
> I have a Python 3 script (built on top of gi.respository.Gtk) that runs on
> Linux and macOS 11.5.  It's at https://stromberg.dnsalias.org/~strombrg/hcm/
> if you're curious.
> 
> It works the way I want on Linux, but on macOS I seem to have to start it
> from the command line, like:
>     hcm --gui
> ...because I don't know how to create a macOS "app" that goes under
> /Applications.
> 
> I don't really care about having a single executable on macOS, and I don't
> really care about creating a .dmg or .pkg file. I'd be perfectly happy just
> running "make install" and putting a #!'d script under /Applications with
> appropriate metadata - but if it's easier to do a single executable, .dmg
> or .pkg, I'd be fine with that.
> 
> I've experimented with a few different options for this (months ago),
> mostly py2app, but it doesn't appear to like gi.repository.Gtk much.
> 
> What's the most straightforward way of installing a Python script under
> /Applications on macOS?  I'd -love- to find a way of doing something
> analogous to Linux' desktop-file-install - that is, something that isn't
> click-happy, but GUI's are acceptable too if they'll get the job done.

There are notes on the topic here, see if they help:

https://docs.python.org/3/using/mac.html#how-to-run-a-python-script



From barry at barrys-emacs.org  Mon Jan 24 12:48:42 2022
From: barry at barrys-emacs.org (Barry)
Date: Mon, 24 Jan 2022 17:48:42 +0000
Subject: mac app from a python script?
In-Reply-To: <Ye4O21CwXTkniBhH@cskk.homeip.net>
References: <Ye4O21CwXTkniBhH@cskk.homeip.net>
Message-ID: <08CFBEFE-1749-4519-977E-C9BBD63D97F2@barrys-emacs.org>



> On 24 Jan 2022, at 02:31, Cameron Simpson <cs at cskk.id.au> wrote:
> 
> ?On 23Jan2022 21:37, Barry Scott <barry at barrys-emacs.org> wrote:
>> I do not have experience with great, but you might try pyinstaller.
>> I use it to make a PyQt Mac app successfully.
> 
> And I have used py2app/py2applet to create a PyQt Mac app.

Current Py2app does not work anymore for PyQt5. I have an question on
the Mac python list about this.

Barry

> 
> AIUI, they both do the same basic task: bundle a Python interpreter and 
> all the implied libraries from your script (i.e. including PyQt) into a 
> Mac App tree, with an executable in the right place to invoke your 
> module using the shipped interpreter.
> 
> So they make a Thing.app directory. A .dmg is just a disc image people 
> ship these things in, often merely containing the .app and a symlink or 
> alias to /Applications for the user to drag the app to.
> 
> Cheers,
> Cameron Simpson <cs at cskk.id.au>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From narshad.380 at gmail.com  Mon Jan 24 03:25:34 2022
From: narshad.380 at gmail.com (NArshad)
Date: Mon, 24 Jan 2022 00:25:34 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
References: <1845217957.793613.1642565848222@mail.yahoo.com>
 <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
Message-ID: <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>

On Sun, 23 Jan 2022 09:17:38 +1100, Chris Angelico <>
declaimed the following:


>
>Absolutely agree with making a console app first. Though I rather
>suspect the OP doesn't want to write any code at all.
>

I am not writing any code because I don?t know what code to do next. Still, I have made a dictionary now searching for what to do next in which one choice is MS SSIS and the second is numpy or pandas which AGross has written. If you want to see the code of making a dictionary then below it is:

xyz = {
    col[0].value: [cell.value for cell in col[1:]]
    for col in sheet.columns
}
print(xyz)

Now the problem is what to do next. If I had known, I must have submitted the whole project at my earliest convenience without coming over here in google groups. 



-Dennis Lee Bieber:
>
>And again... You will not find anything like you want... NOBODY is
>going to write a web application using a spreadsheet as the primary data
>storage. A spreadsheet, and custom transformation code, MIGHT be used to
>initially populate a database. (M$ SQL Server Integration Services is a
>whole system for defining import/transformation/clean-up "functions" for
>data sources to data base). A spreadsheet might be available as a
>report/extraction format from the database.
>

The problem is I don?t want Excel spreadsheet as a report/extraction format I want to UPDATE the Excel spreadsheet automatically with the latest values which are only in the two column cells and I don?t know which cells. Is it possible using SSIS? 

How you know so much about guns??

Why are you not in the favor of pandas if not openpyxl but if my problem is getting solved with MS SSIS then it's fine to leave openpyxl and pandas?



Avi Gross:
>
>Now forget the plain text file and consider the following. Read the EXCEL file one unit/cell at a time 
>in whatever column and row the data starts. This is doable, albeit not necessarily ideal, and 
>basically can be the same algorithm with a substitution in the part that gets the next line to getting 
>the cell below it. Testing for the end of the file may be different too.
>But what I think makes a bit more sense is to set up the server to have a rather long-term process 
>that sits there and waits for requests. It starts by reading all the data and perhaps modifying it 
>once in the ways I suggest above. In python, you would now have some data structure such as a 
>list or set or even dictionary or some variation from numpy or pandas. It does not matter much. The 
>data structure holding all books, or maybe all unique book names, would be there and ready and 
>waiting.
>Your program now sleeps until woken up and it is given the book name being searched for. It now 
>simply needs to apply whatever transformations you want to the received name and then do one of 
>many kinds of lookup to see if it is in the data structure representing the titles known to the library. 
>Using a set or dictionary means no search as it hashes the result and rapidly finds out if it has 
>previously been stored. Other methods like lists have choices between a brute force linear search if 
>the data remains in the original order or letting python do the work by asking if it is "in" the list to 
>more sophisticated methods like keeping it sorted and doing a binary search.
>I am guessing that for your need, a relatively unsophisticated method may work fine and it can be 
>changed if your project scales up to millions of books and multiple people searching at the same 
>time.
>

What you have written is difficult to find on google search and others. That's why writing all this to get something for search. 

From wlfraed at ix.netcom.com  Mon Jan 24 11:29:52 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Mon, 24 Jan 2022 11:29:52 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
 <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>
Message-ID: <5lgtugll2ascb0phbue0agrfuqmvjp4b7b@4ax.com>

On Mon, 24 Jan 2022 00:25:34 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>
>I am not writing any code because I don?t know what code to do next. Still, I have made a dictionary now searching for what to do next in which one choice is MS SSIS and the second is numpy or pandas which AGross has written. If you want to see the code of making a dictionary then below it is:
>
>xyz = {
>    col[0].value: [cell.value for cell in col[1:]]
>    for col in sheet.columns
>}
>print(xyz)
>

	Since none of us have seen a reasonable representation of the
spreadsheet (export it as CSV and paste the first 5-10 lines into a post)
we have no idea if the above even produces anything useful.

	You appear to be grabbing onto catchphrases in the hope that they will
provide you with some miraculous "I call this, that, and another, and look
-- it's done".

>Now the problem is what to do next. If I had known, I must have submitted the whole project at my earliest convenience without coming over here in google groups. 
>

	How would you do this assignment on paper? Print out your spreadsheet
and get a large pad of paper... Then write down each step you have to take
to process "one user request" in your system (make mock-ups of any
input/output screens). Make (horrors) a flow-chart showing the branch
points (book was found, book was not found).

	When you get the steps (aka "algorithm") documented well enough that
someone else can follow them on paper, you are ready to translate each of
those steps into code.

>The problem is I don?t want Excel spreadsheet as a report/extraction format I want to UPDATE the Excel spreadsheet automatically with the latest values which are only in the two column cells and I don?t know which cells. Is it possible using SSIS? 
>

	The Excel spreadsheet is almost the WORST data structure for this
assignment (a variable length record flat file would be the worst; fixed
length record flat file is 1960s business processing but would be more
useful as it allows for in-place updates).

	" I don?t know which cells" -- So how would you do this by hand, if
someone gave you a print-out of the spreadsheet? When you can describe the
operations in sufficient detail for someone else to follow them, you are
ready to convert them into code.

	M$ SSIS, as I mentioned, is a system for importing, TRANSFORMING, and
clean-up of data from external sources -- for inclusion into a M$ SQL
Server database. You insist you don't want to consider database
implementation, so SSIS will do nothing for you (besides, you'd have to
learn how to program ITS work-loads).

>How you know so much about guns??

	Irrelevant... Though I've owned firearms since the 1970s (well, late
60s if you count the Christmas gift of a .22 rifle while in the 7th grade).

>
>Why are you not in the favor of pandas if not openpyxl but if my problem is getting solved with MS SSIS then it's fine to leave openpyxl and pandas?

One:	pandas is still a case of doing import/transform/export; just
because it has functions to directly read (and I presume, write) .xlsx
format spreadsheet files you are still transforming the data into a pandas
"dataframe" object. pandas is using openpyxl to do that read/write.

Two:	pandas is optimized for numerical and "time series" processing
(think time-stamped tables of readings from, say, a weather station) on
which you want to produce smoothed trends. The documentation is explicit
that it is a bit slow when doing ad-hoc row/column indexing.

Three:	in terms of increasing complexity openpyxl will be the simplest,
and M$ SSIS is way above anything you might find useful without studying a
book.


>What you have written is difficult to find on google search and others. That's why writing all this to get something for search. 

	Because you WON'T find it on a search engine -- especially not at your
"tutorial" level.

	You need to 1) Know your programming language (since you are here, that
would be Python) including understanding any compound data structures it
supports (lists, dictionaries) and, at the least, procedural language
features (you can get by without having to do OOP, though most all
libraries are object based, you just need to invoke them, not write OOP
objects themselves); 2) Understand how to fit data structures with
algorithms; 3) be able to translate activities (as performed by a person)
into algorithms, and then from algorithms into language specific code.

	At a very high level, your assignment requires:

1	obtain a title from a user
2	search a file for that title
3	report results of the search

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
		BYSTANDERS WILL WANT TO LOOK AWAY NOW
		JUST GO TO THE NEXT POST

	The following is something I hacked together yesterday, and is untested
(I don't have the outer framework set up yet). So far, the only thing it
does is INITIALIZE an SQLite3 database (create the schema definition) and
install a default Admin registration for logging into the eventual
application.

	The schema is based upon the most complete description we've seen of
your data and intended processing. As such, it has an entry for each copy
of any given title (to track condition and if it is in-stock or
checked-out; check-outs are tracked as a separate relation in order to
allow for historical records).

	The use case/CONOPS for regular users is:

	The application will present the user with an option to log in using a
username and password, or to register a new log-in name. Registration will
verify the username is not being used by another, after which it will
obtain a password and real name from the user.

	Upon a successful user log-in, the application will present the user
with the option to reserve a book, or to cancel a previously made
reservation. 

	For the "reserve" option, the application will present the user with a
field in which to enter search terms. Search terms are words which may
appear in the book title or author names. If the search returns more than
five items, the count of candidate books will be displayed and the
application will ask the user to enter additional or different search
terms. If the search returns five or less items, the items will be
displayed and the application will ask the user to select one of them. Upon
the user indicating which item is desired, the application will verify that
at least one copy of the item is "Available" and, if available, set the
item to "Reserved", and create a reservation record identifying the copy,
the logged in user, and the date of the reservation.

...

	Unreserve is similar, except the search only includes items for which
the user has an active reservation record.

...

	ADMIN functions include: granting admin privileges to registered users;
adding new books and copy counts (or increasing copy counts if a new supply
has been obtained); clearing out stale reservations (in this scheme, the
reservation is just the request by a user for a book; if they don't pick up
the book within a few days, the reservation should be cancelled); checking
out books (checking out is when the reserving user actually takes
possession of the specific book copy [numbered] that they reserved);
producing report of overdue books; checking in books (when the book has
been returned). Other functions as the whim takes me...

-=-=-=-	database.py
"""
    Database.py     Encapsulates functions for working with the
BookDepository DataBase

"""

import sqlite3 as db
import os
import os.path as op
import datetime

import creationSql

STANDARDPRAGMAS = [ "PRAGMA foreign_references = ON;"   ]
DATABASE = "BookDepository.SQ3"

def initializeDB(replace = False):
    status = []
    if op.exists(DATABASE):
        if replace:
            old = DATABASE + "-%s" %
datetime.date.isoformat((datetime.date.today()))
            os.rename(DATABASE, old)
            status.append("Existing %s renamed to %s" % (DATABASE, old))
        else:
            status.append("Can not initialize database -- %s already
exists")
            return status
    con = db.connect(DATABASE, detect_types=db.PARSE_DECLTYPES)
    status.append("Created empty database %s" % DATABASE)
    for pragma in STANDARDPRAGMAS:
        con.execute(pragma)
        status.append("Set PRAGMAS")
    for tbl in creationSql.TABLES:
        con.execute(tbl)
        status.append("Created tables")
    status.append("Database schema defined")
    # TODO: need to create a password hashing function and random password
    password = "DbaAdm1n"
    user = "BRAdmin"
    con.execute("""INSERT INTO USERLOGIN (Username, Password, Last_Name,
First_Name, Admin) 
                    VALUES (%s, %s, %s, %s, %s)""",
                (user, password, "Book Depository DB Admin", None, True))
    status.append("Database Administrator account created -- Please record
these values")
    status.append("\t\tUSER LOGIN:\t%s" % user)
    status.append("\t\tPassword:\t%s" % password)
    status.append("\n\n*****\tRESTART BookDepositoy application to
login\n")
    con.commit()
    con.close()
    return status

-=-=-=-	creationSql.py
"""
    creationSql.py      Defines SQL statements to create an empty database

    SQL syntax is that of SQLite3

"""

USERLOGIN = """
--  UserLogin contains registration information for users of the
--  BookDepository system. This includes a login username,
--  hashed password, and real name (last/first), 
--  along with a flag indicating if the user has administrative
--  privileges (non-admin users may only make or cancel
--  reservations for specific books

CREATE TABLE IF NOT EXISTS UserLogin (
	ID          INTEGER     PRIMARY KEY,
	UserName    TEXT        UNIQUE          NOT NULL,
	Password    TEXT                        NOT NULL,
	Last_Name   TEXT                        NOT NULL,
	First_Name  TEXT,
	Admin       INTEGER     DEFAULT FALSE   NOT NULL
);
"""

PUBLISHER = """
--  Publisher contains the common name of book publishers

CREATE TABLE IF NOT EXISTS Publisher (
	ID          INTEGER     PRIMARY KEY,
	Publisher   TEXT                        NOT NULL
);
"""

AUTHOR = """
--  Author contains the name (last/first) of authors
CREATE TABLE IF NOT EXISTS Author (
	ID          INTEGER     PRIMARY KEY,
	Last_Name   TEXT                    NOT NULL,
	First_Name  TEXT
);
"""

BOOK = """
--  Book contains the ISBN (or alternate call number) for
--  book titles (SQLite3 generic TEXT type supports both
--  under one field; a more traditional RDBM would be
--  better served by creating an ISBN CHAR(13) [also for
--  ISBN-10 format], and a separate alternate-call number),
--  Title, Publisher reference, and copyright date (as
--  text, as these are normally just month and year)

CREATE TABLE IF NOT EXISTS Book (
	ID              INTEGER     PRIMARY KEY,
	ISBN_Call       TEXT                    NOT NULL,
	Title           TEXT                    NOT NULL,
	Publisher_ID    INTEGER                 NOT NULL 
		REFERENCES Publisher(ID)
			ON DELETE   RESTRICT
			ON UPDATE   CASCADE,
	Copyright_Date  TEXT
);
"""

COPY = """
--  Copy contains a record for each copy of each book, 
--  the check-out status for this copy (available,
--  out, reserved), and notes (condition) of the copy

CREATE TABLE IF NOT EXISTS Copy (
	ID              INTEGER     PRIMARY KEY,
	Book_ID         INTEGER                     NOT NULL
		REFERENCES Book(ID)
			ON DELETE   CASCADE
			ON UPDATE   CASCADE,
	Copy_Number     INTEGER                     NOT NULL,
	Status          TEXT        DEFAULT 'A'     NOT NULL 
		CHECK (upper(Status) in ('A', 'O', 'R')),
	Notes           TEXT,
	UNIQUE (Book_ID, Copy_Number)
);
"""

CHECKOUT = """
--  Checkout links specific copies of books to
--  registered users, and tracks reservation date,
--  checked out date (when user received the book),
--  and due date. There is also a flag indicating
--  if the record is active, or historical.

CREATE TABLE IF NOT EXISTS Checkout (
	ID          INTEGER     PRIMARY KEY,
	Copy_ID     INTEGER 
		REFERENCES Copy(ID)
			ON DELETE   SET NULL
			ON UPDATE   CASCADE,
	User_ID     INTEGER 
		REFERENCES UserLogin(ID)
			ON DELETE   SET NULL
			ON UPDATE   CASCADE,
	Reserved    DATE NOT NULL,
	Checkedout  DATE        DEFAULT NULL,
	Due         DATE        DEFAULT NULL,
	Active      INTEGER     DEFAULT TRUE    NOT NULL
);
"""

BOOK_AUTHOR = """
--  Book_Author links authors to book titles

CREATE TABLE IF NOT EXISTS Book_Author (
	ID          INTEGER     PRIMARY KEY,
	Book_ID     INTEGER                 NOT NULL
		REFERENCES Book(ID)
			ON DELETE CASCADE
			ON UPDATE CASCADE,
	Author_ID   INTEGER                 NOT NULL
		REFERENCES Author(ID)
			ON DELETE RESTRICT
			ON UPDATE CASCADE,
	UNIQUE (Book_ID, Author_ID)
);
"""

#   TABLES lists the individual SQL statements for each
#   database table. They are ordered such that
#   referenced tables are before the referencing
#   table
TABLES = [  USERLOGIN,  PUBLISHER,  AUTHOR,
            BOOK,       COPY,       CHECKOUT,
            BOOK_AUTHOR     ]

-=-=-=-

	Good thing this is a text only forum, or I'd toss in an
Entity-Relationship diagram for the database.


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From cs at cskk.id.au  Mon Jan 24 15:51:25 2022
From: cs at cskk.id.au (Cameron Simpson)
Date: Tue, 25 Jan 2022 07:51:25 +1100
Subject: mac app from a python script?
In-Reply-To: <08CFBEFE-1749-4519-977E-C9BBD63D97F2@barrys-emacs.org>
References: <08CFBEFE-1749-4519-977E-C9BBD63D97F2@barrys-emacs.org>
Message-ID: <Ye8RTQcuzawss35U@cskk.homeip.net>

On 24Jan2022 17:48, Barry Scott <barry at barrys-emacs.org> wrote:
>> On 24 Jan 2022, at 02:31, Cameron Simpson <cs at cskk.id.au> wrote:
>> ?On 23Jan2022 21:37, Barry Scott <barry at barrys-emacs.org> wrote:
>>> I do not have experience with great, but you might try pyinstaller.
>>> I use it to make a PyQt Mac app successfully.
>>
>> And I have used py2app/py2applet to create a PyQt Mac app.
>
>Current Py2app does not work anymore for PyQt5. I have an question on
>the Mac python list about this.

Ah. My last build was March 2021 (also PyQt5 - well, PySide2, which I 
believe uses Qt5 behind the scenes).  And I've certainly had troubles.

Thanks for the heads up.

Cheers,
Cameron Simpson <cs at cskk.id.au>

From drsalists at gmail.com  Mon Jan 24 21:54:06 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Mon, 24 Jan 2022 18:54:06 -0800
Subject: mac app from a python script?
In-Reply-To: <838B9F3E-E7C9-4AB9-80CC-700AA83A1FA5@barrys-emacs.org>
References: <CAGGBd_pQ9MWTPYPLBXzRDp32zFCh06d+9GQ47vxnPp6KFuqVMg@mail.gmail.com>
 <838B9F3E-E7C9-4AB9-80CC-700AA83A1FA5@barrys-emacs.org>
Message-ID: <CAGGBd_qQ+Txw7Lxq6UOmRUTWgBiRgCyS7uVcBDdffVn459M7gA@mail.gmail.com>

On Sun, Jan 23, 2022 at 1:37 PM Barry <barry at barrys-emacs.org> wrote:

>
> I do not have experience with great, but you might try pyinstaller.
> I use it to make a PyQt Mac app successfully.
>
> It?s command line plus setup script.
>

I wound up doing:
1) pyinstaller, as normal, but this created a broken all-encompassing
binary of my script.  At least it gave me the metadata I needed though.
2) overwriting /Applications/hcm.app/Contents/MacOS/hcm with a proper
#!/usr/bin/env python3 script

This mostly works.  It's a kinda ugly hack, and it doesn't stay in the dock
after starting it.

There should be a way of installing a python GUI in the macOS Applications
list, without having to bundle everything up into a big binary.

From tony.flury at btinternet.com  Tue Jan 25 09:42:56 2022
From: tony.flury at btinternet.com (Tony Flury)
Date: Tue, 25 Jan 2022 14:42:56 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <CAPTjJmoNk2gLNBXYhXNn=ryL9eFcRrP8wdW9bRvf3wSUmEJUjA@mail.gmail.com>
References: <042d2815-628c-e6b1-1eda-1e385dc26bcc@btinternet.com>
 <CAPTjJmq==4CHdPU2GiXefPG+OUG-jA6LRqWdxDxkdzjdTccANw@mail.gmail.com>
 <mailman.226.1642590598.3079.python-list@python.org>
 <j4u69iF3qpvU1@mid.individual.net>
 <CAPTjJmoNk2gLNBXYhXNn=ryL9eFcRrP8wdW9bRvf3wSUmEJUjA@mail.gmail.com>
Message-ID: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>


On 20/01/2022 23:12, Chris Angelico wrote:
> On Fri, 21 Jan 2022 at 10:10, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> On 20/01/22 12:09 am, Chris Angelico wrote:
>>> At this point, the refcount has indeed been increased.
>>>
>>>>            return self;
>>>>       }
>>> And then you say "my return value is this object".
>>>
>>> So you're incrementing the refcount, then returning it without
>>> incrementing the refcount. Your code is actually equivalent to "return
>>> self".
>> Chris, you're not making any sense. This is C code, so there's no
>> way that "return x" can change the reference count of x.
> Yeah, I wasn't clear there. It was equivalent to *the Python code*
> "return self". My apologies.
>
>>   > The normal thing to do is to add a reference to whatever you're
>>   > returning. For instance, Py_RETURN_NONE will incref None and then
>>   > return it.
>>   >
>>
>> The OP understands that this is not a normal thing to do. He's
>> trying to deliberately leak a reference for the purpose of diagnosing
>> a problem.
>>
>> It would be interesting to see what the actual refcount is after
>> calling this function.

After calling this without a double increment in the function the ref 
count is still only 1 - which means that the 'return self' effectively 
does a double decrement. My original message includes the Python code 
which calls this 'leaky' function and you can see that despite the 
'leaky POC' doing an increment ref count drops back to one after the return.

You are right this is not a normal thing to do, I am trying to 
understand the behaviour so my library does the correct thing in all 
cases - for example - imagine you have two nodes in a tree :

A --- > B

And your Python code has a named reference to A, and B also maintains a 
reference to A as it's parent.

In this case I would expect A to have a reference count of 2 (counted as 
3 through sys.getrefcount() - one for the named reference in the Python 
code - and one for the link from B back to A; I would also expect B to 
have a reference count here of 1 (just the reference from A - assuming 
nothing else referenced B).

My original code was incrementing the ref counts of A and B and then 
returning A. within the Python test code A had a refcount of 1 (and not 
the expected 2), but the refcount from B was correct as far as I could tell.


> Yes, and that's why I was saying it would need a *second* incref.
>
> ChrisA

Thank you to all of you for trying to help - I accept that the only way 
to make the code work is to do a 2nd increment.

I don't understand why doing a 'return self' would result in a double 
decrement - that seems utterly bizzare behaviour - it obviously works, 
but why.



-- 
Anthony Flury
email : anthony.flury at btinternet.com


From barry at barrys-emacs.org  Tue Jan 25 17:28:04 2022
From: barry at barrys-emacs.org (Barry)
Date: Tue, 25 Jan 2022 22:28:04 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
Message-ID: <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>



> On 25 Jan 2022, at 14:50, Tony Flury via Python-list <python-list at python.org> wrote:
> 
> ?
>> On 20/01/2022 23:12, Chris Angelico wrote:
>>> On Fri, 21 Jan 2022 at 10:10, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>>> On 20/01/22 12:09 am, Chris Angelico wrote:
>>>> At this point, the refcount has indeed been increased.
>>>> 
>>>>>           return self;
>>>>>      }
>>>> And then you say "my return value is this object".
>>>> 
>>>> So you're incrementing the refcount, then returning it without
>>>> incrementing the refcount. Your code is actually equivalent to "return
>>>> self".
>>> Chris, you're not making any sense. This is C code, so there's no
>>> way that "return x" can change the reference count of x.
>> Yeah, I wasn't clear there. It was equivalent to *the Python code*
>> "return self". My apologies.
>> 
>>>  > The normal thing to do is to add a reference to whatever you're
>>>  > returning. For instance, Py_RETURN_NONE will incref None and then
>>>  > return it.
>>>  >
>>> 
>>> The OP understands that this is not a normal thing to do. He's
>>> trying to deliberately leak a reference for the purpose of diagnosing
>>> a problem.
>>> 
>>> It would be interesting to see what the actual refcount is after
>>> calling this function.
> 
> After calling this without a double increment in the function the ref count is still only 1 - which means that the 'return self' effectively does a double decrement. My original message includes the Python code which calls this 'leaky' function and you can see that despite the 'leaky POC' doing an increment ref count drops back to one after the return.
> 
> You are right this is not a normal thing to do, I am trying to understand the behaviour so my library does the correct thing in all cases - for example - imagine you have two nodes in a tree :
> 
> A --- > B
> 
> And your Python code has a named reference to A, and B also maintains a reference to A as it's parent.
> 
> In this case I would expect A to have a reference count of 2 (counted as 3 through sys.getrefcount() - one for the named reference in the Python code - and one for the link from B back to A; I would also expect B to have a reference count here of 1 (just the reference from A - assuming nothing else referenced B).
> 
> My original code was incrementing the ref counts of A and B and then returning A. within the Python test code A had a refcount of 1 (and not the expected 2), but the refcount from B was correct as far as I could tell.
> 
> 
>> Yes, and that's why I was saying it would need a *second* incref.
>> 
>> ChrisA
> 
> Thank you to all of you for trying to help - I accept that the only way to make the code work is to do a 2nd increment.
> 
> I don't understand why doing a 'return self' would result in a double decrement - that seems utterly bizzare behaviour - it obviously works, but why.

The return self in C will not change the ref count.

I would suggest setting a break point in your code and stepping out of the function and seeing that python?s code does to the ref count.

Barry
> 
> 
> 
> -- 
> Anthony Flury
> email : anthony.flury at btinternet.com
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list
> 


From barry at barrys-emacs.org  Tue Jan 25 17:23:43 2022
From: barry at barrys-emacs.org (Barry)
Date: Tue, 25 Jan 2022 22:23:43 +0000
Subject: mac app from a python script?
In-Reply-To: <CAGGBd_qQ+Txw7Lxq6UOmRUTWgBiRgCyS7uVcBDdffVn459M7gA@mail.gmail.com>
References: <CAGGBd_qQ+Txw7Lxq6UOmRUTWgBiRgCyS7uVcBDdffVn459M7gA@mail.gmail.com>
Message-ID: <139083F1-E578-4C12-BE00-9F82020539A8@barrys-emacs.org>



> On 25 Jan 2022, at 02:56, Dan Stromberg <drsalists at gmail.com> wrote:
> 
> ?
> 
>> On Sun, Jan 23, 2022 at 1:37 PM Barry <barry at barrys-emacs.org> wrote:
>> 
>> I do not have experience with great, but you might try pyinstaller.
>> I use it to make a PyQt Mac app successfully.
>> 
>> It?s command line plus setup script.
> 
> I wound up doing:
> 1) pyinstaller, as normal, but this created a broken all-encompassing binary of my script.  At least it gave me the metadata I needed though.

You mean it created a .app bundle?

That is the way that macOS makes it trivia to install apps
Just by drag and drop in /Applications.

Barry

> 2) overwriting /Applications/hcm.app/Contents/MacOS/hcm with a proper #!/usr/bin/env python3 script


> 
> This mostly works.  It's a kinda ugly hack, and it doesn't stay in the dock after starting it.
> 
> There should be a way of installing a python GUI in the macOS Applications list, without having to bundle everything up into a big binary.

From tony.flury at btinternet.com  Tue Jan 25 18:50:35 2022
From: tony.flury at btinternet.com (Tony Flury)
Date: Tue, 25 Jan 2022 23:50:35 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
 <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
Message-ID: <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>


On 25/01/2022 22:28, Barry wrote:
>
>> On 25 Jan 2022, at 14:50, Tony Flury via Python-list<python-list at python.org>  wrote:
>>
>> ?
>>> On 20/01/2022 23:12, Chris Angelico wrote:
>>>> On Fri, 21 Jan 2022 at 10:10, Greg Ewing<greg.ewing at canterbury.ac.nz>  wrote:
>>>> On 20/01/22 12:09 am, Chris Angelico wrote:
>>>>> At this point, the refcount has indeed been increased.
>>>>>
>>>>>>            return self;
>>>>>>       }
>>>>> And then you say "my return value is this object".
>>>>>
>>>>> So you're incrementing the refcount, then returning it without
>>>>> incrementing the refcount. Your code is actually equivalent to "return
>>>>> self".
>>>> Chris, you're not making any sense. This is C code, so there's no
>>>> way that "return x" can change the reference count of x.
>>> Yeah, I wasn't clear there. It was equivalent to *the Python code*
>>> "return self". My apologies.
>>>
>>>>   > The normal thing to do is to add a reference to whatever you're
>>>>   > returning. For instance, Py_RETURN_NONE will incref None and then
>>>>   > return it.
>>>>   >
>>>>
>>>> The OP understands that this is not a normal thing to do. He's
>>>> trying to deliberately leak a reference for the purpose of diagnosing
>>>> a problem.
>>>>
>>>> It would be interesting to see what the actual refcount is after
>>>> calling this function.
>> After calling this without a double increment in the function the ref count is still only 1 - which means that the 'return self' effectively does a double decrement. My original message includes the Python code which calls this 'leaky' function and you can see that despite the 'leaky POC' doing an increment ref count drops back to one after the return.
>>
>> You are right this is not a normal thing to do, I am trying to understand the behaviour so my library does the correct thing in all cases - for example - imagine you have two nodes in a tree :
>>
>> A --- > B
>>
>> And your Python code has a named reference to A, and B also maintains a reference to A as it's parent.
>>
>> In this case I would expect A to have a reference count of 2 (counted as 3 through sys.getrefcount() - one for the named reference in the Python code - and one for the link from B back to A; I would also expect B to have a reference count here of 1 (just the reference from A - assuming nothing else referenced B).
>>
>> My original code was incrementing the ref counts of A and B and then returning A. within the Python test code A had a refcount of 1 (and not the expected 2), but the refcount from B was correct as far as I could tell.
>>
>>
>>> Yes, and that's why I was saying it would need a *second* incref.
>>>
>>> ChrisA
>> Thank you to all of you for trying to help - I accept that the only way to make the code work is to do a 2nd increment.
>>
>> I don't understand why doing a 'return self' would result in a double decrement - that seems utterly bizzare behaviour - it obviously works, but why.
> The return self in C will not change the ref count.
>
> I would suggest setting a break point in your code and stepping out of the function and seeing that python?s code does to the ref count.
>
> Barry

Barry,

something odd is going on because the Python code isn't doing anything 
that would cause the reference count to go from 3 inside the C function 
to 1 once the method call is complete.

As far as I know the only things that impact the reference counts are :

  * Increments due to assigning a new name or adding it to a container.
  * Increment due to passing the object to a function (since that binds
    a new name)
  * Decrements due to deletion of a name
  * Decrement due to going out of scope
  * Decrement due to being removed from a container.

None of those things are happening in the python code.

As posted in the original message - immediately before the call to the C 
function/method sys.getrefcount reports the count to be 2 (meaning it is 
actually a 1).

Inside the C function the ref count is incremented and the Py_REFCNT 
macro reports the count as 3 inside the C function as expected (1 for 
the name in the Python code, 1 for the argument as passed to the C 
function, and 1 for the increment), so outside the function one would 
expect the ref count to now be 2 (since the reference caused by calling 
the function is then reversed).

However - Immediately outside the C function and back in the Python code 
sys.getrefcount reports the count to be 2 again - meaning it is now 
really 1. So that means that the refcount has been decremented twice 
in-between the return of the C function and the execution of the 
immediate next python statement. I understand one of those decrements - 
the parameter's ref count is incremented on the way in so the same 
object is decremented on the way out (so that calls don't leak 
references) but I don't understand where the second decrement is coming 
from.

Again there is nothing in the Python code that would cause that 
decrement - the decrement behavior is in the Python runtime.

>>
>>
>> -- 
>> Anthony Flury
>> email :anthony.flury at btinternet.com
>>
>> -- 
>> https://mail.python.org/mailman/listinfo/python-list
>>
-- 
Anthony Flury
email :anthony.flury at btinternet.com

From python at mrabarnett.plus.com  Tue Jan 25 20:29:28 2022
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 26 Jan 2022 01:29:28 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
 <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
 <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
Message-ID: <f5510b80-6ebb-fc45-3a1a-82d8bba37968@mrabarnett.plus.com>

On 2022-01-25 23:50, Tony Flury via Python-list wrote:
> 
> On 25/01/2022 22:28, Barry wrote:
>>
>>> On 25 Jan 2022, at 14:50, Tony Flury via Python-list<python-list at python.org>  wrote:
>>>
>>> ?
>>>> On 20/01/2022 23:12, Chris Angelico wrote:
>>>>> On Fri, 21 Jan 2022 at 10:10, Greg Ewing<greg.ewing at canterbury.ac.nz>  wrote:
>>>>> On 20/01/22 12:09 am, Chris Angelico wrote:
>>>>>> At this point, the refcount has indeed been increased.
>>>>>>
>>>>>>>            return self;
>>>>>>>       }
>>>>>> And then you say "my return value is this object".
>>>>>>
>>>>>> So you're incrementing the refcount, then returning it without
>>>>>> incrementing the refcount. Your code is actually equivalent to "return
>>>>>> self".
>>>>> Chris, you're not making any sense. This is C code, so there's no
>>>>> way that "return x" can change the reference count of x.
>>>> Yeah, I wasn't clear there. It was equivalent to *the Python code*
>>>> "return self". My apologies.
>>>>
>>>>>   > The normal thing to do is to add a reference to whatever you're
>>>>>   > returning. For instance, Py_RETURN_NONE will incref None and then
>>>>>   > return it.
>>>>>   >
>>>>>
>>>>> The OP understands that this is not a normal thing to do. He's
>>>>> trying to deliberately leak a reference for the purpose of diagnosing
>>>>> a problem.
>>>>>
>>>>> It would be interesting to see what the actual refcount is after
>>>>> calling this function.
>>> After calling this without a double increment in the function the ref count is still only 1 - which means that the 'return self' effectively does a double decrement. My original message includes the Python code which calls this 'leaky' function and you can see that despite the 'leaky POC' doing an increment ref count drops back to one after the return.
>>>
>>> You are right this is not a normal thing to do, I am trying to understand the behaviour so my library does the correct thing in all cases - for example - imagine you have two nodes in a tree :
>>>
>>> A --- > B
>>>
>>> And your Python code has a named reference to A, and B also maintains a reference to A as it's parent.
>>>
>>> In this case I would expect A to have a reference count of 2 (counted as 3 through sys.getrefcount() - one for the named reference in the Python code - and one for the link from B back to A; I would also expect B to have a reference count here of 1 (just the reference from A - assuming nothing else referenced B).
>>>
>>> My original code was incrementing the ref counts of A and B and then returning A. within the Python test code A had a refcount of 1 (and not the expected 2), but the refcount from B was correct as far as I could tell.
>>>
>>>
>>>> Yes, and that's why I was saying it would need a *second* incref.
>>>>
>>>> ChrisA
>>> Thank you to all of you for trying to help - I accept that the only way to make the code work is to do a 2nd increment.
>>>
>>> I don't understand why doing a 'return self' would result in a double decrement - that seems utterly bizzare behaviour - it obviously works, but why.
>> The return self in C will not change the ref count.
>>
>> I would suggest setting a break point in your code and stepping out of the function and seeing that python?s code does to the ref count.
>>
>> Barry
> 
> Barry,
> 
> something odd is going on because the Python code isn't doing anything
> that would cause the reference count to go from 3 inside the C function
> to 1 once the method call is complete.
> 
> As far as I know the only things that impact the reference counts are :
> 
>    * Increments due to assigning a new name or adding it to a container.
>    * Increment due to passing the object to a function (since that binds
>      a new name)
>    * Decrements due to deletion of a name
>    * Decrement due to going out of scope
>    * Decrement due to being removed from a container.
> 
> None of those things are happening in the python code.
> 
> As posted in the original message - immediately before the call to the C
> function/method sys.getrefcount reports the count to be 2 (meaning it is
> actually a 1).
> 
> Inside the C function the ref count is incremented and the Py_REFCNT
> macro reports the count as 3 inside the C function as expected (1 for
> the name in the Python code, 1 for the argument as passed to the C
> function, and 1 for the increment), so outside the function one would
> expect the ref count to now be 2 (since the reference caused by calling
> the function is then reversed).
> 
> However - Immediately outside the C function and back in the Python code
> sys.getrefcount reports the count to be 2 again - meaning it is now
> really 1. So that means that the refcount has been decremented twice
> in-between the return of the C function and the execution of the
> immediate next python statement. I understand one of those decrements -
> the parameter's ref count is incremented on the way in so the same
> object is decremented on the way out (so that calls don't leak
> references) but I don't understand where the second decrement is coming
> from.
> 
> Again there is nothing in the Python code that would cause that
> decrement - the decrement behavior is in the Python runtime.
> 
The function returns a result, an object.

The calling code is discarding the result, so it's being DECREFed.

For example:

     def foo():
         return Node()

returns a new node, so its refcount is 1.

Calling 'foo' as statement:

     foo()

discards the result; the result is DECREFed back to 0 and garbage collected.

If you wanted your C function to return None, you'd have:

     Py_INCREF(Py_NONE);
     return Py_None;

or, more succinctly:

     Py_RETURN_NONE;

But you're returning the object itself, and you're INCREFing it first, 
which is what you need to do anyway.

The 'extra' DECREF is coming from the result (i.e. self) being discarded.

If it wasn't DECREFed, a function could create a new object (refcount == 
1) and return it, and if the function was being called as a statement, 
it would be discarded with the refcount still == 1, leading to a memory 
leak.

From drsalists at gmail.com  Tue Jan 25 21:41:25 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 25 Jan 2022 18:41:25 -0800
Subject: mac app from a python script?
In-Reply-To: <139083F1-E578-4C12-BE00-9F82020539A8@barrys-emacs.org>
References: <CAGGBd_qQ+Txw7Lxq6UOmRUTWgBiRgCyS7uVcBDdffVn459M7gA@mail.gmail.com>
 <139083F1-E578-4C12-BE00-9F82020539A8@barrys-emacs.org>
Message-ID: <CAGGBd_rY4Hy-Mqop2kmsT8utc_M=YhyFLy5Pc36fgOqW+ofpaQ@mail.gmail.com>

On Tue, Jan 25, 2022 at 2:23 PM Barry <barry at barrys-emacs.org> wrote:

>
> On 25 Jan 2022, at 02:56, Dan Stromberg <drsalists at gmail.com> wrote:
>
> ?
>
> On Sun, Jan 23, 2022 at 1:37 PM Barry <barry at barrys-emacs.org> wrote:
>
>>
>> I do not have experience with great, but you might try pyinstaller.
>> I use it to make a PyQt Mac app successfully.
>>
>> It?s command line plus setup script.
>>
>
> I wound up doing:
> 1) pyinstaller, as normal, but this created a broken all-encompassing
> binary of my script.  At least it gave me the metadata I needed though.
>
>
> You mean it created a .app bundle?
>
> That is the way that macOS makes it trivia to install apps
> Just by drag and drop in /Applications.
>

Yes, it created an hcm.app for me.  But the executable it created didn't
work.  Hence the hack.

From drsalists at gmail.com  Wed Jan 26 00:17:14 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Tue, 25 Jan 2022 21:17:14 -0800
Subject: mac app from a python script?
In-Reply-To: <CAGGBd_rY4Hy-Mqop2kmsT8utc_M=YhyFLy5Pc36fgOqW+ofpaQ@mail.gmail.com>
References: <CAGGBd_qQ+Txw7Lxq6UOmRUTWgBiRgCyS7uVcBDdffVn459M7gA@mail.gmail.com>
 <139083F1-E578-4C12-BE00-9F82020539A8@barrys-emacs.org>
 <CAGGBd_rY4Hy-Mqop2kmsT8utc_M=YhyFLy5Pc36fgOqW+ofpaQ@mail.gmail.com>
Message-ID: <CAGGBd_qHeOv3iH1QU=zW2kuHuZ7Up-xZEf73ek7X61i=SPR-Kg@mail.gmail.com>

On Tue, Jan 25, 2022 at 6:41 PM Dan Stromberg <drsalists at gmail.com> wrote:

>
> On Tue, Jan 25, 2022 at 2:23 PM Barry <barry at barrys-emacs.org> wrote:
>
>>
>> On 25 Jan 2022, at 02:56, Dan Stromberg <drsalists at gmail.com> wrote:
>>
>> ?
>>
>> On Sun, Jan 23, 2022 at 1:37 PM Barry <barry at barrys-emacs.org> wrote:
>>
>>>
>>> I do not have experience with great, but you might try pyinstaller.
>>> I use it to make a PyQt Mac app successfully.
>>>
>>> It?s command line plus setup script.
>>>
>>
>> I wound up doing:
>> 1) pyinstaller, as normal, but this created a broken all-encompassing
>> binary of my script.  At least it gave me the metadata I needed though.
>>
>>
>> You mean it created a .app bundle?
>>
>> That is the way that macOS makes it trivia to install apps
>> Just by drag and drop in /Applications.
>>
>
> Yes, it created an hcm.app for me.  But the executable it created didn't
> work.  Hence the hack.
>

More specifically:
$ /Applications/hcm.app/Contents/MacOS/hcm --gui
cmd output started 2022 Tue Jan 25 09:00:33 PM PST
Traceback (most recent call last):
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 146, in
<module>
    _run()
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 129, in
_run
    exec(compile(source, path, "exec"), globals(), globals())
  File "/Applications/hcm.app/Contents/Resources/hcm.py", line 1950, in
<module>
    import gi
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in
_load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "gi/__init__.pyc", line 40, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in
_load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "gi/_gi.pyc", line 14, in <module>
  File "gi/_gi.pyc", line 10, in __load
  File "imp.pyc", line 342, in load_dynamic
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 627, in
_load_backward_compatible
  File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 36, in
load_module
    return imp.load_module(
  File "imp.pyc", line 244, in load_module
  File "imp.pyc", line 216, in load_package
  File "<frozen importlib._bootstrap>", line 710, in _load
AttributeError: 'NoneType' object has no attribute 'name'
2022-01-25 21:00:34.576 hcm[62695:1322031] hcm Error
^C^\Quit: 3
above cmd output done    2022 Tue Jan 25 09:00:42 PM PST
dstromberg at Daniels-Mini:~/src/home-svn/hcm/trunk x86_64-apple-darwin20.6.0
61933

$ file /Applications/hcm.app/Contents/MacOS/hcm
cmd output started 2022 Tue Jan 25 09:00:54 PM PST
/Applications/hcm.app/Contents/MacOS/hcm: Mach-O 64-bit executable x86_64

But if I replace /Applications/hcm.app/Contents/MacOS/hcm with a symlink to
a wrapper shell script, hcm runs fine from the Applications menu.

It seems that gi.repository.Gtk applications are not packaged correctly by
pyinstaller and py2app.  Some Python modules require a little assistance to
be packaged up into a Mach-O executable neatly by such tools.  But it's
just not that important to me to have a Mach-O of my app.

From tony.flury at btinternet.com  Wed Jan 26 03:03:36 2022
From: tony.flury at btinternet.com (Tony Flury)
Date: Wed, 26 Jan 2022 08:03:36 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <f5510b80-6ebb-fc45-3a1a-82d8bba37968@mrabarnett.plus.com>
References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
 <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
 <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
 <f5510b80-6ebb-fc45-3a1a-82d8bba37968@mrabarnett.plus.com>
Message-ID: <3d73447c-c793-fb9e-c690-5b6ab2951e31@btinternet.com>


On 26/01/2022 01:29, MRAB wrote:
> On 2022-01-25 23:50, Tony Flury via Python-list wrote:
>>
>> On 25/01/2022 22:28, Barry wrote:
>>>
>>>> On 25 Jan 2022, at 14:50, Tony Flury via 
>>>> Python-list<python-list at python.org>? wrote:
>>>>
>>>> ?
>>>>> On 20/01/2022 23:12, Chris Angelico wrote:
>>>>>> On Fri, 21 Jan 2022 at 10:10, Greg 
>>>>>> Ewing<greg.ewing at canterbury.ac.nz>? wrote:
>>>>>> On 20/01/22 12:09 am, Chris Angelico wrote:
>>>>>>> At this point, the refcount has indeed been increased.
>>>>>>>
>>>>>>>> ?????????? return self;
>>>>>>>> ????? }
>>>>>>> And then you say "my return value is this object".
>>>>>>>
>>>>>>> So you're incrementing the refcount, then returning it without
>>>>>>> incrementing the refcount. Your code is actually equivalent to 
>>>>>>> "return
>>>>>>> self".
>>>>>> Chris, you're not making any sense. This is C code, so there's no
>>>>>> way that "return x" can change the reference count of x.
>>>>> Yeah, I wasn't clear there. It was equivalent to *the Python code*
>>>>> "return self". My apologies.
>>>>>
>>>>>> ? > The normal thing to do is to add a reference to whatever you're
>>>>>> ? > returning. For instance, Py_RETURN_NONE will incref None and 
>>>>>> then
>>>>>> ? > return it.
>>>>>> ? >
>>>>>>
>>>>>> The OP understands that this is not a normal thing to do. He's
>>>>>> trying to deliberately leak a reference for the purpose of 
>>>>>> diagnosing
>>>>>> a problem.
>>>>>>
>>>>>> It would be interesting to see what the actual refcount is after
>>>>>> calling this function.
>>>> After calling this without a double increment in the function the 
>>>> ref count is still only 1 - which means that the 'return self' 
>>>> effectively does a double decrement. My original message includes 
>>>> the Python code which calls this 'leaky' function and you can see 
>>>> that despite the 'leaky POC' doing an increment ref count drops 
>>>> back to one after the return.
>>>>
>>>> You are right this is not a normal thing to do, I am trying to 
>>>> understand the behaviour so my library does the correct thing in 
>>>> all cases - for example - imagine you have two nodes in a tree :
>>>>
>>>> A --- > B
>>>>
>>>> And your Python code has a named reference to A, and B also 
>>>> maintains a reference to A as it's parent.
>>>>
>>>> In this case I would expect A to have a reference count of 2 
>>>> (counted as 3 through sys.getrefcount() - one for the named 
>>>> reference in the Python code - and one for the link from B back to 
>>>> A; I would also expect B to have a reference count here of 1 (just 
>>>> the reference from A - assuming nothing else referenced B).
>>>>
>>>> My original code was incrementing the ref counts of A and B and 
>>>> then returning A. within the Python test code A had a refcount of 1 
>>>> (and not the expected 2), but the refcount from B was correct as 
>>>> far as I could tell.
>>>>
>>>>
>>>>> Yes, and that's why I was saying it would need a *second* incref.
>>>>>
>>>>> ChrisA
>>>> Thank you to all of you for trying to help - I accept that the only 
>>>> way to make the code work is to do a 2nd increment.
>>>>
>>>> I don't understand why doing a 'return self' would result in a 
>>>> double decrement - that seems utterly bizzare behaviour - it 
>>>> obviously works, but why.
>>> The return self in C will not change the ref count.
>>>
>>> I would suggest setting a break point in your code and stepping out 
>>> of the function and seeing that python?s code does to the ref count.
>>>
>>> Barry
>>
>> Barry,
>>
>> something odd is going on because the Python code isn't doing anything
>> that would cause the reference count to go from 3 inside the C function
>> to 1 once the method call is complete.
>>
>> As far as I know the only things that impact the reference counts are :
>>
>> ?? * Increments due to assigning a new name or adding it to a container.
>> ?? * Increment due to passing the object to a function (since that binds
>> ???? a new name)
>> ?? * Decrements due to deletion of a name
>> ?? * Decrement due to going out of scope
>> ?? * Decrement due to being removed from a container.
>>
>> None of those things are happening in the python code.
>>
>> As posted in the original message - immediately before the call to the C
>> function/method sys.getrefcount reports the count to be 2 (meaning it is
>> actually a 1).
>>
>> Inside the C function the ref count is incremented and the Py_REFCNT
>> macro reports the count as 3 inside the C function as expected (1 for
>> the name in the Python code, 1 for the argument as passed to the C
>> function, and 1 for the increment), so outside the function one would
>> expect the ref count to now be 2 (since the reference caused by calling
>> the function is then reversed).
>>
>> However - Immediately outside the C function and back in the Python code
>> sys.getrefcount reports the count to be 2 again - meaning it is now
>> really 1. So that means that the refcount has been decremented twice
>> in-between the return of the C function and the execution of the
>> immediate next python statement. I understand one of those decrements -
>> the parameter's ref count is incremented on the way in so the same
>> object is decremented on the way out (so that calls don't leak
>> references) but I don't understand where the second decrement is coming
>> from.
>>
>> Again there is nothing in the Python code that would cause that
>> decrement - the decrement behavior is in the Python runtime.
>>
> The function returns a result, an object.
>
> The calling code is discarding the result, so it's being DECREFed.
>
> For example:
>
> ??? def foo():
> ??????? return Node()
>
> returns a new node, so its refcount is 1.
>
> Calling 'foo' as statement:
>
> ??? foo()
>
> discards the result; the result is DECREFed back to 0 and garbage 
> collected.
>
> If you wanted your C function to return None, you'd have:
>
> ??? Py_INCREF(Py_NONE);
> ??? return Py_None;
>
> or, more succinctly:
>
> ??? Py_RETURN_NONE;
>
> But you're returning the object itself, and you're INCREFing it first, 
> which is what you need to do anyway.
>
> The 'extra' DECREF is coming from the result (i.e. self) being discarded.
>
> If it wasn't DECREFed, a function could create a new object (refcount 
> == 1) and return it, and if the function was being called as a 
> statement, it would be discarded with the refcount still == 1, leading 
> to a memory leak.

So according to that I should increment twice if and only if the calling 
code is using the result - which you can't tell in the C code - which is 
very odd behaviour.

There is clearly something very deep here that I am simply not 
understanding.

-- 
Anthony Flury
email : anthony.flury at btinternet.com


From rosuav at gmail.com  Wed Jan 26 03:20:30 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Wed, 26 Jan 2022 19:20:30 +1100
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <3d73447c-c793-fb9e-c690-5b6ab2951e31@btinternet.com>
References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
 <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
 <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
 <f5510b80-6ebb-fc45-3a1a-82d8bba37968@mrabarnett.plus.com>
 <3d73447c-c793-fb9e-c690-5b6ab2951e31@btinternet.com>
Message-ID: <CAPTjJmo_ai2rOCVQtfH2hWBaFNzAy2kyYY4swakCCELU3fBJ8w@mail.gmail.com>

On Wed, 26 Jan 2022 at 19:04, Tony Flury via Python-list
<python-list at python.org> wrote:
>
> So according to that I should increment twice if and only if the calling
> code is using the result - which you can't tell in the C code - which is
> very odd behaviour.

No, the return value from your C function will *always* have a
reference taken. Whether the return value is "used" or just dropped,
there's always going to be one ref used by the returning itself.

The standard way to return a value is always to incref it, then return
the pointer. That is exactly equivalent to Python saying "return
<thing>".

Incrementing twice is ONLY because you want to leak a reference.

ChrisA

From greg.ewing at canterbury.ac.nz  Wed Jan 26 01:57:54 2022
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 26 Jan 2022 19:57:54 +1300
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <mailman.259.1643154681.3079.python-list@python.org>
References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
 <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
 <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
 <mailman.259.1643154681.3079.python-list@python.org>
Message-ID: <j5c9nkFp2neU1@mid.individual.net>

The convention for refcounting in CPython is that a function
takes borrowed references as arguments and returns a new
reference.

The 'self' argument passed in is a borrowed reference. If you
want to return it, you need to create a new reference by
increfing it.

So what you have written is just the correct way to write
a do-nothing function that returns its argument, i.e. the
equivalent of the Python function

    def f(self):
       return self

As others have said, if you want to leak a reference, you
need an extra incref besides the one that's a normal part
of the return convention.

-- 
Greg


From tony.flury at btinternet.com  Wed Jan 26 08:15:01 2022
From: tony.flury at btinternet.com (Tony Flury)
Date: Wed, 26 Jan 2022 13:15:01 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <CAPTjJmo_ai2rOCVQtfH2hWBaFNzAy2kyYY4swakCCELU3fBJ8w@mail.gmail.com>
References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
 <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
 <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
 <f5510b80-6ebb-fc45-3a1a-82d8bba37968@mrabarnett.plus.com>
 <3d73447c-c793-fb9e-c690-5b6ab2951e31@btinternet.com>
 <CAPTjJmo_ai2rOCVQtfH2hWBaFNzAy2kyYY4swakCCELU3fBJ8w@mail.gmail.com>
Message-ID: <a438147e-a493-7bc7-5f6a-5dc6a340eff2@btinternet.com>


On 26/01/2022 08:20, Chris Angelico wrote:
> On Wed, 26 Jan 2022 at 19:04, Tony Flury via Python-list
> <python-list at python.org> wrote:
>> So according to that I should increment twice if and only if the calling
>> code is using the result - which you can't tell in the C code - which is
>> very odd behaviour.
> No, the return value from your C function will *always* have a
> reference taken. Whether the return value is "used" or just dropped,
> there's always going to be one ref used by the returning itself.
>
> The standard way to return a value is always to incref it, then return
> the pointer. That is exactly equivalent to Python saying "return
> <thing>".
>
> Incrementing twice is ONLY because you want to leak a reference.
>
> ChrisA

Chris,

You keep saying I am leaking a reference - my original code (not the POC 
in the email) wasn't intending to leak a reference, it was incrementing 
the reference count in order to accurately count references, from other 
objects and i needed to double increment there so that the reference 
count remained correct outside of the C code.

I did try to be clear - my intention was never to leak a reference (I 
have been writing s/w long enough to know leaks are bad) - my POC code 
in the original message was the only code which deliberately leaked a 
reference in order to simply illustrate the problem.

I do appreciate the help you have tried to give - so thank you.

-- 
Anthony Flury
email : anthony.flury at btinternet.com


From barry at barrys-emacs.org  Wed Jan 26 17:35:38 2022
From: barry at barrys-emacs.org (Barry)
Date: Wed, 26 Jan 2022 22:35:38 +0000
Subject: mac app from a python script?
In-Reply-To: <CAGGBd_qHeOv3iH1QU=zW2kuHuZ7Up-xZEf73ek7X61i=SPR-Kg@mail.gmail.com>
References: <CAGGBd_qHeOv3iH1QU=zW2kuHuZ7Up-xZEf73ek7X61i=SPR-Kg@mail.gmail.com>
Message-ID: <B65F3620-D6AF-4037-BF1A-075DEFA2D351@barrys-emacs.org>



> On 26 Jan 2022, at 05:17, Dan Stromberg <drsalists at gmail.com> wrote:
> 
> ?
> 
>> On Tue, Jan 25, 2022 at 6:41 PM Dan Stromberg <drsalists at gmail.com> wrote:
>> 
>>> On Tue, Jan 25, 2022 at 2:23 PM Barry <barry at barrys-emacs.org> wrote:
>>> 
>>>>> On 25 Jan 2022, at 02:56, Dan Stromberg <drsalists at gmail.com> wrote:
>>>>> 
>>>> ?
>>>> 
>>>>> On Sun, Jan 23, 2022 at 1:37 PM Barry <barry at barrys-emacs.org> wrote:
>>>>> 
>>>>> I do not have experience with great, but you might try pyinstaller.
>>>>> I use it to make a PyQt Mac app successfully.
>>>>> 
>>>>> It?s command line plus setup script.
>>>> 
>>>> I wound up doing:
>>>> 1) pyinstaller, as normal, but this created a broken all-encompassing binary of my script.  At least it gave me the metadata I needed though.
>>> 
>>> You mean it created a .app bundle?
>>> 
>>> That is the way that macOS makes it trivia to install apps
>>> Just by drag and drop in /Applications.
>> 
>> Yes, it created an hcm.app for me.  But the executable it created didn't work.  Hence the hack.
> 
> More specifically:
> $ /Applications/hcm.app/Contents/MacOS/hcm --gui
> cmd output started 2022 Tue Jan 25 09:00:33 PM PST
> Traceback (most recent call last):
>   File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 146, in <module>
>     _run()
>   File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 129, in _run
>     exec(compile(source, path, "exec"), globals(), globals())
>   File "/Applications/hcm.app/Contents/Resources/hcm.py", line 1950, in <module>
>     import gi
>   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
>   File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
>   File "<frozen zipimport>", line 259, in load_module
>   File "gi/__init__.pyc", line 40, in <module>
>   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
>   File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
>   File "<frozen zipimport>", line 259, in load_module
>   File "gi/_gi.pyc", line 14, in <module>
>   File "gi/_gi.pyc", line 10, in __load
>   File "imp.pyc", line 342, in load_dynamic
>   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
>   File "<frozen importlib._bootstrap>", line 627, in _load_backward_compatible
>   File "/Applications/hcm.app/Contents/Resources/__boot__.py", line 36, in load_module
>     return imp.load_module(
>   File "imp.pyc", line 244, in load_module
>   File "imp.pyc", line 216, in load_package
>   File "<frozen importlib._bootstrap>", line 710, in _load
> AttributeError: 'NoneType' object has no attribute 'name'
> 2022-01-25 21:00:34.576 hcm[62695:1322031] hcm Error
> ^C^\Quit: 3
> above cmd output done    2022 Tue Jan 25 09:00:42 PM PST
> dstromberg at Daniels-Mini:~/src/home-svn/hcm/trunk x86_64-apple-darwin20.6.0 61933
> 
> $ file /Applications/hcm.app/Contents/MacOS/hcm
> cmd output started 2022 Tue Jan 25 09:00:54 PM PST
> /Applications/hcm.app/Contents/MacOS/hcm: Mach-O 64-bit executable x86_64

It?s intended to be started as an app.

What if you double click the app? Does it work?
Also you can use the open command to run use the app name you give it.

> 
> But if I replace /Applications/hcm.app/Contents/MacOS/hcm with a symlink to a wrapper shell script, hcm runs fine from the Applications menu.
> 
> It seems that gi.repository.Gtk applications are not packaged correctly by pyinstaller and py2app.  Some Python modules require a little assistance to be packaged up into a Mach-O executable neatly by such tools.  But it's just not that important to me to have a Mach-O of my app.

Understood, was just curious.

Barry

From barry at barrys-emacs.org  Wed Jan 26 17:41:40 2022
From: barry at barrys-emacs.org (Barry)
Date: Wed, 26 Jan 2022 22:41:40 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
References: <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
Message-ID: <58EDEC0D-8698-48EF-A5B4-B203224DD4EE@barrys-emacs.org>



> On 25 Jan 2022, at 23:50, Tony Flury <tony.flury at btinternet.com> wrote:
> 
> ?
> 
> 
>> On 25/01/2022 22:28, Barry wrote:
>> 
>>>> On 25 Jan 2022, at 14:50, Tony Flury via Python-list <python-list at python.org> wrote:
>>>> 
>>>> ?
>>>>> On 20/01/2022 23:12, Chris Angelico wrote:
>>>>>>> On Fri, 21 Jan 2022 at 10:10, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>>>>>>> On 20/01/22 12:09 am, Chris Angelico wrote:
>>>>>>> At this point, the refcount has indeed been increased.
>>>>>>> 
>>>>>>>>           return self;
>>>>>>>>      }
>>>>>>> And then you say "my return value is this object".
>>>>>>> 
>>>>>>> So you're incrementing the refcount, then returning it without
>>>>>>> incrementing the refcount. Your code is actually equivalent to "return
>>>>>>> self".
>>>>>> Chris, you're not making any sense. This is C code, so there's no
>>>>>> way that "return x" can change the reference count of x.
>>>>> Yeah, I wasn't clear there. It was equivalent to *the Python code*
>>>>> "return self". My apologies.
>>>>> 
>>>>>  > The normal thing to do is to add a reference to whatever you're
>>>>>  > returning. For instance, Py_RETURN_NONE will incref None and then
>>>>>  > return it.
>>>>>  >
>>>>> 
>>>>> The OP understands that this is not a normal thing to do. He's
>>>>> trying to deliberately leak a reference for the purpose of diagnosing
>>>>> a problem.
>>>>> 
>>>>> It would be interesting to see what the actual refcount is after
>>>>> calling this function.
>>> After calling this without a double increment in the function the ref count is still only 1 - which means that the 'return self' effectively does a double decrement. My original message includes the Python code which calls this 'leaky' function and you can see that despite the 'leaky POC' doing an increment ref count drops back to one after the return.
>>> 
>>> You are right this is not a normal thing to do, I am trying to understand the behaviour so my library does the correct thing in all cases - for example - imagine you have two nodes in a tree :
>>> 
>>> A --- > B
>>> 
>>> And your Python code has a named reference to A, and B also maintains a reference to A as it's parent.
>>> 
>>> In this case I would expect A to have a reference count of 2 (counted as 3 through sys.getrefcount() - one for the named reference in the Python code - and one for the link from B back to A; I would also expect B to have a reference count here of 1 (just the reference from A - assuming nothing else referenced B).
>>> 
>>> My original code was incrementing the ref counts of A and B and then returning A. within the Python test code A had a refcount of 1 (and not the expected 2), but the refcount from B was correct as far as I could tell.
>>> 
>>> 
>>>> Yes, and that's why I was saying it would need a *second* incref.
>>>> 
>>>> ChrisA
>>> Thank you to all of you for trying to help - I accept that the only way to make the code work is to do a 2nd increment.
>>> 
>>> I don't understand why doing a 'return self' would result in a double decrement - that seems utterly bizzare behaviour - it obviously works, but why.
>> The return self in C will not change the ref count.
>> 
>> I would suggest setting a break point in your code and stepping out of the function and seeing that python?s code does to the ref count.
>> 
>> Barry
> Barry,
> 
> something odd is going on because the Python code isn't doing anything that would cause the reference count to go from 3 inside the C function to 1 once the method call is complete.
> 
> As far as I know the only things that impact the reference counts are : 
> 
> Increments due to assigning a new name or adding it to a container.
> Increment due to passing the object to a function (since that binds a new name) 
> Decrements due to deletion of a name
> Decrement due to going out of scope
> Decrement due to being removed from a container.
> None of those things are happening in the python code.
> 

Run python and your code under a debugger and check the ref count of the object as you step through the code.

Don?t just step through your code but also step through the C python code.
That will allow you to see how this works at a low level. 
Setting a watch point on the ref count will allow you run the code and just break as the ref count changes.

That is what I do when a see odd c api behaviour.

Barry
> As posted in the original message - immediately before the call to the C function/method sys.getrefcount reports the count to be 2 (meaning it is actually a 1).
> 
> Inside the C function the ref count is incremented and the Py_REFCNT macro reports the count as 3 inside the C function as expected (1 for the name in the Python code, 1 for the argument as passed to the C function, and 1 for the increment), so outside the function one would expect the ref count to now be 2 (since the reference caused by calling the function is then reversed).
> 
> However - Immediately outside the C function and back in the Python code sys.getrefcount reports the count to be 2 again - meaning it is now really 1. So that means that the refcount has been decremented twice in-between the return of the C function and the execution of the immediate next python statement. I understand one of those decrements - the parameter's ref count is incremented on the way in so the same object is decremented on the way out (so that calls don't leak references) but I don't understand where the second decrement is coming from.
> 
> Again there is nothing in the Python code that would cause that decrement - the decrement behavior is in the Python runtime.
> 
>>> 
>>> 
>>> -- 
>>> Anthony Flury
>>> email : anthony.flury at btinternet.com
>>> 
>>> -- 
>>> https://mail.python.org/mailman/listinfo/python-list
>>> 
> -- 
> Anthony Flury
> email : anthony.flury at btinternet.com

From Marco.Sulla.Python at gmail.com  Wed Jan 26 18:16:12 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Thu, 27 Jan 2022 00:16:12 +0100
Subject: Pandas or Numpy
In-Reply-To: <ga2rugp1e4ibur3sa8b724fo1h78ttvv45@4ax.com>
References: <ssjsi3$3h0$1@gioia.aioe.org>
 <ga2rugp1e4ibur3sa8b724fo1h78ttvv45@4ax.com>
Message-ID: <CABbU2U96LkqUsX5-T6MjDFtRAnz5aN+UA7aWb9hM5dvxCodiFA@mail.gmail.com>

On Mon, 24 Jan 2022 at 05:37, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>         Note that the comparison warns that /indexing/ in pandas can be slow.
> If your manipulation is always "apply operationX to columnY" it should be
> okay -- but "apply operationX to the nth row of columnY", and repeat for
> other rows, is going to be slow.

In my small way, I can confirm. In one of my previous works, we used
numpy and Pandas. Writing the code in Pandas is quick, but they just
realised that was really slow, and they tried to transform as much
Panda code to numpy code as possible.

Furthermore, I saw that they were so accustomed with Pandas that they
used it for all, even for a simple csv creation, when the csv builtin
module is enough.

From drsalists at gmail.com  Wed Jan 26 20:26:51 2022
From: drsalists at gmail.com (Dan Stromberg)
Date: Wed, 26 Jan 2022 17:26:51 -0800
Subject: mac app from a python script?
In-Reply-To: <B65F3620-D6AF-4037-BF1A-075DEFA2D351@barrys-emacs.org>
References: <CAGGBd_qHeOv3iH1QU=zW2kuHuZ7Up-xZEf73ek7X61i=SPR-Kg@mail.gmail.com>
 <B65F3620-D6AF-4037-BF1A-075DEFA2D351@barrys-emacs.org>
Message-ID: <CAGGBd_pzOGJm=QJ=te5i0Uz+vh46pjuuL86_j3_wL+J_KcjqSQ@mail.gmail.com>

On Wed, Jan 26, 2022 at 2:35 PM Barry <barry at barrys-emacs.org> wrote:

>
>
> On 26 Jan 2022, at 05:17, Dan Stromberg <drsalists at gmail.com> wrote:
>
>
> On Tue, Jan 25, 2022 at 6:41 PM Dan Stromberg <drsalists at gmail.com> wrote:
>
>>
>> On Tue, Jan 25, 2022 at 2:23 PM Barry <barry at barrys-emacs.org> wrote:
>>
>>>
>>> On 25 Jan 2022, at 02:56, Dan Stromberg <drsalists at gmail.com> wrote:
>>>
>>> ?
>>>
>>> On Sun, Jan 23, 2022 at 1:37 PM Barry <barry at barrys-emacs.org> wrote:
>>>
>>>>
>>>> I do not have experience with great, but you might try pyinstaller.
>>>> I use it to make a PyQt Mac app successfully.
>>>>
>>>> It?s command line plus setup script.
>>>>
>>>
>>> I wound up doing:
>>> 1) pyinstaller, as normal, but this created a broken all-encompassing
>>> binary of my script.  At least it gave me the metadata I needed though.
>>>
>>>
>>> You mean it created a .app bundle?
>>>
>>> That is the way that macOS makes it trivia to install apps
>>> Just by drag and drop in /Applications.
>>>
>>
>> Yes, it created an hcm.app for me.  But the executable it created didn't
>> work.  Hence the hack.
>>
>
> $ file /Applications/hcm.app/Contents/MacOS/hcm
> cmd output started 2022 Tue Jan 25 09:00:54 PM PST
> /Applications/hcm.app/Contents/MacOS/hcm: Mach-O 64-bit executable x86_64
>
>
> It?s intended to be started as an app.
>
> What if you double click the app? Does it work?
>
No, it does not start that way either.


> Also you can use the open command to run use the app name you give it.
>
 Thanks for the tip.

From tony.flury at btinternet.com  Thu Jan 27 02:46:11 2022
From: tony.flury at btinternet.com (Tony Flury)
Date: Thu, 27 Jan 2022 07:46:11 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <58EDEC0D-8698-48EF-A5B4-B203224DD4EE@barrys-emacs.org>
References: <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
 <58EDEC0D-8698-48EF-A5B4-B203224DD4EE@barrys-emacs.org>
Message-ID: <2d1960e6-17bd-f060-4c66-1aa61ca9c48c@btinternet.com>


On 26/01/2022 22:41, Barry wrote:
>
>
> Run python and your code under a debugger and check the ref count of 
> the object as you step through the code.
>
> Don?t just step through your code but also step through the C python code.
> That will allow you to see how this works at a low level.
> Setting a watch point on the ref count will allow you run the code and 
> just break as the ref count changes.
>
> That is what I do when a see odd c api behaviour.
>
> Barry


Thanks - I have tried a few times on a few projects to run a debugger in 
mixed language mode and never had any success.

I will have to try again.


>> As posted in the original message - immediately before the call to 
>> the C function/method sys.getrefcount reports the count to be 2 
>> (meaning it is actually a 1).
>>
>> Inside the C function the ref count is incremented and the Py_REFCNT 
>> macro reports the count as 3 inside the C function as expected (1 for 
>> the name in the Python code, 1 for the argument as passed to the C 
>> function, and 1 for the increment), so outside the function one would 
>> expect the ref count to now be 2 (since the reference caused by 
>> calling the function is then reversed).
>>
>> However - Immediately outside the C function and back in the Python 
>> code sys.getrefcount reports the count to be 2 again - meaning it is 
>> now really 1. So that means that the refcount has been decremented 
>> twice in-between the return of the C function and the execution of 
>> the immediate next python statement. I understand one of those 
>> decrements - the parameter's ref count is incremented on the way in 
>> so the same object is decremented on the way out (so that calls don't 
>> leak references) but I don't understand where the second decrement is 
>> coming from.
>>
>> Again there is nothing in the Python code that would cause that 
>> decrement - the decrement behavior is in the Python runtime.
>>
>>>> -- 
>>>> Anthony Flury
>>>> email :anthony.flury at btinternet.com
>>>>
>>>> -- 
>>>> https://mail.python.org/mailman/listinfo/python-list
>>>>
>> -- 
>> Anthony Flury
>> email :anthony.flury at btinternet.com

-- 
Anthony Flury
email : anthony.flury at btinternet.com


From tony.flury at btinternet.com  Thu Jan 27 02:58:07 2022
From: tony.flury at btinternet.com (Tony Flury)
Date: Thu, 27 Jan 2022 07:58:07 +0000
Subject: Starting using Python
In-Reply-To: <CALdZMUFDBHQz4PFHO9MS9pm1=sKSnjCg3aUsXdQsgjOnR36yyA@mail.gmail.com>
References: <CALdZMUFDBHQz4PFHO9MS9pm1=sKSnjCg3aUsXdQsgjOnR36yyA@mail.gmail.com>
Message-ID: <cd74438d-ff47-cc94-4a24-2c6e6af36937@btinternet.com>


On 03/01/2022 12:45, Joao Marques wrote:
> Good morning: I have a very simple question: I want to start writing
> programs in Python so I went to the Microsoft Store and installed
> Python3.9. No problem so far. I would prefer to have a gui interface, an
> interface that I can use file-->Open and File-->Save as, as I see it on
> different videos. How can I get it? Because my problem is to run the
> programs I have already written and saved on a *.py file in my own working
> directory, not in the Python's CWD directory.
> Can you please help?
> I am running Windows 10 Pro version 20H2
>
> Regards,
> Joao


The simplest Python GUI editor is IDLE, which should come installed with 
with Python. On windows just open the start menu and type 'IDLE'

You can save your python files anywhere - you shouldn't need to save it 
in Python's working directory. Python doesn't impose any particular file 
system until you start implement features such as packages which are not 
something a beginner should ever worry about.

-- 
Anthony Flury
email : anthony.flury at btinternet.com


From barry at barrys-emacs.org  Thu Jan 27 14:38:11 2022
From: barry at barrys-emacs.org (Barry Scott)
Date: Thu, 27 Jan 2022 19:38:11 +0000
Subject: Puzzling behaviour of Py_IncRef
In-Reply-To: <2d1960e6-17bd-f060-4c66-1aa61ca9c48c@btinternet.com>
References: <8201bfa3-8626-a509-bbde-801566e2bc33@btinternet.com>
 <58EDEC0D-8698-48EF-A5B4-B203224DD4EE@barrys-emacs.org>
 <2d1960e6-17bd-f060-4c66-1aa61ca9c48c@btinternet.com>
Message-ID: <B850DED3-A37B-4968-9801-574DD9AA13E2@barrys-emacs.org>



> On 27 Jan 2022, at 07:46, Tony Flury <tony.flury at btinternet.com> wrote:
> 
> 
> On 26/01/2022 22:41, Barry wrote:
>> 
>> 
>> Run python and your code under a debugger and check the ref count of the object as you step through the code.
>> 
>> Don?t just step through your code but also step through the C python code.
>> That will allow you to see how this works at a low level.
>> Setting a watch point on the ref count will allow you run the code and just break as the ref count changes.
>> 
>> That is what I do when a see odd c api behaviour.
>> 
>> Barry
> 
> 
> Thanks - I have tried a few times on a few projects to run a debugger in mixed language mode and never had any success.
> 
> I will have to try again.

You mean debugging Python and C/C++? In this case your python code is a simple test script and it's C that you care about.
Should not be difficult. I tend to use linux as my lead debug platform as its the easiest to work with. But Windows and macOS
also have very good debuggers.

Barry

> 
> 
>>> As posted in the original message - immediately before the call to the C function/method sys.getrefcount reports the count to be 2 (meaning it is actually a 1).
>>> 
>>> Inside the C function the ref count is incremented and the Py_REFCNT macro reports the count as 3 inside the C function as expected (1 for the name in the Python code, 1 for the argument as passed to the C function, and 1 for the increment), so outside the function one would expect the ref count to now be 2 (since the reference caused by calling the function is then reversed).
>>> 
>>> However - Immediately outside the C function and back in the Python code sys.getrefcount reports the count to be 2 again - meaning it is now really 1. So that means that the refcount has been decremented twice in-between the return of the C function and the execution of the immediate next python statement. I understand one of those decrements - the parameter's ref count is incremented on the way in so the same object is decremented on the way out (so that calls don't leak references) but I don't understand where the second decrement is coming from.
>>> 
>>> Again there is nothing in the Python code that would cause that decrement - the decrement behavior is in the Python runtime.
>>> 
>>>>> -- 
>>>>> Anthony Flury
>>>>> email :anthony.flury at btinternet.com
>>>>> 
>>>>> -- 
>>>>> https://mail.python.org/mailman/listinfo/python-list
>>>>> 
>>> -- 
>>> Anthony Flury
>>> email :anthony.flury at btinternet.com
> 
> -- 
> Anthony Flury
> email : anthony.flury at btinternet.com
> 


From etemcetin.toptani at std.yeditepe.edu.tr  Thu Jan 27 15:31:39 2022
From: etemcetin.toptani at std.yeditepe.edu.tr (=?UTF-8?B?RVRFTSDDh0VUxLBOIFRPUFRBTsSw?=)
Date: Thu, 27 Jan 2022 23:31:39 +0300
Subject: Short Survey About Requirements Engineering
Message-ID: <CAJ48VLhsj=4=PvYSDTMxWcUHqUwAcF2A6UiqjzngTv=8m60ZSQ@mail.gmail.com>

Dear Sir or Madam,


We prepared a short survey to understand practitioners? perspectives
towards the requirements engineering. Our survey basically aims to clarify
on many aspects of the requirements engineering applied in industry, including
(i) requirements gathering and specifications, (ii) requirements
modifications, (iii) requirements analysis, and (iv) requirements
transformation. The survey results will be submitted to a reputable journal
on software engineering.


The survey takes about 2-10 minutes to participate, we would be so grateful
if you could separate your time. Also, please circulate the e-mail to
anyone who may be interested in.


The survey link: https://forms.gle/DhLqr15GXVhJhzzy6


All the best,

 Etem ?etin Toptani

-- 
*Bu mesaj? yazd?rmadan ?nce ?evreye verebilece?iniz zararlar? bir kez daha 
d???n?n?z.?*
*Think of the environment once more before printing out this 
message.*

-- 
*Bu mesaj? yazd?rmadan ?nce ?evreye verebilece?iniz zararlar? bir kez daha 
d???n?n?z.?*
*Think of the environment once more before printing out this 
message.*

From cousinstanley at gmail.com  Fri Jan 28 11:34:39 2022
From: cousinstanley at gmail.com (Cousin Stanley)
Date: Fri, 28 Jan 2022 09:34:39 -0700
Subject: What to write or search on github to get the code for what is
 written below:
References: <e84eugt6j4bl2tirde97oup1r9te70uif1@4ax.com>
 <mailman.224.1642565856.3079.python-list@python.org>
 <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
 <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>
 <5lgtugll2ascb0phbue0agrfuqmvjp4b7b@4ax.com>
Message-ID: <st15v2$fk6$1@dont-email.me>

Dennis Lee Bieber wrote:

> ....
> How would you do this assignment on paper ?
> ....

  Your patience and willingness to help and guide someone else
  with such a complete and understanable post is hihgly commendable.

  Thanks .... 
  
-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona


From wlfraed at ix.netcom.com  Fri Jan 28 15:21:35 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Fri, 28 Jan 2022 15:21:35 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
 <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>
 <5lgtugll2ascb0phbue0agrfuqmvjp4b7b@4ax.com> <st15v2$fk6$1@dont-email.me>
Message-ID: <e3j8vg1nu02f8bnpkiqm3it0pit4nrv5c8@4ax.com>

On Fri, 28 Jan 2022 09:34:39 -0700, Cousin Stanley
<cousinstanley at gmail.com> declaimed the following:


>
>  Your patience and willingness to help and guide someone else
>  with such a complete and understanable post is hihgly commendable.
>
	Ignoring the code spam I presume <G>

	I seem to have scared off the OP with that post. Suspect the assignment
came due, and they've nothing to show for it.

	I went on with the SQLite3 interpretation of the whole system,
including using pysimplegui* (wx variant) and have implemented the login
and user registration pages (though I haven't added a hash function for
password storage yet), and made a start on the actual operations of the
application (regular user: reserve book, unreserve book, list reservations,
list checkouts [my interpretation over this long thread was that user makes
reservation, then has to report to some desk to receive the book -- I treat
the latter as the checkout stage]; admin users get: checkout for client,
checkin for client, expire stale reservations, list overdue, grant admin
privilege, add book, delete book).

C:\Users\Wulfraed\Documents\_Hg-Repositories\Python Progs>pygount -s py -f
summary BookDepository
Language   Files    %     Code    %     Comment    %
---------  -----  ------  ----  ------  -------  ------
Python         6  100.00   350  100.00      108  100.00
---------  -----  ------  ----  ------  -------  ------
Sum total      6           350              108

C:\Users\Wulfraed\Documents\_Hg-Repositories\Python Progs>pygount -s py
BookDepository
18      Python  BookDepository  BookDepository\BookDepository.py
113     Python  BookDepository  BookDepository\creationSql.py
94      Python  BookDepository  BookDepository\database.py
2       Python  BookDepository  BookDepository\gui.py
119     Python  BookDepository  BookDepository\gui_login.py
112     Python  BookDepository  BookDepository\gui_main.py

C:\Users\Wulfraed\Documents\_Hg-Repositories\Python Progs>

	Taking a few days off from this exercise. 350LOC in a week, including
reading the pysimplegui documentation as I encounter things to implement vs
the <10 lines presented by the OP in over three weeks. Out of stubbornness
I'll probably continue this in the next few days.




* pysimplegui actual feels somewhat comfortable to me... being similar to
my 30 year old Amiga in terms of coding GUIs (no master event loop invoking
call-backs, rather an explicit loop with comparison for event/widget and
dispatch to handler functions)


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From HooDunnit at didly42KahZidly.net  Fri Jan 28 16:31:20 2022
From: HooDunnit at didly42KahZidly.net (Cousin Stanley)
Date: Fri, 28 Jan 2022 14:31:20 -0700
Subject: What to write or search on github to get the code for what is
 written below:
References: <2b437a57-c01e-4c27-b10c-7352e1dafaf4n@googlegroups.com>
 <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
 <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>
 <5lgtugll2ascb0phbue0agrfuqmvjp4b7b@4ax.com> <st15v2$fk6$1@dont-email.me>
 <e3j8vg1nu02f8bnpkiqm3it0pit4nrv5c8@4ax.com>
Message-ID: <st1nbb$jn1$1@dont-email.me>

Dennis Lee Bieber wrote:

> Ignoring the code spam I presume <G>
> .... 

  I'm an sqlite user myself and was glad to see 
  the code you posted and have a couple of tiny example
  book/author sql3 databases but nothing resembling 
  an actual library check in/out program .... 

  I've never used PySimpleGUI 
  but it does look interesting .... 

    https://pysimplegui.readthedocs.io/en/latest/

    https://pypi.org/search/?q=pysimplegui

  After you complete your BookDepository DataBase development
  perhaps you would consider making it publically available 
  for others to learn from ....


-- 
Stanley C. Kitching
Human Being
Phoenix, Arizona


From wlfraed at ix.netcom.com  Sat Jan 29 12:02:36 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Sat, 29 Jan 2022 12:02:36 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
 <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>
 <5lgtugll2ascb0phbue0agrfuqmvjp4b7b@4ax.com> <st15v2$fk6$1@dont-email.me>
 <e3j8vg1nu02f8bnpkiqm3it0pit4nrv5c8@4ax.com> <st1nbb$jn1$1@dont-email.me>
Message-ID: <mvpavgdftdm4eeh6jb1ejhltqdvdm30s3r@4ax.com>

On Fri, 28 Jan 2022 14:31:20 -0700, Cousin Stanley
<HooDunnit at didly42KahZidly.net> declaimed the following:

>Dennis Lee Bieber wrote:
>
>> Ignoring the code spam I presume <G>
>> .... 
>
>  I'm an sqlite user myself and was glad to see 
>  the code you posted and have a couple of tiny example
>  book/author sql3 databases but nothing resembling 
>  an actual library check in/out program .... 
>

	I'd modify the database significantly for library usage -- among other
things I'd remove the "Copy" table; if the library has multiple copies of a
book, it would have multiple entries at the "Book" level. There would also
be fields for short description, genre/category, maybe shelving.

	The impression I got from the OPs assorted (and rather vague) posts is
something closer to school classrooms issuing books to students (vs college
level where students have to buy them, which implies no need to track who
and when), so having massive amounts of textbooks in various conditions.
The separate reservation/checkout is something I inferred, and is really a
strange concept -- at least when I was in school, the textbooks were
distributed by the instructor on the first day of class, and collected at
the end of the course.

	Say a small high-school; 600 students -> 150 each 9th, 10th, 11th, 12th
grade. 30 students per class (on the high side, yes), makes five sessions
in the day... and, of course, means 150 copies of a text book. Reality may
not be quite that bad as some courses are electives or have options
(foreign language: French, German, Italian, Spanish -- probably few take
Italian, so maybe only 30-60 books, etc.).

>  I've never used PySimpleGUI 
>  but it does look interesting .... 
>

	I'm not sure how I stumbled upon it... I think the older ActiveState
Python I have installed was one of those "include everything that doesn't
cause a conflict" configurations, with tons of 3rd-party packages... And I
just saw the name in a package list. I had considered doing just a console
application with curses but... Windows makes curses difficult to even get
installed.

	PySimpleGUI's explicit event dispatch loops made ad-hoc forms simpler
to implement over having one master implicit event loop (app.run() or
similar).

	The alternative would have been using Flask or Django (and configuring
my Raspberry-Pi nginx server for them -- right now it just serves static
web pages)

>  After you complete your BookDepository DataBase development
>  perhaps you would consider making it publically available 
>  for others to learn from ....

	Unlikely -- I'd have to put in a lot more documentation to explain my
choices and design choices (I'm NOT using SQLAlchemy -- as I'd stated some
posts ago, it just confuses me; instead I'm coding explicit SQL and cursor
processing). The GUI is quick&dirty -- there is no doubt much that even
PySimpleGUI can do that I've not delved into (I really would prefer the
documentation to be available in PDF format, so I could print it in booklet
form and read while away from the computer).

	There are a number of library management systems written in Python
already (though I encountered one site that only had two chapters in place
-- about enough to install Python and packages [chapter 1] and do a
"hello-world" [chapter 2] -- and nothing else).

https://data-flair.training/blogs/library-management-system-python-project/
Tkinter and MySQL
(I don't like Tkinter)

https://techvidvan.com/tutorials/python-library-management-system/
Django
(assumes school library "addstudent" function)

https://rrtutors.com/tutorials/python-library-management-system-project
MySQL, console
(Some things I'd improve -- they used a chain of IF statements to process
menu input where a dictionary look-up would be shorter and easier to
modify)

https://itsourcecode.com/free-projects/python-projects/library-management-system-project-in-python-and-mysql/
MySQL and Tkinter
(Another "school" focused example -- users are "students")

	Many of them use quite a simplified schema... Storing publishers and
authors directly within the Book record -- which means the possibility for
typos from one book to another, etc. I'd pull those out into separate
tables and link via foreign keys.



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From frank at chagford.com  Sun Jan 30 01:02:11 2022
From: frank at chagford.com (Frank Millman)
Date: Sun, 30 Jan 2022 08:02:11 +0200
Subject: A bit of nostalgia
Message-ID: <cb7ddfa2-a6be-6430-a13f-ef282437f91b@chagford.com>

Hi all

Sadly my ex-boss died recently. I learnt a huge amount from him. I also 
enjoyed and got valuable insights from his stories of 'the good old 
days', so I thought I would share a couple of them.

Back in the 1960's (long before we met) he worked for and eventually 
became manager of a large ICL bureau here in South Africa, running 
accounting and other applications for a number of corporate customers.

One day some new equipment arrived from UK. The engineers spent some 
time moving things around in the computer room and installing the new 
kit, but nobody told the software department what was going on. When it 
was finished, they were told 'Right, now you have disk drives.'

Up to that point, their programs used magnetic tape for data storage. A 
typical debtors run would involve loading a master file onto one tape 
drive, and a transaction file, sorted in account number sequence, onto a 
second tape drive. To print statements, the program would read the first 
master record, then read the first transaction record, if it found a 
match keep reading transactions until it hit a new account number, etc. 
When all reports were run, the last step was to create a new master file 
for the following month. This required mounting a scratch tape onto a 
third tape drive, and the program would run in a similar fashion but 
write a new record with an updated balance to the scratch tape on every 
change of account number.

When they asked how they were supposed to use the new disk drives, the 
answer was 'No idea, but there are the manuals' (no doubt several inches 
thick). After some discussion someone came up with an idea - load the 
master file onto one disk drive, the transaction file onto a second one, 
and a blank disk onto a third one. They ran it, it worked, and it ran 
much faster, so everyone was happy. Over time they realised that they 
could write the master record back onto the first drive, and slowly they 
adapted to a new way of developing applications.

The second story involves a rumour coming down the grapevine from UK 
that a new product was coming out that would put all their jobs at risk. 
It would make programming so easy that there would be no need to hire 
specialist programmers any more. Eventually this new product was 
released. It was called Cobol.

Up to that point they programmed everything in assembler. I recall my 
boss telling me that the ICL assembler was called PLAN, which was an 
acronym, but I forget what it stood for.

Frank Millman

From ojomooluwatolami675 at gmail.com  Mon Jan 31 09:18:47 2022
From: ojomooluwatolami675 at gmail.com (ojomooluwatolami675 at gmail.com)
Date: Mon, 31 Jan 2022 14:18:47 +0000
Subject: Problem upgrading pip and installing pygame
Message-ID: <7D88A74E-D482-40E7-B4FC-F48326EF2DA5@gmail.com>


Good morning, Python. I am having trouble installing pygame. it keeps saying to upgrade my pip version which I have done several times. then when I tried importing python to see if it has worked, in the ide it says the module does not exist or something along that line. what do I do please? Thanks.

Sent from my iPhone

From narshad.380 at gmail.com  Mon Jan 31 04:32:15 2022
From: narshad.380 at gmail.com (NArshad)
Date: Mon, 31 Jan 2022 01:32:15 -0800 (PST)
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <mvpavgdftdm4eeh6jb1ejhltqdvdm30s3r@4ax.com>
References: <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
 <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>
 <5lgtugll2ascb0phbue0agrfuqmvjp4b7b@4ax.com>
 <st15v2$fk6$1@dont-email.me> <e3j8vg1nu02f8bnpkiqm3it0pit4nrv5c8@4ax.com>
 <st1nbb$jn1$1@dont-email.me> <mvpavgdftdm4eeh6jb1ejhltqdvdm30s3r@4ax.com>
Message-ID: <1366d2e7-242c-4890-8e98-4c37313531e8n@googlegroups.com>

What about CGI?
Do you know any Library Management System based on CGI just like the one on Django?


From rosuav at gmail.com  Mon Jan 31 10:41:17 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 1 Feb 2022 02:41:17 +1100
Subject: What to write or search on github to get the code for what is
 written below:
In-Reply-To: <1366d2e7-242c-4890-8e98-4c37313531e8n@googlegroups.com>
References: <30cec0cf-e452-4ac5-996b-38e96ffbc7den@googlegroups.com>
 <utroug9qthef55dv4db2b1gjngqvq3l780@4ax.com>
 <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
 <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>
 <5lgtugll2ascb0phbue0agrfuqmvjp4b7b@4ax.com>
 <st15v2$fk6$1@dont-email.me> <e3j8vg1nu02f8bnpkiqm3it0pit4nrv5c8@4ax.com>
 <st1nbb$jn1$1@dont-email.me> <mvpavgdftdm4eeh6jb1ejhltqdvdm30s3r@4ax.com>
 <1366d2e7-242c-4890-8e98-4c37313531e8n@googlegroups.com>
Message-ID: <CAPTjJmpQ-k02S=V2GELz-X-g2d9eYffRzbrRe=BqdCxUUce97A@mail.gmail.com>

On Tue, 1 Feb 2022 at 02:38, NArshad <narshad.380 at gmail.com> wrote:
>
> What about CGI?
> Do you know any Library Management System based on CGI just like the one on Django?
>

Have you done any research, or are you just picking up a new acronym
to see if you can suck some more volunteer time out of this list?

ChrisA

From wlfraed at ix.netcom.com  Mon Jan 31 11:59:04 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Mon, 31 Jan 2022 11:59:04 -0500
Subject: Problem upgrading pip and installing pygame
References: <7D88A74E-D482-40E7-B4FC-F48326EF2DA5@gmail.com>
 <mailman.0.1643643441.2976.python-list@python.org>
Message-ID: <r15gvglmbev50ovgcjvcjs8pf6gq383f96@4ax.com>

On Mon, 31 Jan 2022 14:18:47 +0000, ojomooluwatolami675 at gmail.com declaimed
the following:

>
>Good morning, Python. I am having trouble installing pygame. it keeps saying to upgrade my pip version which I have done several times. then when I tried importing python to see if it has worked, in the ide it says the module does not exist or something along that line. what do I do please? Thanks.

	Where to begin... "importing python" is never done -- python is the
language and its interpreter/compiler.

	Which "ide"?

	Show us the EXACT operations you are performing and the results... [NOT
SCREEN CAPTURES -- this forum strips non-text attachments; select the TEXT
and cut&paste that]


-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From wlfraed at ix.netcom.com  Mon Jan 31 12:50:52 2022
From: wlfraed at ix.netcom.com (Dennis Lee Bieber)
Date: Mon, 31 Jan 2022 12:50:52 -0500
Subject: What to write or search on github to get the code for what is
 written below:
References: <CAPTjJmrO_Vp+ZC+WZ=_mFhizCSRH9dAkPWsAmzjVGCErQvFAmg@mail.gmail.com>
 <mailman.238.1642889872.3079.python-list@python.org>
 <92fpug9pvts5dttcorrrgjbqrl2s17r71v@4ax.com>
 <10976640-0472-42b0-9ffb-5765b37df4acn@googlegroups.com>
 <5lgtugll2ascb0phbue0agrfuqmvjp4b7b@4ax.com> <st15v2$fk6$1@dont-email.me>
 <e3j8vg1nu02f8bnpkiqm3it0pit4nrv5c8@4ax.com> <st1nbb$jn1$1@dont-email.me>
 <mvpavgdftdm4eeh6jb1ejhltqdvdm30s3r@4ax.com>
 <1366d2e7-242c-4890-8e98-4c37313531e8n@googlegroups.com>
Message-ID: <0b5gvglak1kjso724esg57nu47b6hbj6gm@4ax.com>

On Mon, 31 Jan 2022 01:32:15 -0800 (PST), NArshad <narshad.380 at gmail.com>
declaimed the following:

>What about CGI?
>Do you know any Library Management System based on CGI just like the one on Django?

	Pure CGI is 30 year old technology...
https://en.wikipedia.org/wiki/Common_Gateway_Interface

	To use it will require a properly configured web-server and lots of
individual script files. For a monolithic Python application you may want
to study https://en.wikipedia.org/wiki/Web_Server_Gateway_Interface (which
will still require a properly configured web-server).

https://docs.python.org/3/library/cgi.html

	You'll still have write the HTML for the pages to be displayed
(possibly via template engines as used by Django, Flask, Pylons, Zope), and
maybe CSS (pure HTML just describes /what/ to render, leaving it up to the
browser to do that rendering -- CSS adds lots of overhead to control how
the rendering is done).





	You have now spent a MONTH asking people to provide you with pre-built
solutions to a problem description that is so vaguely specified that no one
here would even consider a solution on that model. You have been provided
links to possible approaches and technologies which might be used by you to
write code for your requirements. 

	In this month you have never shown us actual sample data (export your
spreadsheet as CSV so you can paste text into a post). You have shown us
less than 10 lines of code (which, again, can not be evaluated as they were
incomplete and you provided no data against which to run them). You grab at
any acronym mentioned in replies as if it will magically create your
solution but don't seem to spend time learning about those technologies to
understand what they do (CGI is a method by which a web page can send data
to a web server, and the web server then starts a program passing the data
on to the program... THAT IS ALL IT DOES -- you still have to create the
HTML forms AND the scripts that process the data).

	Your original post(s) stated you were under a time crunch and didn't
have time to write everything... But you seem to have lots of time to keep
asking others to find a solution for you to copy. A rank beginner in
Python* should have been able to write minimal code to access the data,
even if the user interface is just a text console:

		ENTER USERNAME> ....
		ENTER TITLE OF INTEREST> ...
		# copies are available for /title/
		RESERVE COPY (Y/N)> ...

****************************************************
	After all this time, my recommendation is that you go to whoever gave
you this assignment and flat out tell them you are unable to provide the
solution and need to be replaced.

	It's that, or do a massive editing session to produce a few hundred
lines of code that at least attempts to solve part of your assignment and
hope for an extension on the due date. You have a number of things to
consider:
	User Interface:
		1	text console (maybe with use of curses to create forms)
		2	local GUI app (Tkinter, GTK, or wX libraries)
		3	web-based app (HTML, web-framework, web-server config)
	Data Storage:
		1	your insistence on an Excel spreadsheet
			(the most complex solution as it does not handle
			concurrent access -- so you need some means of 
			locking it to only one user for the entire time of a
			session)
		2	relational database
			(one time import, and need to provide functions to allow
			for complete management of the data, not just one
			"reserve" function. option for on-demand export for
			post-process reporting)
	User Management:
		you need some means to control who can perform operations
		on the data -- this could be additional tables in a database.

****************************************************



*
{Given the first edition O'Reilly Python book, and an Amiga port of Python
1.x -- it took me a week to produce a rudimentary SMTP sending daemon using
my ISP SMTPd for relay... I had to do this as the first SMTP sending
program I'd obtained often hung up trying to send mails (this was back in
the day when it was common to send mail directly to the destination host;
but it didn't do MX lookup; some addresses didn't run receiving SMTPd); the
second program did relay via ISP but never processed CC: or BCC: addresses,
only TO: addresses. Oh, this was also back in the days when mail clients
only handled local mailboxes, and did not send/receive messages -- one had
to invoke scripts to queue outgoing messages for sending, and others to
fetch from POP3 servers. I'll admit I couldn't do that now -- the need for
SSL/TLS protocols complicates things.}





-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
	wlfraed at ix.netcom.com    http://wlfraed.microdiversity.freeddns.org/

From mats at wichmann.us  Mon Jan 31 13:08:13 2022
From: mats at wichmann.us (Mats Wichmann)
Date: Mon, 31 Jan 2022 11:08:13 -0700
Subject: Problem upgrading pip and installing pygame
In-Reply-To: <7D88A74E-D482-40E7-B4FC-F48326EF2DA5@gmail.com>
References: <7D88A74E-D482-40E7-B4FC-F48326EF2DA5@gmail.com>
Message-ID: <e19920a5-7c08-4ade-92c1-c946323d4be0@wichmann.us>

On 1/31/22 07:18, ojomooluwatolami675 at gmail.com wrote:
> 
> Good morning, Python. I am having trouble installing pygame. it keeps saying to upgrade my pip version which I have done several times. then when I tried importing python to see if it has worked, in the ide it says the module does not exist or something along that line. what do I do please? Thanks.
> 
> Sent from my iPhone


When Python can't find modules it's *always* a path problem - the
install went somewhere on the system, but not to the places the Python
you are using is looking in.

One suggestion that usually helps is to use pip as a module, then it
will match exactly the Python used, because you're actually using that
Python.  So for example, if the way you invoke Python is through the
Windows Python launcher "py", then use this line:

py -m pip install --upgrade pip pygame


should put things in the right place.

adjust according to your situation, which we can't guess at.




From dieter at handshake.de  Mon Jan 31 14:02:46 2022
From: dieter at handshake.de (Dieter Maurer)
Date: Mon, 31 Jan 2022 20:02:46 +0100
Subject: http.client and dns lookups
In-Reply-To: <n7uhcixevq.ln2@news.c0t0d0s0.de>
References: <n7uhcixevq.ln2@news.c0t0d0s0.de>
Message-ID: <25080.12886.84182.252875@ixdm.fritz.box>

Michael Welle wrote at 2022-1-30 09:18 +0100:
> ...
 The machine this is running on regularly switches
>its network configuration without restarting the Python application. Now
>it turns out that the application is still using an old, outdated dns
>server after such a network configuration switch.

It is unlikely that Python performs the host -> IP address translation
itself. Almost surely, the translation is delegated to functions
of the underlying C runtime library, e.g. "gethostbyname".

From Marco.Sulla.Python at gmail.com  Mon Jan 31 16:52:57 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Mon, 31 Jan 2022 22:52:57 +0100
Subject: Segfault after deepcopy in a C extension
Message-ID: <CABbU2U-8wjwkzpoFjvfzLGvO-d_P25tzq-=qy8rEGijnBthjcQ@mail.gmail.com>

Well, this is more or less what I'm trying to do.

I have an immutable object. I would have copy.deepcopy() will return the
object itself if it's hashable. If not, it must return a deepcopy of it.

So I tried to implement a __deepcopy__ for the object. It segfaults if the
object is not hashable. And I don't understand why. gdb gives me an
incomprehensible backtrace. So I tried the old "print at each line", but
the segfault does not happen in the function. It happens when I quit REPL
or if I try to see the deepcopy.

What can I do to further debug it?

If someone is interested, this is the code:

PyObject* frozendict_deepcopy(PyObject* self, PyObject* memo) {
    if (PyAnyFrozenDict_CheckExact(self)) {
        frozendict_hash(self);

        if (PyErr_Occurred()) {
            PyErr_Clear();
        }
        else {
            Py_INCREF(self);
            return self;
        }
    }

    if (! PyAnyFrozenDict_Check(self)) {
        Py_RETURN_NOTIMPLEMENTED;
    }

    PyObject* d = PyDict_New();

    if (d == NULL) {
        return NULL;
    }

    PyObject* copy_module_name = NULL;
    PyObject* copy_module = NULL;
    PyObject* deepcopy_fun = NULL;
    PyObject* args = NULL;
    PyObject* res = NULL;

    if (PyDict_Merge(d, self, 1)) {
        goto end;
    }
    copy_module_name = PyUnicode_FromString("copy");

    if (copy_module_name == NULL) {
        goto end;
    }

    copy_module = PyImport_Import(copy_module_name);

    if (copy_module == NULL) {
        goto end;
    }

    deepcopy_fun = PyObject_GetAttrString(copy_module, "deepcopy");

    if (deepcopy_fun == NULL) {
        goto end;
    }

    args = PyTuple_New(2);

    if (args == NULL) {
        goto end;
    }

    PyTuple_SET_ITEM(args, 0, d);
    PyTuple_SET_ITEM(args, 1, memo);

    res = PyObject_CallObject(deepcopy_fun, args);

end:
    Py_XDECREF(args);
    Py_XDECREF(deepcopy_fun);
    Py_XDECREF(copy_module);
    Py_XDECREF(copy_module_name);
    Py_DECREF(d);

    return res;
}

From rosuav at gmail.com  Mon Jan 31 16:59:33 2022
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 1 Feb 2022 08:59:33 +1100
Subject: Segfault after deepcopy in a C extension
In-Reply-To: <CABbU2U-8wjwkzpoFjvfzLGvO-d_P25tzq-=qy8rEGijnBthjcQ@mail.gmail.com>
References: <CABbU2U-8wjwkzpoFjvfzLGvO-d_P25tzq-=qy8rEGijnBthjcQ@mail.gmail.com>
Message-ID: <CAPTjJmqhignngFL6CymrjCwmT=7JHh6YJXQP3SBC1LzxKPs4nA@mail.gmail.com>

On Tue, 1 Feb 2022 at 08:54, Marco Sulla <Marco.Sulla.Python at gmail.com> wrote:
>     PyObject* d = PyDict_New();
>     args = PyTuple_New(2);
>     PyTuple_SET_ITEM(args, 0, d);
>     PyTuple_SET_ITEM(args, 1, memo);
>     Py_DECREF(d);
>

https://docs.python.org/3/c-api/tuple.html#c.PyTuple_SET_ITEM

SET_ITEM steals a reference, so you'll need to not also decref the
dict yourself.

ChrisA

From Marco.Sulla.Python at gmail.com  Mon Jan 31 18:10:49 2022
From: Marco.Sulla.Python at gmail.com (Marco Sulla)
Date: Tue, 1 Feb 2022 00:10:49 +0100
Subject: Segfault after deepcopy in a C extension
In-Reply-To: <CAPTjJmqhignngFL6CymrjCwmT=7JHh6YJXQP3SBC1LzxKPs4nA@mail.gmail.com>
References: <CABbU2U-8wjwkzpoFjvfzLGvO-d_P25tzq-=qy8rEGijnBthjcQ@mail.gmail.com>
 <CAPTjJmqhignngFL6CymrjCwmT=7JHh6YJXQP3SBC1LzxKPs4nA@mail.gmail.com>
Message-ID: <CABbU2U8SCXqFf6ndkoUAsXjXHbiWCcnyfdgZB6LLjAAGToNQrg@mail.gmail.com>

....and I had to Py_INCREF(memo)! Thank you A LOT!

On Mon, 31 Jan 2022 at 23:01, Chris Angelico <rosuav at gmail.com> wrote:

> On Tue, 1 Feb 2022 at 08:54, Marco Sulla <Marco.Sulla.Python at gmail.com>
> wrote:
> >     PyObject* d = PyDict_New();
> >     args = PyTuple_New(2);
> >     PyTuple_SET_ITEM(args, 0, d);
> >     PyTuple_SET_ITEM(args, 1, memo);
> >     Py_DECREF(d);
> >
>
> https://docs.python.org/3/c-api/tuple.html#c.PyTuple_SET_ITEM
>
> SET_ITEM steals a reference, so you'll need to not also decref the
> dict yourself.
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>