From tomerv at gmail.com Mon Jul 1 08:32:32 2019 From: tomerv at gmail.com (tomerv at gmail.com) Date: Mon, 1 Jul 2019 05:32:32 -0700 (PDT) Subject: compiling 3.7.0 from source with custom libffi path In-Reply-To: References: Message-ID: <8181788f-19b5-4192-b3ca-de3d57f2a178@googlegroups.com> On Monday, September 24, 2018 at 11:48:59 AM UTC+3, Fetchinson . wrote: > I'm trying to compile python 3.7.0 from source with a custom libffi > path and the compiler/linker doesn't seem to pick up the right > version. The system libffi doesn't have the development files so I've > installed the latest libffi (also from source) to /opt/custom but > still I get > > INFO: Could not locate ffi libs and/or headers > > Failed to build these modules: > _ctypes > > Although I compile python with --prefix=/opt/custom because that's the > location I'd like to install it too. So how do I tell the build system > where to find my custom libffi? > > Cheers, > Daniel > > > > -- > Psss, psss, put it down! - http://www.cafepress.com/putitdown After some messing around with the build scripts, I figured it out. This works for me: 0. Choose installation path for FFI -> $LIBFFI_PATH 1. Configure, build, install libffi: 1a. Download libffi from https://sourceware.org/libffi/ and untar it 1b. ./configure --prefix $LIBFFI_PATH 1c. make 1d. make install 2. Configure CPython: (I'm using tcsh syntax, it is slightly different with bash) 2a. setenv PKG_CONFIG_PATH $LIBFFI_PATH/lib/pkgconfig/ 2b. setenv LDFLAGS "-L $LIBFFI_PATH/lib64/" 2c. /configure --with-pydebug 3. Build 3a. setenv LD_LIBRARY_PATH "${LIBFFI_PATH}/lib64/" 3b. make -s -j2 Good luck! From jmariano65 at gmail.com Mon Jul 1 11:23:47 2019 From: jmariano65 at gmail.com (=?UTF-8?Q?jos=C3=A9_mariano?=) Date: Mon, 1 Jul 2019 08:23:47 -0700 (PDT) Subject: Do I need a parser? In-Reply-To: References: <5e122997-3e60-4cff-9e5d-6b69601d3805@googlegroups.com> Message-ID: <82bef4e1-e157-4cf0-9931-81c74a9e5c37@googlegroups.com> Dear All, Thank you very much for your valuable input. Thanks Alan for your kind words. I'm not Spanish, I'm Portuguese, but I know what you mean. Thomas, I was able to track down the author but he is not willing to release the source code. The executable is free but apparently the source is not. >From your comments I came up with this general configuration for the new software. Please tell me what you think. The new software would use a settings files in one "standard" format. I like INI. It's note very powerful, but is easy to read and enough for the matter at hand. I could then use configparser to parse the settings to the main module. One separate module would convert the original format into the new one. The same for the script files. The new format would be plain python, in one separated file, that could be imported into the main file. A separated module would convert the old script format to the new one (python). Both modules could be run explicitly or the main software would take as input the original files and run the conversion modules. In this way, old users could use their existing scripts and settings files and new users that are willing to learn some basic python could use the new format. So, besides the new software, i would need to write one module to convert the settings from the old format to the new one, maybe using the ideas of Thomas, and another module to convert the sripts. Is this a good approach. You comment are very welcome. Thanks Mariano From simon.pomarede at gmail.com Mon Jul 1 11:53:25 2019 From: simon.pomarede at gmail.com (Simon) Date: Mon, 1 Jul 2019 08:53:25 -0700 (PDT) Subject: Looking for tips and gotchas for working with Python 3.5 zipapp feature In-Reply-To: References: Message-ID: > The other main limitation (not so much a gotcha as a consequence of > how the OS works) is that you can't load C extensions (pyd or so > files) from a zipfile. If you need to do that, you'll have to bundle > the C extensions to work around that limitation, but that's pretty > advanced usage. > Very interesting post (and one of the only ones on the internet about that... that is why I am posting here even though the thread is quite old). I am exactly in the "pretty advanced usage": I want to create a zip that embed numpy. In this case, I have to bundle the C extension. How can I do that? Thx, Simon From python at bdurham.com Mon Jul 1 13:15:42 2019 From: python at bdurham.com (Malcolm Greene) Date: Mon, 01 Jul 2019 11:15:42 -0600 Subject: =?UTF-8?Q?Re:_Looking_for_tips_and_gotchas_for_working_with_Python_3.5_z?= =?UTF-8?Q?ipapp_feature?= In-Reply-To: References: Message-ID: <1724f068-bdb5-41c4-aea7-1a394dbf4acb@www.fastmail.com> > I am exactly in the "pretty advanced usage": I want to create a zip that embed numpy. In this case, I have to bundle the C extension. How can I do that? 1. PyInstaller 2. PyOxide (new technology, may or may not support Numpy) Let us know how you make out. Malcolm From awilliam at whitemice.org Mon Jul 1 16:13:57 2019 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Mon, 01 Jul 2019 16:13:57 -0400 Subject: pyodbc -> MS-SQL Server Named Instance ? Message-ID: <1562012037.3662.1.camel@whitemice.org> I am trying to connect to a Named Instance on an MS-SQL server using pyODBC. The ODBC driver works, as I can connection without issue to a non- named-instance SQL-Server used by another application. What is the DSN (connection) string magick to connect to a named instance? I can connect from my JDBC client (DbVisualizer) by specifying the instance name and port 1434. -- Adam Tauno Williams GPG D95ED383 OpenGroupware Developer From markos at c2o.pro.br Mon Jul 1 15:08:53 2019 From: markos at c2o.pro.br (Markos) Date: Mon, 1 Jul 2019 16:08:53 -0300 Subject: Matplotlib import image as float32 In-Reply-To: <721c6dce-46db-01f9-2759-699903aaa415@c2o.pro.br> References: <721c6dce-46db-01f9-2759-699903aaa415@c2o.pro.br> Message-ID: <8142748a-9214-5eee-3974-c81a886f251b@c2o.pro.br> Hi, I observed that matplotlib reads an image file (PNG) as float32: Please, how to read this file as int8 to get RGB in range of 0-255? Thank you, Markos >import numpy as np >import matplotlib.pyplot as plt >import matplotlib.image as mpimg >imagem = mpimg.imread('lenna.png') >print (imagem) [[[0.8862745? 0.5372549? 0.49019608] ? [0.8862745? 0.5372549? 0.49019608] ? [0.8745098? 0.5372549? 0.52156866] ? ... >print (imagem.dtype) float32 From rosuav at gmail.com Mon Jul 1 17:03:43 2019 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 2 Jul 2019 07:03:43 +1000 Subject: Matplotlib import image as float32 In-Reply-To: <8142748a-9214-5eee-3974-c81a886f251b@c2o.pro.br> References: <721c6dce-46db-01f9-2759-699903aaa415@c2o.pro.br> <8142748a-9214-5eee-3974-c81a886f251b@c2o.pro.br> Message-ID: On Tue, Jul 2, 2019 at 6:59 AM Markos wrote: > > Hi, > > I observed that matplotlib reads an image file (PNG) as float32: > > Please, how to read this file as int8 to get RGB in range of 0-255? > > Thank you, > > Markos > > >import numpy as np > > >import matplotlib.pyplot as plt > > >import matplotlib.image as mpimg > > >imagem = mpimg.imread('lenna.png') > > >print (imagem) > > [[[0.8862745 0.5372549 0.49019608] > [0.8862745 0.5372549 0.49019608] > [0.8745098 0.5372549 0.52156866] > ... > > >print (imagem.dtype) > > float32 > Seems like matplotlib is rescaling everything to be on the range 0 to 1. https://matplotlib.org/users/image_tutorial.html ChrisA From cs at cskk.id.au Mon Jul 1 18:58:20 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 2 Jul 2019 08:58:20 +1000 Subject: Do I need a parser? In-Reply-To: <82bef4e1-e157-4cf0-9931-81c74a9e5c37@googlegroups.com> References: <82bef4e1-e157-4cf0-9931-81c74a9e5c37@googlegroups.com> Message-ID: <20190701225820.GA73832@cskk.homeip.net> On 01Jul2019 08:23, jos? mariano wrote: >The new software would use a settings files in one "standard" format. I >like INI. It's note very powerful, but is easy to read and enough for >the matter at hand. I could then use configparser to parse the settings >to the main module. One separate module would convert the original >format into the new one. I agree with this. I also like INI for configurations which are not complex (==> do not have deeply nested structure). Another advantage of the configparser module it that has a fine mapping interface (like a dict of dicts) _and_ it will write itself out. So if your programme can run "bare", you can get it to write out a default INI file for the user to modify subsequently. >The same for the script files. The new format would be plain python, in one separated file, that could be imported into the main file. A separated module would convert the old script format to the new one (python). I am a bit uncomfortable with fully executable python script files; it is far too easy for someone to send you "here is a nice script file" which does ANYTHING. They are unsecurable. It is better (though much more work) to devise a simple script language which you parse and turn into some kind of rule system. That way the semantics are constrained. Cheers, Cameron Simpson From frank at chagford.com Tue Jul 2 01:36:55 2019 From: frank at chagford.com (Frank Millman) Date: Tue, 2 Jul 2019 07:36:55 +0200 Subject: pyodbc -> MS-SQL Server Named Instance ? In-Reply-To: <1562012037.3662.1.camel@whitemice.org> References: <1562012037.3662.1.camel@whitemice.org> Message-ID: On 2019-07-01 10:13 PM, Adam Tauno Williams wrote: > I am trying to connect to a Named Instance on an MS-SQL server using > pyODBC. > > The ODBC driver works, as I can connection without issue to a non- > named-instance SQL-Server used by another application. > > What is the DSN (connection) string magick to connect to a named > instance? > > I can connect from my JDBC client (DbVisualizer) by specifying the > instance name and port 1434. > > This is what I use - conn = pyodbc.connect( driver='sql server', server=r'localhost\sqlexpress', database=self.database, user=self.user, password=self.pwd, trusted_connection=True) SQL Server is running on the same host as my python program, so it may be a simpler setup than yours. Frank Millman From nimbiotics at gmail.com Tue Jul 2 08:53:46 2019 From: nimbiotics at gmail.com (Mario R. Osorio) Date: Tue, 2 Jul 2019 05:53:46 -0700 (PDT) Subject: Do I need a parser? In-Reply-To: <5e122997-3e60-4cff-9e5d-6b69601d3805@googlegroups.com> References: <5e122997-3e60-4cff-9e5d-6b69601d3805@googlegroups.com> Message-ID: <48548748-0324-46f0-b507-6cc09b80b9f9@googlegroups.com> On Saturday, June 29, 2019 at 8:40:06 AM UTC-4, jos? mariano wrote: > Dear all, > > I'm sure that this subject has been addressed many times before on this forum, but my poor knowledge of English and of computer jargon and concepts results on not being able to find the answer i'm looking for when I search the forum. > > So here is my problem: I have this open source project for the scientific community were i want to duplicate an old MS-DOS application written in Fortran. I don't have the source code. The idea is to re-write the software in Python. Originally, the old application would would need to input files: one config file, written with a specific format (see below) and a second one, the so-called scrip file, that defines the sequence of operations to be performed by the main software, also written in a specific format. > > To make the transition to the new application as painless as possible to the users, because most of them have their collection of scrips (and settings) developed over the years and are not willing to learn a new script language, I would like to make the new app 100% compatible with the old input files. > > The operation of the new software would be like this: From the shell, run "my_new_software old_script_file.***". The new software would load the old_script, parse it (?), set the internal variables, load the script and run it. > > So, to get to my questions: > > - To load and read the config file I need a parser, right? Is their a parser library where we can define the syntax of the language to use? Are there better (meaning easier) ways to accomplish the same result? > > - For the interpretation of the script file, I don't have any clue how to this... One important thing, the script language admits some simple control flow statements like do-wile, again written using a specific sintax. > > Thanks a lot for the help and sorry for the long post. > > Mariano > > > > Example of a config (settings) file > ======================== > ..... > CONDAD -11 > BURAD2 4 SALT1 1.0 KNO3 > ELEC5 -2.0 mV 400 58 0. 0 > ..... > > > Example of a script > =================== > !Conductivity titration > cmnd bur1 f > set vinit 100 > set endpt 2000 > set mvinc 20 > set drftim 1 > set rdcrit cond 0.5 per_min > set dosinc bur1 0.02 1000 > set titdir up > titratc cond bur1 Jose, You've had already very good advise; all I wanted to do is recommend the use of PyParsing (https://github.com/pyparsing/pyparsing). It helped me a lot in creating and parsing a simple grammar I had to implement in a personal project. If you've ever used lex or yacc you'll appreciate PyParsing's simplicity. Of course, as with anything else; there is a (rather small) learning curve From awilliam at whitemice.org Tue Jul 2 09:41:00 2019 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Tue, 02 Jul 2019 09:41:00 -0400 Subject: pyodbc -> MS-SQL Server Named Instance ? In-Reply-To: References: <1562012037.3662.1.camel@whitemice.org> Message-ID: <1562074860.4466.1.camel@whitemice.org> On Tue, 2019-07-02 at 07:36 +0200, Frank Millman wrote: > On 2019-07-01 10:13 PM, Adam Tauno Williams wrote: > > I am trying to connect to a Named Instance on an MS-SQL server > > using pyODBC. > This is what I use - > > ?????conn = pyodbc.connect( > ?????????driver='sql server', > ?????????server=r'localhost\sqlexpress', > ?????????database=self.database, > ?????????user=self.user, > ?????????password=self.pwd, > ?????????trusted_connection=True) > > SQL Server is running on the same host as my python program, so it > may? be a simpler setup than yours. What ODBC driver are you using? I have - >>> db = pyodbc.connect( ...?????driver='ODBC Driver 13 for SQL Server', ...?????server=r'server.example.com\instancename', ...?????database='dbname', ...?????user='*********', ...?????password='*******', ) Traceback (most recent call last): ? File "", line 6, in pyodbc.OperationalError: ('HYT00', '[HYT00] [Microsoft][ODBC Driver 13 for SQL Server]Login timeout expired (0) (SQLDriverConnect)') [ODBC Driver 13 for SQL Server] Description=Microsoft ODBC Driver 13 for SQL Server Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.9.2 UsageCount=1 -- Executive Committee Chair Michigan Association of Railroad Passengers 537 Shirley St NE Grand Rapids, MI 49503-1754 Phone: 616.581.8010 E-mail: awilliam at whitemice.org GPG#D95ED383 From python at bdurham.com Tue Jul 2 09:48:55 2019 From: python at bdurham.com (Malcolm Greene) Date: Tue, 02 Jul 2019 07:48:55 -0600 Subject: pyodbc -> MS-SQL Server Named Instance ? In-Reply-To: <1562074860.4466.1.camel@whitemice.org> References: <1562012037.3662.1.camel@whitemice.org> <1562074860.4466.1.camel@whitemice.org> Message-ID: <4a9c335e-12fa-4284-9fcb-fadbd7f1d4b4@www.fastmail.com> You may need to update your ODBC driver from version 13 to 17.x. Malcolm From awilliam at whitemice.org Tue Jul 2 11:00:05 2019 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Tue, 02 Jul 2019 11:00:05 -0400 Subject: pyodbc -> MS-SQL Server Named Instance ? In-Reply-To: <1562074860.4466.1.camel@whitemice.org> References: <1562012037.3662.1.camel@whitemice.org> <1562074860.4466.1.camel@whitemice.org> Message-ID: <1562079605.4466.3.camel@whitemice.org> On Tue, 2019-07-02 at 09:41 -0400, Adam Tauno Williams wrote: > On Tue, 2019-07-02 at 07:36 +0200, Frank Millman wrote: > > On 2019-07-01 10:13 PM, Adam Tauno Williams wrote: > > > I am trying to connect to a Named Instance on an MS-SQL server > > > using pyODBC. It appears that the instance location service doesn't work. Digging out the instance->power assignment I can change the error message and create an authentication failed even in the Event Log! It is sorta progress. :( >>> db = pyodbc.connect( ...??????driver='ODBC Driver 13 for SQL Server', ...??????server=r'sql.example.com,51994', ...??????database='dbname', ...??????user='*******', ...??????password='*******', ) Traceback (most recent call last): ? File "", line 6, in pyodbc.Error: ('28000', "[28000] [Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Login failed for user '*******'. (18456) (SQLDriverConnect)") -- Highland Park Neighborhood Association Adam Tauno Williams, Board Chair Grand Rapids, MI. From markos at c2o.pro.br Tue Jul 2 11:06:06 2019 From: markos at c2o.pro.br (Markos) Date: Tue, 2 Jul 2019 12:06:06 -0300 Subject: Matplotlib import image as float32 In-Reply-To: References: <721c6dce-46db-01f9-2759-699903aaa415@c2o.pro.br> <8142748a-9214-5eee-3974-c81a886f251b@c2o.pro.br> Message-ID: Em 01-07-2019 18:03, Chris Angelico escreveu: > On Tue, Jul 2, 2019 at 6:59 AM Markos wrote: >> Hi, >> >> I observed that matplotlib reads an image file (PNG) as float32: >> >> Please, how to read this file as int8 to get RGB in range of 0-255? >> >> Thank you, >> >> Markos >> >>> import numpy as np >>> import matplotlib.pyplot as plt >>> import matplotlib.image as mpimg >>> imagem = mpimg.imread('lenna.png') >>> print (imagem) >> [[[0.8862745 0.5372549 0.49019608] >> [0.8862745 0.5372549 0.49019608] >> [0.8745098 0.5372549 0.52156866] >> ... >> >>> print (imagem.dtype) >> float32 >> > Seems like matplotlib is rescaling everything to be on the range 0 to 1. > > https://matplotlib.org/users/image_tutorial.html > > ChrisA Hi Chris, I found a workaround at: www.programcreek.com/python/example/99442/matplotlib.image.imread I don't know if is the most elegant? but it worked. I replaced: imagem = mpimg.imread('lenna.png') by the command: imagem = (mpimg.imread('lenna.png') * 255).astype('uint8') Thanks, Markos From awilliam at whitemice.org Tue Jul 2 11:30:47 2019 From: awilliam at whitemice.org (Adam Tauno Williams) Date: Tue, 02 Jul 2019 11:30:47 -0400 Subject: pyodbc -> MS-SQL Server Named Instance ? In-Reply-To: <1562079605.4466.3.camel@whitemice.org> References: <1562012037.3662.1.camel@whitemice.org> <1562074860.4466.1.camel@whitemice.org> <1562079605.4466.3.camel@whitemice.org> Message-ID: <1562081447.4466.5.camel@whitemice.org> On Tue, 2019-07-02 at 11:00 -0400, Adam Tauno Williams wrote: > On Tue, 2019-07-02 at 09:41 -0400, Adam Tauno Williams wrote: > > On Tue, 2019-07-02 at 07:36 +0200, Frank Millman wrote: > > > On 2019-07-01 10:13 PM, Adam Tauno Williams wrote: > > > > I am trying to connect to a Named Instance on an MS-SQL server > > > > using pyODBC. > > It appears that the instance location service doesn't work.??Digging > out the instance->power assignment I can change the error message and > create an authentication failed even in the Event Log! > > It is sorta progress. :( > > > > > db = pyodbc.connect( > > ...??????driver='ODBC Driver 13 for SQL Server', > ...??????server=r'sql.example.com,51994', > ...??????database='dbname', > ...??????user='*******', > ...??????password='*******', ) > Traceback (most recent call last): > ? File "", line 6, in > pyodbc.Error: ('28000', "[28000] [Microsoft][ODBC Driver 13 for SQL > Server][SQL Server]Login failed for user '*******'. (18456) > (SQLDriverConnect)") Interesting, it does appear to be a driver problem. The old FreeTDS connection ( pymsssql ) works right away. WORKS: pymssql.connect( host='sql.example.com\instanceName', user='******',password='********', database='dbName') -- Adam Tauno Williams GPG D95ED383 OpenGroupware Developer From tjol at tjol.eu Tue Jul 2 14:47:01 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 2 Jul 2019 20:47:01 +0200 Subject: Do I need a parser? In-Reply-To: <82bef4e1-e157-4cf0-9931-81c74a9e5c37@googlegroups.com> References: <5e122997-3e60-4cff-9e5d-6b69601d3805@googlegroups.com> <82bef4e1-e157-4cf0-9931-81c74a9e5c37@googlegroups.com> Message-ID: <62cbdeb7-c5b7-9614-ae96-a7d57a73cad2@tjol.eu> On 01/07/2019 17:23, jos? mariano wrote: > Dear All, > > Thank you very much for your valuable input. > Thanks Alan for your kind words. I'm not Spanish, I'm Portuguese, but I know what you mean. > Thomas, I was able to track down the author but he is not willing to release the source code. The executable is free but apparently the source is not. > > From your comments I came up with this general configuration for the new software. Please tell me what you think. > > The new software would use a settings files in one "standard" format. I like INI. It's note very powerful, but is easy to read and enough for the matter at hand. I could then use configparser to parse the settings to the main module. One separate module would convert the original format into the new one. > > The same for the script files. The new format would be plain python, in one separated file, that could be imported into the main file. A separated module would convert the old script format to the new one (python). > > Both modules could be run explicitly or the main software would take as input the original files and run the conversion modules. > > In this way, old users could use their existing scripts and settings files and new users that are willing to learn some basic python could use the new format. > > So, besides the new software, i would need to write one module to convert the settings from the old format to the new one, maybe using the ideas of Thomas, and another module to convert the sripts. > > Is this a good approach. You comment are very welcome. Sounds reasonable to me. Personally I prefer YAML (parsed e.g. with the ruamel.yaml module) to INI because it's more flexible: you get (deeper) nesting and data types (i.e. numbers and strings are different) - but configparser is a fine module to use for config files. Some people like using Python modules as config files. Obviously, as Cameron points out, using Python instead of a custom scripting language has security implications, that go with the increased power that the user has. However, in a laboratory setting, where the users are scientists who have physical access to the machine anyway, I really don't think this is a concern. And the added flexibility may be worth a lot down the line! (I would say that though, as I suggested it) Don't forget that many scientists frequently use Python anyway. - Thomas From torriem at gmail.com Tue Jul 2 17:16:25 2019 From: torriem at gmail.com (Michael Torrie) Date: Tue, 2 Jul 2019 15:16:25 -0600 Subject: Do I need a parser? In-Reply-To: <62cbdeb7-c5b7-9614-ae96-a7d57a73cad2@tjol.eu> References: <5e122997-3e60-4cff-9e5d-6b69601d3805@googlegroups.com> <82bef4e1-e157-4cf0-9931-81c74a9e5c37@googlegroups.com> <62cbdeb7-c5b7-9614-ae96-a7d57a73cad2@tjol.eu> Message-ID: <8ab07cff-df4f-fcb6-167a-15aad9abd31b@gmail.com> On 07/02/2019 12:47 PM, Thomas Jollans wrote: > Obviously, as Cameron points out, using Python instead of a custom > scripting language has security implications, that go with the increased > power that the user has. I've always found this kind of thing to be a bit strange. I do understand where Cameron was coming from, but still. The python interpreter itself will happily run any python program you give it, even one that will erase all your files. As well as compilers, or bash. Yet we don't often go around claiming that they are security issues. As with all things, the security implications depend entirely on how the program is running, who's running the program, and where the input is coming from. And this is where Cameron's warning comes into play. I guess where people get into trouble is when they have their nice little python program that exec()'s a user's input file, and then they think, hey that'd be cool to run this as a web service. Whoops. So maybe better to avoid it in the first place. > However, in a laboratory setting, where the users are scientists who > have physical access to the machine anyway, I really don't think this > is a concern. And the added flexibility may be worth a lot down the > line! It's not even a matter of physical access. Any script can do anything that the user can do. I'm pretty sure python scripts cannot be setuid. In these limited circumstances, I can't think of any security implications other than the usual "don't run programs from people you don't know." Which is directed at the user, not the programmer. From nad at python.org Tue Jul 2 19:15:05 2019 From: nad at python.org (Ned Deily) Date: Tue, 2 Jul 2019 19:15:05 -0400 Subject: [RELEASE] Python 3.6.9 security-fix release is now available Message-ID: <33DCD8BF-01EE-4BED-A67F-B7F44CE83261@python.org> Python 3.6.9 is now available. 3.6.9 is the first security-only-fix release of Python 3.6. Python 3.6 has now entered the security fix phase of its life cycle. Only security-related issues are accepted and addressed during this phase. We plan to provide security fixes for Python 3.6 as needed through 2021, five years following its initial release. Security fix releases are produced periodically as needed and only provided in source code form; binary installers are not provided. We urge you to consider upgrading to the latest release of Python 3.7, the current fully-supported version. You can find the release files, a link to the changelog, and more information here: https://www.python.org/downloads/release/python-369/ https://www.python.org/downloads/ -- Ned Deily nad at python.org -- [] From nad at python.org Tue Jul 2 19:31:42 2019 From: nad at python.org (Ned Deily) Date: Tue, 2 Jul 2019 19:31:42 -0400 Subject: [RELEASE] Python 3.7.4rc2 is now available for testing Message-ID: Python 3.7.4rc2 is now available. 3.7.4rc2 is the second release preview of the next maintenance release of Python 3.7, the latest feature release of Python. Assuming no further critical problems are found prior to 2019-07-08, no code changes are planned between this release candidate and the final release. The release candidate is intended to give you the opportunity to test the new security and bug fixes in 3.7.4. Because of the small number of changes between rc1, the original release preview, and rc2, we are planning on an abbreviated exposure cycle so we can get the final release to you as soon as possible. We strongly encourage you to test your projects and report issues found to https://bugs.python.org/ as soon as possible. Please keep in mind that this is a preview release and, thus, is not recommended for production environments. You can find the release files, a link to the changelog, and more information here: https://www.python.org/downloads/release/python-374rc2/ -- Ned Deily nad at python.org -- [] From frank at chagford.com Wed Jul 3 01:11:43 2019 From: frank at chagford.com (Frank Millman) Date: Wed, 3 Jul 2019 07:11:43 +0200 Subject: pyodbc -> MS-SQL Server Named Instance ? In-Reply-To: <1562074860.4466.1.camel@whitemice.org> References: <1562012037.3662.1.camel@whitemice.org> <1562074860.4466.1.camel@whitemice.org> Message-ID: On 2019-07-02 3:41 PM, Adam Tauno Williams wrote: > On Tue, 2019-07-02 at 07:36 +0200, Frank Millman wrote: >> On 2019-07-01 10:13 PM, Adam Tauno Williams wrote: >>> I am trying to connect to a Named Instance on an MS-SQL server >>> using pyODBC. >> This is what I use - >> >> ?????conn = pyodbc.connect( >> ?????????driver='sql server', >> ?????????server=r'localhost\sqlexpress', >> ?????????database=self.database, >> ?????????user=self.user, >> ?????????password=self.pwd, >> ?????????trusted_connection=True) >> >> SQL Server is running on the same host as my python program, so it >> may? be a simpler setup than yours. > > What ODBC driver are you using? > [...] I did reply to this yesterday but for some reason it did not appear. For the record, here it is again - I am not using an ODBC driver at all. My connection string 'just works'. This is the version of Sql Server I am using - C:\Users\User>sqlcmd 1> select @@version 2> go Microsoft SQL Server 2017 (RTM-GDR) (KB4494351) - 14.0.2014.14 (X64) Apr 5 2019 09:18:51 Copyright (C) 2017 Microsoft Corporation Express Edition (64-bit) on Windows 10 Pro 10.0 (Build 17134: ) Frank From frank at chagford.com Wed Jul 3 01:11:43 2019 From: frank at chagford.com (Frank Millman) Date: Wed, 3 Jul 2019 07:11:43 +0200 Subject: pyodbc -> MS-SQL Server Named Instance ? In-Reply-To: <1562074860.4466.1.camel@whitemice.org> References: <1562012037.3662.1.camel@whitemice.org> <1562074860.4466.1.camel@whitemice.org> Message-ID: On 2019-07-02 3:41 PM, Adam Tauno Williams wrote: > On Tue, 2019-07-02 at 07:36 +0200, Frank Millman wrote: >> On 2019-07-01 10:13 PM, Adam Tauno Williams wrote: >>> I am trying to connect to a Named Instance on an MS-SQL server >>> using pyODBC. >> This is what I use - >> >> ?????conn = pyodbc.connect( >> ?????????driver='sql server', >> ?????????server=r'localhost\sqlexpress', >> ?????????database=self.database, >> ?????????user=self.user, >> ?????????password=self.pwd, >> ?????????trusted_connection=True) >> >> SQL Server is running on the same host as my python program, so it >> may? be a simpler setup than yours. > > What ODBC driver are you using? > [...] I did reply to this yesterday but for some reason it did not appear. For the record, here it is again - I am not using an ODBC driver at all. My connection string 'just works'. This is the version of Sql Server I am using - C:\Users\User>sqlcmd 1> select @@version 2> go Microsoft SQL Server 2017 (RTM-GDR) (KB4494351) - 14.0.2014.14 (X64) Apr 5 2019 09:18:51 Copyright (C) 2017 Microsoft Corporation Express Edition (64-bit) on Windows 10 Pro 10.0 (Build 17134: ) Frank From mal at europython.eu Wed Jul 3 05:25:26 2019 From: mal at europython.eu (M.-A. Lemburg) Date: Wed, 3 Jul 2019 11:25:26 +0200 Subject: EuroPython 2019: Find a new job at the conference Message-ID: We?d like to draw your attention to our job board, with plenty of job ads from our sponsors: EuroPython 2019 Job Board * https://ep2019.europython.eu/sponsor/job-board/ * Our sponsors would love to get in touch with you, so please have a look and visit them at their booth or contact them via the links and email addresses given on the page. Job ad emails ------------- We will also send out job ad emails to attendees who have agreed to receiving these emails. If you are interested, please log in, go to your profile and enable the recruiting email option in the privacy section: https://ep2019.europython.eu/user-panel/privacy-settings/ Note that we will not give your email addresses to sponsors, but only send out these emails on behalf of them. Dates and Venues ---------------- EuroPython will be held from July 8-14 2019 in Basel, Switzerland, at the Congress Center Basel (CCB) for the main conference days (Wed-Fri) and the FHNW Muttenz for the workshops/trainings/sprints days (Mon-Tue, Sat-Sun). The schedule is available at: https://ep2019.europython.eu/events/schedule/ Tickets can be purchased on our registration page: https://ep2019.europython.eu/registration/buy-tickets/ For more details, please have a look at our website and the FAQ: https://ep2019.europython.eu/faq Help spread the word -------------------- Please help us spread this message by sharing it on your social networks as widely as possible. Thank you ! Link to the blog post: https://blog.europython.eu/post/186020999932/europython-2019-find-a-new-job-at-the-conference Tweet: https://twitter.com/europython/status/1146342161354543106 Enjoy, -- EuroPython 2019 Team https://ep2019.europython.eu/ https://www.europython-society.org/ From ijbrewster at alaska.edu Wed Jul 3 12:37:50 2019 From: ijbrewster at alaska.edu (Israel Brewster) Date: Wed, 3 Jul 2019 08:37:50 -0800 Subject: Multiprocessing and memory management Message-ID: I have a script that benefits greatly from multiprocessing (it?s generating a bunch of images from data). Of course, as expected each process uses a chunk of memory, and the more processes there are, the more memory used. The amount used per process can vary from around 3 GB (yes, gigabytes) to over 40 or 50 GB, depending on the amount of data being processed (usually closer to 10GB, the 40/50 is fairly rare). This puts me in a position of needing to balance the number of processes with memory usage, such that I maximize resource utilization (running one process at a time would simply take WAY to long) while not overloading RAM (which at best would slow things down due to swap). Obviously this process will be run on a machine with lots of RAM, but as I don?t know how large the datasets that will be fed to it are, I wanted to see if I could build some intelligence into the program such that it doesn?t overload the memory. A couple of approaches I thought of: 1) Determine the total amount of RAM in the machine (how?), assume an average of 10GB per process, and only launch as many processes as calculated to fit. Easy, but would run the risk of under-utilizing the processing capabilities and taking longer to run if most of the processes were using significantly less than 10GB 2) Somehow monitor the memory usage of the various processes, and if one process needs a lot, pause the others until that one is complete. Of course, I?m not sure if this is even possible. 3) Other approaches? --- Israel Brewster Software Engineer Alaska Volcano Observatory Geophysical Institute - UAF 2156 Koyukuk Drive Fairbanks AK 99775-7320 Work: 907-474-5172 cell: 907-328-9145 From shakeelihf at gmail.com Wed Jul 3 06:14:14 2019 From: shakeelihf at gmail.com (Shakeel Ahmad) Date: Wed, 3 Jul 2019 15:14:14 +0500 Subject: FCNLT Module in Windows Message-ID: <5d1c7ff9.1c69fb81.17f67.c978@mx.google.com> Sent from Mail for Windows 10 From gherron at digipen.edu Wed Jul 3 14:24:18 2019 From: gherron at digipen.edu (Gary Herron) Date: Wed, 3 Jul 2019 11:24:18 -0700 Subject: Multiprocessing and memory management In-Reply-To: References: Message-ID: On 7/3/19 9:37 AM, ijbrewster at alaska.edu wrote: > I have a script that benefits greatly from multiprocessing (it?s generating a bunch of images from data). Of course, as expected each process uses a chunk of memory, and the more processes there are, the more memory used. The amount used per process can vary from around 3 GB (yes, gigabytes) to over 40 or 50 GB, depending on the amount of data being processed (usually closer to 10GB, the 40/50 is fairly rare). This puts me in a position of needing to balance the number of processes with memory usage, such that I maximize resource utilization (running one process at a time would simply take WAY to long) while not overloading RAM (which at best would slow things down due to swap). > > Obviously this process will be run on a machine with lots of RAM, but as I don?t know how large the datasets that will be fed to it are, I wanted to see if I could build some intelligence into the program such that it doesn?t overload the memory. A couple of approaches I thought of: > > 1) Determine the total amount of RAM in the machine (how?), assume an average of 10GB per process, and only launch as many processes as calculated to fit. Easy, but would run the risk of under-utilizing the processing capabilities and taking longer to run if most of the processes were using significantly less than 10GB Try psutil to get information about memory (and cpu usage and lots more).? For example: >>> import psutil >>> psutil.virtual_memory() svmem(total=16769519616, available=9151971328, percent=45.4, used=7031549952, free=4486520832, active=9026158592, inactive=2238566400, buffers=312815616, cached=4938633216, shared=234295296, slab=593375232) Home page: https://github.com/giampaolo/psutil > > 2) Somehow monitor the memory usage of the various processes, and if one process needs a lot, pause the others until that one is complete. Of course, I?m not sure if this is even possible. > > 3) Other approaches? > > > --- > Israel Brewster > Software Engineer > Alaska Volcano Observatory > Geophysical Institute - UAF > 2156 Koyukuk Drive > Fairbanks AK 99775-7320 > Work: 907-474-5172 > cell: 907-328-9145 > -- Dr. Gary Herron Professor of Computer Science DigiPen Institute of Technology (425) 895-4418 From mal at europython.eu Wed Jul 3 14:40:39 2019 From: mal at europython.eu (M.-A. Lemburg) Date: Wed, 3 Jul 2019 20:40:39 +0200 Subject: EuroPython 2019: Late Bird Rates and Day Passes Message-ID: <1a4fb5db-0d1f-af29-afbc-cdf60f1db449@europython.eu> We will be switching to the late bird rates for tickets on Saturday (July 6), so this is your last chance to get tickets at the regular rate, which is about 30% less than the late bird rate. EuroPython 2019 Tickets * https://ep2019.europython.eu/registration/buy-tickets/ * Late Bird Tickets ----------------- We will have the following categories of late bird ticket prices for the conference tickets: - Business conference ticket: EUR 775.00 excl. VAT, EUR 834.68 incl. 7.7% Swiss VAT (for people using Python to make a living) - Personal conference day pass: EUR 520.00 incl. 7.7% Swiss VAT (for people enjoying Python from home) - Student conference ticket: not available (please get day passes or a personal ticket instead) Day Passes ---------- As in the past, we will also sell day passes for the conference. These allow attending the conference for a single day (July 8 - 14; valid on the day you pick up the day pass badge): - Business conference day pass: EUR 390.00 excl. VAT, EUR 420.03 incl. 7.7% Swiss VAT (for people using Python to make a living) - Personal conference day pass: EUR 260.00 incl. 7.7% Swiss VAT (for people enjoying Python from home) - Student conference day pass: EUR 110.00 incl. 7.7% Swiss VAT (only available for pupils, students and postdoctoral researchers; please bring your student card or declaration from University, stating your affiliation, starting and end dates of your contract) Day passes are available starting today, Wednesday, July 3. Please see the registration page for full details of what is included in the ticket price. Also note that neither late bird tickets, nor day passes are refundable. Dates and Venues ---------------- EuroPython will be held from July 8-14 2019 in Basel, Switzerland, at the Congress Center Basel (CCB) for the main conference days (Wed-Fri) and the FHNW Muttenz for the workshops/trainings/sprints days (Mon-Tue, Sat-Sun). The schedule is available at: https://ep2019.europython.eu/events/schedule/ Tickets can be purchased on our registration page: https://ep2019.europython.eu/registration/buy-tickets/ For more details, please have a look at our website and the FAQ: https://ep2019.europython.eu/faq Help spread the word -------------------- Please help us spread this message by sharing it on your social networks as widely as possible. Thank you ! Link to the blog post: https://blog.europython.eu/post/186029753397/europython-2019-late-bird-rates-and-day-passes Tweet: https://twitter.com/europython/status/1146486458162864130 Enjoy, -- EuroPython 2019 Team https://ep2019.europython.eu/ https://www.europython-society.org/ From hjp-python at hjp.at Wed Jul 3 15:48:09 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Wed, 3 Jul 2019 21:48:09 +0200 Subject: Multiprocessing and memory management In-Reply-To: References: Message-ID: <20190703194809.d6od3453jvlob4oy@hjp.at> On 2019-07-03 08:37:50 -0800, Israel Brewster wrote: > 1) Determine the total amount of RAM in the machine (how?), assume an > average of 10GB per process, and only launch as many processes as > calculated to fit. Easy, but would run the risk of under-utilizing the > processing capabilities and taking longer to run if most of the > processes were using significantly less than 10GB > > 2) Somehow monitor the memory usage of the various processes, and if > one process needs a lot, pause the others until that one is complete. > Of course, I?m not sure if this is even possible. If you use Linux or another unixoid OS, you can pause and resume processes with the STOP and CONT signals. Just keep in mind that the paused processes still need virtual memory and cannot complete - so you need enough swap space. > 3) Other approaches? Is the memory usage at all predictable? I.e. can you estimate the usage from the size of the input data? Or after the process has been running for a short time? In that case you could monitor the free space and use your estimates to determine whether you can start another process now or need to wait until later (until a process terminates or maybe only until you get a better estimate). hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ikorot01 at gmail.com Wed Jul 3 16:19:36 2019 From: ikorot01 at gmail.com (Igor Korot) Date: Wed, 3 Jul 2019 15:19:36 -0500 Subject: FCNLT Module in Windows In-Reply-To: <5d1c7ff9.1c69fb81.17f67.c978@mx.google.com> References: <5d1c7ff9.1c69fb81.17f67.c978@mx.google.com> Message-ID: Hi, On Wed, Jul 3, 2019 at 12:56 PM Shakeel Ahmad wrote: > > ACK > > Sent from Mail for Windows 10 > > -- > https://mail.python.org/mailman/listinfo/python-list From none at invalid.com Wed Jul 3 16:37:43 2019 From: none at invalid.com (mm0fmf) Date: Wed, 3 Jul 2019 21:37:43 +0100 Subject: Handle foreign character web input In-Reply-To: References: <8dea3921-63db-df9f-5482-b6732c4344ee@tjol.eu> <24016ef6-4122-4293-b2e1-8f2812807dfd@googlegroups.com> Message-ID: On 30/06/2019 15:04, Chris Angelico wrote: > ut it looks like our old "Py3's > Unicode is buggy" troll is back Yes... check the from field in the posts for confirmation. But it's obvious from the message content anyway. From ikorot01 at gmail.com Wed Jul 3 17:06:22 2019 From: ikorot01 at gmail.com (Igor Korot) Date: Wed, 3 Jul 2019 16:06:22 -0500 Subject: Handle foreign character web input In-Reply-To: <8dea3921-63db-df9f-5482-b6732c4344ee@tjol.eu> References: <8dea3921-63db-df9f-5482-b6732c4344ee@tjol.eu> Message-ID: Hi, Thomas, On Sat, Jun 29, 2019 at 11:06 AM Thomas Jollans wrote: > > On 28/06/2019 22:25, Tobiah wrote: > > A guy comes in and enters his last name as R?nngren. > With a capital ? in the middle? That's unusual. > > > > So what did the browser really give me; is it encoded > > in some way, like latin-1? Does it depend on whether > > the name was cut and pasted from a Word doc. etc? > > Should I handle these internally as unicode? Right > > now my database tables are latin-1 and things seem > > to usually work, but not always. > > > If your database is using latin-1, German and French names will work, > but Croatian and Polish names often won't. Not to mention people using > other writing systems. > > So G?nther and Fran?ois are ok, but Boles?aw turns into Boles?aw and > don't even think about anybody called ???????? or ????. As others pointed out - it is very easy to do transliteration especially if its' not a user registration that will be done. But I would simply not do that at all - create your forms in English and accept English spellings only. Most people that do computers this days can enter phonetic spelling of their first/last names (even in Chinese/Japanese/Hebrew). And all European names can be transliterated to English. Besides as the OP said - if someone comes to him and will try to enter the non-English name. The OP might not even have the appropriate keyboard layout to input such a name. And if this is an (time consuming) event all (s)he can do is ask for phonetic spelling. Thank you. > > > > > > Also, what do people do when searching for a record. > > Is there some way to get 'Ronngren' to match the other > > possible foreign spellings? > > -- > https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Wed Jul 3 17:38:22 2019 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Jul 2019 07:38:22 +1000 Subject: Handle foreign character web input In-Reply-To: References: <8dea3921-63db-df9f-5482-b6732c4344ee@tjol.eu> Message-ID: On Thu, Jul 4, 2019 at 7:08 AM Igor Korot wrote: > > Hi, Thomas, > > On Sat, Jun 29, 2019 at 11:06 AM Thomas Jollans wrote: > > > > On 28/06/2019 22:25, Tobiah wrote: > > > A guy comes in and enters his last name as R?nngren. > > With a capital ? in the middle? That's unusual. > > > > > > So what did the browser really give me; is it encoded > > > in some way, like latin-1? Does it depend on whether > > > the name was cut and pasted from a Word doc. etc? > > > Should I handle these internally as unicode? Right > > > now my database tables are latin-1 and things seem > > > to usually work, but not always. > > > > > > If your database is using latin-1, German and French names will work, > > but Croatian and Polish names often won't. Not to mention people using > > other writing systems. > > > > So G?nther and Fran?ois are ok, but Boles?aw turns into Boles?aw and > > don't even think about anybody called ???????? or ????. > > As others pointed out - it is very easy to do transliteration especially if > its' not a user registration that will be done. > > But I would simply not do that at all - create your forms in English and > accept English spellings only. > Most people that do computers this days can enter phonetic spelling > of their first/last names (even in Chinese/Japanese/Hebrew). > > And all European names can be transliterated to English. > > Besides as the OP said - if someone comes to him and will > try to enter the non-English name. The OP might not even have the appropriate > keyboard layout to input such a name. And if this is an (time consuming) event > all (s)he can do is ask for phonetic spelling. > > Thank you. > What you basically just said was "I wish all those ugly foreign names would just go away". Honestly, that's not really an acceptable solution; you assume that you can transliterate any name into "English" in some perfect way, which is acceptable to everyone in the world. And you also assume that this transformation will be completely consistent, so you can ask someone his/her name and always get back the same thing. If you want to do a Latinization and accent strip for the sake of a search, that's fine; but make sure you retain the name as people want it to be retained. Don't be bigoted. https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ ChrisA From ikorot01 at gmail.com Wed Jul 3 19:13:20 2019 From: ikorot01 at gmail.com (Igor Korot) Date: Wed, 3 Jul 2019 18:13:20 -0500 Subject: Handle foreign character web input In-Reply-To: References: <8dea3921-63db-df9f-5482-b6732c4344ee@tjol.eu> Message-ID: Hi, Chris, On Wed, Jul 3, 2019 at 4:41 PM Chris Angelico wrote: > > On Thu, Jul 4, 2019 at 7:08 AM Igor Korot wrote: > > > > Hi, Thomas, > > > > On Sat, Jun 29, 2019 at 11:06 AM Thomas Jollans wrote: > > > > > > On 28/06/2019 22:25, Tobiah wrote: > > > > A guy comes in and enters his last name as R?nngren. > > > With a capital ? in the middle? That's unusual. > > > > > > > > So what did the browser really give me; is it encoded > > > > in some way, like latin-1? Does it depend on whether > > > > the name was cut and pasted from a Word doc. etc? > > > > Should I handle these internally as unicode? Right > > > > now my database tables are latin-1 and things seem > > > > to usually work, but not always. > > > > > > > > > If your database is using latin-1, German and French names will work, > > > but Croatian and Polish names often won't. Not to mention people using > > > other writing systems. > > > > > > So G?nther and Fran?ois are ok, but Boles?aw turns into Boles?aw and > > > don't even think about anybody called ???????? or ????. > > > > As others pointed out - it is very easy to do transliteration especially if > > its' not a user registration that will be done. > > > > But I would simply not do that at all - create your forms in English and > > accept English spellings only. > > Most people that do computers this days can enter phonetic spelling > > of their first/last names (even in Chinese/Japanese/Hebrew). > > > > And all European names can be transliterated to English. > > > > Besides as the OP said - if someone comes to him and will > > try to enter the non-English name. The OP might not even have the appropriate > > keyboard layout to input such a name. And if this is an (time consuming) event > > all (s)he can do is ask for phonetic spelling. > > > > Thank you. > > > What you basically just said was "I wish all those ugly foreign names > would just go away". Honestly, that's not really an acceptable > solution; you assume that you can transliterate any name into > "English" in some perfect way, which is acceptable to everyone in the > world. And you also assume that this transformation will be completely > consistent, so you can ask someone his/her name and always get back > the same thing. > > If you want to do a Latinization and accent strip for the sake of a > search, that's fine; but make sure you retain the name as people want > it to be retained. Don't be bigoted. > > https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ I'm not opposing this, in fact I'm all for keeping the native name somewhere in the DB. But as I said, imaging the following situation: You are somewhere in Germany and you have a German version of OS (any OS) . You also have a German keyboard (hardware) with German keys. Now you are assigned to go to some international events where people all over the world will be coming to your presentation and they will be registering on you machine Also imagine that the company policy prohibits you from modifying the system settings. My solution: I would probably grab a lot of registering paper and ask people to enter English transliteration of the names on the machine so when you come back to the office you can properly enter their names using all those different keyboards (maybe virtual ones) to associate them with their English counterparts. Just curious - what would you do? Thank you. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list From rosuav at gmail.com Wed Jul 3 18:47:06 2019 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Jul 2019 08:47:06 +1000 Subject: Handle foreign character web input In-Reply-To: References: <8dea3921-63db-df9f-5482-b6732c4344ee@tjol.eu> Message-ID: On Thu, Jul 4, 2019 at 8:12 AM Igor Korot wrote: > > Hi, Chris, > > On Wed, Jul 3, 2019 at 4:41 PM Chris Angelico wrote: > > > > On Thu, Jul 4, 2019 at 7:08 AM Igor Korot wrote: > > > > > > Hi, Thomas, > > > > > > On Sat, Jun 29, 2019 at 11:06 AM Thomas Jollans wrote: > > > > > > > > On 28/06/2019 22:25, Tobiah wrote: > > > > > A guy comes in and enters his last name as R?nngren. > > > > With a capital ? in the middle? That's unusual. > > > > > > > > > > So what did the browser really give me; is it encoded > > > > > in some way, like latin-1? Does it depend on whether > > > > > the name was cut and pasted from a Word doc. etc? > > > > > Should I handle these internally as unicode? Right > > > > > now my database tables are latin-1 and things seem > > > > > to usually work, but not always. > > > > > > > > > > > > If your database is using latin-1, German and French names will work, > > > > but Croatian and Polish names often won't. Not to mention people using > > > > other writing systems. > > > > > > > > So G?nther and Fran?ois are ok, but Boles?aw turns into Boles?aw and > > > > don't even think about anybody called ???????? or ????. > > > > > > As others pointed out - it is very easy to do transliteration especially if > > > its' not a user registration that will be done. > > > > > > But I would simply not do that at all - create your forms in English and > > > accept English spellings only. > > > Most people that do computers this days can enter phonetic spelling > > > of their first/last names (even in Chinese/Japanese/Hebrew). > > > > > > And all European names can be transliterated to English. > > > > > > Besides as the OP said - if someone comes to him and will > > > try to enter the non-English name. The OP might not even have the appropriate > > > keyboard layout to input such a name. And if this is an (time consuming) event > > > all (s)he can do is ask for phonetic spelling. > > > > > > Thank you. > > > > > What you basically just said was "I wish all those ugly foreign names > > would just go away". Honestly, that's not really an acceptable > > solution; you assume that you can transliterate any name into > > "English" in some perfect way, which is acceptable to everyone in the > > world. And you also assume that this transformation will be completely > > consistent, so you can ask someone his/her name and always get back > > the same thing. > > > > If you want to do a Latinization and accent strip for the sake of a > > search, that's fine; but make sure you retain the name as people want > > it to be retained. Don't be bigoted. > > > > https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ > > I'm not opposing this, in fact I'm all for keeping the native name somewhere in > the DB. > > But as I said, imaging the following situation: > > You are somewhere in Germany and you have a German version of OS > (any OS) > . > You also have a German keyboard (hardware) with German keys. > > Now you are assigned to go to some international events where people > all over the world will be coming to your presentation and they will be > registering on you machine > > Also imagine that the company policy prohibits you from modifying the > system settings. > > My solution: > I would probably grab a lot of registering paper and ask people to enter > English transliteration of the names on the machine so when you come > back to the office you can properly enter their names using all those > different keyboards (maybe virtual ones) to associate them with > their English counterparts. > > Just curious - what would you do? > I would use a Compose key (if available) or a software input method (always available, as long as you have an internet connection and browser, and often available locally too). With your method, how would you enter it back at the office, if all you have is an English transliteration? How do you transform it back into the original characters? ChrisA From jfong at ms4.hinet.net Wed Jul 3 19:57:40 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Wed, 3 Jul 2019 16:57:40 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? Message-ID: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> I have the test0.py below. I expect to see 'abcd' showing in the notepad window: --------- import subprocess as sp p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) p0.communicate(input=b'abcd') --------- But nothing happens. The notepad is completely empty. What have I missed? --Jach PS. I am using python 3.4 on Windows Vista From cs at cskk.id.au Wed Jul 3 20:36:36 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 4 Jul 2019 10:36:36 +1000 Subject: What's wrong on using Popen's communicate method? In-Reply-To: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> Message-ID: <20190704003636.GA31710@cskk.homeip.net> On 03Jul2019 16:57, Jach Fong wrote: >I have the test0.py below. I expect to see 'abcd' showing in the notepad window: >--------- >import subprocess as sp >p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) >p0.communicate(input=b'abcd') >--------- >But nothing happens. The notepad is completely empty. What have I missed? >--Jach > >PS. I am using python 3.4 on Windows Vista Well I am not a Windows person, but in most GUI environments a desktop app such as Notepad does not read from its standard input - it reads keyboard stuff though the GUI interface instead. So it is ignoring your input data - this approach will generally fail with any desktop app on any platform. You may need to investigate generating synthetic keystrokes somehow. Or alternatively write your data to a text file and hand the name of the text file to notepad as a file to edit. Cheers, Cameron Simpson From rosuav at gmail.com Wed Jul 3 20:36:47 2019 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Jul 2019 10:36:47 +1000 Subject: What's wrong on using Popen's communicate method? In-Reply-To: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> Message-ID: On Thu, Jul 4, 2019 at 10:01 AM wrote: > > I have the test0.py below. I expect to see 'abcd' showing in the notepad window: > --------- > import subprocess as sp > p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > p0.communicate(input=b'abcd') > --------- > But nothing happens. The notepad is completely empty. What have I missed? > The "communicate" method sends text to the standard input pipe. This has nothing to do with the GUI, and most Windows GUI programs take no notice of it. You'll need something GUI-aware for this. Is Notepad just an example, or are you actually trying to control MS Notepad? ChrisA From jfong at ms4.hinet.net Wed Jul 3 21:25:14 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Wed, 3 Jul 2019 18:25:14 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> Message-ID: Chris Angelico? 2019?7?4???? UTC+8??8?37?13???? > On Thu, Jul 4, 2019 at 10:01 AM wrote: > > > > I have the test0.py below. I expect to see 'abcd' showing in the notepad window: > > --------- > > import subprocess as sp > > p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > > p0.communicate(input=b'abcd') > > --------- > > But nothing happens. The notepad is completely empty. What have I missed? > > > > The "communicate" method sends text to the standard input pipe. This > has nothing to do with the GUI, and most Windows GUI programs take no > notice of it. You'll need something GUI-aware for this. > > Is Notepad just an example, or are you actually trying to control MS Notepad? > > ChrisA Yes, the notepad is just an example. My real attempt is to operate the external programs through Python. I know there are some "keyboard simulation" packages in Pypi which may work on this situation. But I prefer not bother to install them if Python's build-ins can do it. By the way, after Popen invokes an external program, is there a way of making it on-foucs when there are multiple Popen instances? --Jach From aldwinaldwindev at gmail.com Wed Jul 3 21:20:27 2019 From: aldwinaldwindev at gmail.com (Aldwin Pollefeyt) Date: Thu, 4 Jul 2019 09:20:27 +0800 Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> Message-ID: This is same as echo abcd | notepad.exe in Command Prompt. You won't see the abcd neither. On Thu, Jul 4, 2019 at 8:41 AM Chris Angelico wrote: > On Thu, Jul 4, 2019 at 10:01 AM wrote: > > > > I have the test0.py below. I expect to see 'abcd' showing in the notepad > window: > > --------- > > import subprocess as sp > > p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > > p0.communicate(input=b'abcd') > > --------- > > But nothing happens. The notepad is completely empty. What have I missed? > > > > The "communicate" method sends text to the standard input pipe. This > has nothing to do with the GUI, and most Windows GUI programs take no > notice of it. You'll need something GUI-aware for this. > > Is Notepad just an example, or are you actually trying to control MS > Notepad? > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Wed Jul 3 21:57:21 2019 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 4 Jul 2019 11:57:21 +1000 Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> Message-ID: On Thu, Jul 4, 2019 at 11:31 AM wrote: > > Chris Angelico? 2019?7?4???? UTC+8??8?37?13???? > > On Thu, Jul 4, 2019 at 10:01 AM wrote: > > > > > > I have the test0.py below. I expect to see 'abcd' showing in the notepad window: > > > --------- > > > import subprocess as sp > > > p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > > > p0.communicate(input=b'abcd') > > > --------- > > > But nothing happens. The notepad is completely empty. What have I missed? > > > > > > > The "communicate" method sends text to the standard input pipe. This > > has nothing to do with the GUI, and most Windows GUI programs take no > > notice of it. You'll need something GUI-aware for this. > > > > Is Notepad just an example, or are you actually trying to control MS Notepad? > > > > ChrisA > > Yes, the notepad is just an example. My real attempt is to operate the external programs through Python. I know there are some "keyboard simulation" packages in Pypi which may work on this situation. But I prefer not bother to install them if Python's build-ins can do it. > > By the way, after Popen invokes an external program, is there a way of making it on-foucs when there are multiple Popen instances? > Definitely look into GUI manipulation tools. What you're doing is nothing to do with the subprocess module (at least, not with the way most Windows apps are built). ChrisA From best_lay at yahoo.com Wed Jul 3 23:15:41 2019 From: best_lay at yahoo.com (Wildman) Date: Wed, 03 Jul 2019 22:15:41 -0500 Subject: What's wrong on using Popen's communicate method? References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <20190704003636.GA31710@cskk.homeip.net> Message-ID: On Thu, 04 Jul 2019 10:36:36 +1000, Cameron Simpson wrote: > On 03Jul2019 16:57, Jach Fong wrote: >>I have the test0.py below. I expect to see 'abcd' showing in the notepad window: >>--------- >>import subprocess as sp >>p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) >>p0.communicate(input=b'abcd') >>--------- >>But nothing happens. The notepad is completely empty. What have I missed? >>--Jach >> >>PS. I am using python 3.4 on Windows Vista > > Well I am not a Windows person, but in most GUI environments a desktop > app such as Notepad does not read from its standard input - it reads > keyboard stuff though the GUI interface instead. So it is ignoring your > input data - this approach will generally fail with any desktop app on > any platform. > > You may need to investigate generating synthetic keystrokes somehow. > > Or alternatively write your data to a text file and hand the name of the > text file to notepad as a file to edit. > > Cheers, > Cameron Simpson I block googlegroups so I don't see the original post. I am talking to the OP. Take a look at the SendKeys module. https://pypi.org/project/SendKeys/ -- GNU/Linux user #557453 The cow died so I don't need your bull! From jfong at ms4.hinet.net Thu Jul 4 00:38:47 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Wed, 3 Jul 2019 21:38:47 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <20190704003636.GA31710@cskk.homeip.net> Message-ID: <9b5fa9f5-a2b7-49d9-a159-9cf03df94759@googlegroups.com> Wildman? 2019?7?4???? UTC+8??11?15?57???? > On Thu, 04 Jul 2019 10:36:36 +1000, Cameron Simpson wrote: > > > On 03Jul2019 16:57, Jach Fong wrote: > >>I have the test0.py below. I expect to see 'abcd' showing in the notepad window: > >>--------- > >>import subprocess as sp > >>p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > >>p0.communicate(input=b'abcd') > >>--------- > >>But nothing happens. The notepad is completely empty. What have I missed? > >>--Jach > >> > >>PS. I am using python 3.4 on Windows Vista > > > > Well I am not a Windows person, but in most GUI environments a desktop > > app such as Notepad does not read from its standard input - it reads > > keyboard stuff though the GUI interface instead. So it is ignoring your > > input data - this approach will generally fail with any desktop app on > > any platform. > > > > You may need to investigate generating synthetic keystrokes somehow. > > > > Or alternatively write your data to a text file and hand the name of the > > text file to notepad as a file to edit. > > > > Cheers, > > Cameron Simpson > > I block googlegroups so I don't see the original post. I am > talking to the OP. > > Take a look at the SendKeys module. > > https://pypi.org/project/SendKeys/ > > -- > GNU/Linux user #557453 > The cow died so I don't need your bull! Thank you. But the download there seems is for Python 2.7? From tjreedy at udel.edu Thu Jul 4 12:12:57 2019 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 4 Jul 2019 12:12:57 -0400 Subject: What's wrong on using Popen's communicate method? In-Reply-To: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> Message-ID: On 7/3/2019 7:57 PM, jfong at ms4.hinet.net wrote: > I have the test0.py below. I expect to see 'abcd' showing in the notepad window: > --------- > import subprocess as sp > p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > p0.communicate(input=b'abcd') > --------- > But nothing happens. The notepad is completely empty. What have I missed? > > --Jach > > PS. I am using python 3.4 on Windows Vista Upgrade to 3.7 or 3.8 to get hundreds of bug fixes, let alone new features. Both subprocess and multiprocessing have gotten fixes. subprocess is meant for running an external program in batch mode. It receives the one input byte string, sends output, and closes. For interaction, try multiprocessing. Or do what IDLE does, which is to open a two-way socket to the parent program. (Managing this is not fun.) IDLE was initially written before multiprocessing. It has been suggested to me that it should switch to multiprocessing. (But last I read, multiprocessing and tkinter (tcl/tk) do not play well together on macOS.) If the subprocess runs a gui, the user should be able to switch focus by clicking on a subprocess window. -- Terry Jan Reedy From hjp-python at hjp.at Thu Jul 4 13:00:04 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Thu, 4 Jul 2019 19:00:04 +0200 Subject: Handle foreign character web input In-Reply-To: References: <8dea3921-63db-df9f-5482-b6732c4344ee@tjol.eu> Message-ID: <20190704170004.tijbpx35cc2q7qm6@hjp.at> On 2019-07-03 18:13:20 -0500, Igor Korot wrote: > But as I said, imaging the following situation: > > You are somewhere in Germany and you have a German version of OS > (any OS) > . > You also have a German keyboard (hardware) with German keys. > > Now you are assigned to go to some international events where people > all over the world will be coming to your presentation and they will be > registering on you machine > > Also imagine that the company policy prohibits you from modifying the > system settings. > > My solution: > I would probably grab a lot of registering paper and ask people to enter > English transliteration of the names on the machine so when you come > back to the office you can properly enter their names using all those > different keyboards (maybe virtual ones) to associate them with > their English counterparts. > > Just curious - what would you do? I would set up a web registration form, so people can use their own device (phone, laptop, whatever) to enter their name. Presumably they know how to enter their name on their device. I definitely don't know how to enter a hand-written (or even printed) Chinese name on any keyboard (I managed to do that recently, but that was a lot of work and way into "a fun challenge to do once, not something I want to repeat" territory). hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From tjol at tjol.eu Thu Jul 4 10:57:28 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 4 Jul 2019 16:57:28 +0200 Subject: Multiprocessing and memory management In-Reply-To: References: Message-ID: <72d6b28c-b935-4218-e8bf-4473e95db0b4@tjol.eu> On 03/07/2019 18.37, Israel Brewster wrote: > I have a script that benefits greatly from multiprocessing (it?s generating a bunch of images from data). Of course, as expected each process uses a chunk of memory, and the more processes there are, the more memory used. The amount used per process can vary from around 3 GB (yes, gigabytes) to over 40 or 50 GB, depending on the amount of data being processed (usually closer to 10GB, the 40/50 is fairly rare). This puts me in a position of needing to balance the number of processes with memory usage, such that I maximize resource utilization (running one process at a time would simply take WAY to long) while not overloading RAM (which at best would slow things down due to swap). > > Obviously this process will be run on a machine with lots of RAM, but as I don?t know how large the datasets that will be fed to it are, I wanted to see if I could build some intelligence into the program such that it doesn?t overload the memory. A couple of approaches I thought of: > > 1) Determine the total amount of RAM in the machine (how?), assume an average of 10GB per process, and only launch as many processes as calculated to fit. Easy, but would run the risk of under-utilizing the processing capabilities and taking longer to run if most of the processes were using significantly less than 10GB > > 2) Somehow monitor the memory usage of the various processes, and if one process needs a lot, pause the others until that one is complete. Of course, I?m not sure if this is even possible. > > 3) Other approaches? > Are you familiar with Dask? I don't know it myself other than through hearsay, but I have a feeling it may have a ready-to-go solution to your problem. You'd have to look into dask in more detail than I have... From lukasz at langa.pl Thu Jul 4 18:05:52 2019 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Fri, 5 Jul 2019 00:05:52 +0200 Subject: [RELEASE] Python 3.8.0b2 is now available for testing Message-ID: <75CB7BD4-8224-49B7-9F10-EAAA67259C86@langa.pl> After a few days of delay, but somewhat cutely timed with the US Independence Day, I present you Python 3.8.0b2: https://www.python.org/downloads/release/python-380b2/ This release is the second of four planned beta release previews. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. The next pre-release of Python 3.8 will be 3.8.0b3, currently scheduled for 2019-07-29. Call to action We strongly encourage maintainers of third-party Python projects to test with 3.8 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2019-09-30). Our goal is have no ABI changes after beta 3 and no code changes after 3.8.0rc1, the release candidate. To achieve that, it will be extremely important to get as much exposure for 3.8 as possible during the beta phase. Please keep in mind that this is a preview release and its use is not recommended for production environments. No more non-bugfixes allowed on the ?3.8? branch The time has come, team. Please help make Python 3.8 as stable as possible and keep all features not currently landed for Python 3.9. Don?t fret, it?ll come faster than you think. - ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From chema at rinzewind.org Fri Jul 5 10:11:16 2019 From: chema at rinzewind.org (=?UTF-8?Q?Jos=C3=A9_Mar=C3=ADa_Mateos?=) Date: Fri, 05 Jul 2019 10:11:16 -0400 Subject: Random signal capture when using multiprocessing Message-ID: <7cb3d2da-bb60-4c1a-ac57-081c09ade68f@www.fastmail.com> Hi, This is a minimal proof of concept for something that has been bugging me for a few days: ``` $ cat signal_multiprocessing_poc.py import random import multiprocessing import signal import time def signal_handler(signum, frame): raise Exception(f"Unexpected signal {signum}!") def process_message(args): time.sleep(random.random() / 100.) if __name__ == "__main__": signal.signal(signal.SIGTERM, signal_handler) n_round = 1 while n_round < 10: job_list = [x for x in range(random.randint(100, 400))] print(f"Running round {n_round} with {len(job_list)} jobs") with multiprocessing.Pool(8) as p1: p1.map(process_message, job_list) n_round += 1 ``` So basically I have some subprocesses that don't do anything, just sleep for a few milliseconds, and I capture SIGTERM signals. I don't expect This is the output: ``` $ python signal_multiprocessing_poc.py Running round 1 with 244 jobs Running round 2 with 151 jobs Running round 3 with 173 jobs Running round 4 with 124 jobs Running round 5 with 249 jobs Running round 6 with 359 jobs Process ForkPoolWorker-48: Traceback (most recent call last): File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/pool.py", line 110, in worker task = get() File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/queues.py", line 352, in get res = self._reader.recv_bytes() File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/synchronize.py", line 98, in __exit__ return self._semlock.__exit__(*args) File "signal_multiprocessing_poc.py", line 7, in signal_handler raise Exception(f"Unexpected signal {signum}!") Exception: Unexpected signal 15! Running round 7 with 185 jobs Running round 8 with 246 jobs Running round 9 with 217 jobs Process ForkPoolWorker-68: Traceback (most recent call last): File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/process.py", line 99, in run self._target(*self._args, **self._kwargs) File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/pool.py", line 110, in worker task = get() File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/queues.py", line 352, in get res = self._reader.recv_bytes() File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/synchronize.py", line 98, in __exit__ return self._semlock.__exit__(*args) File "signal_multiprocessing_poc.py", line 7, in signal_handler raise Exception(f"Unexpected signal {signum}!") Exception: Unexpected signal 15! ``` Can anyone help me understand why the random SIGTERM gets captured? I guess some of the children processes receive the signal for termination and that's why I see it, but I don't understand the randomness of it. Of course, as a coworker just pointed out, this works (telling the children processes to ignore the signal handler): ``` import random import multiprocessing import signal import time def signal_handler(signum, frame): raise Exception(f"Unexpected signal {signum}!") def init_worker(): signal.signal(signal.SIGTERM, signal.SIG_IGN) signal.signal(signal.SIGINT, signal.SIG_IGN) def process_message(args): time.sleep(random.random() / 100.) if __name__ == "__main__": signal.signal(signal.SIGTERM, signal_handler) signal.signal(signal.SIGINT, signal_handler) n_round = 1 while n_round < 20: job_list = [x for x in range(random.randint(100, 400))] print(f"Running round {n_round} with {len(job_list)} jobs") with multiprocessing.Pool(8,init_worker) as p1: p1.map(process_message, job_list) n_round += 1 ``` Thanks for your help, -- Jos? Mar?a (Chema) Mateos || https://rinzewind.org From tjol at tjol.eu Fri Jul 5 13:26:29 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 5 Jul 2019 19:26:29 +0200 Subject: Matplotlib import image as float32 In-Reply-To: <8142748a-9214-5eee-3974-c81a886f251b@c2o.pro.br> References: <721c6dce-46db-01f9-2759-699903aaa415@c2o.pro.br> <8142748a-9214-5eee-3974-c81a886f251b@c2o.pro.br> Message-ID: <568d17b3-aa67-dd39-e5af-113a12d89b85@tjol.eu> On 01/07/2019 21:08, Markos wrote: > Hi, > > I observed that matplotlib reads an image file (PNG) as float32: > > Please, how to read this file as int8 to get RGB in range of 0-255? You may want to try a different library. scikit-image's imread function will give you the image as an integer array. This uses the imageio module under the hood. OpenCV uses int8's for everything IIRC, but using that only makes sense if you also want its image processing functionality -- Thomas > > Thank you, > > Markos > >> import numpy as np > >> import matplotlib.pyplot as plt > >> import matplotlib.image as mpimg > >> imagem = mpimg.imread('lenna.png') > >> print (imagem) > > [[[0.8862745? 0.5372549? 0.49019608] > ? [0.8862745? 0.5372549? 0.49019608] > ? [0.8745098? 0.5372549? 0.52156866] > ? ... > >> print (imagem.dtype) > > float32 > From mal at europython.eu Fri Jul 5 15:40:02 2019 From: mal at europython.eu (M.-A. Lemburg) Date: Fri, 5 Jul 2019 21:40:02 +0200 Subject: EuroPython 2019: Attendee briefing Message-ID: EuroPython 2019 will start on Monday. Again we will have more than a thousand attendees signed up and we're looking forward to an exciting conference packed with more than 130 sessions from Monday - Sunday, July 8 - 14. Here's a summary of helpful information for all attendees: 1. Picking up your badge using your ticket ID 2. Conference App 3. Twitter handles 4. Social Event / Conference Dinner 5. Self-Organisation and Public Channels 6. Catering on Workshop and Training Days (Monday / Tuesday) 7. Drinking water ----------------------------------------------------------------------- 1. Picking up your badge using your ticket ID As in past year, we are again using the ticket ID for badge pickup. You can find your ticket ID in your account listed under https://ep2019.europython.eu/user-panel/ (on the left next to your tickets) Please have this ID at hand when picking up your badge. We are planning to open doors at 08:30 on Monday. Our regular registration desk hours are : 09:00 - 17:00 from Tuesday - Friday. ----------------------------------------------------------------------- 2. Conference App We have created a dedicated mobile app to support your conference experience. The mobile app gives you access to the conference schedule (even offline), helps you in planning your conference experience and provides a rich social engagement platform for all attendees. You can create a profile within the app (or link this to your existing social accounts), share messages and photos, and easily reach out to other fellow attendees. Please note: You need to open a new account inside the app, it's not connected to your EuroPython account. If you have had an account in previous year, the app should recognize this and you don't have to set it up again. Download the Attendify app here: https://ep2019.europython.eu/mobile-app/ search for "EuroPython 2019" and enter the code "europython2019" to access the app For more details have a look at: https://ep2019.europython.eu/events/conference-app/ ----------------------------------------------------------------------- 3. Social media handles Our official social media handle is: #EuroPython ----------------------------------------------------------------------- 4. Social Event We are organizing a EuroPython Social Event at the FHNW Muttenz: on Thursday, 19:00 - 22:30 We serve finger food and cold drinks. We'll also have Python crafted live music and a DJ for more conventional music. You need an extra ticket for this event, it's not included in the conference ticket. The program and tickets can be found here: https://ep2019.europython.eu/events/social-event/ ----------------------------------------------------------------------- 5. Self-Organisation and Public Channels We have a Telegram group attendees can chat with each other, it's open for everyone to join: https://telegram.me/europython We have set up a wiki page for attendees to self-organize and give other attendees helpful tips: https://wiki.python.org/moin/EuroPython2019 Help on how to use it: https://wiki.python.org/moin/FrontPage#use More info on our social channels. ----------------------------------------------------------------------- 6. Catering on Workshop and Training Days (Monday / Tuesday) Please note that catering on the Monday and Tuesday is only included for attendees who have a training or combined ticket. If you don?t have such a ticket, you can still attend the free sponsored trainings, the Beginners? Day and the many PewPew workshops (remember, each attendee will get a PewPew device for free) on Monday and Tuesday, but you will either have to pay in cash / with card at the caterer or arrange food/drinks from the grocery store at the ground floor. For coffee breaks you can go to the ground floor, to the 12th floor of the FHNW building, or outside at the beach bar (nice weather only) and buy drinks. More information: https://blog.europython.eu/post/185080400427/europython-2019-monday-and-tuesday-activities-for https://blog.europython.eu/post/185584014022/europython-2019-warning-spoiler-alert https://blog.europython.eu/post/185723885512/europython-2019-beginners-day-workshop ----------------------------------------------------------------------- 7. Drinking water At the FHNW, you can bring your own water, if you like, and fill up your bottle at the taps. Drinks are only available during the coffee breaks and lunch. At the CCB, we will have water dispensers throughout the day, so you can bring your bottle and fill it up as needed. If you have questions, please contact our helpdesk at europython.eu. Replies to this email will not be read. Dates and Venues ---------------- EuroPython will be held from July 8-14 2019 in Basel, Switzerland, at the Congress Center Basel (CCB) for the main conference days (Wed-Fri) and the FHNW Muttenz for the workshops/trainings/sprints days (Mon-Tue, Sat-Sun). The schedule is available at: https://ep2019.europython.eu/events/schedule/ Tickets can be purchased on our registration page: https://ep2019.europython.eu/registration/buy-tickets/ For more details, please have a look at our website and the FAQ: https://ep2019.europython.eu/faq Help spread the word -------------------- Please help us spread this message by sharing it on your social networks as widely as possible. Thank you ! Link to the blog post: https://blog.europython.eu/post/186076969427/europython-2019-attendee-briefing Tweet: https://twitter.com/europython/status/1147226144338579462 Enjoy, -- EuroPython 2019 Team https://ep2019.europython.eu/ https://www.europython-society.org/ From jfong at ms4.hinet.net Fri Jul 5 22:25:21 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Fri, 5 Jul 2019 19:25:21 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> Message-ID: <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Terry Reedy? 2019?7?5???? UTC+8??12?13?25???? > On 7/3/2019 7:57 PM, jfong at ms4.hinet.net wrote: > > I have the test0.py below. I expect to see 'abcd' showing in the notepad window: > > --------- > > import subprocess as sp > > p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > > p0.communicate(input=b'abcd') > > --------- > > But nothing happens. The notepad is completely empty. What have I missed? > > > > --Jach > > > > PS. I am using python 3.4 on Windows Vista > > Upgrade to 3.7 or 3.8 to get hundreds of bug fixes, let alone new > features. Both subprocess and multiprocessing have gotten fixes. I can't because my OS is Vista and v3.4 is the last it can run:-( Also the pywin32 can't be installed for it requires v3.5 and up. > subprocess is meant for running an external program in batch mode. It > receives the one input byte string, sends output, and closes. For > interaction, try multiprocessing. > > Or do what IDLE does, which is to open a two-way socket to the parent > program. (Managing this is not fun.) IDLE was initially written before > multiprocessing. It has been suggested to me that it should switch to > multiprocessing. (But last I read, multiprocessing and tkinter (tcl/tk) > do not play well together on macOS.) Can multiprocessing make debugging of eventloop app easier? > If the subprocess runs a gui, the user should be able to switch focus by > clicking on a subprocess window. I finally get a solution to switch between two notepads in Python. Here is the codes for someone he may be interested. ----------- import subprocess, time, ctypes user32 = ctypes.windll.user32 p0 = subprocess.Popen('notepad.exe') time.sleep(1) handle0 = user32.GetForegroundWindow() # the handler of foreground window thread0 = user32.GetWindowThreadProcessId(handle0, 0) # get its thread id p1 = subprocess.Popen('notepad.exe') time.sleep(1) handle1 = user32.GetForegroundWindow() thread1 = user32.GetWindowThreadProcessId(handle1, 0) for i in range(3): # suppose the notepad1 is the foreground window time.sleep(5) # delay enough time for typing test user32.AttachThreadInput(thread1, thread0, True) user32.SetForegroundWindow(handle0) time.sleep(5) user32.AttachThreadInput(thread0, thread1, True) user32.SetForegroundWindow(handle1) print('end') --Jach From rosuav at gmail.com Fri Jul 5 22:47:45 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Jul 2019 12:47:45 +1000 Subject: What's wrong on using Popen's communicate method? In-Reply-To: <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Message-ID: On Sat, Jul 6, 2019 at 12:31 PM wrote: > > Terry Reedy? 2019?7?5???? UTC+8??12?13?25???? > > On 7/3/2019 7:57 PM, jfong at ms4.hinet.net wrote: > > > I have the test0.py below. I expect to see 'abcd' showing in the notepad window: > > > --------- > > > import subprocess as sp > > > p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > > > p0.communicate(input=b'abcd') > > > --------- > > > But nothing happens. The notepad is completely empty. What have I missed? > > > > > > --Jach > > > > > > PS. I am using python 3.4 on Windows Vista > > > > Upgrade to 3.7 or 3.8 to get hundreds of bug fixes, let alone new > > features. Both subprocess and multiprocessing have gotten fixes. > > I can't because my OS is Vista and v3.4 is the last it can run:-( Also the pywin32 can't be installed for it requires v3.5 and up. > So... upgrade to a better operating system? ChrisA From jfong at ms4.hinet.net Fri Jul 5 23:03:08 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Fri, 5 Jul 2019 20:03:08 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Message-ID: <8607b724-6f87-4af7-9a2a-5fec8631ca3d@googlegroups.com> Chris Angelico? 2019?7?6???? UTC+8??10?48?10???? > On Sat, Jul 6, 2019 at 12:31 PM wrote: > > > > Terry Reedy? 2019?7?5???? UTC+8??12?13?25???? > > > On 7/3/2019 7:57 PM, jfong at ms4.hinet.net wrote: > > > > I have the test0.py below. I expect to see 'abcd' showing in the notepad window: > > > > --------- > > > > import subprocess as sp > > > > p0 = sp.Popen('notepad.exe', stdin=sp.PIPE) > > > > p0.communicate(input=b'abcd') > > > > --------- > > > > But nothing happens. The notepad is completely empty. What have I missed? > > > > > > > > --Jach > > > > > > > > PS. I am using python 3.4 on Windows Vista > > > > > > Upgrade to 3.7 or 3.8 to get hundreds of bug fixes, let alone new > > > features. Both subprocess and multiprocessing have gotten fixes. > > > > I can't because my OS is Vista and v3.4 is the last it can run:-( Also the pywin32 can't be installed for it requires v3.5 and up. > > > > So... upgrade to a better operating system? > > ChrisA From Vista to Win10? That's a three generation gap! Buy a new PC might be a better solution:-) --Jach From torriem at gmail.com Sat Jul 6 00:01:31 2019 From: torriem at gmail.com (Michael Torrie) Date: Fri, 5 Jul 2019 22:01:31 -0600 Subject: What's wrong on using Popen's communicate method? In-Reply-To: <8607b724-6f87-4af7-9a2a-5fec8631ca3d@googlegroups.com> References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> <8607b724-6f87-4af7-9a2a-5fec8631ca3d@googlegroups.com> Message-ID: <6ec37c6c-be11-7d84-891e-77046d6bbe10@gmail.com> On 07/05/2019 09:03 PM, jfong at ms4.hinet.net wrote: > From Vista to Win10? That's a three generation gap! Buy a new PC might be a better solution:-) Maybe. Windows 10 would probably run okay on that machine. It might be 10 years old, but if it's 64-bit, Win 10 should run on it. You could also buy a hard drive and download the ISO from Microsoft and give it a try. If it works you can buy the license for it (sigh). Of course there are also other good alternatives too. From rosuav at gmail.com Sat Jul 6 00:32:23 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Jul 2019 14:32:23 +1000 Subject: What's wrong on using Popen's communicate method? In-Reply-To: <6ec37c6c-be11-7d84-891e-77046d6bbe10@gmail.com> References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> <8607b724-6f87-4af7-9a2a-5fec8631ca3d@googlegroups.com> <6ec37c6c-be11-7d84-891e-77046d6bbe10@gmail.com> Message-ID: On Sat, Jul 6, 2019 at 2:03 PM Michael Torrie wrote: > > On 07/05/2019 09:03 PM, jfong at ms4.hinet.net wrote: > > From Vista to Win10? That's a three generation gap! Buy a new PC might be a better solution:-) > > Maybe. Windows 10 would probably run okay on that machine. It might be > 10 years old, but if it's 64-bit, Win 10 should run on it. > > You could also buy a hard drive and download the ISO from Microsoft and > give it a try. If it works you can buy the license for it (sigh). > > Of course there are also other good alternatives too. When I said "upgrade to a better operating system", I wasn't being specific. There are plenty of better OSes than Windows Vista; Win 10 is just one of them. Upgrading to the latest release of Debian GNU/Linux would almost certainly solve your problem, as there is plenty of support in any current Linux distro for managing other programs. Of course, every decision has consequences, so it's up to you to decide whether it's better to upgrade your existing hardware to a newer Windows, switch to Linux, switch to BSD, buy new hardware, or something else entirely. ChrisA From dieter at handshake.de Sat Jul 6 01:33:42 2019 From: dieter at handshake.de (dieter) Date: Sat, 06 Jul 2019 07:33:42 +0200 Subject: Random signal capture when using multiprocessing References: <7cb3d2da-bb60-4c1a-ac57-081c09ade68f@www.fastmail.com> Message-ID: <87r273iv61.fsf@handshake.de> Jos? Mar?a Mateos writes: > This is a minimal proof of concept for something that has been bugging me for a few days: > .... > So basically I have some subprocesses that don't do anything, just sleep for a few milliseconds, and I capture SIGTERM signals. I don't expect > ... > Running round 6 with 359 jobs > Process ForkPoolWorker-48: > Traceback (most recent call last): > ... > File "signal_multiprocessing_poc.py", line 7, in signal_handler > raise Exception(f"Unexpected signal {signum}!") > Exception: Unexpected signal 15! > ... "multiprocessing.util" may register an exit function which calls "terminate" which signals SIGTERM. There is also an "os.kill(..., SIGTERM)" in "multiprocessing.popen_fork". I would put some print at those places to determine if the SIGTERM comes from "multiprocessing". From eryksun at gmail.com Sat Jul 6 02:13:04 2019 From: eryksun at gmail.com (eryk sun) Date: Sat, 6 Jul 2019 01:13:04 -0500 Subject: What's wrong on using Popen's communicate method? In-Reply-To: <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Message-ID: On 7/5/19, jfong at ms4.hinet.net wrote: > Terry Reedy? 2019?7?5???? UTC+8??12?13?25???? > >> Upgrade to 3.7 or 3.8 to get hundreds of bug fixes, let alone new >> features. Both subprocess and multiprocessing have gotten fixes. > > I can't because my OS is Vista and v3.4 is the last it can run:-( Also > the pywin32 can't be installed for it requires v3.5 and up. Major versions of Python support Windows versions that have extended support from Microsoft at the time of the first release. For Vista, extended support ended on 2017-04-11. Python 3.6 was released on 2016-12-23, so it supports Vista. Python 3.7, on the other hand, does not because it was released on 2018-06-27. Windows 7 has extended support until 2020-01-14, so 3.8 supports it. However, system update KB2533623 is required for enhancements to the system loader that 3.8 depends on for loading extension modules and DLLs. > I finally get a solution to switch between two notepads in Python. Here > is the codes for someone he may be interested. If I have time I'll write a better version since the code you have is fragile. But I think you should take others' advice to use a cross-platform GUI automation library, so you won't be locked into a Windows-only solution. From rosuav at gmail.com Sat Jul 6 02:54:42 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 6 Jul 2019 16:54:42 +1000 Subject: Random signal capture when using multiprocessing In-Reply-To: <7cb3d2da-bb60-4c1a-ac57-081c09ade68f@www.fastmail.com> References: <7cb3d2da-bb60-4c1a-ac57-081c09ade68f@www.fastmail.com> Message-ID: On Sat, Jul 6, 2019 at 12:13 AM Jos? Mar?a Mateos wrote: > > Hi, > > This is a minimal proof of concept for something that has been bugging me for a few days: > > ``` > $ cat signal_multiprocessing_poc.py > > import random > import multiprocessing > import signal > import time > > def signal_handler(signum, frame): > raise Exception(f"Unexpected signal {signum}!") > > def process_message(args): > time.sleep(random.random() / 100.) > > if __name__ == "__main__": > signal.signal(signal.SIGTERM, signal_handler) > n_round = 1 > while n_round < 10: > job_list = [x for x in range(random.randint(100, 400))] > print(f"Running round {n_round} with {len(job_list)} jobs") > with multiprocessing.Pool(8) as p1: > p1.map(process_message, job_list) > n_round += 1 > > ``` > > So basically I have some subprocesses that don't do anything, just sleep for a few milliseconds, and I capture SIGTERM signals. I don't expect > > This is the output: > > ``` > $ python signal_multiprocessing_poc.py > Running round 1 with 244 jobs > Running round 2 with 151 jobs > Running round 3 with 173 jobs > Running round 4 with 124 jobs > Running round 5 with 249 jobs > Running round 6 with 359 jobs > Process ForkPoolWorker-48: > Traceback (most recent call last): > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap > self.run() > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/process.py", line 99, in run > self._target(*self._args, **self._kwargs) > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/pool.py", line 110, in worker > task = get() > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/queues.py", line 352, in get > res = self._reader.recv_bytes() > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/synchronize.py", line 98, in __exit__ > return self._semlock.__exit__(*args) > File "signal_multiprocessing_poc.py", line 7, in signal_handler > raise Exception(f"Unexpected signal {signum}!") > Exception: Unexpected signal 15! > Running round 7 with 185 jobs > Running round 8 with 246 jobs > Running round 9 with 217 jobs > Process ForkPoolWorker-68: > Traceback (most recent call last): > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap > self.run() > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/process.py", line 99, in run > self._target(*self._args, **self._kwargs) > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/pool.py", line 110, in worker > task = get() > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/queues.py", line 352, in get > res = self._reader.recv_bytes() > File "/home/j_mariamateos/miniconda3/lib/python3.7/multiprocessing/synchronize.py", line 98, in __exit__ > return self._semlock.__exit__(*args) > File "signal_multiprocessing_poc.py", line 7, in signal_handler > raise Exception(f"Unexpected signal {signum}!") > Exception: Unexpected signal 15! > ``` > > Can anyone help me understand why the random SIGTERM gets captured? I guess some of the children processes receive the signal for termination and that's why I see it, but I don't understand the randomness of it. > > Of course, as a coworker just pointed out, this works (telling the children processes to ignore the signal handler): > > ``` > import random > import multiprocessing > import signal > import time > > def signal_handler(signum, frame): > raise Exception(f"Unexpected signal {signum}!") > > def init_worker(): > signal.signal(signal.SIGTERM, signal.SIG_IGN) > signal.signal(signal.SIGINT, signal.SIG_IGN) > > def process_message(args): > time.sleep(random.random() / 100.) > > if __name__ == "__main__": > > signal.signal(signal.SIGTERM, signal_handler) > signal.signal(signal.SIGINT, signal_handler) > n_round = 1 > > while n_round < 20: > job_list = [x for x in range(random.randint(100, 400))] > print(f"Running round {n_round} with {len(job_list)} jobs") > with multiprocessing.Pool(8,init_worker) as p1: > p1.map(process_message, job_list) > n_round += 1 > ``` > In the interests of test consistency, I removed all the random number generation from the test, locking in some numbers instead: import multiprocessing import signal import time def signal_handler(signum, frame): raise Exception(f"Unexpected signal {signum}!") def process_message(args): time.sleep(0.005) if __name__ == "__main__": signal.signal(signal.SIGTERM, signal_handler) for n_round in range(1, 10): job_list = range(250) print(f"Running round {n_round} with {len(job_list)} jobs") with multiprocessing.Pool(8) as p1: p1.map(process_message, job_list) The SIGTERMs are still happening, and it seems to be somewhat unpredictable. I can have a couple of runs of the program with none, then two come up in the same run. Once, I actually got a chained exception. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "mpterm.py", line 16, in with multiprocessing.Pool(8) as p1: File "/usr/local/lib/python3.9/multiprocessing/context.py", line 119, in Pool return Pool(processes, initializer, initargs, maxtasksperchild, File "/usr/local/lib/python3.9/multiprocessing/pool.py", line 218, in __init__ p.join() File "/usr/local/lib/python3.9/multiprocessing/process.py", line 147, in join assert self._parent_pid == os.getpid(), 'can only join a child process' AssertionError: can only join a child process (Yes, that says python3.9 there, but I don't think this is version-dependent; the same thing happens on 3.8, 3.7, and 3.6. On 3.5, of course, f-strings aren't available.) But if I comment out the signal.signal line, there seem to be no ill effects. I suspect that what you're seeing here is the multiprocessing module managing its own subprocesses, telling some of them to shut down. I added a print call to multiprocessing/popen_fork.py inside _send_signal (line 53 or thereabouts depending on Python version) and saw a *lot* of termination signals being sent; only a few actually triggered the exception message. My guess is that most of the time, the SIGTERM is smoothly handled as part of the Pool's __exit__ method, but sometimes the child process is blocked on something, and has to be told to shut down; and then normally, the signal gets caught and handled just fine, but since you're explicitly hooking it, you get to see it. ChrisA From jfong at ms4.hinet.net Sat Jul 6 05:22:29 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sat, 6 Jul 2019 02:22:29 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Message-ID: eryk sun? 2019?7?6???? UTC+8??2?13?23???? > On 7/5/19, jfong at ms4.hinet.net wrote: > > Terry Reedy? 2019?7?5???? UTC+8??12?13?25???? > > > >> Upgrade to 3.7 or 3.8 to get hundreds of bug fixes, let alone new > >> features. Both subprocess and multiprocessing have gotten fixes. > > > > I can't because my OS is Vista and v3.4 is the last it can run:-( Also > > the pywin32 can't be installed for it requires v3.5 and up. > > Major versions of Python support Windows versions that have extended > support from Microsoft at the time of the first release. For Vista, > extended support ended on 2017-04-11. Python 3.6 was released on > 2016-12-23, so it supports Vista. Python 3.7, on the other hand, does > not because it was released on 2018-06-27. I tried the python-3.6.1.exe, downloaded from www.python.org. Its release date is March 21, 2017. But it doesn't run, Vista has no response at all. --Jach > Windows 7 has extended support until 2020-01-14, so 3.8 supports it. > However, system update KB2533623 is required for enhancements to the > system loader that 3.8 depends on for loading extension modules and > DLLs. > > > I finally get a solution to switch between two notepads in Python. Here > > is the codes for someone he may be interested. > > If I have time I'll write a better version since the code you have is > fragile. But I think you should take others' advice to use a > cross-platform GUI automation library, so you won't be locked into a > Windows-only solution. From eryksun at gmail.com Sat Jul 6 06:26:23 2019 From: eryksun at gmail.com (eryk sun) Date: Sat, 6 Jul 2019 05:26:23 -0500 Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Message-ID: On 7/6/19, jfong at ms4.hinet.net wrote: > eryk sun? 2019?7?6???? UTC+8??2?13?23???? > >> Major versions of Python support Windows versions that have extended >> support from Microsoft at the time of the first release. For Vista, >> extended support ended on 2017-04-11. Python 3.6 was released on >> 2016-12-23, so it supports Vista. Python 3.7, on the other hand, does >> not because it was released on 2018-06-27. > > I tried the python-3.6.1.exe, downloaded from www.python.org. Its > release date is March 21, 2017. But it doesn't run, Vista has no response at > all. Starting with version 3.5, Python requires the Universal C Runtime, which is an OS component that was introduced in Windows 10 and backported to previous versions, including Windows Vista (service pack 2). Python's installer should attempt to install the CRT update, but from what I've seen this isn't reliable. Ideally, it shouldn't be necessary to manually install the CRT update since Windows Update includes it. Enable optional updates, and keep your system updated. That said, if you can't or won't use Windows Update, then you can download and install standalone updates. CRT Update https://support.microsoft.com/en-us/help/3118401 The above requires Vista service pack 2 (SP2). The system properties dialog should say whether SP1 or SP2 is installed. Your system should be running SP2 if Windows Update is enabled. If you're not even running SP1, you'll have to install SP1 and then SP2. The system properties dialog will tell you whether you're running 32-bit or 64-bit Windows. x86 32-bit SP1 https://www.microsoft.com/en-us/download/details.aspx?id=30 x64 64-bit SP2 https://www.microsoft.com/en-us/download/details.aspx?id=21299 x86 32-bit SP2 https://www.microsoft.com/en-us/download/details.aspx?id=16468 x64 64-bit SP2 https://www.microsoft.com/en-us/download/details.aspx?id=17669 From chema at rinzewind.org Sat Jul 6 10:41:13 2019 From: chema at rinzewind.org (=?iso-8859-1?Q?Jos=E9_Mar=EDa?= Mateos) Date: Sat, 6 Jul 2019 10:41:13 -0400 Subject: Random signal capture when using multiprocessing In-Reply-To: References: <7cb3d2da-bb60-4c1a-ac57-081c09ade68f@www.fastmail.com> Message-ID: <20190706144113.GB3206@equipaje> On Sat, Jul 06, 2019 at 04:54:42PM +1000, Chris Angelico wrote: > But if I comment out the signal.signal line, there seem to be no ill > effects. I suspect that what you're seeing here is the multiprocessing > module managing its own subprocesses, telling some of them to shut > down. I added a print call to multiprocessing/popen_fork.py inside > _send_signal (line 53 or thereabouts depending on Python version) and > saw a *lot* of termination signals being sent; only a few actually > triggered the exception message. My guess is that most of the time, > the SIGTERM is smoothly handled as part of the Pool's __exit__ method, > but sometimes the child process is blocked on something, and has to > be told to shut down; and then normally, the signal gets caught and > handled just fine, but since you're explicitly hooking it, you get to > see it. Ok, that makes all the sense in the world, thanks for digging into this. -- Jos? Mar?a (Chema) Mateos || https://rinzewind.org/ From jfong at ms4.hinet.net Sat Jul 6 21:03:52 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sat, 6 Jul 2019 18:03:52 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> <8607b724-6f87-4af7-9a2a-5fec8631ca3d@googlegroups.com> <6ec37c6c-be11-7d84-891e-77046d6bbe10@gmail.com> Message-ID: <523583d6-58c3-475b-bfb9-bf9f1844ef29@googlegroups.com> Michael Torrie? 2019?7?6???? UTC+8??12?02?02???? > On 07/05/2019 09:03 PM, jfong at ms4.hinet.net wrote: > > From Vista to Win10? That's a three generation gap! Buy a new PC might be a better solution:-) > > Maybe. Windows 10 would probably run okay on that machine. It might be > 10 years old, but if it's 64-bit, Win 10 should run on it. > > You could also buy a hard drive and download the ISO from Microsoft and > give it a try. If it works you can buy the license for it (sigh). > > Of course there are also other good alternatives too. Thank you for your suggestion. This PC was purchased in year 2006 from Dell. The CPU is AMD Athlon 64 X2 4000+. I doubt Win10 will support such an old CPU:-) and the chipset driver might be a big problem too. --Jach From jfong at ms4.hinet.net Sat Jul 6 21:16:46 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sat, 6 Jul 2019 18:16:46 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Message-ID: eryk sun? 2019?7?6???? UTC+8??6?26?42???? > On 7/6/19, jfong at ms4.hinet.net wrote: > > eryk sun? 2019?7?6???? UTC+8??2?13?23???? > > > >> Major versions of Python support Windows versions that have extended > >> support from Microsoft at the time of the first release. For Vista, > >> extended support ended on 2017-04-11. Python 3.6 was released on > >> 2016-12-23, so it supports Vista. Python 3.7, on the other hand, does > >> not because it was released on 2018-06-27. > > > > I tried the python-3.6.1.exe, downloaded from www.python.org. Its > > release date is March 21, 2017. But it doesn't run, Vista has no response at > > all. > > Starting with version 3.5, Python requires the Universal C Runtime, > which is an OS component that was introduced in Windows 10 and > backported to previous versions, including Windows Vista (service pack > 2). Python's installer should attempt to install the CRT update, but > from what I've seen this isn't reliable. > > Ideally, it shouldn't be necessary to manually install the CRT update > since Windows Update includes it. Enable optional updates, and keep > your system updated. That said, if you can't or won't use Windows > Update, then you can download and install standalone updates. > > CRT Update > https://support.microsoft.com/en-us/help/3118401 > > The above requires Vista service pack 2 (SP2). The system properties > dialog should say whether SP1 or SP2 is installed. Your system should > be running SP2 if Windows Update is enabled. If you're not even > running SP1, you'll have to install SP1 and then SP2. The system > properties dialog will tell you whether you're running 32-bit or > 64-bit Windows. > > x86 32-bit SP1 > https://www.microsoft.com/en-us/download/details.aspx?id=30 > x64 64-bit SP2 > https://www.microsoft.com/en-us/download/details.aspx?id=21299 > > x86 32-bit SP2 > https://www.microsoft.com/en-us/download/details.aspx?id=16468 > x64 64-bit SP2 > https://www.microsoft.com/en-us/download/details.aspx?id=17669 According to the info you give, I have done the following steps: 1. Upgrade to SP2 first. 2. Install the CRT update. But the result is still the same, nothing happens after double click the file python-3.6.1.exe. I deeply appreciate your help. It looks like that Vista has to live with Python 3.4 for its rest of life:-) --Jach From eryksun at gmail.com Sat Jul 6 22:13:18 2019 From: eryksun at gmail.com (eryk sun) Date: Sat, 6 Jul 2019 21:13:18 -0500 Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Message-ID: On 7/6/19, jfong at ms4.hinet.net wrote: > > According to the info you give, I have done the following steps: > 1. Upgrade to SP2 first. > 2. Install the CRT update. > But the result is still the same, nothing happens after double click the > file python-3.6.1.exe. Did you reboot after installing the updates? Afterward, did you confirm in the system properties that you're running SP2? Did you check that "%SystemRoot%\System32\ucrtbase.dll" exists, plus a bunch of "api-ms-win-crt-*.dll" DLLs in the same directory? > I deeply appreciate your help. It looks like that Vista has to live with > Python 3.4 for its rest of life:-) That is not true in general for Vista. It's particular to your computer. Check the installer logs in your %TEMP% directory for errors, which will typically be an HRESULT error code that starts with 0x8, such as 0x80070002. Try downloading and installing the 64-bit version: https://www.python.org/ftp/python/3.6.8/python-3.6.8-amd64.exe From jfong at ms4.hinet.net Sat Jul 6 22:57:19 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sat, 6 Jul 2019 19:57:19 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> Message-ID: <4b0e5521-a8a7-4386-9265-73e40bb4612e@googlegroups.com> eryk sun? 2019?7?7???? UTC+8??10?13?34???? > On 7/6/19, jfong at ms4.hinet.net wrote: > > > > According to the info you give, I have done the following steps: > > 1. Upgrade to SP2 first. > > 2. Install the CRT update. > > But the result is still the same, nothing happens after double click the > > file python-3.6.1.exe. > > Did you reboot after installing the updates? Afterward, did you > confirm in the system properties that you're running SP2? Did you > check that "%SystemRoot%\System32\ucrtbase.dll" exists, plus a bunch > of "api-ms-win-crt-*.dll" DLLs in the same directory? > > > I deeply appreciate your help. It looks like that Vista has to live with > > Python 3.4 for its rest of life:-) > > That is not true in general for Vista. It's particular to your > computer. Check the installer logs in your %TEMP% directory for > errors, which will typically be an HRESULT error code that starts with > 0x8, such as 0x80070002. > > Try downloading and installing the 64-bit version: > > https://www.python.org/ftp/python/3.6.8/python-3.6.8-amd64.exe Yes, it's SP2 showing up in Start->Computer->Properities window. Also in System32 directory, there is ucrtbase.dll and a bunch of api-ms-win-crt-*.dll too. In TEMP, there is only one file "chrome_installer.log" dated at the time of doing the upgrade (strange?, why chrome?). No errors in it. I can't. My Vista is 32bit. Anyway I will give v3.6.8 a try later. --Jach From eryksun at gmail.com Sun Jul 7 00:00:30 2019 From: eryksun at gmail.com (eryk sun) Date: Sat, 6 Jul 2019 23:00:30 -0500 Subject: What's wrong on using Popen's communicate method? In-Reply-To: <4b0e5521-a8a7-4386-9265-73e40bb4612e@googlegroups.com> References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> <4b0e5521-a8a7-4386-9265-73e40bb4612e@googlegroups.com> Message-ID: On 7/6/19, jfong at ms4.hinet.net wrote: > > In TEMP, there is only one file "chrome_installer.log" dated at the time of > doing the upgrade (strange?, why chrome?). No errors in it. So the installer doesn't even execute. Do you maybe have an anti-malware program that's blocking its execution? Definitely try the 3.6.8 installer. I didn't pay attention to the fact that you were trying to install a very outdated version of 3.6. There may have been bugs in the installer that were fixed. Always use the most recent version possible. From jfong at ms4.hinet.net Sun Jul 7 00:56:41 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sat, 6 Jul 2019 21:56:41 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> <4b0e5521-a8a7-4386-9265-73e40bb4612e@googlegroups.com> Message-ID: eryk sun? 2019?7?7???? UTC+8??12?00?47???? > On 7/6/19, jfong at ms4.hinet.net wrote: > > > > In TEMP, there is only one file "chrome_installer.log" dated at the time of > > doing the upgrade (strange?, why chrome?). No errors in it. > > So the installer doesn't even execute. Do you maybe have an > anti-malware program that's blocking its execution? Definitely try > the 3.6.8 installer. I didn't pay attention to the fact that you were > trying to install a very outdated version of 3.6. There may have been > bugs in the installer that were fixed. Always use the most recent > version possible. I turn off the anti-virus and tried v3.6.8, get the same result. Usually the anti-virus program will warn me if something was blocked. --Jach From eryksun at gmail.com Sun Jul 7 03:51:04 2019 From: eryksun at gmail.com (eryk sun) Date: Sun, 7 Jul 2019 02:51:04 -0500 Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> <4b0e5521-a8a7-4386-9265-73e40bb4612e@googlegroups.com> Message-ID: On 7/6/19, jfong at ms4.hinet.net wrote: > > I turn off the anti-virus and tried v3.6.8, get the same result. Usually the > anti-virus program will warn me if something was blocked. In case the executable is corrupt, clear your browser cache and download the 32-bit version again: https://www.python.org/ftp/python/3.6.8/python-3.6.8.exe Try to run the installer from an elevated command prompt. If it fails to run, check again for log files in your %TEMP% directory, which typically expands to "C:\Users\\AppData\Local\Temp". If the installer is still not running -- not even enough to create a single log file -- then apparently something is configured incorrectly or missing on your computer that's causing it to fail hard. I'd have to run it under a debugger to find the cause. Anyway, if you're stuck with ancient 3.4, you can at least install "pywin32-221.win32-py3.4.exe" from the old SourceForge repo: https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221 From jfong at ms4.hinet.net Sun Jul 7 04:52:54 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sun, 7 Jul 2019 01:52:54 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> <4b0e5521-a8a7-4386-9265-73e40bb4612e@googlegroups.com> Message-ID: eryk sun? 2019?7?7???? UTC+8??3?51?57???? > On 7/6/19, jfong at ms4.hinet.net wrote: > > > > I turn off the anti-virus and tried v3.6.8, get the same result. Usually the > > anti-virus program will warn me if something was blocked. > > In case the executable is corrupt, clear your browser cache and > download the 32-bit version again: I have its MD5 checked after each download. > https://www.python.org/ftp/python/3.6.8/python-3.6.8.exe > > Try to run the installer from an elevated command prompt. If it fails > to run, check again for log files in your %TEMP% directory, which > typically expands to "C:\Users\\AppData\Local\Temp". > > If the installer is still not running -- not even enough to create a > single log file -- then apparently something is configured incorrectly > or missing on your computer that's causing it to fail hard. I'd have > to run it under a debugger to find the cause. It's very possible there is some problem in my system. A few months ago suddenly I can't login to my email account through ThunderBird. Now I have to remove my password from its settings and manually type-in every time when I log in. Using a debugger is a big burden to me because I am not familiar with the internal of Windows well. > Anyway, if you're stuck with ancient 3.4, you can at least install > "pywin32-221.win32-py3.4.exe" from the old SourceForge repo: > > https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221 Thank you for the link. I will try it later and hope it will run:-) --Jach From jaymoyer44 at gmail.com Sun Jul 7 06:40:38 2019 From: jaymoyer44 at gmail.com (jaymoyer44 at gmail.com) Date: Sun, 7 Jul 2019 03:40:38 -0700 (PDT) Subject: how to setup for localhost:8000 In-Reply-To: <5ea1e31b-09ba-4b6a-be8b-58886c5c5f81@googlegroups.com> References: <5ea1e31b-09ba-4b6a-be8b-58886c5c5f81@googlegroups.com> Message-ID: <723d1172-c400-41a3-9ac5-668bc49ace1c@googlegroups.com> On Thursday, April 14, 2016 at 1:47:03 PM UTC-4, wrh... at gmail.com wrote: > Hi, > > I am working on window 7 and Python 3.5 to setup a localhost:8000 but it did not get through as shown below: > > python -m http.server > Serving HTTP on 0.0.0.0 port 8000 ... > > But it did not show the results. > > Can someone help me how to setup the localhost? > > Thanks, > Wen-Ruey From jaymoyer44 at gmail.com Sun Jul 7 06:41:37 2019 From: jaymoyer44 at gmail.com (jaymoyer44 at gmail.com) Date: Sun, 7 Jul 2019 03:41:37 -0700 (PDT) Subject: how to setup for localhost:8000 In-Reply-To: <5ea1e31b-09ba-4b6a-be8b-58886c5c5f81@googlegroups.com> References: <5ea1e31b-09ba-4b6a-be8b-58886c5c5f81@googlegroups.com> Message-ID: <70ed80b1-7ce9-4f9b-b844-e4876dd14637@googlegroups.com> On Thursday, April 14, 2016 at 1:47:03 PM UTC-4, wrh... at gmail.com wrote: > Hi, > > I am working on window 7 and Python 3.5 to setup a localhost:8000 but it did not get through as shown below: > > python -m http.server > Serving HTTP on 0.0.0.0 port 8000 ... > > But it did not show the results. > > Can someone help me how to setup the localhost? > > Thanks, > Wen-Ruey On Thursday, April 14, 2016 at 1:47:03 PM UTC-4, wrh... at gmail.com wrote: > Hi, > > I am working on window 7 and Python 3.5 to setup a localhost:8000 but it did not get through as shown below: > > python -m http.server > Serving HTTP on 0.0.0.0 port 8000 ... > > But it did not show the results. > > Can someone help me how to setup the localhost? > > Thanks, > Wen-Ruey From jfong at ms4.hinet.net Sun Jul 7 07:44:59 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Sun, 7 Jul 2019 04:44:59 -0700 (PDT) Subject: What's wrong on using Popen's communicate method? In-Reply-To: References: <2051e9c4-2d42-44a4-8bd7-6aadb9978d3b@googlegroups.com> <96b74a7c-a3d5-4950-afdf-b88e3e89e35c@googlegroups.com> <4b0e5521-a8a7-4386-9265-73e40bb4612e@googlegroups.com> Message-ID: <6d60c069-ee6d-41fe-aee1-6f81cc583ee3@googlegroups.com> eryk sun? 2019?7?7???? UTC+8??3?51?57???? > On 7/6/19, jfong at ms4.hinet.net wrote: > > > > I turn off the anti-virus and tried v3.6.8, get the same result. Usually the > > anti-virus program will warn me if something was blocked. > > In case the executable is corrupt, clear your browser cache and > download the 32-bit version again: > > https://www.python.org/ftp/python/3.6.8/python-3.6.8.exe > > Try to run the installer from an elevated command prompt. If it fails > to run, check again for log files in your %TEMP% directory, which > typically expands to "C:\Users\\AppData\Local\Temp". > > If the installer is still not running -- not even enough to create a > single log file -- then apparently something is configured incorrectly > or missing on your computer that's causing it to fail hard. I'd have > to run it under a debugger to find the cause. > > Anyway, if you're stuck with ancient 3.4, you can at least install > "pywin32-221.win32-py3.4.exe" from the old SourceForge repo: > > https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221 I had overlooked your suggestion of installing it under elevated command prompt. At the last moment I was noticed and gave it a try. Guess what? I run Python-3.6.8.exe successfully and installed it. A real surprise! Thank you for everything you had done to help. --Jach From nabru at nabru.io Sun Jul 7 12:57:12 2019 From: nabru at nabru.io (nabru) Date: Sun, 7 Jul 2019 16:57:12 +0000 (UTC) Subject: how to setup for localhost:8000 References: <5ea1e31b-09ba-4b6a-be8b-58886c5c5f81@googlegroups.com> <723d1172-c400-41a3-9ac5-668bc49ace1c@googlegroups.com> Message-ID: jaymoyer44 [jaymoyer44 at gmail.com] wrote: > On Thursday, April 14, 2016 at 1:47:03 PM UTC-4, wrh... at gmail.com wrote: >> Hi, >> >> I am working on window 7 and Python 3.5 to setup a localhost:8000 but >> it did not get through as shown below: >> > python -m http.server >> Serving HTTP on 0.0.0.0 port 8000 ... >> >> But it did not show the results. >> >> Can someone help me how to setup the localhost? >> >> Thanks, >> Wen-Ruey If it says 'serving HTTP...' try visiting localhost:8000 on your browser to see what happens. -- ~nabru From alister.ware at ntlworld.com Sun Jul 7 13:27:58 2019 From: alister.ware at ntlworld.com (Alister) Date: Sun, 07 Jul 2019 17:27:58 GMT Subject: how to setup for localhost:8000 References: <5ea1e31b-09ba-4b6a-be8b-58886c5c5f81@googlegroups.com> <723d1172-c400-41a3-9ac5-668bc49ace1c@googlegroups.com> Message-ID: On Sun, 07 Jul 2019 16:57:12 +0000, nabru wrote: > jaymoyer44 [jaymoyer44 at gmail.com] wrote: > >> On Thursday, April 14, 2016 at 1:47:03 PM UTC-4, wrh... at gmail.com >> wrote: >>> Hi, >>> >>> I am working on window 7 and Python 3.5 to setup a localhost:8000 but >>> it did not get through as shown below: >>> > python -m http.server >>> Serving HTTP on 0.0.0.0 port 8000 ... >>> >>> But it did not show the results. >>> >>> Can someone help me how to setup the localhost? >>> >>> Thanks, >>> Wen-Ruey > > If it says 'serving HTTP...' try visiting localhost:8000 on your browser > to see what happens. failing that try showing your code so that we do not have to be psychic! -- Nirvana? That's the place where the powers that be and their friends hang out. -- Zonker Harris From arj.python at gmail.com Sun Jul 7 14:28:50 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 7 Jul 2019 22:28:50 +0400 Subject: Images on Dynamically Generated Buttons Glitch Message-ID: Greetings, i have this snippet: --- import json import os from tkinter import * from tkinter.ttk import * root = Tk() jload = json.load buttons = list() programs = os.listdir('programs') for i, program in enumerate(programs): jsonpath = 'programs/{}/cmdlaunch.json'.format(program) info = jload(open(jsonpath)) photo = PhotoImage(file = 'icons/'+info['icon']) photoimage = photo.subsample(3, 3) buttons.append(Button(root, text = 'Click Me !', image = photoimage, compound = LEFT)) buttons[-1].pack() root.mainloop() --- All buttons are appearing but only the last button is displaying the image, any idea? -- Abdur-Rahmaan Janhangeer http://www.pythonmembers.club Mauritius From python at mrabarnett.plus.com Sun Jul 7 15:10:28 2019 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 7 Jul 2019 20:10:28 +0100 Subject: Images on Dynamically Generated Buttons Glitch In-Reply-To: References: Message-ID: <3b59e56d-d8a8-1947-2cfe-2a8dc8890260@mrabarnett.plus.com> On 2019-07-07 19:28, Abdur-Rahmaan Janhangeer wrote: > Greetings, i have this snippet: > --- > import json > import os > from tkinter import * > from tkinter.ttk import * > > root = Tk() > jload = json.load > buttons = list() > programs = os.listdir('programs') > for i, program in enumerate(programs): > jsonpath = 'programs/{}/cmdlaunch.json'.format(program) > info = jload(open(jsonpath)) > photo = PhotoImage(file = 'icons/'+info['icon']) > photoimage = photo.subsample(3, 3) > buttons.append(Button(root, text = 'Click Me !', image = photoimage, > compound = LEFT)) > buttons[-1].pack() > > root.mainloop() > --- > > All buttons are appearing but only the last button is displaying the image, > any idea? > You need to keep a reference to the image that you passed when creating the button; tkinter itself doesn't keep a reference to it: import json import os from tkinter import * from tkinter.ttk import * root = Tk() jload = json.load buttons = list() images = [] programs = os.listdir('programs') for i, program in enumerate(programs): jsonpath = 'programs/{}/cmdlaunch.json'.format(program) info = jload(open(jsonpath)) photo = PhotoImage(file = 'icons/'+info['icon']) photoimage = photo.subsample(3, 3) buttons.append(Button(root, text = 'Click Me !', image = photoimage, compound = LEFT)) buttons[-1].pack() images.append(photoimage) root.mainloop() In your code, 'photoimage' still refers to the last image, but there are no references to the previous images, so they're discarded. (Why is tkinter like that? Who knows! :-)) From arj.python at gmail.com Sun Jul 7 22:05:34 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 8 Jul 2019 06:05:34 +0400 Subject: Images on Dynamically Generated Buttons Glitch In-Reply-To: <3b59e56d-d8a8-1947-2cfe-2a8dc8890260@mrabarnett.plus.com> References: <3b59e56d-d8a8-1947-2cfe-2a8dc8890260@mrabarnett.plus.com> Message-ID: You mean we should create the image variables aside like one for each button? XD. Thanks! Abdur-Rahmaan Janhangeer Mauritius From python at mrabarnett.plus.com Sun Jul 7 23:03:14 2019 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 8 Jul 2019 04:03:14 +0100 Subject: Images on Dynamically Generated Buttons Glitch In-Reply-To: References: <3b59e56d-d8a8-1947-2cfe-2a8dc8890260@mrabarnett.plus.com> Message-ID: <9a225716-bb7b-3c76-df39-6cb45f8d23a2@mrabarnett.plus.com> On 2019-07-08 03:05, Abdur-Rahmaan Janhangeer wrote: > You mean we should create the image variables aside like one for each > button? XD. Thanks! > You need to ensure that there's a reference somewhere to every image that you're displaying, whether it's on a button or some other widget, because tkinter itself doesn't keep a reference to the image. Sometimes the most convenient way is to put them in a list, but just pick what makes the most sense for your use case. From arj.python at gmail.com Sun Jul 7 23:52:27 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 8 Jul 2019 07:52:27 +0400 Subject: Images on Dynamically Generated Buttons Glitch In-Reply-To: <9a225716-bb7b-3c76-df39-6cb45f8d23a2@mrabarnett.plus.com> References: <3b59e56d-d8a8-1947-2cfe-2a8dc8890260@mrabarnett.plus.com> <9a225716-bb7b-3c76-df39-6cb45f8d23a2@mrabarnett.plus.com> Message-ID: Did it but that's ?=?$/?*#?4?53@ Abdur-Rahmaan Janhangeer Mauritius From arj.python at gmail.com Mon Jul 8 01:59:16 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 8 Jul 2019 09:59:16 +0400 Subject: Tkinter Button Command Taking Only Newest Reference In Callback Parameter Message-ID: Greetings, I'm doing this project: https://github.com/Abdur-rahmaanJ/cmdlaunch/blob/master/cmdlaunch/cmdlaunch.py Everything works fine, except for one thing. On line 60, the button is taking an object in it's command parameter If you run the file, you'll see that all the different names are written on the gui just the callback is takest the newest icon object for all three buttons I know just a reference issue but it's been **bugging** me. Thanks All -- Abdur-Rahmaan Janhangeer Mauritius From cspealma at redhat.com Mon Jul 8 09:40:09 2019 From: cspealma at redhat.com (Calvin Spealman) Date: Mon, 8 Jul 2019 09:40:09 -0400 Subject: Tkinter Button Command Taking Only Newest Reference In Callback Parameter In-Reply-To: References: Message-ID: This is a really common scoping issue to get tripped up on. I'll try to help you understand and work around it. for i, icon in enumerate(self.icons): print('// entering', icon.program) print('icons ... ... ... ...') pretty(icon.info, icon.program) x = Button(root, text=icon.info['name'] +'\n' + icon.info['version'], image=icon.photo, compound=LEFT, command=lambda: self.button_exec(icon)) On the first line you enter a for loop where each iteration assigns a different pair of values to the names "i" and "icon". On the last line you create a lambda, an anonymous function, which you pass as the command callback for the button. Of course, no code inside the lambda is actually executed at this point. It is only after the entire for loop has completed, later when the user sees the buttons and clicks on one of them, that one of the lambda callbacks for the appropriate button will be executed. Now, when this happens, what is the value of "icon"? This involves a closure, which is how we describe the effect of the lambda (or any function) being created inside the scope of another function. The scope here which contains the "icon" variable is kept after the for loop and its container function is complete, because the lambda you create holds on to it in order to resolve variable names from. So all the lambda functions read the "icon" variable from this same scope, and at the time the loop completes that variable is assigned to the LAST value from the loop. That's why all the buttons get the same icon. There is a simple work around, however, called "rebinding". It is a trick you can do with default values for lambda or any function. Simply pass the icon as a default value for a parameter to the lambda, and the value at that time for one iteration of the loop will be bound to the callback you create. Just change this lambda: self.button_exec(icon) to this lambda icon=icon: self.button_exec(icon) I hope both the explanation and the example help! On Mon, Jul 8, 2019 at 1:59 AM Abdur-Rahmaan Janhangeer < arj.python at gmail.com> wrote: > Greetings, > > I'm doing this project: > > > https://github.com/Abdur-rahmaanJ/cmdlaunch/blob/master/cmdlaunch/cmdlaunch.py > > Everything works fine, except for one thing. > > On line 60, the button is taking an object in it's command parameter > > If you run the file, you'll see that all the different names are written on > the gui just the callback is takest the newest icon object for all three > buttons > > I know just a reference issue but it's been **bugging** me. > > Thanks All > > -- > Abdur-Rahmaan Janhangeer > Mauritius > -- > https://mail.python.org/mailman/listinfo/python-list > -- CALVIN SPEALMAN SENIOR QUALITY ENGINEER cspealma at redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] TRIED. TESTED. TRUSTED. From jmariano65 at gmail.com Mon Jul 8 12:01:12 2019 From: jmariano65 at gmail.com (=?UTF-8?Q?jos=C3=A9_mariano?=) Date: Mon, 8 Jul 2019 09:01:12 -0700 (PDT) Subject: Do I need a parser? In-Reply-To: <5e122997-3e60-4cff-9e5d-6b69601d3805@googlegroups.com> References: <5e122997-3e60-4cff-9e5d-6b69601d3805@googlegroups.com> Message-ID: <9b0bec80-47f0-435e-b08d-5bd40a984ae5@googlegroups.com> On Saturday, 29 June 2019 13:40:06 UTC+1, jos? mariano wrote: > Dear all, > > I'm sure that this subject has been addressed many times before on this forum, but my poor knowledge of English and of computer jargon and concepts results on not being able to find the answer i'm looking for when I search the forum. > > So here is my problem: I have this open source project for the scientific community were i want to duplicate an old MS-DOS application written in Fortran. I don't have the source code. The idea is to re-write the software in Python. Originally, the old application would would need to input files: one config file, written with a specific format (see below) and a second one, the so-called scrip file, that defines the sequence of operations to be performed by the main software, also written in a specific format. > > To make the transition to the new application as painless as possible to the users, because most of them have their collection of scrips (and settings) developed over the years and are not willing to learn a new script language, I would like to make the new app 100% compatible with the old input files. > > The operation of the new software would be like this: From the shell, run "my_new_software old_script_file.***". The new software would load the old_script, parse it (?), set the internal variables, load the script and run it. > > So, to get to my questions: > > - To load and read the config file I need a parser, right? Is their a parser library where we can define the syntax of the language to use? Are there better (meaning easier) ways to accomplish the same result? > > - For the interpretation of the script file, I don't have any clue how to this... One important thing, the script language admits some simple control flow statements like do-wile, again written using a specific sintax. > > Thanks a lot for the help and sorry for the long post. > > Mariano > > > > Example of a config (settings) file > ======================== > ..... > CONDAD -11 > BURAD2 4 SALT1 1.0 KNO3 > ELEC5 -2.0 mV 400 58 0. 0 > ..... > > > Example of a script > =================== > !Conductivity titration > cmnd bur1 f > set vinit 100 > set endpt 2000 > set mvinc 20 > set drftim 1 > set rdcrit cond 0.5 per_min > set dosinc bur1 0.02 1000 > set titdir up > titratc cond bur1 Hello All, I have a pretty good idea of what route I should take. Thank you very much for all your valuable input. Regards, Mariano From arj.python at gmail.com Mon Jul 8 02:00:04 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 8 Jul 2019 10:00:04 +0400 Subject: Tkinter Button Command Taking Only Newest Reference In Callback Parameter In-Reply-To: References: Message-ID: excuses for typos! From arj.python at gmail.com Mon Jul 8 15:44:33 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Mon, 8 Jul 2019 23:44:33 +0400 Subject: Tkinter Button Command Taking Only Newest Reference In Callback Parameter In-Reply-To: References: Message-ID: "Thanks" * INFINITY hum weirdly it worked ... not yet understood why! Here is a post on it: https://www.linkedin.com/feed/update/urn:li:activity:6554081428492161024/ Thanks for all! On Mon, Jul 8, 2019 at 5:40 PM Calvin Spealman wrote: > This is a really common scoping issue to get tripped up on. I'll try to > help you understand and work around it. > > for i, icon in enumerate(self.icons): > print('// entering', icon.program) > print('icons ... ... ... ...') > pretty(icon.info, icon.program) > x = Button(root, text=icon.info['name'] +'\n' + icon.info['version'], > image=icon.photo, > compound=LEFT, > command=lambda: self.button_exec(icon)) > > On the first line you enter a for loop where each iteration assigns a > different pair of values to the names "i" and "icon". > On the last line you create a lambda, an anonymous function, which you > pass as the command callback for the button. Of course, no code inside the > lambda is actually executed at this point. It is only after the entire for > loop has completed, later when the user sees the buttons and clicks on one > of them, that one of the lambda callbacks for the appropriate button will > be executed. > Now, when this happens, what is the value of "icon"? This involves a > closure, which is how we describe the effect of the lambda (or any > function) being created inside the scope of another function. The scope > here which contains the "icon" variable is kept after the for loop and its > container function is complete, because the lambda you create holds on to > it in order to resolve variable names from. So all the lambda functions > read the "icon" variable from this same scope, and at the time the loop > completes that variable is assigned to the LAST value from the loop. That's > why all the buttons get the same icon. > There is a simple work around, however, called "rebinding". It is a trick > you can do with default values for lambda or any function. Simply pass the > icon as a default value for a parameter to the lambda, and the value at > that time for one iteration of the loop will be bound to the callback you > create. > > Just change this > > lambda: self.button_exec(icon) > > to this > > lambda icon=icon: self.button_exec(icon) > > I hope both the explanation and the example help! > > > On Mon, Jul 8, 2019 at 1:59 AM Abdur-Rahmaan Janhangeer < > arj.python at gmail.com> wrote: > >> Greetings, >> >> I'm doing this project: >> >> >> https://github.com/Abdur-rahmaanJ/cmdlaunch/blob/master/cmdlaunch/cmdlaunch.py >> >> Everything works fine, except for one thing. >> >> On line 60, the button is taking an object in it's command parameter >> >> If you run the file, you'll see that all the different names are written >> on >> the gui just the callback is takest the newest icon object for all three >> buttons >> >> I know just a reference issue but it's been **bugging** me. >> >> Thanks All >> >> -- >> Abdur-Rahmaan Janhangeer >> Mauritius >> -- >> https://mail.python.org/mailman/listinfo/python-list >> > > > -- > > CALVIN SPEALMAN > > SENIOR QUALITY ENGINEER > > cspealma at redhat.com M: +1.336.210.5107 > [image: https://red.ht/sig] > TRIED. TESTED. TRUSTED. > -- Abdur-Rahmaan Janhangeer http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ Mauritius From barry at python.org Mon Jul 8 19:34:50 2019 From: barry at python.org (Barry Warsaw) Date: Mon, 8 Jul 2019 16:34:50 -0700 Subject: [RELEASE] Python 3.8.0b2 is now available for testing In-Reply-To: <75CB7BD4-8224-49B7-9F10-EAAA67259C86@langa.pl> References: <75CB7BD4-8224-49B7-9F10-EAAA67259C86@langa.pl> Message-ID: <0F97C3D4-99B6-4D73-9092-249B1DC94B0C@python.org> I?ve updated the official images to include 3.8.0b2: https://gitlab.com/python-devs/ci-images/tree/master Cheers, -Barry > On Jul 4, 2019, at 15:05, ?ukasz Langa wrote: > > Signed PGP part > After a few days of delay, but somewhat cutely timed with the US Independence Day, I present you Python 3.8.0b2: > > https://www.python.org/downloads/release/python-380b2/ > > This release is the second of four planned beta release previews. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. The next pre-release of Python 3.8 will be 3.8.0b3, currently scheduled for 2019-07-29. > > Call to action > > We strongly encourage maintainers of third-party Python projects to test with 3.8 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2019-09-30). Our goal is have no ABI changes after beta 3 and no code changes after 3.8.0rc1, the release candidate. To achieve that, it will be extremely important to get as much exposure for 3.8 as possible during the beta phase. > > Please keep in mind that this is a preview release and its use is not recommended for production environments. > > No more non-bugfixes allowed on the ?3.8? branch > > The time has come, team. Please help make Python 3.8 as stable as possible and keep all features not currently landed for Python 3.9. Don?t fret, it?ll come faster than you think. > > > - ? > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From nad at python.org Mon Jul 8 19:57:58 2019 From: nad at python.org (Ned Deily) Date: Mon, 8 Jul 2019 19:57:58 -0400 Subject: [RELEASE] Python 3.7.4 is now available Message-ID: <5F03BD58-F398-4765-A220-C7A97E13082E@python.org> Python 3.7.4 is now available. 3.7.4 is the next maintenance release of Python 3.7, the latest feature release of Python. You can find the release files, a link to the changelog, and more information here: https://www.python.org/downloads/release/python-374/ See the "What?s New In Python 3.7" document for more information about the many new features and optimizations included in the 3.7 series. Detailed information about the changes made in 3.7.4 can be found in its change log: https://docs.python.org/3.7/whatsnew/changelog.html#python-3-7-4-final https://docs.python.org/3.7/whatsnew/3.7.html Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation: https://www.python.org/psf/ -- Ned Deily nad at python.org -- [] From weaponx4120 at gmail.com Tue Jul 9 12:13:51 2019 From: weaponx4120 at gmail.com (pro_ bro) Date: Tue, 9 Jul 2019 21:43:51 +0530 Subject: unable to import pip installed packages and modules in python 3.7.3 In-Reply-To: References: Message-ID: On Mon, 8 Jul 2019 22:37 pro_ bro, wrote: > environmental variables are all good ,(i also set path for > site-packages,scripts ) > when i install any package using pip it says installation is successful, > but when i try to import the package(module) it says "NO MODULE NAMED > (MODULE NAME)" > THIS IS HAPPENING TO ALL PIP INSTALLED MODULES > PLEASE CHECK THE IMAGES AND > """"""HELP ME OVERCOME THIS PROBLEM"""""" > From barry at barrys-emacs.org Tue Jul 9 13:23:42 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Tue, 9 Jul 2019 18:23:42 +0100 Subject: unable to import pip installed packages and modules in python 3.7.3 In-Reply-To: References: Message-ID: <78BDDB9F-3031-4971-ADFE-9377B9498A03@barrys-emacs.org> If you included an image of the screen it was striped from your email. Use the mouse to select the text in the cmd window and paste that into the email. Show the command you issued and all the messages it prints. There i no need for upper case text. Barry > On 9 Jul 2019, at 17:13, pro_ bro wrote: > > On Mon, 8 Jul 2019 22:37 pro_ bro, wrote: > >> environmental variables are all good ,(i also set path for >> site-packages,scripts ) >> when i install any package using pip it says installation is successful, >> but when i try to import the package(module) it says "NO MODULE NAMED >> (MODULE NAME)" >> THIS IS HAPPENING TO ALL PIP INSTALLED MODULES >> PLEASE CHECK THE IMAGES AND >> """"""HELP ME OVERCOME THIS PROBLEM"""""" >> > -- > https://mail.python.org/mailman/listinfo/python-list > From python at mrabarnett.plus.com Tue Jul 9 14:04:19 2019 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 9 Jul 2019 19:04:19 +0100 Subject: unable to import pip installed packages and modules in python 3.7.3 In-Reply-To: References: Message-ID: <701f2e38-49be-1b5c-4e1e-73fefbbcc30d@mrabarnett.plus.com> On 2019-07-09 17:13, pro_ bro wrote: > On Mon, 8 Jul 2019 22:37 pro_ bro, wrote: > >> environmental variables are all good ,(i also set path for >> site-packages,scripts ) >> when i install any package using pip it says installation is successful, >> but when i try to import the package(module) it says "NO MODULE NAMED >> (MODULE NAME)" >> THIS IS HAPPENING TO ALL PIP INSTALLED MODULES >> PLEASE CHECK THE IMAGES AND >> """"""HELP ME OVERCOME THIS PROBLEM"""""" >> This list is text-only. Images are automatically removed. Copy and paste any error messages and include details such as which operating system and which version of Python you're using. Incidentally, on Windows the recommended way to start Python from the command is via the py launcher: py.exe You can install modules by calling the pip module from it: py.exe -m pip install MODULE_NAME The advantage is that you know that it's being installed into the correct version of Python if there's more than one version on the system. From scott at scolby.com Tue Jul 9 14:28:15 2019 From: scott at scolby.com (Scott Colby) Date: Tue, 09 Jul 2019 14:28:15 -0400 Subject: Difficulties configuring LTO on macOS with clang Message-ID: Hello all, I am having difficulty building Python with `--with-lto` on macOS 10.14.5. With a clean checkout of the CPython source at the tagged release for 3.7.4, I ran: $ ./configure --prefix=/opt/python3.7.4 --enable-optimizations --enable-ipv6 --with-lto --with-openssl=$(brew --prefix openssl) CC=clang checking for git... found checking build system type... x86_64-apple-darwin18.6.0 checking host system type... x86_64-apple-darwin18.6.0 checking for python3.7... python3.7 checking for --enable-universalsdk... no checking for --with-universal-archs... no checking MACHDEP... checking for --without-gcc... no checking for --with-icc... no checking for gcc... clang checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether clang accepts -g... yes checking for clang option to accept ISO C89... none needed checking how to run the C preprocessor... clang -E checking for grep that handles long lines and -e... /usr/bin/grep checking for a sed that does not truncate output... /usr/bin/sed checking for --with-cxx-main=... no checking for clang++... no configure: By default, distutils will build C++ extension modules with "clang++". If this is not intended, then set CXX on the configure command line. checking for the platform triplet based on compiler characteristics... darwin checking for -Wl,--no-as-needed... no checking for egrep... /usr/bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking minix/config.h usability... no checking minix/config.h presence... no checking for minix/config.h... no checking whether it is safe to define __EXTENSIONS__... yes checking for the Android API level... not Android checking for --with-suffix... checking for case-insensitive build directory... yes checking LIBRARY... libpython$(VERSION)$(ABIFLAGS).a checking LINKCC... $(PURIFY) $(MAINCC) checking for GNU ld... no checking for --enable-shared... no checking for --enable-profiling... no checking LDLIBRARY... libpython$(VERSION)$(ABIFLAGS).a checking for ar... ar checking for readelf... no checking for a BSD-compatible install... /usr/local/bin/ginstall -c checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p checking for --with-pydebug... no checking for --with-assertions... no checking for --enable-optimizations... yes checking for --with-lto... yes checking target system type... x86_64-apple-darwin18.6.0 checking for -llvm-ar... no checking for llvm-ar... '' configure: error: llvm-ar is required for a --with-lto build with clang but could not be found. If I change the command to `... CC=cc` configuration appears to happen normally. On my version of macOS, cc is a symlink to clang. Why is this happening? What should I be doing differently to explicitly specify clang as my compiler? Thank you, Scott Colby From barry at barrys-emacs.org Tue Jul 9 15:39:42 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Tue, 9 Jul 2019 20:39:42 +0100 Subject: Difficulties configuring LTO on macOS with clang In-Reply-To: References: Message-ID: <76870224-4A91-48BF-B483-4EA9931BF293@barrys-emacs.org> > On 9 Jul 2019, at 19:28, Scott Colby wrote: > > Hello all, > > I am having difficulty building Python with `--with-lto` on macOS 10.14.5. With a clean checkout of the CPython source at the tagged release for 3.7.4, I ran: > > $ ./configure --prefix=/opt/python3.7.4 --enable-optimizations --enable-ipv6 --with-lto --with-openssl=$(brew --prefix openssl) CC=clang > checking for git... found > checking build system type... x86_64-apple-darwin18.6.0 > checking host system type... x86_64-apple-darwin18.6.0 > checking for python3.7... python3.7 > checking for --enable-universalsdk... no > checking for --with-universal-archs... no > checking MACHDEP... checking for --without-gcc... no > checking for --with-icc... no > checking for gcc... clang > checking whether the C compiler works... yes > checking for C compiler default output file name... a.out > checking for suffix of executables... > checking whether we are cross compiling... no > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether clang accepts -g... yes > checking for clang option to accept ISO C89... none needed > checking how to run the C preprocessor... clang -E > checking for grep that handles long lines and -e... /usr/bin/grep > checking for a sed that does not truncate output... /usr/bin/sed > checking for --with-cxx-main=... no > checking for clang++... no > configure: > > By default, distutils will build C++ extension modules with "clang++". > If this is not intended, then set CXX on the configure command line. > > checking for the platform triplet based on compiler characteristics... darwin > checking for -Wl,--no-as-needed... no > checking for egrep... /usr/bin/grep -E > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for unistd.h... yes > checking minix/config.h usability... no > checking minix/config.h presence... no > checking for minix/config.h... no > checking whether it is safe to define __EXTENSIONS__... yes > checking for the Android API level... not Android > checking for --with-suffix... > checking for case-insensitive build directory... yes > checking LIBRARY... libpython$(VERSION)$(ABIFLAGS).a > checking LINKCC... $(PURIFY) $(MAINCC) > checking for GNU ld... no > checking for --enable-shared... no > checking for --enable-profiling... no > checking LDLIBRARY... libpython$(VERSION)$(ABIFLAGS).a > checking for ar... ar > checking for readelf... no > checking for a BSD-compatible install... /usr/local/bin/ginstall -c > checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p > checking for --with-pydebug... no > checking for --with-assertions... no > checking for --enable-optimizations... yes > checking for --with-lto... yes > checking target system type... x86_64-apple-darwin18.6.0 > checking for -llvm-ar... no > checking for llvm-ar... '' > configure: error: llvm-ar is required for a --with-lto build with clang but could not be found. I see /opt/local/bin/llvm-ar-mp-4.0 which ends up running the llvm-ar from /opt/local/libexec/llvm-4.0/bin Just a guess but maybe you need to add /opt/local/libexec/llvm-4.0/bin/ to your path as its where llvm-ar is. Barry > > If I change the command to `... CC=cc` configuration appears to happen normally. > > On my version of macOS, cc is a symlink to clang. > > Why is this happening? What should I be doing differently to explicitly specify clang as my compiler? > > Thank you, > Scott Colby > -- > https://mail.python.org/mailman/listinfo/python-list > From hlakyi77 at yahoo.com Tue Jul 9 20:24:23 2019 From: hlakyi77 at yahoo.com (Hla Kyi) Date: Wed, 10 Jul 2019 00:24:23 +0000 (UTC) Subject: Cannot delete or modify References: <93873048.5064312.1562718263313.ref@mail.yahoo.com> Message-ID: <93873048.5064312.1562718263313@mail.yahoo.com> Dear Sir / Madam, ? ? I have subscribed mailing Python-list.? ??I have installed and used " Python 3.7.2". I try to modify or uninstall it.?Please see the attached screen shot.? ? Please help me how to uninstall and modify. Best Regards. Kyi - Modify.jpg49kB python-list-request at python.org From PythonList at DancesWithMice.info Tue Jul 9 21:22:17 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Wed, 10 Jul 2019 13:22:17 +1200 Subject: Cannot delete or modify In-Reply-To: <93873048.5064312.1562718263313@mail.yahoo.com> References: <93873048.5064312.1562718263313.ref@mail.yahoo.com> <93873048.5064312.1562718263313@mail.yahoo.com> Message-ID: <315a4972-e42f-9fb8-0680-1075273c8113@DancesWithMice.info> Hi, welcome to the Python community! On 10/07/19 12:24 PM, Hla Kyi via Python-list wrote: > Dear Sir / Madam, > ? ? I have subscribed mailing Python-list.? ??I have installed and used " Python 3.7.2". I try to modify or uninstall it.?Please see the attached screen shot.? ? Please help me how to uninstall and modify. > > Best Regards. > Kyi The mailing list strips all graphics from messages. Please describe the problem you are having when you say "modify". It would also help if you identified your PC's operating system... -- Regards =dn From raju.mailinglists at gmail.com Wed Jul 10 08:57:29 2019 From: raju.mailinglists at gmail.com (kamaraju kusumanchi) Date: Wed, 10 Jul 2019 08:57:29 -0400 Subject: OT: Is there a name for this transformation? Message-ID: Given a csv file with the following contents 20180701, A 20180702, A, B 20180703, A, B, C 20180704, B, C 20180705, C I would like to transform the underlying data into a dataframe such as date, A, B, C 20180701, True, False, False 20180702, True, True, False 20180703, True, True, True 20180704, False, True, True 20180705, False, False, True the idea is that the first field in each line of the csv is the row index of the dataframe. The subsequent fields will be its column names and the values in the dataframe tell whether that element is present or not in the line. Is there a name for this transformation? Any existing code/library that can transform data back and forth between the two formats? I can write one myself if there is none but trying to avoid reinventing the wheel if possible. thanks in advance raju -- Kamaraju S Kusumanchi | http://raju.shoutwiki.com/wiki/Blog From scott at scolby.com Wed Jul 10 10:25:59 2019 From: scott at scolby.com (Scott Colby) Date: Wed, 10 Jul 2019 10:25:59 -0400 Subject: Difficulties configuring LTO on macOS with clang In-Reply-To: <76870224-4A91-48BF-B483-4EA9931BF293@barrys-emacs.org> References: <76870224-4A91-48BF-B483-4EA9931BF293@barrys-emacs.org> Message-ID: Unfortunately that path is absent on my system. Could the issue be that the llvm version of ar is not present at all on macOS? What remains confusing in that case is that by using CC=cc (and I assume just ar) the LTO compilation succeeds, despite cc being a symlink to clang anyway. Thanks, Scott On Tue, Jul 9, 2019, at 15:39, Barry Scott wrote: > > > > On 9 Jul 2019, at 19:28, Scott Colby wrote: > > > > Hello all, > > > > I am having difficulty building Python with `--with-lto` on macOS 10.14.5. With a clean checkout of the CPython source at the tagged release for 3.7.4, I ran: > > > > $ ./configure --prefix=/opt/python3.7.4 --enable-optimizations --enable-ipv6 --with-lto --with-openssl=$(brew --prefix openssl) CC=clang > > checking for git... found > > checking build system type... x86_64-apple-darwin18.6.0 > > checking host system type... x86_64-apple-darwin18.6.0 > > checking for python3.7... python3.7 > > checking for --enable-universalsdk... no > > checking for --with-universal-archs... no > > checking MACHDEP... checking for --without-gcc... no > > checking for --with-icc... no > > checking for gcc... clang > > checking whether the C compiler works... yes > > checking for C compiler default output file name... a.out > > checking for suffix of executables... > > checking whether we are cross compiling... no > > checking for suffix of object files... o > > checking whether we are using the GNU C compiler... yes > > checking whether clang accepts -g... yes > > checking for clang option to accept ISO C89... none needed > > checking how to run the C preprocessor... clang -E > > checking for grep that handles long lines and -e... /usr/bin/grep > > checking for a sed that does not truncate output... /usr/bin/sed > > checking for --with-cxx-main=... no > > checking for clang++... no > > configure: > > > > By default, distutils will build C++ extension modules with "clang++". > > If this is not intended, then set CXX on the configure command line. > > > > checking for the platform triplet based on compiler characteristics... darwin > > checking for -Wl,--no-as-needed... no > > checking for egrep... /usr/bin/grep -E > > checking for ANSI C header files... yes > > checking for sys/types.h... yes > > checking for sys/stat.h... yes > > checking for stdlib.h... yes > > checking for string.h... yes > > checking for memory.h... yes > > checking for strings.h... yes > > checking for inttypes.h... yes > > checking for stdint.h... yes > > checking for unistd.h... yes > > checking minix/config.h usability... no > > checking minix/config.h presence... no > > checking for minix/config.h... no > > checking whether it is safe to define __EXTENSIONS__... yes > > checking for the Android API level... not Android > > checking for --with-suffix... > > checking for case-insensitive build directory... yes > > checking LIBRARY... libpython$(VERSION)$(ABIFLAGS).a > > checking LINKCC... $(PURIFY) $(MAINCC) > > checking for GNU ld... no > > checking for --enable-shared... no > > checking for --enable-profiling... no > > checking LDLIBRARY... libpython$(VERSION)$(ABIFLAGS).a > > checking for ar... ar > > checking for readelf... no > > checking for a BSD-compatible install... /usr/local/bin/ginstall -c > > checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p > > checking for --with-pydebug... no > > checking for --with-assertions... no > > checking for --enable-optimizations... yes > > checking for --with-lto... yes > > checking target system type... x86_64-apple-darwin18.6.0 > > checking for -llvm-ar... no > > checking for llvm-ar... '' > > configure: error: llvm-ar is required for a --with-lto build with clang but could not be found. > > I see /opt/local/bin/llvm-ar-mp-4.0 which ends up running the llvm-ar from > /opt/local/libexec/llvm-4.0/bin > > Just a guess but maybe you need to add /opt/local/libexec/llvm-4.0/bin/ to your path > as its where llvm-ar is. > > Barry > > > > > If I change the command to `... CC=cc` configuration appears to happen normally. > > > > On my version of macOS, cc is a symlink to clang. > > > > Why is this happening? What should I be doing differently to explicitly specify clang as my compiler? > > > > Thank you, > > Scott Colby > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > > From joshkay200612 at gmail.com Wed Jul 10 10:35:43 2019 From: joshkay200612 at gmail.com (joshua kay) Date: Wed, 10 Jul 2019 07:35:43 -0700 Subject: issue Message-ID: Hello, i am encountering a issue with python 3.7.2rc1 64 bit currently My operating system is windows 10 and every time i try to run my code the shell just says: == RESTART: C:\Users\joshu\OneDrive\Desktop\python\projects\multi choice.py == I how scanned the internet looking for a solution until i was made aware by the python installer about the support team, please tell me a solution for my issue thank you, Joshua kay From cspealma at redhat.com Wed Jul 10 11:09:42 2019 From: cspealma at redhat.com (Calvin Spealman) Date: Wed, 10 Jul 2019 11:09:42 -0400 Subject: issue In-Reply-To: References: Message-ID: If you could share the code you are trying to run, which is causing the issue, someone might be able to help. Without that, it is doubtful you'll get any help. On Wed, Jul 10, 2019 at 11:08 AM joshua kay wrote: > Hello, > i am encountering a issue with python 3.7.2rc1 64 bit currently > My operating system is windows 10 and every time i try to run my code the > shell just says: > == RESTART: C:\Users\joshu\OneDrive\Desktop\python\projects\multi choice.py > == > > I how scanned the internet looking for a solution until i was made aware by > the python installer about the support team, please tell me a solution for > my issue > > thank you, > Joshua kay > -- > https://mail.python.org/mailman/listinfo/python-list > -- CALVIN SPEALMAN SENIOR QUALITY ENGINEER cspealma at redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] TRIED. TESTED. TRUSTED. From dfnsonfsduifb at gmx.de Wed Jul 10 13:50:06 2019 From: dfnsonfsduifb at gmx.de (Johannes Bauer) Date: Wed, 10 Jul 2019 19:50:06 +0200 Subject: Pythonic custom multi-line parsers Message-ID: Hi list, I'm looking for ideas as to a pretty, Pythonic solution for a specific problem that I am solving over and over but where I'm never happy about the solution in the end. It always works, but never is pretty. So see this as an open-ended brainstorming question. Here's the task: There's a custom file format. Each line can be parsed individually and, given the current context, the meaning of each individual line is always clearly distinguishable. I'll give an easy example to demonstrate: moo = koo bar = foo foo := abc def baz = abc Let's say the root context knows only two regexes and give them names: keyvalue: \w+ = \w+ start-multiblock: \w+ := The keyvalue is contained in itself, when the line is successfully parsed all the information is present. The start-multiblock however gives us only part of the puzzle, namely the name of the following block. In the multiblock context, there's different regexes that can happen (actually only one): multiblock-item: \s\w+ Now obviously whe the block is finished, there's no delimiter. It's implicit by the multiblock-item regex not matching and therefore we backtrack to the previous parser (root parser) and can successfully parse the last line baz = abc. Especially consider that even though this is a simple example, generally you'll have multiple contexts, many more regexes and especially nesting inside these contexts. Without having to use a parser generator (for those the examples I deal with are usually too much overhead) what I usually end up doing is building a state machine by hand. I.e., I memorize the context, match those and upon no match manually delegate the input data to backtracked matchers. This results in AWFULLY ugly code. I'm wondering what your ideas are to solve this neatly in a Pythonic fashion without having to rely on third-party dependencies. Cheers, Joe From tjreedy at udel.edu Wed Jul 10 14:23:42 2019 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 10 Jul 2019 14:23:42 -0400 Subject: issue In-Reply-To: References: Message-ID: On 7/10/2019 10:35 AM, joshua kay wrote: > Hello, > i am encountering a issue with python 3.7.2rc1 64 bit currently > My operating system is windows 10 and every time i try to run my code the > shell just says: > == RESTART: C:\Users\joshu\OneDrive\Desktop\python\projects\multi choice.py > == You are running your code from an IDLE editor. The above is what you *should* see when you hit F5 or Run => Run Module. This is not a problem. If the above is all you ever see, not even a '>>> ' prompt, then the almost certain problem is that your program runs 'forever' without producing output. Examples: "while True: pass" will run the cpu at full speed, not good; "s = input()" will wait for input forever. Try putting some print statements in your code to determine where it is stuck. -- Terry Jan Reedy From rgaddi at highlandtechnology.invalid Wed Jul 10 14:29:35 2019 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Wed, 10 Jul 2019 11:29:35 -0700 Subject: Pythonic custom multi-line parsers In-Reply-To: References: Message-ID: On 7/10/19 10:50 AM, Johannes Bauer wrote: > Hi list, > > I'm looking for ideas as to a pretty, Pythonic solution for a specific > problem that I am solving over and over but where I'm never happy about > the solution in the end. It always works, but never is pretty. So see > this as an open-ended brainstorming question. > > Here's the task: There's a custom file format. Each line can be parsed > individually and, given the current context, the meaning of each > individual line is always clearly distinguishable. I'll give an easy > example to demonstrate: > > > moo = koo > bar = foo > foo := > abc > def > baz = abc > > Let's say the root context knows only two regexes and give them names: > > keyvalue: \w+ = \w+ > start-multiblock: \w+ := > > The keyvalue is contained in itself, when the line is successfully > parsed all the information is present. The start-multiblock however > gives us only part of the puzzle, namely the name of the following > block. In the multiblock context, there's different regexes that can > happen (actually only one): > > multiblock-item: \s\w+ > > Now obviously whe the block is finished, there's no delimiter. It's > implicit by the multiblock-item regex not matching and therefore we > backtrack to the previous parser (root parser) and can successfully > parse the last line baz = abc. > > Especially consider that even though this is a simple example, generally > you'll have multiple contexts, many more regexes and especially nesting > inside these contexts. > > Without having to use a parser generator (for those the examples I deal > with are usually too much overhead) what I usually end up doing is > building a state machine by hand. I.e., I memorize the context, match > those and upon no match manually delegate the input data to backtracked > matchers. > > This results in AWFULLY ugly code. I'm wondering what your ideas are to > solve this neatly in a Pythonic fashion without having to rely on > third-party dependencies. > > Cheers, > Joe > That's pretty much what I do. I generally make the parser a class and each state a method. Every line the parser takes out of the file it passes to self.statefn, which processes the line in the current context and updates self.statefn to a different method if necessary. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From hjp-python at hjp.at Wed Jul 10 15:05:56 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Wed, 10 Jul 2019 21:05:56 +0200 Subject: OT: Is there a name for this transformation? In-Reply-To: References: Message-ID: <20190710190556.auafngbtakbzig42@hjp.at> On 2019-07-10 08:57:29 -0400, kamaraju kusumanchi wrote: > Given a csv file with the following contents > > 20180701, A > 20180702, A, B > 20180703, A, B, C > 20180704, B, C > 20180705, C > > I would like to transform the underlying data into a dataframe such as > > date, A, B, C > 20180701, True, False, False > 20180702, True, True, False > 20180703, True, True, True > 20180704, False, True, True > 20180705, False, False, True > > the idea is that the first field in each line of the csv is the row > index of the dataframe. The subsequent fields will be its column names > and the values in the dataframe tell whether that element is present > or not in the line. > > Is there a name for this transformation? This type of output is usually called a cross table, but I don't know whether this specific transformation has a name (if you had only one of A, B, and C per line it would be a kind of pivot operation). > Any existing code/library > that can transform data back and forth between the two formats? I can > write one myself if there is none but trying to avoid reinventing the > wheel if possible. I need to produce cross tables frequently, but I never bothered to make it into the library because the part that is common (maintaining two hashes and dumping them) is so much less than the parts which are different (data source and format, what information to extract, output format). The basic idea is that you use a dict of dict of (whatever) to represent your output matrix: row keys are the first level, column keys are the second level. Cell type in your case is bool, so you could use a set instead of a dict of bool. Often you want to keep information about each row (e.g. order of appearance, or a count), so you'll use a second dict for that. For output you get a list of columns in the right order and then iterate over the 1st level keys of your dict and the list of columns to access each cell. hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From barry at barrys-emacs.org Wed Jul 10 16:22:02 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Wed, 10 Jul 2019 21:22:02 +0100 Subject: Difficulties configuring LTO on macOS with clang In-Reply-To: References: <76870224-4A91-48BF-B483-4EA9931BF293@barrys-emacs.org> Message-ID: > On 10 Jul 2019, at 15:25, Scott Colby wrote: > > Unfortunately that path is absent on my system. Could the issue be that the llvm version of ar is not present at all on macOS? What remains confusing in that case is that by using CC=cc (and I assume just ar) the LTO compilation succeeds, despite cc being a symlink to clang anyway. Oh that may be because I have MacPorts installed on my mac I guess you have macPorts installed as well? I see you are using brew openssl. Maybe the macPorts tools and the brew tools are conflicting? What if you remove /opt/local/bin from your PATH does that help configure find the tools it needs? Barry > > Thanks, > Scott > > On Tue, Jul 9, 2019, at 15:39, Barry Scott wrote: >> >> >> > On 9 Jul 2019, at 19:28, Scott Colby wrote: >> > >> > Hello all, >> > >> > I am having difficulty building Python with `--with-lto` on macOS 10.14.5. With a clean checkout of the CPython source at the tagged release for 3.7.4, I ran: >> > >> > $ ./configure --prefix=/opt/python3.7.4 --enable-optimizations --enable-ipv6 --with-lto --with-openssl=$(brew --prefix openssl) CC=clang >> > checking for git... found >> > checking build system type... x86_64-apple-darwin18.6.0 >> > checking host system type... x86_64-apple-darwin18.6.0 >> > checking for python3.7... python3.7 >> > checking for --enable-universalsdk... no >> > checking for --with-universal-archs... no >> > checking MACHDEP... checking for --without-gcc... no >> > checking for --with-icc... no >> > checking for gcc... clang >> > checking whether the C compiler works... yes >> > checking for C compiler default output file name... a.out >> > checking for suffix of executables... >> > checking whether we are cross compiling... no >> > checking for suffix of object files... o >> > checking whether we are using the GNU C compiler... yes >> > checking whether clang accepts -g... yes >> > checking for clang option to accept ISO C89... none needed >> > checking how to run the C preprocessor... clang -E >> > checking for grep that handles long lines and -e... /usr/bin/grep >> > checking for a sed that does not truncate output... /usr/bin/sed >> > checking for --with-cxx-main=... no >> > checking for clang++... no >> > configure: >> > >> > By default, distutils will build C++ extension modules with "clang++". >> > If this is not intended, then set CXX on the configure command line. >> > >> > checking for the platform triplet based on compiler characteristics... darwin >> > checking for -Wl,--no-as-needed... no >> > checking for egrep... /usr/bin/grep -E >> > checking for ANSI C header files... yes >> > checking for sys/types.h... yes >> > checking for sys/stat.h... yes >> > checking for stdlib.h... yes >> > checking for string.h... yes >> > checking for memory.h... yes >> > checking for strings.h... yes >> > checking for inttypes.h... yes >> > checking for stdint.h... yes >> > checking for unistd.h... yes >> > checking minix/config.h usability... no >> > checking minix/config.h presence... no >> > checking for minix/config.h... no >> > checking whether it is safe to define __EXTENSIONS__... yes >> > checking for the Android API level... not Android >> > checking for --with-suffix... >> > checking for case-insensitive build directory... yes >> > checking LIBRARY... libpython$(VERSION)$(ABIFLAGS).a >> > checking LINKCC... $(PURIFY) $(MAINCC) >> > checking for GNU ld... no >> > checking for --enable-shared... no >> > checking for --enable-profiling... no >> > checking LDLIBRARY... libpython$(VERSION)$(ABIFLAGS).a >> > checking for ar... ar >> > checking for readelf... no >> > checking for a BSD-compatible install... /usr/local/bin/ginstall -c >> > checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p >> > checking for --with-pydebug... no >> > checking for --with-assertions... no >> > checking for --enable-optimizations... yes >> > checking for --with-lto... yes >> > checking target system type... x86_64-apple-darwin18.6.0 >> > checking for -llvm-ar... no >> > checking for llvm-ar... '' >> > configure: error: llvm-ar is required for a --with-lto build with clang but could not be found. >> >> I see /opt/local/bin/llvm-ar-mp-4.0 which ends up running the llvm-ar from >> /opt/local/libexec/llvm-4.0/bin >> >> Just a guess but maybe you need to add /opt/local/libexec/llvm-4.0/bin/ to your path >> as its where llvm-ar is. >> >> Barry >> >> > >> > If I change the command to `... CC=cc` configuration appears to happen normally. >> > >> > On my version of macOS, cc is a symlink to clang. >> > >> > Why is this happening? What should I be doing differently to explicitly specify clang as my compiler? >> > >> > Thank you, >> > Scott Colby >> > -- >> > https://mail.python.org/mailman/listinfo/python-list >> > From jasonanyilian at gmail.com Wed Jul 10 18:37:22 2019 From: jasonanyilian at gmail.com (CrazyVideoGamez) Date: Wed, 10 Jul 2019 15:37:22 -0700 (PDT) Subject: How Do You Replace Variables With Their Values? Message-ID: How do you replace a variable with its value in python 3.7.2? For example, say I have: dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} # Don't ask where I got the dinner from for meal in dinner.keys(): meal = list(dinner[meal] But I only get one list called "meal" and I'm just changing it with the code above (you can find that by printing it out). How can I make separate lists called 'Starters', 'Main Course', and 'Desert'? From jasonanyilian at gmail.com Wed Jul 10 18:43:06 2019 From: jasonanyilian at gmail.com (CrazyVideoGamez) Date: Wed, 10 Jul 2019 15:43:06 -0700 (PDT) Subject: How Do You Replace Variables With Their Values? Message-ID: How do you replace a variable with its value in python 3.7.2? For example, say I have: dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} # Don't ask where I got the dinner from for meal in dinner.keys(): meal = list(dinner[meal]) But I only get one list called "meal" and I'm just changing it with the code above (you can find that by printing it out). How can I make separate lists called 'Starters', 'Main Course', and 'Desert'? From MICHAEL.LANE at msp.gouv.qc.ca Wed Jul 10 13:40:46 2019 From: MICHAEL.LANE at msp.gouv.qc.ca (MICHAEL LANE) Date: Wed, 10 Jul 2019 13:40:46 -0400 Subject: load extention spatialite in sqlite on windows References: <5D26231E020000E20005385C@msp.gouv.qc.ca> Message-ID: <5D26231E020000E20005385C@msp.gouv.qc.ca> Hi, I tried to reproduce this(https://geoalchemy-2.readthedocs.io/en/latest/spatialite_tutorial.html) in windows. How can I do that? I'm using Windows 10, my project use Pyramid Framework with Python 3.6.8. Even though I put the dll file in a directory defined in %PATH% or try with/without full path, with/without file extension, with '//' or '\', with libspatialite-4.dll or using excute with all combination mentionned before... dbapi_conn.execute("SELECT load_extension('mod_spatialite.dll')") For example i tried this kind of combinaison: def load_spatialite(dbapi_conn, connection_record): dbapi_conn.load_extension('C:\\mydll\\mod_spatialite.dll') # dbapi_conn.load_extension('C:\\mydll\\libspatialite-4.dll') # dbapi_conn.load_extension('mod_spatialite') # dbapi_conn.load_extension('mod_spatialite.dll') # dbapi_conn.execute("SELECT load_extension('C:\\mydll\\mod_spatialite.dll')") # dbapi_conn.execute("SELECT load_extension('mod_spatialite.dll')") # dbapi_conn.execute("SELECT load_extension('mod_spatialite')") # dbapi_conn.load_extension('C:/mydll\mod_spatialite.dll') Can make it work, always getting: dbapi_conn.load_extension('C:\\mydll\\mod_spatialite.dll')sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) OsError 0xc1 (193) (Background on this error at: http://sqlalche.me/e/e3q8) ( 'http://www.google.com/url?qdhttp%3A%2F%2Fsqlalche.me%2Fe%2Fe3q8sadDsntzd1usgdAFQjCNERkh6zEzVSQLk5naEvDfK-IlHbLg';return true;" onmousedown="this.href='http://www.google.com/url?qdhttp%3A%2F%2Fsqlalche.me%2Fe%2Fe3q8sadDsntzd1usgdAFQjCNERkh6zEzVSQLk5naEvDfK-IlHbLg';return true;" style="CURSOR: pointer; TEXT-DECORATION: none; BORDER-TOP: 0px; BORDER-RIGHT: 0px; BORDER-BOTTOM: 0px; COLOR: rgb(102,17,204); PADDING-BOTTOM: 0px; PADDING-TOP: 0px; PADDING-LEFT: 0px; MARGIN: 0px; BORDER-LEFT: 0px; PADDING-RIGHT: 0px" href="http://sqlalche.me/e/e3q8)" rel=nofollow target=_blank) I've been told to use import sqlite3 so I make it a try with this code: conn = sqlite3.connect(":memory:") conn.enable_load_extension(True) conn.load_extension('c:\\mydll\\mod_spatialite') Try too with 'c:/mydll/mod_sptatialite' But I got the same error. conn.load_extension('c:\\mydll\\mod_spatialite') sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) OsError 0xc1 (193) (Background on this error at: http://sqlalche.me/e/e3q8) My directory contain the unzip of: http://www.gaia-gis.it/gaia-sins/windows-bin-x86/mod_spatialite-4.3.0a-win-x86.7z The directory is set in %PATH% Any hint will be appreciated. Thanks Michael From rosuav at gmail.com Wed Jul 10 19:09:19 2019 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 11 Jul 2019 09:09:19 +1000 Subject: load extention spatialite in sqlite on windows In-Reply-To: <5D26231E020000E20005385C@msp.gouv.qc.ca> References: <5D26231E020000E20005385C@msp.gouv.qc.ca> <5D26231E020000E20005385C@msp.gouv.qc.ca> Message-ID: On Thu, Jul 11, 2019 at 8:57 AM MICHAEL LANE via Python-list wrote: > > conn.load_extension('c:\\mydll\\mod_spatialite') > sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) OsError 0xc1 (193) > (Background on this error at: http://sqlalche.me/e/e3q8) > Seems this is SQLite's way of throwing up its arms and saying "I have no idea what went wrong". The error code 0xC1 or 193 decimal can be interpreted from Microsoft's documentation: https://docs.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499- """ ERROR_BAD_EXE_FORMAT 193 (0xC1) %1 is not a valid Win32 application. """ I would guess that this is referring to the DLL itself (it might not be, but that's a good place to start), and the first thing I'd do is check if it's even a Windows DLL file - maybe something broke in the download or something. If it looks like a legit file, the next thing I'd look at is whether it's a 32-bit or 64-bit DLL; you have to match the word size of your SQLite binary. Unfortunately SQLite didn't give much info, so we're a bit in the dark here. This may require some research. ChrisA From tjreedy at udel.edu Wed Jul 10 21:26:30 2019 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 10 Jul 2019 21:26:30 -0400 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: Message-ID: On 7/10/2019 6:37 PM, CrazyVideoGamez wrote: the below twice. Please post just once and be patient for at least a couple of hours -- or days if not subscribed. > How do you replace a variable with its value in python 3.7.2? Use the variable (name) in an expression. But this is not what you ask below. > For example, say I have: > > dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} > > # Don't ask where I got the dinner from I think most of us have seen examples like this in programming books. > for meal in dinner.keys(): > meal = list(dinner[meal] > > But I only get one list called "meal" Because it is just one identifier, and you rebind it to multiple values. > and I'm just changing it with the code above (you can find that by printing it out). How can I make separate lists called 'Starters', 'Main Course', and 'Desert'? starters = dinner['Starters'] main_course = dinner['Main Course'] desert = dinner['Desert'] -- Terry Jan Reedy From hlakyi77 at yahoo.com Wed Jul 10 21:49:57 2019 From: hlakyi77 at yahoo.com (Hla Kyi) Date: Thu, 11 Jul 2019 01:49:57 +0000 (UTC) Subject: Problem to delete or modify References: <856232322.5620543.1562809797952.ref@mail.yahoo.com> Message-ID: <856232322.5620543.1562809797952@mail.yahoo.com> Dear Sir/ Madam, ? ? 1. I try to modify, some of the check boxes can not be selected.? ? 2. I try to uninstall, "successfully uninstall" message is come out. When I exit it "if you have any problem, please contact??python-list at python.org?" message is come out.?? ? I install Python 3.7.2 at my laptop in Windows 10 operating system. Best Regards, Kyi? From ben+python at benfinney.id.au Wed Jul 10 22:33:59 2019 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 11 Jul 2019 12:33:59 +1000 Subject: How Do You Replace Variables With Their Values? References: Message-ID: <86v9w9qoyw.fsf@benfinney.id.au> Terry Reedy writes: > On 7/10/2019 6:37 PM, CrazyVideoGamez wrote: > > > and I'm just changing it with the code above (you can find that by > > printing it out). How can I make separate lists called 'Starters', > > 'Main Course', and 'Desert'? > > starters = dinner['Starters'] > main_course = dinner['Main Course'] > desert = dinner['Desert'] The question comes, though, why you (CrazyVideoGamez) are doing this. You have the lists immediately accesible as dictionary elements, by name. Why do you need to also have them bound to separate names; what problem are you trying to solve that you think this will help? -- \ ?If [a technology company] has confidence in their future | `\ ability to innovate, the importance they place on protecting | _o__) their past innovations really should decline.? ?Gary Barnett | Ben Finney From frank at chagford.com Thu Jul 11 00:59:03 2019 From: frank at chagford.com (Frank Millman) Date: Thu, 11 Jul 2019 06:59:03 +0200 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: Message-ID: On 2019-07-11 12:43 AM, CrazyVideoGamez wrote: > How do you replace a variable with its value in python 3.7.2? For example, say I have: > > dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} > > # Don't ask where I got the dinner from > > for meal in dinner.keys(): > meal = list(dinner[meal]) > > But I only get one list called "meal" and I'm just changing it with the code above (you can find that by printing it out). How can I make separate lists called 'Starters', 'Main Course', and 'Desert'? > 1. Iterating over a dictionary returns each key. So instead of 'for meal in dinner.keys()' you can just say 'for meal in dinner'. 2. It is not a good idea to use the variable name 'meal' for two purposes. You use it to get each key, and then it gets over-ridden with the result of 'list(dinner[meal])'. Then on the next iteration it gets over-ridden again with the next key. 3. The result of 'dinner[meal]' is already a list, so there is no need to say 'list(dinner[meal])'. Technically there is a difference - your approach creates a new list, instead of just creating a reference to the original one, but I doubt if that was your intention. 4. There is potentially more than one list, but for each iteration you over-ride the previous one, so at the end, only the last one remains. The solution is to create a 'list of lists'. Putting all this together - courses = [] for course in dinner: courses.append(dinner[course]) This gives you a list, called 'courses', containing three lists, one for each 'course' containing the options for that course. However, in the process, you have lost the names of the courses, namely 'Starters', 'Main course', and 'Desert'. So to answer your original question "How can I make separate lists called 'Starters', 'Main Course', and 'Desert'?", the code that you started with is exactly what you asked for. I think you were asking how to create a variable called 'Starters' containing the list of starters. It can be done, using the built-in function 'setattr()', but I don't think that would be useful. If you knew in advance that one of the options was called 'Starters', you could just say Starters = ['Fried Calamari', 'Potted crab']. But if you did not know that in advance, how would you know what your variable was called? Frank Millman From frank at chagford.com Thu Jul 11 00:59:03 2019 From: frank at chagford.com (Frank Millman) Date: Thu, 11 Jul 2019 06:59:03 +0200 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: Message-ID: On 2019-07-11 12:43 AM, CrazyVideoGamez wrote: > How do you replace a variable with its value in python 3.7.2? For example, say I have: > > dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} > > # Don't ask where I got the dinner from > > for meal in dinner.keys(): > meal = list(dinner[meal]) > > But I only get one list called "meal" and I'm just changing it with the code above (you can find that by printing it out). How can I make separate lists called 'Starters', 'Main Course', and 'Desert'? > 1. Iterating over a dictionary returns each key. So instead of 'for meal in dinner.keys()' you can just say 'for meal in dinner'. 2. It is not a good idea to use the variable name 'meal' for two purposes. You use it to get each key, and then it gets over-ridden with the result of 'list(dinner[meal])'. Then on the next iteration it gets over-ridden again with the next key. 3. The result of 'dinner[meal]' is already a list, so there is no need to say 'list(dinner[meal])'. Technically there is a difference - your approach creates a new list, instead of just creating a reference to the original one, but I doubt if that was your intention. 4. There is potentially more than one list, but for each iteration you over-ride the previous one, so at the end, only the last one remains. The solution is to create a 'list of lists'. Putting all this together - courses = [] for course in dinner: courses.append(dinner[course]) This gives you a list, called 'courses', containing three lists, one for each 'course' containing the options for that course. However, in the process, you have lost the names of the courses, namely 'Starters', 'Main course', and 'Desert'. So to answer your original question "How can I make separate lists called 'Starters', 'Main Course', and 'Desert'?", the code that you started with is exactly what you asked for. I think you were asking how to create a variable called 'Starters' containing the list of starters. It can be done, using the built-in function 'setattr()', but I don't think that would be useful. If you knew in advance that one of the options was called 'Starters', you could just say Starters = ['Fried Calamari', 'Potted crab']. But if you did not know that in advance, how would you know what your variable was called? Frank Millman From aldwinaldwindev at gmail.com Thu Jul 11 00:51:48 2019 From: aldwinaldwindev at gmail.com (Aldwin Pollefeyt) Date: Thu, 11 Jul 2019 12:51:48 +0800 Subject: How Do You Replace Variables With Their Values? In-Reply-To: <86v9w9qoyw.fsf@benfinney.id.au> References: <86v9w9qoyw.fsf@benfinney.id.au> Message-ID: dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} # Don't ask where I got the dinner from for meal in dinner.keys(): exec(meal.replace(' ','_') + ' = list(dinner[meal])') print(Starters) print(Main_Course) print(Desert) OUTPUT: ['Fried Calamari', 'Potted crab'] ['Fish', 'Meat'] ['Cake', 'Banana Split'] On Thu, Jul 11, 2019 at 10:34 AM Ben Finney wrote: > Terry Reedy writes: > > > On 7/10/2019 6:37 PM, CrazyVideoGamez wrote: > > > > > and I'm just changing it with the code above (you can find that by > > > printing it out). How can I make separate lists called 'Starters', > > > 'Main Course', and 'Desert'? > > > > starters = dinner['Starters'] > > main_course = dinner['Main Course'] > > desert = dinner['Desert'] > > The question comes, though, why you (CrazyVideoGamez) are doing this. > > You have the lists immediately accesible as dictionary elements, by > name. > > Why do you need to also have them bound to separate names; what problem > are you trying to solve that you think this will help? > > -- > \ ?If [a technology company] has confidence in their future | > `\ ability to innovate, the importance they place on protecting | > _o__) their past innovations really should decline.? ?Gary Barnett | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list > From debasreebnrj at gmail.com Thu Jul 11 01:54:21 2019 From: debasreebnrj at gmail.com (Debasree Banerjee) Date: Thu, 11 Jul 2019 06:54:21 +0100 Subject: Seeking help regarding Python code Message-ID: Hi, I have a dataset like this: RecTime NO2_RAW NO2 LAQN_NO2 10980 06/6/19 01:45 17.95444444 53.46255556 17.7 10981 06/6/19 01:45 17.94444444 53.44344444 17.7 10982 06/6/19 01:45 17.92111111 53.39877778 17.7 I want to calculate the difference between LAQN_NO2 and NO2_RAW everyday at 04:00 and add that value to NO2_RAW values in all rows on a particular day. I have 2 months of data and I want to do this for each day. So, I want a table like this RecTime NO2_RAW NO2 LAQN_NO2 NO2_Cal 10980 06/6/19 01:45 17.95444444 53.46255556 17.7 (LAQN_NO2-NO2_RAW)+NO2_RAW 10981 06/6/19 01:45 17.94444444 53.44344444 17.7 10982 06/6/19 01:45 17.92111111 53.39877778 17.7 Can someone please help? Thanks Debasree From hobson42 at gmail.com Thu Jul 11 14:26:28 2019 From: hobson42 at gmail.com (Ian Hobson) Date: Thu, 11 Jul 2019 19:26:28 +0100 Subject: Seeking help regarding Python code In-Reply-To: References: Message-ID: <6de94d65-ae52-fd36-923b-02634b078a54@gmail.com> Hi, Afraid the formatting gremlins got to your data before we saw it, so I am taking a guess at what you want to achieve. On 11/07/19 06:54, Debasree Banerjee wrote: > > I want to calculate the difference between LAQN_NO2 and NO2_RAW everyday at > 04:00 and add that value to NO2_RAW values in all rows on a particular day. > I have 2 months of data and I want to do this for each day. What is the difference value at 04:00? Your times all appear to be the same. You doubt you mean you run the program at 04:00 every day, because that will not update any rows that don't yet exist, and anyway NO2_RAW + (LAQN_NO2 - NO2_RAW) simply equals LAQN_NO2. Once calculated, I would either store it in a new field (and add to NO2_RAW on every use), or copy the whole day's dataset to a new table. That way, should you have to re-run, you have not lost the old value of NO2_RAW. Writing the code will come after you understand exactly what to achieve, and how a computer could do it. Regards Ian (Another one). From tjreedy at udel.edu Thu Jul 11 14:36:06 2019 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 11 Jul 2019 14:36:06 -0400 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: <86v9w9qoyw.fsf@benfinney.id.au> Message-ID: On 7/11/2019 12:51 AM, Aldwin Pollefeyt wrote: > dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main > Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} > > # Don't ask where I got the dinner from > > for meal in dinner.keys(): > exec(meal.replace(' ','_') + ' = list(dinner[meal])') If dinner came from an untrusted source, and the OP does not say differently, this would be a stupid thing to do. -- Terry Jan Reedy From rhodri at kynesim.co.uk Thu Jul 11 13:45:29 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 11 Jul 2019 18:45:29 +0100 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: <86v9w9qoyw.fsf@benfinney.id.au> Message-ID: On 11/07/2019 05:51, Aldwin Pollefeyt wrote: > dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main > Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} > > # Don't ask where I got the dinner from > > for meal in dinner.keys(): > exec(meal.replace(' ','_') + ' = list(dinner[meal])') > > print(Starters) > print(Main_Course) > print(Desert) > > OUTPUT: > ['Fried Calamari', 'Potted crab'] > ['Fish', 'Meat'] > ['Cake', 'Banana Split'] If you think you need to do this, you are almost certainly wrong. Ew! -- Rhodri James *-* Kynesim Ltd From rosuav at gmail.com Thu Jul 11 14:41:07 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Jul 2019 04:41:07 +1000 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: <86v9w9qoyw.fsf@benfinney.id.au> Message-ID: On Fri, Jul 12, 2019 at 4:37 AM Terry Reedy wrote: > > On 7/11/2019 12:51 AM, Aldwin Pollefeyt wrote: > > dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main > > Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} > > > > # Don't ask where I got the dinner from > > > > for meal in dinner.keys(): > > exec(meal.replace(' ','_') + ' = list(dinner[meal])') > > If dinner came from an untrusted source, and the OP does not say > differently, this would be a stupid thing to do. > Yes, but if your dinner came from an untrusted source, you have bigger problems than remote code execution exploits. But exec is a bad idea even without security concerns. Python has FAR better metaprogramming features (mutating globals() comes to mind here) - but even that's probably not necessary. Needs more info from the OP. ChrisA From pkpearson at nowhere.invalid Thu Jul 11 17:12:10 2019 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 11 Jul 2019 21:12:10 GMT Subject: Seeking help regarding Python code References: Message-ID: On Thu, 11 Jul 2019 06:54:21 +0100, Debasree Banerjee wrote: > > I have a dataset like this: > > RecTime > > NO2_RAW > > NO2 > > LAQN_NO2 > > 10980 > > 06/6/19 01:45 > > 17.95444444 [snip] > > Can someone please help? Your question might appear intelligibly on the mailing list (I can't tell), but it is mangled into unintelligibility on the newsgroup. My guess is that instead of double-spaced mixed-type data fields, you intended to present an input table like this: RecTime NO2_RAW NO2 LAQN_NO2 10980 06/6/19 01:45 17.95444444 53.46255556 17.7 10981 06/6/19 01:45 17.94444444 53.44344444 17.7 10982 06/6/19 01:45 17.92111111 53.39877778 17.7 I really can't tell what you want the output to look like. -- To email me, substitute nowhere->runbox, invalid->com. From hlakyi77 at yahoo.com Thu Jul 11 21:52:43 2019 From: hlakyi77 at yahoo.com (Hla Kyi) Date: Fri, 12 Jul 2019 01:52:43 +0000 (UTC) Subject: Problem to delete or modify Python program and to read "signature.asc" References: <615809785.48117.1562896363681.ref@mail.yahoo.com> Message-ID: <615809785.48117.1562896363681@mail.yahoo.com> Dear Sir/ Madam, ? ? 1. I try to modify, some of the check boxes can not be selected.? ? 2. I try to uninstall, "successfully uninstall" message is come out. When I exit it "if you have any problem, please contact??python-list at python.org?" message is come out.?? ? I install Python 3.7.2 at my laptop in Windows 10 operating system.? ? I cannot get reply yet. I saw?"signature.asc" and I cannot read Best Regards, Kyi? From PythonList at DancesWithMice.info Thu Jul 11 23:28:22 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Fri, 12 Jul 2019 15:28:22 +1200 Subject: Problem to delete or modify Python program and to read "signature.asc" In-Reply-To: <615809785.48117.1562896363681@mail.yahoo.com> References: <615809785.48117.1562896363681.ref@mail.yahoo.com> <615809785.48117.1562896363681@mail.yahoo.com> Message-ID: <29cb4e0d-83cd-6a21-f040-41e9eb65c2f6@DancesWithMice.info> On 12/07/19 1:52 PM, Hla Kyi via Python-list wrote: > Dear Sir/ Madam, > ? ? 1. I try to modify, some of the check boxes can not be selected.? ? 2. I try to uninstall, "successfully uninstall" message is come out. When I exit it "if you have any problem, please contact??python-list at python.org?" message is come out.?? ? I install Python 3.7.2 at my laptop in Windows 10 operating system.? ? I cannot get reply yet. I saw?"signature.asc" and I cannot read > > Best Regards, > Kyi > Not true. I replied within one hour of your post, yesterday. <<< Please describe the problem you are having when you say "modify". It would also help if you identified your PC's operating system... >>> The information you have given is insufficient to identify a particular problem. -- Regards =dn From ben+python at benfinney.id.au Fri Jul 12 00:09:28 2019 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 12 Jul 2019 14:09:28 +1000 Subject: How Do You Replace Variables With Their Values? References: <86v9w9qoyw.fsf@benfinney.id.au> Message-ID: <86r26vrj0n.fsf@benfinney.id.au> Aldwin Pollefeyt writes: > dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main > Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} > > # Don't ask where I got the dinner from > > for meal in dinner.keys(): > exec(meal.replace(' ','_') + ' = list(dinner[meal])') > > print(Starters) > print(Main_Course) > print(Desert) Why do you think this is needed? Why (there may be some reason! but you have not told us what that is) can your program not just:: print(dinner['Starters']) print(dinner['Main Course']) print(dinner['Desert']) > OUTPUT: > ['Fried Calamari', 'Potted crab'] > ['Fish', 'Meat'] > ['Cake', 'Banana Split'] The above code produces this output, without any need for binding new names. So what is it you are actually trying to achieve, and why do you think the new bindings are necessary? -- \ ?The number of UNIX installations has grown to 10, with more | `\ expected.? ?Unix Programmer's Manual, 2nd Ed., 1972-06-12 | _o__) | Ben Finney From aldwinaldwindev at gmail.com Fri Jul 12 00:28:42 2019 From: aldwinaldwindev at gmail.com (Aldwin Pollefeyt) Date: Fri, 12 Jul 2019 12:28:42 +0800 Subject: How Do You Replace Variables With Their Values? In-Reply-To: <86r26vrj0n.fsf@benfinney.id.au> References: <86v9w9qoyw.fsf@benfinney.id.au> <86r26vrj0n.fsf@benfinney.id.au> Message-ID: Wow, I'm so sorry I answered on the question : "How do you replace a variable with its value". For what i understood with the example values, CrazyVideoGamez wants 3 variables named like the meal-names in dictionary. Yes, it's not secure unless you work with your own dataset (just like sending your own created commands with set=True in subprocess). Yes there might be better solutions for the real problem. But maybe the user really has a purpose for it, in a secure environment with own datatset, it's a valid answer for "How do you replace a variable with its value". On Fri, Jul 12, 2019 at 12:10 PM Ben Finney wrote: > Aldwin Pollefeyt writes: > > > dinner = {'Starters':['Fried Calamari', 'Potted crab'],'Main > > Course':['Fish', 'Meat'], 'Desert':['Cake', 'Banana Split']} > > > > # Don't ask where I got the dinner from > > > > for meal in dinner.keys(): > > exec(meal.replace(' ','_') + ' = list(dinner[meal])') > > > > print(Starters) > > print(Main_Course) > > print(Desert) > > Why do you think this is needed? Why (there may be some reason! but you > have not told us what that is) can your program not just:: > > print(dinner['Starters']) > print(dinner['Main Course']) > print(dinner['Desert']) > > > OUTPUT: > > ['Fried Calamari', 'Potted crab'] > > ['Fish', 'Meat'] > > ['Cake', 'Banana Split'] > > The above code produces this output, without any need for binding new > names. So what is it you are actually trying to achieve, and why do you > think the new bindings are necessary? > > -- > \ ?The number of UNIX installations has grown to 10, with more | > `\ expected.? ?Unix Programmer's Manual, 2nd Ed., 1972-06-12 | > _o__) | > Ben Finney > > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Fri Jul 12 01:07:46 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 12 Jul 2019 15:07:46 +1000 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: <86v9w9qoyw.fsf@benfinney.id.au> <86r26vrj0n.fsf@benfinney.id.au> Message-ID: On Fri, Jul 12, 2019 at 2:30 PM Aldwin Pollefeyt wrote: > > Wow, I'm so sorry I answered on the question : "How do you replace a > variable with its value". For what i understood with the example values, > CrazyVideoGamez wants 3 variables named like the meal-names in dictionary. > Yes, it's not secure unless you work with your own dataset (just like > sending your own created commands with set=True in subprocess). Yes there > might be better solutions for the real problem. But maybe the user really > has a purpose for it, in a secure environment with own datatset, it's a > valid answer for "How do you replace a variable with its value". > What you gave was dangerous advice, and yes, there IS a better solution - and an easier one. If you want to create variables dynamically, then just create them! for meal, parts in dinner.items(): globals()[meal.replace(' ','_')] = dinner[meal] Python has a rich set of metaprogramming tools. Don't just always reach for exec and caveat it with "it's okay if you trust everything". ChrisA From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Fri Jul 12 10:12:11 2019 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Fri, 12 Jul 2019 15:12:11 +0100 Subject: super or not super? Message-ID: Hi all! Is there any difference between using the base class name or super to call __init__ from base class? class C1: def __init__(self): ... class C2(C1): def __init__(self): C1.__init__(self) or super().__init__() ?? ... I have been using super, but I see some scripts where the base class name is used. Thanks for any comments. From mok888 at gmail.com Fri Jul 12 10:36:54 2019 From: mok888 at gmail.com (mok888 at gmail.com) Date: Fri, 12 Jul 2019 07:36:54 -0700 (PDT) Subject: Books for Python 3.7 Message-ID: Can anyone help me. New to Python. Installed version 3.7 I purchased the "Python for Dummies" book But this book was written for an older version of Python. All the examples and samples don't work with version 3.7 Can anyone direct me to which is the latest book to buy to properly learn Python. Thanks From sjmsoft at gmail.com Fri Jul 12 11:04:35 2019 From: sjmsoft at gmail.com (sjmsoft at gmail.com) Date: Fri, 12 Jul 2019 08:04:35 -0700 (PDT) Subject: Books for Python 3.7 In-Reply-To: References: Message-ID: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> On Friday, July 12, 2019 at 11:37:08 AM UTC-3, mok... at gmail.com wrote: > Can anyone help me. > New to Python. > Installed version 3.7 > I purchased the "Python for Dummies" book But this book was written for an older version of Python. > All the examples and samples don't work with version 3.7 > Can anyone direct me to which is the latest book to buy to properly learn Python. > Thanks You don't say what Python release your book covers, but the first thing to know is that Python 3 has many incompatibilities with Python 2. I don't think any book that covers Python 2 is useful for a new Python programmer using Python 3. You also don't say whether you're a beginning programmer or a veteran wanting to learn a new language. This might affect your book choice, as some are directed more toward beginning programmers. One good choice is Dive Into Python, available for free on-line (google "dive into python 3"). I'm sure other folks could suggest additional suitable Python 3 books. HTH, Steve J. Martin From rhodri at kynesim.co.uk Fri Jul 12 11:20:13 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Fri, 12 Jul 2019 16:20:13 +0100 Subject: super or not super? In-Reply-To: References: Message-ID: <60817a37-27ce-701a-de94-d9c06005c7cd@kynesim.co.uk> On 12/07/2019 15:12, Paulo da Silva wrote: > Hi all! > > Is there any difference between using the base class name or super to > call __init__ from base class? > > class C1: > def __init__(self): > ... > > class C2(C1): > def __init__(self): > C1.__init__(self) or super().__init__() ?? > ... > > I have been using super, but I see some scripts where the base class > name is used. For a simple single inheritance case like this there isn't any difference, but using super() a good habit to get into. Mostly it makes code maintenance easier: if you suddenly decide to change your base class to C3 (say for debug purposes), you only have to change one line: class C3: def __init__(self): ... class C2(C3): def __init__(self): super().__init__() Using the base class by name can lead to errors like this: class C2(C3): def __init__(self): C1.__init__(self) # Whoops, forgot this one super() also has major advantages if you are stuck with multiple inheritance. Raymond Hettinger has an excellent article on this here: https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ -- Rhodri James *-* Kynesim Ltd From mok888 at gmail.com Fri Jul 12 11:27:49 2019 From: mok888 at gmail.com (Richard Mok) Date: Fri, 12 Jul 2019 08:27:49 -0700 (PDT) Subject: Books for Python 3.7 In-Reply-To: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> References: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> Message-ID: <67d530e1-262f-4972-847c-18c6f5bf5152@googlegroups.com> On Friday, July 12, 2019 at 11:04:57 AM UTC-4, sjm... at gmail.com wrote: > On Friday, July 12, 2019 at 11:37:08 AM UTC-3, mok... at gmail.com wrote: > > Can anyone help me. > > New to Python. > > Installed version 3.7 > > I purchased the "Python for Dummies" book But this book was written for an older version of Python. > > All the examples and samples don't work with version 3.7 > > Can anyone direct me to which is the latest book to buy to properly learn Python. > > Thanks > > You don't say what Python release your book covers, but the first thing to know is that Python 3 has many incompatibilities with Python 2. I don't think any book that covers Python 2 is useful for a new Python programmer using Python 3. > > You also don't say whether you're a beginning programmer or a veteran wanting to learn a new language. This might affect your book choice, as some are directed more toward beginning programmers. > > One good choice is Dive Into Python, available for free on-line (google "dive into python 3"). I'm sure other folks could suggest additional suitable Python 3 books. > > HTH, > Steve J. Martin Hi Steve Thanks for the super fast response. It does not mention on the book which version of Python it is using. I assume it is for 2.7 Its written by Stef and Aahz Maruch I am a seasoned programmer but only on main frames computer using RPG, Cobol etc. I have no experience programming on a PC at all. That's why I bought the Dummies book, so that I can start from the very beginning. Will check out the Dive Into Python..... Thanks From tjreedy at udel.edu Fri Jul 12 11:40:27 2019 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 12 Jul 2019 11:40:27 -0400 Subject: Books for Python 3.7 In-Reply-To: <67d530e1-262f-4972-847c-18c6f5bf5152@googlegroups.com> References: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> <67d530e1-262f-4972-847c-18c6f5bf5152@googlegroups.com> Message-ID: On 7/12/2019 11:27 AM, Richard Mok wrote: > It does not mention on the book which version of Python it is using. That would likely mean 2.x. Easy way to tell: 2.x has 'print x' statements. 3.x has 'print(x)' function calles. -- Terry Jan Reedy From tjol at tjol.eu Fri Jul 12 10:30:11 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 12 Jul 2019 16:30:11 +0200 Subject: super or not super? In-Reply-To: References: Message-ID: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> On 12/07/2019 16.12, Paulo da Silva wrote: > Hi all! > > Is there any difference between using the base class name or super to > call __init__ from base class? There is, when multiple inheritance is involved. super() can call different 'branches' of the inheritance tree if necessary. Let me demonstrate: class A1: ??? def __init__(self): ??????? super().__init__() ??????? print('A1 called') class B1(A1): ??? def __init__(self): ??????? super().__init__() ??????? print('B1 called') class C1(A1): ??? def __init__(self): ??????? super().__init__() ??????? print('C1 called') class D1(B1,C1): ??? def __init__(self): ??????? super().__init__() ??????? print('D1 called') class A2: ??? def __init__(self): ??????? object.__init__(self) ??????? print('A2 called') class B2(A2): ??? def __init__(self): ??????? A2.__init__(self) ??????? print('B2 called') class C2(A2): ??? def __init__(self): ??????? A2.__init__(self) ??????? print('C2 called') class D2(B2,C2): ??? def __init__(self): ??????? super().__init__() ??????? print('D2 called') if __name__ == '__main__': ??? D1() ??? print('---') ??? D2() ############## % python3 super_demo.py A1 called C1 called B1 called D1 called --- A2 called B2 called D2 called > > class C1: > def __init__(self): > ... > > class C2(C1): > def __init__(self): > C1.__init__(self) or super().__init__() ?? > ... > > I have been using super, but I see some scripts where the base class > name is used. Just use super(), especially in __init__. From python at mrabarnett.plus.com Fri Jul 12 12:59:37 2019 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 12 Jul 2019 17:59:37 +0100 Subject: Books for Python 3.7 In-Reply-To: References: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> <67d530e1-262f-4972-847c-18c6f5bf5152@googlegroups.com> Message-ID: <888e2537-9438-aa3c-4d49-3987e1f056b1@mrabarnett.plus.com> On 2019-07-12 16:40, Terry Reedy wrote: > On 7/12/2019 11:27 AM, Richard Mok wrote: > >> It does not mention on the book which version of Python it is using. > > That would likely mean 2.x. Easy way to tell: > 2.x has 'print x' statements. 3.x has 'print(x)' function calles. > I had a brief look online and saw a preview. It was written in 2006 and Appendix B stops at Python 2.5. A lot has happened since then! From mok888 at gmail.com Fri Jul 12 13:44:29 2019 From: mok888 at gmail.com (RIchy M) Date: Fri, 12 Jul 2019 10:44:29 -0700 (PDT) Subject: Books for Python 3.7 In-Reply-To: References: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> <67d530e1-262f-4972-847c-18c6f5bf5152@googlegroups.com> <888e2537-9438-aa3c-4d49-3987e1f056b1@mrabarnett.plus.com> Message-ID: On Friday, July 12, 2019 at 1:00:01 PM UTC-4, MRAB wrote: > On 2019-07-12 16:40, Terry Reedy wrote: > > On 7/12/2019 11:27 AM, Richard Mok wrote: > > > >> It does not mention on the book which version of Python it is using. > > > > That would likely mean 2.x. Easy way to tell: > > 2.x has 'print x' statements. 3.x has 'print(x)' function calles. > > > I had a brief look online and saw a preview. It was written in 2006 and > Appendix B stops at Python 2.5. A lot has happened since then! I already stopped studying from this book. Now just reading the 3.7 tutorial that came with the install. But I am a beginner... Finding it tough to learn like that. I need like a beginners guide to Python 3.7 sort of book. From formisc at gmail.com Fri Jul 12 14:45:19 2019 From: formisc at gmail.com (Andrew Z) Date: Fri, 12 Jul 2019 14:45:19 -0400 Subject: Books for Python 3.7 In-Reply-To: References: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> <67d530e1-262f-4972-847c-18c6f5bf5152@googlegroups.com> <888e2537-9438-aa3c-4d49-3987e1f056b1@mrabarnett.plus.com> Message-ID: Richy, What specific part you consider hard? If i may suggest, get a (pet) project as you read it. On Fri, Jul 12, 2019, 13:46 RIchy M wrote: > On Friday, July 12, 2019 at 1:00:01 PM UTC-4, MRAB wrote: > > On 2019-07-12 16:40, Terry Reedy wrote: > > > On 7/12/2019 11:27 AM, Richard Mok wrote: > > > > > >> It does not mention on the book which version of Python it is using. > > > > > > That would likely mean 2.x. Easy way to tell: > > > 2.x has 'print x' statements. 3.x has 'print(x)' function calles. > > > > > I had a brief look online and saw a preview. It was written in 2006 and > > Appendix B stops at Python 2.5. A lot has happened since then! > > I already stopped studying from this book. > Now just reading the 3.7 tutorial that came with the install. > But I am a beginner... > Finding it tough to learn like that. > I need like a beginners guide to Python 3.7 sort of book. > -- > https://mail.python.org/mailman/listinfo/python-list > From mok888 at gmail.com Fri Jul 12 15:25:31 2019 From: mok888 at gmail.com (RIchy M) Date: Fri, 12 Jul 2019 12:25:31 -0700 (PDT) Subject: Books for Python 3.7 In-Reply-To: References: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> <67d530e1-262f-4972-847c-18c6f5bf5152@googlegroups.com> <888e2537-9438-aa3c-4d49-3987e1f056b1@mrabarnett.plus.com> Message-ID: <15bc571c-7dae-4798-a084-267dcf62d608@googlegroups.com> On Friday, July 12, 2019 at 2:45:48 PM UTC-4, Andrew Z wrote: > Richy, > What specific part you consider hard? > If i may suggest, get a (pet) project as you read it. > > On Fri, Jul 12, 2019, 13:46 RIchy M wrote: > > > On Friday, July 12, 2019 at 1:00:01 PM UTC-4, MRAB wrote: > > > On 2019-07-12 16:40, Terry Reedy wrote: > > > > On 7/12/2019 11:27 AM, Richard Mok wrote: > > > > > > > >> It does not mention on the book which version of Python it is using. > > > > > > > > That would likely mean 2.x. Easy way to tell: > > > > 2.x has 'print x' statements. 3.x has 'print(x)' function calles. > > > > > > > I had a brief look online and saw a preview. It was written in 2006 and > > > Appendix B stops at Python 2.5. A lot has happened since then! > > > > I already stopped studying from this book. > > Now just reading the 3.7 tutorial that came with the install. > > But I am a beginner... > > Finding it tough to learn like that. > > I need like a beginners guide to Python 3.7 sort of book. > > -- > > https://mail.python.org/mailman/listinfo/python-list > > Hi Andrew I feel that that guide is written for people who already have some basic knowledge of Python. I am on the beginners level only. What is a (pet) project? From info at wingware.com Fri Jul 12 15:37:42 2019 From: info at wingware.com (Wingware) Date: Fri, 12 Jul 2019 15:37:42 -0400 Subject: ANN: Wing Python IDE 7.0.4 Released Message-ID: <5D28E186.1060204@wingware.com> Wing Python IDE 7.0.4 has been released. Some of the highlights of this release include: Fix debugging notebooks with newer Jupyter versions Fix setting up a Django project with the default Python Executable Don't lose retained Debug I/O buffers after 60 seconds Avoid several incorrect code warnings Fix refactoring or finding points of use in code that has type hints Fix Move Program Counter in remote files Fix remote searching and Find Uses to include files that are not open in the editor Avoid displaying spurious Disk File is Newer dialogs when saving remote files Fix comparing two directories Avoid hanging up in file comparisons with large difference blocks Fix several problems in the extension scripting API Fix many other bugs You can obtain this release with Check for Updates in Wing 7's Help menu or download it now: https://wingware.com/downloads **New in Wing 7** Wing 7 introduces an improved code warnings and code quality inspection system that includes built-in error detection and tight integration with Pylint, pep8, and mypy. This release also adds a new data frame and array viewer, a MATLAB keyboard personality, easy inline debug data display with Shift-Space, improved stack data display, support for PEP 3134 chained exceptions, callouts for search and other code navigation features, four new color palettes, improved bookmarking, a high-level configuration menu, magnified presentation mode, a new update manager, stepping over import internals, simplified remote agent installation, and much more. For details see What's New in Wing 7: https://wingware.com/wingide/whatsnew **Try Wing 7 Now!** Wing 7 is a an exciting new step for Wingware's Python IDE product line. Find out how Wing 7 can turbocharge your Python development by trying it today. Download Wing 7.0.4: https://wingware.com/downloads Wing 7 installs side by side with earlier versions of Wing, so there is no need to remove old versions in order to try it. Wing 7 will read and convert your old preferences, settings, and projects. Projects should be saved to a new name since earlier versions of Wing cannot read Wing 7 projects. See Upgrading https://wingware.com/doc/install/upgrading for details and Migrating from Older Versions https://wingware.com/doc/install/migrating for a list of compatibility notes. From formisc at gmail.com Fri Jul 12 17:58:49 2019 From: formisc at gmail.com (Andrew Z) Date: Fri, 12 Jul 2019 17:58:49 -0400 Subject: Books for Python 3.7 In-Reply-To: <15bc571c-7dae-4798-a084-267dcf62d608@googlegroups.com> References: <4535e7c6-b3ac-48a1-8dde-0816c7a79618@googlegroups.com> <67d530e1-262f-4972-847c-18c6f5bf5152@googlegroups.com> <888e2537-9438-aa3c-4d49-3987e1f056b1@mrabarnett.plus.com> <15bc571c-7dae-4798-a084-267dcf62d608@googlegroups.com> Message-ID: Then look at, for example, tutorialpoint.com for basic concepts - loops, data structures, objects . Pet- project- something you want to build. For example, my current petproject is a android based clock with a voice recognition. Use case - clock should understand 2-3 commands to set time 8nterval and start/stop countdown. I have 0 knowledge in android and kotlin. So for me to get to voice recognition part i need to learn basics of the android and kotlin. On Fri, Jul 12, 2019, 15:35 RIchy M wrote: > On Friday, July 12, 2019 at 2:45:48 PM UTC-4, Andrew Z wrote: > > Richy, > > What specific part you consider hard? > > If i may suggest, get a (pet) project as you read it. > > > > On Fri, Jul 12, 2019, 13:46 RIchy M wrote: > > > > > On Friday, July 12, 2019 at 1:00:01 PM UTC-4, MRAB wrote: > > > > On 2019-07-12 16:40, Terry Reedy wrote: > > > > > On 7/12/2019 11:27 AM, Richard Mok wrote: > > > > > > > > > >> It does not mention on the book which version of Python it is > using. > > > > > > > > > > That would likely mean 2.x. Easy way to tell: > > > > > 2.x has 'print x' statements. 3.x has 'print(x)' function calles. > > > > > > > > > I had a brief look online and saw a preview. It was written in 2006 > and > > > > Appendix B stops at Python 2.5. A lot has happened since then! > > > > > > I already stopped studying from this book. > > > Now just reading the 3.7 tutorial that came with the install. > > > But I am a beginner... > > > Finding it tough to learn like that. > > > I need like a beginners guide to Python 3.7 sort of book. > > > -- > > > https://mail.python.org/mailman/listinfo/python-list > > > > > Hi Andrew > > I feel that that guide is written for people who already have some basic > knowledge of Python. > I am on the beginners level only. > > What is a (pet) project? > -- > https://mail.python.org/mailman/listinfo/python-list > From arj.python at gmail.com Sat Jul 13 05:54:09 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sat, 13 Jul 2019 13:54:09 +0400 Subject: Accumulate , Range and Zeros Message-ID: Greetings, Given this snippet from itertools import * import operator x = [1, 2, 3] # [0, 1, 2, 3, ..., 10] y = accumulate(x, operator.mul) print(list(y)) why does x = list(range(5)) produces only zeros? -- Abdur-Rahmaan Janhangeer http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ Mauritius From frank at chagford.com Sat Jul 13 06:49:45 2019 From: frank at chagford.com (Frank Millman) Date: Sat, 13 Jul 2019 12:49:45 +0200 Subject: Accumulate , Range and Zeros In-Reply-To: References: Message-ID: On 2019-07-13 11:54 AM, Abdur-Rahmaan Janhangeer wrote: > Greetings, > > Given this snippet > > from itertools import * > import operator > > > x = [1, 2, 3] # [0, 1, 2, 3, ..., 10] > > y = accumulate(x, operator.mul) > > print(list(y)) > > why does x = list(range(5)) produces only zeros? > That is an easy one. By default, range() starts from 0. Anything multiplied by 0 equals 0. So you can multiply as many numbers as you like, if the first one is 0, the rest will also be 0. QED Frank Millman From frank at chagford.com Sat Jul 13 06:49:45 2019 From: frank at chagford.com (Frank Millman) Date: Sat, 13 Jul 2019 12:49:45 +0200 Subject: Accumulate , Range and Zeros In-Reply-To: References: Message-ID: On 2019-07-13 11:54 AM, Abdur-Rahmaan Janhangeer wrote: > Greetings, > > Given this snippet > > from itertools import * > import operator > > > x = [1, 2, 3] # [0, 1, 2, 3, ..., 10] > > y = accumulate(x, operator.mul) > > print(list(y)) > > why does x = list(range(5)) produces only zeros? > That is an easy one. By default, range() starts from 0. Anything multiplied by 0 equals 0. So you can multiply as many numbers as you like, if the first one is 0, the rest will also be 0. QED Frank Millman From 2QdxY4RzWzUUiLuE at potatochowder.com Sat Jul 13 06:30:35 2019 From: 2QdxY4RzWzUUiLuE at potatochowder.com (Dan Sommers) Date: Sat, 13 Jul 2019 06:30:35 -0400 Subject: Accumulate , Range and Zeros In-Reply-To: References: Message-ID: <61fcc3a1-d975-9b44-5081-9a29180cffdb@potatochowder.com> On 7/13/19 5:54 AM, Abdur-Rahmaan Janhangeer wrote: > > Greetings, > > > > Given this snippet > > > > from itertools import * > > import operator > > > > > > x = [1, 2, 3] # [0, 1, 2, 3, ..., 10] > > > > y = accumulate(x, operator.mul) > > > > print(list(y)) > > > > why does x = list(range(5)) produces only zeros? I see two things going on here. (1) Don't type snippets of code and results from memory, or in bits and pieces from an interactive session. Copy and paste exactly the code that ran and the output that it produced. (2) What is range(5)? Okay, what is list(range(5))? What do you (the person) get when you multiply those five integers together? HTH, Dan From arj.python at gmail.com Sat Jul 13 07:22:46 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sat, 13 Jul 2019 15:22:46 +0400 Subject: Accumulate , Range and Zeros In-Reply-To: References: Message-ID: @Frank Sooooo simple. Thanks! Abdur-Rahmaan Janhangeer http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ Mauritius From tjol at tjol.eu Sat Jul 13 06:44:15 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Sat, 13 Jul 2019 12:44:15 +0200 Subject: Accumulate , Range and Zeros In-Reply-To: References: Message-ID: <696cb0a4-426c-44af-e52f-fb3fc36c08b4@tjol.eu> On 13/07/2019 11:54, Abdur-Rahmaan Janhangeer wrote: > Greetings, > > Given this snippet > > from itertools import * > import operator > > > x = [1, 2, 3] # [0, 1, 2, 3, ..., 10] > > y = accumulate(x, operator.mul) > > print(list(y)) > > why does x = list(range(5)) produces only zeros? What would you expect it to produce? From arj.python at gmail.com Sat Jul 13 08:00:58 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sat, 13 Jul 2019 16:00:58 +0400 Subject: Accumulate , Range and Zeros In-Reply-To: <696cb0a4-426c-44af-e52f-fb3fc36c08b4@tjol.eu> References: <696cb0a4-426c-44af-e52f-fb3fc36c08b4@tjol.eu> Message-ID: @Thomas thought was a generator bug since instead of returning the usual nums it was returning 0 0 0 ... From rosuav at gmail.com Sat Jul 13 08:24:31 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 13 Jul 2019 22:24:31 +1000 Subject: Accumulate , Range and Zeros In-Reply-To: References: <696cb0a4-426c-44af-e52f-fb3fc36c08b4@tjol.eu> Message-ID: On Sat, Jul 13, 2019 at 10:02 PM Abdur-Rahmaan Janhangeer wrote: > > @Thomas thought was a generator bug since instead of returning the usual > nums it was returning 0 0 0 ... When you find a bug or strange bit of behaviour, first make sure you can reproduce it. Then create a small, self-contained program, just enough to demonstrate what's going on. Post that code. Don't make us guess :) ChrisA From arj.python at gmail.com Sat Jul 13 08:58:49 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sat, 13 Jul 2019 16:58:49 +0400 Subject: Accumulate , Range and Zeros In-Reply-To: References: <696cb0a4-426c-44af-e52f-fb3fc36c08b4@tjol.eu> Message-ID: oh not a real bug, i thought the effect had to do with generator working rather a simple times 0 Abdur-Rahmaan Janhangeer Mauritius On Sat, 13 Jul 2019, 16:24 Chris Angelico, wrote: > On Sat, Jul 13, 2019 at 10:02 PM Abdur-Rahmaan Janhangeer > wrote: > > > > @Thomas thought was a generator bug since instead of returning the usual > > nums it was returning 0 0 0 ... > > When you find a bug or strange bit of behaviour, first make sure you > can reproduce it. Then create a small, self-contained program, just > enough to demonstrate what's going on. Post that code. Don't make us > guess :) > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From mok888 at gmail.com Sat Jul 13 10:08:22 2019 From: mok888 at gmail.com (RIchy M) Date: Sat, 13 Jul 2019 07:08:22 -0700 (PDT) Subject: Books for Python 3.7 In-Reply-To: References: Message-ID: On Friday, July 12, 2019 at 7:34:10 PM UTC-4, Louis Krupp wrote: > On Fri, 12 Jul 2019 07:36:54 -0700 (PDT), mok888 at gmail.com wrote: > > >Can anyone help me. > >New to Python. > >Installed version 3.7 > >I purchased the "Python for Dummies" book But this book was written for an older version of Python. > >All the examples and samples don't work with version 3.7 > >Can anyone direct me to which is the latest book to buy to properly learn Python. > >Thanks > > I found _Introducing Python_ useful: > > http://shop.oreilly.com/product/0636920028659.do > > It's also available on Amazon. > > Louis Thanks you all for all the advice. At least now I know there is a Python 2 or 3. I just bought a book and jumped right in without knowing all the facts first. I found a free online course by Guru99 that I am trying out now. I just need some basic knowledge to start of with. And as Andrew Z suggested, its good to have a Pet Project. I will do that as soon as I feel more comfortable with Python. Thanks Louis for the book recommendation..... From mahdiumd at gmail.com Sun Jul 14 12:20:56 2019 From: mahdiumd at gmail.com (Matt Zand) Date: Sun, 14 Jul 2019 12:20:56 -0400 Subject: Python string with character exchange Message-ID: I am new to Python. I am trying to solve below Python question: Given a string, return a new string where the first and last chars have been exchanged. -- Cheers, Matt Zand Cell: 202-420-9192 Work: 240-200-6131 High School Technology Services DC Web Makers Coding Bootcamps From reto at labrat.space Sun Jul 14 14:26:43 2019 From: reto at labrat.space (Reto) Date: Sun, 14 Jul 2019 20:26:43 +0200 Subject: Python string with character exchange In-Reply-To: References: Message-ID: <20190714182643.lyeh3dgobr2wjeqb@feather.localdomain> On Sun, Jul 14, 2019 at 12:20:56PM -0400, Matt Zand wrote: > Given a string, return a new string where the first and last chars have > been exchanged. This sounds awfully like a homework question. What did you try? What concepts are you missing? Did you already look into slicing / getting element from a list? That will be what you need in the end. You can easily get a character from a string like this ``` a = "some random string" print(a[0]) ``` strings are immutable in python, meaning you will need to create a new string. From ian.g.kelly at gmail.com Sun Jul 14 15:43:19 2019 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Sun, 14 Jul 2019 13:43:19 -0600 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: <86v9w9qoyw.fsf@benfinney.id.au> <86r26vrj0n.fsf@benfinney.id.au> Message-ID: On Thu, Jul 11, 2019 at 11:10 PM Chris Angelico wrote: > > On Fri, Jul 12, 2019 at 2:30 PM Aldwin Pollefeyt > wrote: > > > > Wow, I'm so sorry I answered on the question : "How do you replace a > > variable with its value". For what i understood with the example values, > > CrazyVideoGamez wants 3 variables named like the meal-names in dictionary. > > Yes, it's not secure unless you work with your own dataset (just like > > sending your own created commands with set=True in subprocess). Yes there > > might be better solutions for the real problem. But maybe the user really > > has a purpose for it, in a secure environment with own datatset, it's a > > valid answer for "How do you replace a variable with its value". > > > > What you gave was dangerous advice, and yes, there IS a better > solution - and an easier one. If you want to create variables > dynamically, then just create them! > > for meal, parts in dinner.items(): > globals()[meal.replace(' ','_')] = dinner[meal] > > Python has a rich set of metaprogramming tools. Don't just always > reach for exec and caveat it with "it's okay if you trust everything". To be fair, if dinner is untrusted then this new version is still unsafe. You've just allowed it to shadow any global or built-in it wants to. From rosuav at gmail.com Sun Jul 14 15:59:42 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 15 Jul 2019 05:59:42 +1000 Subject: How Do You Replace Variables With Their Values? In-Reply-To: References: <86v9w9qoyw.fsf@benfinney.id.au> <86r26vrj0n.fsf@benfinney.id.au> Message-ID: On Mon, Jul 15, 2019 at 5:45 AM Ian Kelly wrote: > > On Thu, Jul 11, 2019 at 11:10 PM Chris Angelico wrote: > > > > On Fri, Jul 12, 2019 at 2:30 PM Aldwin Pollefeyt > > wrote: > > > > > > Wow, I'm so sorry I answered on the question : "How do you replace a > > > variable with its value". For what i understood with the example values, > > > CrazyVideoGamez wants 3 variables named like the meal-names in > dictionary. > > > Yes, it's not secure unless you work with your own dataset (just like > > > sending your own created commands with set=True in subprocess). Yes > there > > > might be better solutions for the real problem. But maybe the user > really > > > has a purpose for it, in a secure environment with own datatset, it's a > > > valid answer for "How do you replace a variable with its value". > > > > > > > What you gave was dangerous advice, and yes, there IS a better > > solution - and an easier one. If you want to create variables > > dynamically, then just create them! > > > > for meal, parts in dinner.items(): > > globals()[meal.replace(' ','_')] = dinner[meal] > > > > Python has a rich set of metaprogramming tools. Don't just always > > reach for exec and caveat it with "it's okay if you trust everything". > > To be fair, if dinner is untrusted then this new version is still unsafe. > You've just allowed it to shadow any global or built-in it wants to. This is true, but that risk is in the original too. What I'd ACTUALLY do, in this sort of situation, would be to make some sort of namespace object, so I can write foo.Desert rather than using square bracket notation; otherwise, though, I'd just stick with the original. But if you want to unpack an object into a namespace, it's certainly better to assign directly into the namespace than to eval. ChrisA From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sun Jul 14 20:45:27 2019 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 15 Jul 2019 01:45:27 +0100 Subject: super or not super? References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> Message-ID: ?s 15:30 de 12/07/19, Thomas Jollans escreveu: > On 12/07/2019 16.12, Paulo da Silva wrote: >> Hi all! >> >> Is there any difference between using the base class name or super to >> call __init__ from base class? > > There is, when multiple inheritance is involved. super() can call > different 'branches' of the inheritance tree if necessary. > ... Thank you Jollans. I forgot multiple inheritance. I never needed it in python, so far. Regards From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Sun Jul 14 20:47:38 2019 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Mon, 15 Jul 2019 01:47:38 +0100 Subject: super or not super? References: <60817a37-27ce-701a-de94-d9c06005c7cd@kynesim.co.uk> Message-ID: ?s 16:20 de 12/07/19, Rhodri James escreveu: > On 12/07/2019 15:12, Paulo da Silva wrote: > ... > super() also has major advantages if you are stuck with multiple > inheritance.? Raymond Hettinger has an excellent article on this here: > https://rhettinger.wordpress.com/2011/05/26/super-considered-super/ > Thank you. I'll take a look at the suggestion. From rosuav at gmail.com Sun Jul 14 21:11:45 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 15 Jul 2019 11:11:45 +1000 Subject: super or not super? In-Reply-To: References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> Message-ID: On Mon, Jul 15, 2019 at 10:51 AM Paulo da Silva wrote: > > ?s 15:30 de 12/07/19, Thomas Jollans escreveu: > > On 12/07/2019 16.12, Paulo da Silva wrote: > >> Hi all! > >> > >> Is there any difference between using the base class name or super to > >> call __init__ from base class? > > > > There is, when multiple inheritance is involved. super() can call > > different 'branches' of the inheritance tree if necessary. > > ... > > Thank you Jollans. I forgot multiple inheritance. I never needed it in > python, so far. > Something to consider is that super() becomes useful even if someone else uses MI involving your class. Using super() ensures that your class will play nicely in someone else's hierarchy, not just your own. ChrisA From kumarscheen at gmail.com Mon Jul 15 02:44:08 2019 From: kumarscheen at gmail.com (kumarscheen at gmail.com) Date: Sun, 14 Jul 2019 23:44:08 -0700 (PDT) Subject: Want to learn python as I have donne manual testing for 12 years. Please help to share opinion how to start. Thanks Message-ID: <0ec14c02-15fc-436f-8372-5ec4c8005ae0@googlegroups.com> Want to learn python as I have donne manual testing for 12 years. Please help to share opinion how to start. Thanks From blmadhavan at gmail.com Mon Jul 15 02:59:33 2019 From: blmadhavan at gmail.com (Madhavan Bomidi) Date: Sun, 14 Jul 2019 23:59:33 -0700 (PDT) Subject: How to sort the files based on the date? Message-ID: <5419bf74-2411-4283-9969-0d41cb7ba7fa@googlegroups.com> I am using the following command line to sort the files: import glob a = sorted(glob.glob('3RIMG_*.h5') Following is the result: 3RIMG_01APR2018_0514_L2G_AOD.h5 3RIMG_01APR2018_0544_L2G_AOD.h5 3RIMG_01APR2018_0644_L2G_AOD.h5 3RIMG_01APR2018_0714_L2G_AOD.h5 3RIMG_01APR2018_0744_L2G_AOD.h5 3RIMG_01MAY2018_0515_L2G_AOD.h5 3RIMG_01MAY2018_0545_L2G_AOD.h5 3RIMG_01MAY2018_0615_L2G_AOD.h5 3RIMG_01MAY2018_0645_L2G_AOD.h5 3RIMG_01MAY2018_0715_L2G_AOD.h5 3RIMG_01MAY2018_0745_L2G_AOD.h5 3RIMG_01MAY2018_0815_L2G_AOD.h5 3RIMG_02APR2018_0514_L2G_AOD.h5 3RIMG_02APR2018_0544_L2G_AOD.h5 3RIMG_02APR2018_0614_L2G_AOD.h5 3RIMG_02APR2018_0644_L2G_AOD.h5 3RIMG_02APR2018_0714_L2G_AOD.h5 3RIMG_02APR2018_0744_L2G_AOD.h5 3RIMG_02APR2018_0814_L2G_AOD.h5 3RIMG_02MAY2018_0515_L2G_AOD.h5 3RIMG_02MAY2018_0545_L2G_AOD.h5 3RIMG_02MAY2018_0615_L2G_AOD.h5 3RIMG_02MAY2018_0645_L2G_AOD.h5 3RIMG_02MAY2018_0715_L2G_AOD.h5 3RIMG_02MAY2018_0745_L2G_AOD.h5 3RIMG_02MAY2018_0815_L2G_AOD.h5 3RIMG_31MAR2018_0514_L2G_AOD.h5 3RIMG_31MAR2018_0544_L2G_AOD.h5 3RIMG_31MAR2018_0614_L2G_AOD.h5 3RIMG_31MAR2018_0644_L2G_AOD.h5 3RIMG_31MAR2018_0714_L2G_AOD.h5 3RIMG_31MAR2018_0744_L2G_AOD.h5 As you can see from the list of files displayed after sorting, the files are not arranged according to the increasing order of the date since the date is in the DDMMMYYYY format. Can anyone suggest me how I can sort theses files in increasing order of the date on the file name? Thanks in advance From __peter__ at web.de Mon Jul 15 03:20:51 2019 From: __peter__ at web.de (Peter Otten) Date: Mon, 15 Jul 2019 09:20:51 +0200 Subject: How to sort the files based on the date? References: <5419bf74-2411-4283-9969-0d41cb7ba7fa@googlegroups.com> Message-ID: Madhavan Bomidi wrote: > I am using the following command line to sort the files: > > import glob > a = sorted(glob.glob('3RIMG_*.h5') > > Following is the result: > > 3RIMG_01APR2018_0514_L2G_AOD.h5 > 3RIMG_01APR2018_0544_L2G_AOD.h5 > 3RIMG_01APR2018_0644_L2G_AOD.h5 > 3RIMG_01APR2018_0714_L2G_AOD.h5 > 3RIMG_01APR2018_0744_L2G_AOD.h5 > 3RIMG_01MAY2018_0515_L2G_AOD.h5 > 3RIMG_01MAY2018_0545_L2G_AOD.h5 > 3RIMG_01MAY2018_0615_L2G_AOD.h5 > 3RIMG_01MAY2018_0645_L2G_AOD.h5 > 3RIMG_01MAY2018_0715_L2G_AOD.h5 > 3RIMG_01MAY2018_0745_L2G_AOD.h5 > 3RIMG_01MAY2018_0815_L2G_AOD.h5 > 3RIMG_02APR2018_0514_L2G_AOD.h5 > 3RIMG_02APR2018_0544_L2G_AOD.h5 > 3RIMG_02APR2018_0614_L2G_AOD.h5 > 3RIMG_02APR2018_0644_L2G_AOD.h5 > 3RIMG_02APR2018_0714_L2G_AOD.h5 > 3RIMG_02APR2018_0744_L2G_AOD.h5 > 3RIMG_02APR2018_0814_L2G_AOD.h5 > 3RIMG_02MAY2018_0515_L2G_AOD.h5 > 3RIMG_02MAY2018_0545_L2G_AOD.h5 > 3RIMG_02MAY2018_0615_L2G_AOD.h5 > 3RIMG_02MAY2018_0645_L2G_AOD.h5 > 3RIMG_02MAY2018_0715_L2G_AOD.h5 > 3RIMG_02MAY2018_0745_L2G_AOD.h5 > 3RIMG_02MAY2018_0815_L2G_AOD.h5 > 3RIMG_31MAR2018_0514_L2G_AOD.h5 > 3RIMG_31MAR2018_0544_L2G_AOD.h5 > 3RIMG_31MAR2018_0614_L2G_AOD.h5 > 3RIMG_31MAR2018_0644_L2G_AOD.h5 > 3RIMG_31MAR2018_0714_L2G_AOD.h5 > 3RIMG_31MAR2018_0744_L2G_AOD.h5 > > As you can see from the list of files displayed after sorting, the files > are not arranged according to the increasing order of the date since the > date is in the DDMMMYYYY format. > > Can anyone suggest me how I can sort theses files in increasing order of > the date on the file name? Use a key function filenames = sorted(filename, key=get_datetime) get_datetime should extract date/time from the filename into a datetime object. The names will then be ordered according to the datetimes' values: import glob import datetime def get_datetime(filename): parts = filename.split("_") return datetime.datetime.strptime(parts[1] + parts[2], "%d%b%Y%H%M") filenames = sorted(glob.glob('3RIMG_*.h5'), key=get_datetime) for fn in filenames: print(fn) From aldwinaldwindev at gmail.com Mon Jul 15 03:49:33 2019 From: aldwinaldwindev at gmail.com (Aldwin Pollefeyt) Date: Mon, 15 Jul 2019 15:49:33 +0800 Subject: Want to learn python as I have donne manual testing for 12 years. Please help to share opinion how to start. Thanks In-Reply-To: <0ec14c02-15fc-436f-8372-5ec4c8005ae0@googlegroups.com> References: <0ec14c02-15fc-436f-8372-5ec4c8005ae0@googlegroups.com> Message-ID: Try to find small projects to solve with Python instead of using other applications. Hereby my experience: * solve or just represent a riddle or mathematical question. - Youtube channels called standupmaths and numberphile has some interesting videos about algorithms. Then it's fun trying to build those and see what happens. not only numerical results but also plot it into graphs or create images from it - Every year General Intelligence and Security Service of the Netherlands releases a series of puzzles around Christmas. It's fun to try to create some decrypting tools around it and run them on top of the questions. Can even automatically crawl over some wiki pages to create lists of names for possible answers if you suspect it's a crossword full of encrypted nobel prize winners. - or find some websites with riddles and crosswords or just on hackerrank.com * show headlines of a newspaper in the format you want in a terminal using a rss feedparser * a ubuntu desktop indicator with your favorite stock market price or currency exchange or the time you logged into work in the morning * write your own api or webservice around all info that you learned to gather from multiple places * a IRC bot that put's all the info of other scripts that interest you (RSS, stock market, cpu usage, cheatsheets, python bugs, ...) on your own IRC channel (.get stock_info || .get python_snippets strings) so you can all info in one place * write a part of a rubiks cube algorithm to understand the changing parts to solve a small part * solve a sudoku you had trouble with in the newspaper last week and still bothers you * start using some threading to solve puzzles faster if possible. * send yourself a alert email when some disk is 80% full or when some idol of you tweeted again * just follow this mailing list or stackoverflow (python tags) and try to understand the code examples that pass by ... with just some daily tasks or solving questions, in the end, you learn first some basic functions and algorithms, get some info from a webpage (wiki) or a open api (weather info?), connect a socket to pass data, plot some data on a chart/image, show info in your OS's status bar, write your own api, threading, ... On Mon, Jul 15, 2019 at 3:00 PM wrote: > Want to learn python as I have donne manual testing for 12 years. Please > help to share opinion how to start. Thanks > -- > https://mail.python.org/mailman/listinfo/python-list > From blmadhavan at gmail.com Mon Jul 15 04:09:55 2019 From: blmadhavan at gmail.com (Madhavan Bomidi) Date: Mon, 15 Jul 2019 01:09:55 -0700 (PDT) Subject: How to sort the files based on the date? In-Reply-To: References: <5419bf74-2411-4283-9969-0d41cb7ba7fa@googlegroups.com> Message-ID: <87fdee98-9f80-4e7c-ac29-827ee526acc5@googlegroups.com> Thanks Peter. The following lines worked: import glob import datetime def get_datetime (filename): parts = filename.split ("_") return datetime.datetime.strptime (parts[1] + parts[2], "%d%b%Y%H%M") filenames = sorted(glob.glob('3RIMG _ *. h5'), key = get_datetime) From Richard at Damon-Family.org Mon Jul 15 06:45:26 2019 From: Richard at Damon-Family.org (Richard Damon) Date: Mon, 15 Jul 2019 06:45:26 -0400 Subject: How to sort the files based on the date? In-Reply-To: <5419bf74-2411-4283-9969-0d41cb7ba7fa@googlegroups.com> References: <5419bf74-2411-4283-9969-0d41cb7ba7fa@googlegroups.com> Message-ID: On 7/15/19 2:59 AM, Madhavan Bomidi wrote: > I am using the following command line to sort the files: > > import glob > a = sorted(glob.glob('3RIMG_*.h5') > > Following is the result: > > 3RIMG_01APR2018_0514_L2G_AOD.h5 > 3RIMG_01APR2018_0544_L2G_AOD.h5 > 3RIMG_01APR2018_0644_L2G_AOD.h5 > 3RIMG_01APR2018_0714_L2G_AOD.h5 > 3RIMG_01APR2018_0744_L2G_AOD.h5 > 3RIMG_01MAY2018_0515_L2G_AOD.h5 > 3RIMG_01MAY2018_0545_L2G_AOD.h5 > 3RIMG_01MAY2018_0615_L2G_AOD.h5 > 3RIMG_01MAY2018_0645_L2G_AOD.h5 > 3RIMG_01MAY2018_0715_L2G_AOD.h5 > 3RIMG_01MAY2018_0745_L2G_AOD.h5 > 3RIMG_01MAY2018_0815_L2G_AOD.h5. > 3RIMG_02APR2018_0514_L2G_AOD.h5 > 3RIMG_02APR2018_0544_L2G_AOD.h5 > 3RIMG_02APR2018_0614_L2G_AOD.h5 > 3RIMG_02APR2018_0644_L2G_AOD.h5 > 3RIMG_02APR2018_0714_L2G_AOD.h5 > 3RIMG_02APR2018_0744_L2G_AOD.h5 > 3RIMG_02APR2018_0814_L2G_AOD.h5 > 3RIMG_02MAY2018_0515_L2G_AOD.h5 > 3RIMG_02MAY2018_0545_L2G_AOD.h5 > 3RIMG_02MAY2018_0615_L2G_AOD.h5 > 3RIMG_02MAY2018_0645_L2G_AOD.h5 > 3RIMG_02MAY2018_0715_L2G_AOD.h5 > 3RIMG_02MAY2018_0745_L2G_AOD.h5 > 3RIMG_02MAY2018_0815_L2G_AOD.h5 > 3RIMG_31MAR2018_0514_L2G_AOD.h5 > 3RIMG_31MAR2018_0544_L2G_AOD.h5 > 3RIMG_31MAR2018_0614_L2G_AOD.h5 > 3RIMG_31MAR2018_0644_L2G_AOD.h5 > 3RIMG_31MAR2018_0714_L2G_AOD.h5 > 3RIMG_31MAR2018_0744_L2G_AOD.h5 > > As you can see from the list of files displayed after sorting, the files are not arranged according to the increasing order of the date since the date is in the DDMMMYYYY format. > > Can anyone suggest me how I can sort theses files in increasing order of the date on the file name? > > Thanks in advance One quick comment which may or may not be applicable, as it relies on changing the file name format would be to switch the file names to using something like an ISO date format fo the name, ISO dates are in the format YYYY-MM-DD (Month is a two digit number) which has the nice attribute that the alphabetical sort is also a date sort. If you can't do this, then as others have pointed out, you need to parse out the date from the filename, and use that as the sort key. -- Richard Damon From mahdiumd at gmail.com Mon Jul 15 08:55:51 2019 From: mahdiumd at gmail.com (Matt Zand) Date: Mon, 15 Jul 2019 08:55:51 -0400 Subject: Python string with character exchange In-Reply-To: <20190714182643.lyeh3dgobr2wjeqb@feather.localdomain> References: <20190714182643.lyeh3dgobr2wjeqb@feather.localdomain> Message-ID: Thanks Reto. I got it now. Matt On Sun, Jul 14, 2019 at 2:26 PM Reto wrote: > On Sun, Jul 14, 2019 at 12:20:56PM -0400, Matt Zand wrote: > > Given a string, return a new string where the first and last chars have > > been exchanged. > > This sounds awfully like a homework question. > What did you try? > What concepts are you missing? > > Did you already look into slicing / getting element from a list? > That will be what you need in the end. > > You can easily get a character from a string like this > > ``` > a = "some random string" > print(a[0]) > ``` > > strings are immutable in python, meaning you will need to create a new > string. > -- Cheers, Matt Zand Cell: 202-420-9192 Work: 240-200-6131 High School Technology Services DC Web Makers Coding Bootcamps From copeterson07 at gmail.com Mon Jul 15 11:00:18 2019 From: copeterson07 at gmail.com (copeterson07 at gmail.com) Date: Mon, 15 Jul 2019 08:00:18 -0700 (PDT) Subject: Want to learn python as I have donne manual testing for 12 years. Please help to share opinion how to start. Thanks In-Reply-To: <0ec14c02-15fc-436f-8372-5ec4c8005ae0@googlegroups.com> References: <0ec14c02-15fc-436f-8372-5ec4c8005ae0@googlegroups.com> Message-ID: On Monday, July 15, 2019 at 12:44:20 AM UTC-6, kumar... at gmail.com wrote: > Want to learn python as I have donne manual testing for 12 years. Please help to share opinion how to start. Thanks I don't know your skill level with programming, but I have found this https://www.learnpython.org/ website most useful. It allows you to get started quickly without having to setup a python development environment. If you are intending to do test automation, then I would also recommend learning https://robotframework.org/. The Gauge Test Framework is also valuable to learn https://gauge.org/ From pkpearson at nowhere.invalid Mon Jul 15 12:59:12 2019 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 15 Jul 2019 16:59:12 GMT Subject: How to sort the files based on the date? References: <5419bf74-2411-4283-9969-0d41cb7ba7fa@googlegroups.com> Message-ID: On Mon, 15 Jul 2019 09:20:51 +0200, Peter Otten <__peter__ at web.de> wrote: > Madhavan Bomidi wrote: > [snip] >> >> 3RIMG_01APR2018_0514_L2G_AOD.h5 >> 3RIMG_01APR2018_0544_L2G_AOD.h5 >> 3RIMG_01APR2018_0644_L2G_AOD.h5 >> 3RIMG_01APR2018_0714_L2G_AOD.h5 >> 3RIMG_01APR2018_0744_L2G_AOD.h5 [snip] >> >> Can anyone suggest me how I can sort theses files in increasing order of >> the date on the file name? > > Use a key function > > filenames = sorted(filename, key=get_datetime) > > get_datetime should extract date/time from the filename into a datetime > object. The names will then be ordered according to the datetimes' values: > > import glob > import datetime > > def get_datetime(filename): > parts = filename.split("_") > return datetime.datetime.strptime(parts[1] + parts[2], "%d%b%Y%H%M") > > filenames = sorted(glob.glob('3RIMG_*.h5'), key=get_datetime) > > for fn in filenames: > print(fn) Gorgeous. This is the best newsgroup ever. -- To email me, substitute nowhere->runbox, invalid->com. From barry at barrys-emacs.org Mon Jul 15 14:31:34 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Mon, 15 Jul 2019 19:31:34 +0100 Subject: super or not super? In-Reply-To: References: Message-ID: <59BEDF33-9559-4C96-A096-258B2E2A56DC@barrys-emacs.org> > On 12 Jul 2019, at 15:12, Paulo da Silva wrote: > > Hi all! > > Is there any difference between using the base class name or super to > call __init__ from base class? > > class C1: > def __init__(self): > ... > > class C2(C1): > def __init__(self): > C1.__init__(self) or super().__init__() ?? > ... > > I have been using super, but I see some scripts where the base class > name is used. If you get into the habit of always using super() you will avoid the cases where its critical and you did not use it. super() is always right (I think by design) but calling C1.__init__(self) can go wrong and is often hard to debug. Barry > > Thanks for any comments. > -- > https://mail.python.org/mailman/listinfo/python-list > From barry at barrys-emacs.org Mon Jul 15 14:52:15 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Mon, 15 Jul 2019 19:52:15 +0100 Subject: PyPi twine check does not like my README.rst files anymore Message-ID: <719984C3-5731-4587-9DA3-F3D5B662C59A@barrys-emacs.org> I am update some PyPI projects and found that twine was refusing the upload. I have been googling and reading the PyPI guides but did not find anthing that helps me understand what is required. What am I doing wrong? Using the twine check command I get reports about the reasons. All the code on https://github.com/barry-scott/namedstruct where you can see the README.rst rendered in full. Where as at https://pypi.org/project/namedstruct/ only the first line is rendered. Checking distribution dist\namedstruct-1.2.1-py3-none-any.whl: warning: `long_description_content_type` missing. defaulting to `text/x-rst`. Passed Checking distribution dist\namedstruct-1.2.1.tar.gz: warning: `long_description_content_type` missing. defaulting to `text/x-rst`. Failed The project's long_description has invalid markup which will not be rendered on PyPI. The following syntax errors were detected: line 1: Severe: Unexpected section title or transition. The part of setup.py that does this is: setup( name='namedstruct', version=open('version.txt').read().strip(), description='namedstruct encapsulates struct.unpack() with results accessed by name', long_description=long_description, long_description_content_type='text/x-rst', # The project's main homepage. url='https://github.com/barry-scott/namedstruct', Where should I have added the content type? The other reason is the show stopper. It does not allow me to have sections in my README.rst. The README.rst starts with: Module namedstruct ------------------ namedstruct encapsulates struct.unpack() and struct.pack() with results accessed by name. Classes ------- class namedstruct.namedstruct If I remove the sections the README.rst shown only has the first line of the readme twine allows be to upload but only the first line of the README is rendered. Barry From invalid at invalid.com Mon Jul 15 16:17:34 2019 From: invalid at invalid.com (Gys) Date: Mon, 15 Jul 2019 22:17:34 +0200 Subject: Books for Python 3.7 In-Reply-To: References: Message-ID: On 7/12/19 4:36 PM, mok888 at gmail.com wrote: > Can anyone help me. > New to Python. > Installed version 3.7 > I purchased the "Python for Dummies" book But this book was written for an older version of Python. > All the examples and samples don't work with version 3.7 > Can anyone direct me to which is the latest book to buy to properly learn Python. > Thanks Hi Richy M, I'm in somewhat the same situation. I'm retired now at 73. Used Fortran on IBM, Vax, PDP. Visual Basic on Windows for Work Groups to Windows 10 and changed to Linux Mint about 1 year ago. Started Python about that time. I quickly learned that Python is more a ecosystem than a programming language my advice would be to search what ecosystem you want. I always use a pet project, like suggested by Andrew Z, when I start something new. My pet project is Geo Information Systems or Gis for short. So I have Python 3 and the Python Gis tools. I make long cycle tours with OsmAnd on a Samsung Galaxy for navigation and record the Gpx Tracks. I use Python 3 for analysing the tracks. I use Pandas which let me use Excel like Data Sheets in Python : https://pandas.pydata.org/pandas-docs/stable/index.html If you also have MathPlot you have a serious competitor for MathLab. Pandas also includes Numpy for better manipulation of arrays. If you have a look at Anaconda https://www.anaconda.com/distribution/ you can download the suite in one go. I used the official Linux Mint repository to download the things I wanted. I think you will greatly benefit from a IDE I use Spider (also in the Anaconda suite) : https://www.spyder-ide.org/ but there are several others. If you want to create a GUI in your programs than Quantum seems a great Idea. I use PyQt5 which is the Python 3 version of Quantum version 5. I also would like to have a good book, but have not yet decided which one. There is a 50$ book on learning Python; the language reference (?) There is a 50$ book for learning PyQt5 programming of a GUI. There is a 50$ book on using Python in Pandas for analysing tabular data. In the meantime I use Gitlab for finding code snippets. I use these beginner guides : https://www.python.org/doc/ https://www.tutorialspoint.com/python3/ https://overiq.com/python-101/ https://www.pythonforbeginners.com/ https://www.programiz.com/python-programming/tutorial hth Gys From rshepard at appl-ecosys.com Mon Jul 15 17:11:29 2019 From: rshepard at appl-ecosys.com (Rich Shepard) Date: Mon, 15 Jul 2019 14:11:29 -0700 (PDT) Subject: Books for Python 3.7 In-Reply-To: References: Message-ID: On Mon, 15 Jul 2019, Dennis Lee Bieber wrote: > If one lacks both A and B, one ends up with various editions of > "Learning Python", "Programming Python", and "Fluent Python" (among many > others). I have found those books quite helpful over the years. I recommend starting with the latest edition of 'Learning Python'. The other might be helpful depending on how you use Python. The latest edition is quite large but well written, easy to follow, and complete. Regards, Rich From formisc at gmail.com Mon Jul 15 17:24:25 2019 From: formisc at gmail.com (Andrew Z) Date: Mon, 15 Jul 2019 17:24:25 -0400 Subject: Books for Python 3.7 In-Reply-To: References: Message-ID: Gys - hats off. Basically what Dennis is saying- you dont need a book "about python ". Tutorials and general search online will get you further and faster than any book. Blah-blah about myself: my bookshelf has 2 technical books, just because i got them to prepare for certifications. For my trading app, i had to figure out how to work with asyncio module, at the time -2017 , there were no semi- decent explanation for it, let alone books. By 2018 it became "the thing" with a ton of books. Blah-blah= off Good luck. P.s. and if you want to implement your idea really fast and easy - look at the go (golang.org). In my humble opinion- it is super easy and excellent all around. Doing their golangtour is all you need to write a working app. P.p.s. i just started a holy war .. damn. On Mon, Jul 15, 2019, 17:03 Dennis Lee Bieber wrote: > On Mon, 15 Jul 2019 22:17:34 +0200, Gys declaimed > the > following: > > >I also would like to have a good book, but have not yet decided which > >one. There is a 50$ book on learning Python; the language reference (?) > >There is a 50$ book for learning PyQt5 programming of a GUI. There is a > >50$ book on using Python in Pandas for analysing tabular data. > > > > For the language and "batteries" -- every distribution should > provide > the language reference, and the standard library reference. If one has a) > experience with other languages, the LRM should be sufficient for learning > the syntax; b) skill at interpreting technical documents, one should become > familiar with the contents of the SL reference (this does not mean > memorizing all of it -- critical would be the chapters on data types [which > explains what one can do with lists, dictionaries, tuples...] and then get > an idea of the contents of other chapters, so one can look up specifics for > tasks. > > After that, one ends up with print books that tend to focus on > narrow > application domains: XML, WxPython, SQLAlchemy, MatPlotLib, Win32 (just > from scanning my bookshelf). > > If one lacks both A and B, one ends up with various editions of > "Learning Python", "Programming Python", and "Fluent Python" (among many > others). > > > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com > http://wlfraed.microdiversity.freeddns.org/ > > -- > https://mail.python.org/mailman/listinfo/python-list > From PythonList at DancesWithMice.info Mon Jul 15 19:42:26 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Tue, 16 Jul 2019 11:42:26 +1200 Subject: Books for Python 3.7 In-Reply-To: References: Message-ID: <420f78ca-265d-9d71-0a7e-6f099952b36d@DancesWithMice.info> Wlfraed probably knows a thing-or-two about kicking-over ants' nests/wasps' nests... Talking about books is one thing. Judging them by asyncio coverage is quite another - and rather unfair. The use and methods of asyncio have changed frequently and markedly since '3.0'. Books take time to produce, sell, buy, and consume... Recommend OP takes a look at the LeanPub series: Python Apprentice; Journeyman; and Master. They also publish Mike Driscoll - few of which I have read personally [hangs head in embarrassment/shame], but I do follow his "Mouse Vs Python" web site... I much prefer to learn from a dead-tree presentation - and likely gain as much benefit from being able to 'look stuff up', thereafter. However, YMMV! Accordingly, the OP might like to broaden his analysis beyond books (paper or on-line) and take a look at MOOCs (on-line courses). Each platform seems to offer something on Python (some good, some tedious, some little more than puffery) [disclaimer: 'my' courses (non-Python) are hosted on edX]. Just this morning I noted a veritable wave of free courses being released on the Swayam platform (Indian universities) including: The Joy of Computing using Python (https://www.classcentral.com/course/swayam-the-joy-of-computing-using-python-14329). NB sadly I don't have time to attempt/review this myself, but would be intrigued to hear from you, should you... Last comment (to OP): you should be aware of the Python version 'covered'. Am not convinced that v3.7 is that important - to a beginner. Thus, maybe accept v3.5+, and make a practice of reviewing the Python docs - especially the Release Notes if you think version differences are important/worthy of particular concern. On 16/07/19 9:24 AM, Andrew Z wrote: > Gys - hats off. > > Basically what Dennis is saying- you dont need a book "about python ". > Tutorials and general search online will get you further and faster than > any book. > > Blah-blah about myself: > my bookshelf has 2 technical books, just because i got them to prepare for > certifications. > For my trading app, i had to figure out how to work with asyncio module, > at the time -2017 , there were no semi- decent explanation for it, let > alone books. By 2018 it became "the thing" with a ton of books. > Blah-blah= off > > Good luck. > > P.s. and if you want to implement your idea really fast and easy - look at > the go (golang.org). In my humble opinion- it is super easy and excellent > all around. Doing their golangtour is all you need to write a working app. > P.p.s. i just started a holy war .. damn. > > On Mon, Jul 15, 2019, 17:03 Dennis Lee Bieber wrote: > >> On Mon, 15 Jul 2019 22:17:34 +0200, Gys declaimed >> the >> following: >> >>> I also would like to have a good book, but have not yet decided which >>> one. There is a 50$ book on learning Python; the language reference (?) >>> There is a 50$ book for learning PyQt5 programming of a GUI. There is a >>> 50$ book on using Python in Pandas for analysing tabular data. >>> >> >> For the language and "batteries" -- every distribution should >> provide >> the language reference, and the standard library reference. If one has a) >> experience with other languages, the LRM should be sufficient for learning >> the syntax; b) skill at interpreting technical documents, one should become >> familiar with the contents of the SL reference (this does not mean >> memorizing all of it -- critical would be the chapters on data types [which >> explains what one can do with lists, dictionaries, tuples...] and then get >> an idea of the contents of other chapters, so one can look up specifics for >> tasks. >> >> After that, one ends up with print books that tend to focus on >> narrow >> application domains: XML, WxPython, SQLAlchemy, MatPlotLib, Win32 (just >> from scanning my bookshelf). >> >> If one lacks both A and B, one ends up with various editions of >> "Learning Python", "Programming Python", and "Fluent Python" (among many >> others). >> >> >> -- >> Wulfraed Dennis Lee Bieber AF6VN >> wlfraed at ix.netcom.com >> http://wlfraed.microdiversity.freeddns.org/ >> >> -- >> https://mail.python.org/mailman/listinfo/python-list >> -- Regards =dn From python at bdurham.com Mon Jul 15 19:46:26 2019 From: python at bdurham.com (Malcolm Greene) Date: Mon, 15 Jul 2019 17:46:26 -0600 Subject: Books for Python 3.7 In-Reply-To: <420f78ca-265d-9d71-0a7e-6f099952b36d@DancesWithMice.info> References: <420f78ca-265d-9d71-0a7e-6f099952b36d@DancesWithMice.info> Message-ID: Python Cookbook; highly recommended. Malcolm From PythonList at DancesWithMice.info Mon Jul 15 20:15:58 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Tue, 16 Jul 2019 12:15:58 +1200 Subject: Embedding classes' names Message-ID: <38b39ff2-54c7-08ac-c785-89b5fa3f42f1@etelligence.info> When used, do you embed a class's name within its own code, as a literal? In the thread "super or not super?", the OP asked: <<< C1.__init__(self) or super().__init__() >>> One of the answers recommended super() [agreed!] in order to avoid embedding "C1" into the code. The explanation: in case the name of the parent class is ever changed. Which is eminently reasonable (remember, the parent class may be in a separate file, and thus few "cues" to remember to inspect and amend sub-classes' code!) So, what about other situations where one might need to access the class's own name or that of its/a super-class? eg class C2(C1): def __init__(self, fred, barney ): super().__init__( fred ) self.barney = barney def __repr__( self ): return f"C2( { self.fred }, { self.barney }" ### note: 'common practice' of "C2" embedded as constant How 'purist' do you go, cf YAGNI? -- Regards, =dn From jcasale at activenetwerx.com Mon Jul 15 20:13:57 2019 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Tue, 16 Jul 2019 00:13:57 +0000 Subject: Class initialization with multiple inheritance Message-ID: I am trying to find explicit documentation on the initialization logic for a Base class when multiple exist. For the example in the documentation at https://docs.python.org/3/tutorial/classes.html#multiple-inheritance, if Base1 and Base2 both themselves inherited from the same base class, only Base1 would call __init__ on the subclass, Base2 would not. While I know this from experience, I need to locate actual documentation, does Anyone know if it exists? Thanks, jlc From rosuav at gmail.com Mon Jul 15 20:20:09 2019 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Jul 2019 10:20:09 +1000 Subject: Embedding classes' names In-Reply-To: <38b39ff2-54c7-08ac-c785-89b5fa3f42f1@etelligence.info> References: <38b39ff2-54c7-08ac-c785-89b5fa3f42f1@etelligence.info> Message-ID: On Tue, Jul 16, 2019 at 10:17 AM DL Neil wrote: > > When used, do you embed a class's name within its own code, as a literal? > > > In the thread "super or not super?", the OP asked: > <<< > C1.__init__(self) or > super().__init__() > >>> > > One of the answers recommended super() [agreed!] in order to avoid > embedding "C1" into the code. The explanation: in case the name of the > parent class is ever changed. > > Which is eminently reasonable (remember, the parent class may be in a > separate file, and thus few "cues" to remember to inspect and amend > sub-classes' code!) > > So, what about other situations where one might need to access the > class's own name or that of its/a super-class? eg > > class C2(C1): > def __init__(self, fred, barney ): > super().__init__( fred ) > self.barney = barney > > def __repr__( self ): > return f"C2( { self.fred }, { self.barney }" > ### note: 'common practice' of "C2" embedded as constant > > > How 'purist' do you go, cf YAGNI? In the case of __repr__, I would most definitely use self.__class__.__name__, because that way, a subclass can leave repr untouched and still get decent behaviour. ChrisA From PythonList at DancesWithMice.info Mon Jul 15 21:09:29 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Tue, 16 Jul 2019 13:09:29 +1200 Subject: Class initialization with multiple inheritance In-Reply-To: References: Message-ID: <509dc118-07cd-b781-0f81-948d5fa1da4a@DancesWithMice.info> On 16/07/19 12:13 PM, Joseph L. Casale wrote: > I am trying to find explicit documentation on the initialization logic for a > Base class when multiple exist. For the example in the documentation at > https://docs.python.org/3/tutorial/classes.html#multiple-inheritance, > if Base1 and Base2 both themselves inherited from the same base class, > only Base1 would call __init__ on the subclass, Base2 would not. While > I know this from experience, I need to locate actual documentation, does > Anyone know if it exists? There are two "initialisation" 'events', or two points-in-time: 1 import - when the various namespaces are constructed 2 execution - when a method is called. (am assuming the latter) That document describes MRO as "search for attributes inherited from a parent class as depth-first, left-to-right, not searching twice in the same class where there is an overlap in the hierarchy"; and goes on to say "all classes inherit from object, so any case of multiple inheritance provides more than one path to reach object. To keep the base classes from being accessed more than once, the dynamic algorithm linearizes the search order in a way that preserves the left-to-right ordering specified in each class, that calls each parent only once, and that is monotonic". Add those together with the idea of 'search' - when the search algorithm is searching for something, eg which super().__init__(), once it has 'found' the entity (which may actually be 'the first such entity' or 'the first way to find this entity' in the 'diamond' case), it STOPS looking! Accordingly, avoiding a duplicate execution problem. Does that satisfy the question? -- Regards =dn From dieter at handshake.de Tue Jul 16 00:43:57 2019 From: dieter at handshake.de (dieter) Date: Tue, 16 Jul 2019 06:43:57 +0200 Subject: PyPi twine check does not like my README.rst files anymore References: <719984C3-5731-4587-9DA3-F3D5B662C59A@barrys-emacs.org> Message-ID: <877e8imvw2.fsf@handshake.de> Barry Scott writes: > I am update some PyPI projects and found that twine was refusing the upload. > ... > Failed > The project's long_description has invalid markup which will not be rendered on PyPI. The following syntax errors were detected: > line 1: Severe: Unexpected section title or transition. > ... > The other reason is the show stopper. > > It does not allow me to have sections in my README.rst. > > The README.rst starts with: > > Module namedstruct > ------------------ > > namedstruct encapsulates struct.unpack() and struct.pack() with results accessed by name. > ... It may not expect that your "README.rst" starts with a section: PyPI will provide a title; typical descriptions then have some introductory text and only then the section structuring begins. If this is not the cause, look at the source of the checking implementation. It should tell you what is does not like. From ian.g.kelly at gmail.com Tue Jul 16 01:29:53 2019 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Mon, 15 Jul 2019 23:29:53 -0600 Subject: super or not super? In-Reply-To: References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> Message-ID: On Sun, Jul 14, 2019 at 7:14 PM Chris Angelico wrote: > > On Mon, Jul 15, 2019 at 10:51 AM Paulo da Silva > wrote: > > > > ?s 15:30 de 12/07/19, Thomas Jollans escreveu: > > > On 12/07/2019 16.12, Paulo da Silva wrote: > > >> Hi all! > > >> > > >> Is there any difference between using the base class name or super to > > >> call __init__ from base class? > > > > > > There is, when multiple inheritance is involved. super() can call > > > different 'branches' of the inheritance tree if necessary. > > > ... > > > > Thank you Jollans. I forgot multiple inheritance. I never needed it in > > python, so far. > > > > Something to consider is that super() becomes useful even if someone > else uses MI involving your class. Using super() ensures that your > class will play nicely in someone else's hierarchy, not just your own. Just using super() is not enough. You need to take steps if you want to ensure that you class plays nicely with MI. For example, consider the following: class C1: def __init__(self, name): self._name = name class C2(C1): def __init__(self, name, value): super().__init__(name) self._value = value This usage of super is just fine for the single-inheritance shown here. But there are two reasons why this cannot be neatly pulled into an MI hierarchy. Can you spot both of them? From rosuav at gmail.com Tue Jul 16 03:18:57 2019 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Jul 2019 17:18:57 +1000 Subject: super or not super? In-Reply-To: References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> Message-ID: On Tue, Jul 16, 2019 at 3:32 PM Ian Kelly wrote: > > On Sun, Jul 14, 2019 at 7:14 PM Chris Angelico wrote: > > > > On Mon, Jul 15, 2019 at 10:51 AM Paulo da Silva > > wrote: > > > > > > ?s 15:30 de 12/07/19, Thomas Jollans escreveu: > > > > On 12/07/2019 16.12, Paulo da Silva wrote: > > > >> Hi all! > > > >> > > > >> Is there any difference between using the base class name or super to > > > >> call __init__ from base class? > > > > > > > > There is, when multiple inheritance is involved. super() can call > > > > different 'branches' of the inheritance tree if necessary. > > > > ... > > > > > > Thank you Jollans. I forgot multiple inheritance. I never needed it in > > > python, so far. > > > > > > > Something to consider is that super() becomes useful even if someone > > else uses MI involving your class. Using super() ensures that your > > class will play nicely in someone else's hierarchy, not just your own. > > Just using super() is not enough. You need to take steps if you want to > ensure that you class plays nicely with MI. For example, consider the > following: > > class C1: > def __init__(self, name): > self._name = name > > class C2(C1): > def __init__(self, name, value): > super().__init__(name) > self._value = value > > This usage of super is just fine for the single-inheritance shown here. But > there are two reasons why this cannot be neatly pulled into an MI > hierarchy. Can you spot both of them? Well, obviously it's violating LSP by changing the signature of __init__, which means that you have to be aware of its position in the hierarchy. If you want things to move around smoothly, you HAVE to maintain a constant signature (which might mean using *args and/or **kwargs cooperatively). I'm not sure what the second issue is, though. It's kinda hard to imagine a real-world situation where this would come up (some things have names but not values, others have values... what third class could you bring in that would make sense here?), so I'm unsure whether you're pointing to "self._value" as a problem. Personally, I'd use "self.value = value" and not try to pretend that it's private. (Not a fan of "self.__value" because subclasses can't use it usefully AT ALL, so unless you're absolutely perfectly maintaining a consistent public signature - which you're not, see the first point - there's no way you can do anything good with it.) ChrisA From antoon.pardon at vub.be Tue Jul 16 04:02:29 2019 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 16 Jul 2019 10:02:29 +0200 Subject: super or not super? In-Reply-To: References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> Message-ID: <03a3de96-9daf-986e-ec3a-fd3c1fa5abb2@vub.be> On 16/07/19 09:18, Chris Angelico wrote: > On Tue, Jul 16, 2019 at 3:32 PM Ian Kelly wrote: >> On Sun, Jul 14, 2019 at 7:14 PM Chris Angelico wrote: >>> On Mon, Jul 15, 2019 at 10:51 AM Paulo da Silva >>> wrote: >>>> ?s 15:30 de 12/07/19, Thomas Jollans escreveu: >>>>> On 12/07/2019 16.12, Paulo da Silva wrote: >>>>>> Hi all! >>>>>> >>>>>> Is there any difference between using the base class name or super to >>>>>> call __init__ from base class? >>>>> There is, when multiple inheritance is involved. super() can call >>>>> different 'branches' of the inheritance tree if necessary. >>>>> ... >>>> Thank you Jollans. I forgot multiple inheritance. I never needed it in >>>> python, so far. >>>> >>> Something to consider is that super() becomes useful even if someone >>> else uses MI involving your class. Using super() ensures that your >>> class will play nicely in someone else's hierarchy, not just your own. >> Just using super() is not enough. You need to take steps if you want to >> ensure that you class plays nicely with MI. For example, consider the >> following: >> >> class C1: >> def __init__(self, name): >> self._name = name >> >> class C2(C1): >> def __init__(self, name, value): >> super().__init__(name) >> self._value = value >> >> This usage of super is just fine for the single-inheritance shown here. But >> there are two reasons why this cannot be neatly pulled into an MI >> hierarchy. Can you spot both of them? > Well, obviously it's violating LSP by changing the signature of > __init__, which means that you have to be aware of its position in the > hierarchy. If you want things to move around smoothly, you HAVE to > maintain a constant signature (which might mean using *args and/or > **kwargs cooperatively). I guess the second problem is that C1 doesn't call super. Meaning that if someone else uses this in a multiple heritance scheme, and the MRO reaches C1, the call doesn't get propagated to the rest. -- Antoon Pardon From rosuav at gmail.com Tue Jul 16 04:18:20 2019 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 16 Jul 2019 18:18:20 +1000 Subject: super or not super? In-Reply-To: <03a3de96-9daf-986e-ec3a-fd3c1fa5abb2@vub.be> References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> <03a3de96-9daf-986e-ec3a-fd3c1fa5abb2@vub.be> Message-ID: On Tue, Jul 16, 2019 at 6:05 PM Antoon Pardon wrote: > > On 16/07/19 09:18, Chris Angelico wrote: > > On Tue, Jul 16, 2019 at 3:32 PM Ian Kelly wrote: > >> Just using super() is not enough. You need to take steps if you want to > >> ensure that you class plays nicely with MI. For example, consider the > >> following: > >> > >> class C1: > >> def __init__(self, name): > >> self._name = name > >> > >> class C2(C1): > >> def __init__(self, name, value): > >> super().__init__(name) > >> self._value = value > >> > >> This usage of super is just fine for the single-inheritance shown here. But > >> there are two reasons why this cannot be neatly pulled into an MI > >> hierarchy. Can you spot both of them? > > Well, obviously it's violating LSP by changing the signature of > > __init__, which means that you have to be aware of its position in the > > hierarchy. If you want things to move around smoothly, you HAVE to > > maintain a constant signature (which might mean using *args and/or > > **kwargs cooperatively). > > I guess the second problem is that C1 doesn't call super. Meaning that if > someone else uses this in a multiple heritance scheme, and the MRO reaches > C1, the call doesn't get propagated to the rest. > My understanding of this tiny hierarchy is that C1 is the root of the cooperative subtree. Example: class Pizza: # like C1 def __init__(self, *, name="Generic Pizza", **kw): self.name = name if kw: raise TypeError("Unexpected keyword args - did you miss a mixin?") class MeatloversPizza(Pizza): # like C2 def __init__(self, *, pepperoni=True, **kw): super().__init__(**kw) if pepperoni: self.ingredients = "hot" class CheesyCrustPizza(Pizza): # alternate mixin def __init__(self, *, cheesetype="mozzarella", **kw): super().__init__(**kw) self.crust = cheesetype + "-stuffed" class DecadentPizza(MeatloversPizza, CheesyCrustPizza): pass The only change I've made here is to ensure that everyone's cooperatively using **kwargs. You can instantiate a DecadentPizza(cheesetype="cheddar", name="Eat Me", pepperoni=False), and the different classes will pick up the parts they want. This sort of hierarchy works just fine, since MeatloversPizza is using super(). The signature of all __init__ functions is (self, *, **kw), with the variant that the arguments specifically wanted by this class are listed in a more discoverable way (rather than just popping them out of kw before calling super's init). It's therefore fine for MeatloversPizza to chain into CheesyCrustPizza. ChrisA From uri at speedy.net Tue Jul 16 06:08:41 2019 From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=) Date: Tue, 16 Jul 2019 13:08:41 +0300 Subject: super() in Python 3 Message-ID: Hi, 1. When we use super() in Python 3, we don't pass it the first argument (self). Why? What happens if the first argument is not self? def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) I think it would make more sense to use something like self.super().__init__(*args, **kwargs) or something like this. 2. I want to override a function called build_suite in an inherited class. The function receives an argument "test_labels" which I want to change (I define it if it's not defined), but I don't do anything with the argument "extra_tests". Is it possible to include "extra_tests" in *args, **kwargs and how? I think maybe they will release another version in the future without "extra_tests", or with additional arguments, and I don't want to have to change my code then. def build_suite(self, test_labels=None, extra_tests=None, **kwargs): .... return super().build_suite(test_labels=test_labels, extra_tests=extra_tests, **kwargs) (Notice there are no *args) Thanks! ???? uri at speedy.net From antoon.pardon at vub.be Tue Jul 16 06:42:59 2019 From: antoon.pardon at vub.be (Antoon Pardon) Date: Tue, 16 Jul 2019 12:42:59 +0200 Subject: super or not super? In-Reply-To: References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> <03a3de96-9daf-986e-ec3a-fd3c1fa5abb2@vub.be> Message-ID: On 16/07/19 10:18, Chris Angelico wrote: > On Tue, Jul 16, 2019 at 6:05 PM Antoon Pardon wrote: >> On 16/07/19 09:18, Chris Angelico wrote: >>> On Tue, Jul 16, 2019 at 3:32 PM Ian Kelly wrote: >>>> Just using super() is not enough. You need to take steps if you want to >>>> ensure that you class plays nicely with MI. For example, consider the >>>> following: >>>> >>>> class C1: >>>> def __init__(self, name): >>>> self._name = name >>>> >>>> class C2(C1): >>>> def __init__(self, name, value): >>>> super().__init__(name) >>>> self._value = value >>>> >>>> This usage of super is just fine for the single-inheritance shown here. But >>>> there are two reasons why this cannot be neatly pulled into an MI >>>> hierarchy. Can you spot both of them? >>> Well, obviously it's violating LSP by changing the signature of >>> __init__, which means that you have to be aware of its position in the >>> hierarchy. If you want things to move around smoothly, you HAVE to >>> maintain a constant signature (which might mean using *args and/or >>> **kwargs cooperatively). >> I guess the second problem is that C1 doesn't call super. Meaning that if >> someone else uses this in a multiple heritance scheme, and the MRO reaches >> C1, the call doesn't get propagated to the rest. >> > My understanding of this tiny hierarchy is that C1 is the root of the > cooperative subtree. Example: Why should that stop someone else from using C1 in it's own multiple inheritance scheme? Since object is the root of the whole inheritance hierarchy that should be workable. I find it a bit odd to construct something so narrowly cooperative instead of constructing it in a way that allows easier cooperation with future classes. -- Antoon. From cs at cskk.id.au Tue Jul 16 06:57:38 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 16 Jul 2019 20:57:38 +1000 Subject: Embedding classes' names In-Reply-To: References: Message-ID: <20190716105738.GA83475@cskk.homeip.net> On 16Jul2019 10:20, Chris Angelico wrote: >On Tue, Jul 16, 2019 at 10:17 AM DL Neil wrote: >> When used, do you embed a class's name within its own code, as a >> literal? [...] >> So, what about other situations where one might need to access the >> class's own name or that of its/a super-class? eg >> >> class C2(C1): >> def __init__(self, fred, barney ): >> super().__init__( fred ) >> self.barney = barney >> >> def __repr__( self ): >> return f"C2( { self.fred }, { self.barney }" >> ### note: 'common practice' of "C2" embedded as constant >> >> >> How 'purist' do you go, cf YAGNI? > >In the case of __repr__, I would most definitely use >self.__class__.__name__, because that way, a subclass can leave repr >untouched and still get decent behaviour. Yeah, me too, though I spell it "type(self).__name__" for no totally rational reason. Cheers, Cameron Simpson From formisc at gmail.com Tue Jul 16 07:42:50 2019 From: formisc at gmail.com (Andrew Z) Date: Tue, 16 Jul 2019 07:42:50 -0400 Subject: Books for Python 3.7 In-Reply-To: <420f78ca-265d-9d71-0a7e-6f099952b36d@DancesWithMice.info> References: <420f78ca-265d-9d71-0a7e-6f099952b36d@DancesWithMice.info> Message-ID: Thats a good idea. Indeed- take an online course, it is cheap, gets your thru basics and you have an instructor to help if you r stuck. Id vote for online course vs buying a book . On Mon, Jul 15, 2019, 19:43 DL Neil wrote: > Wlfraed probably knows a thing-or-two about kicking-over ants' > nests/wasps' nests... > > Talking about books is one thing. Judging them by asyncio coverage is > quite another - and rather unfair. The use and methods of asyncio have > changed frequently and markedly since '3.0'. Books take time to produce, > sell, buy, and consume... > > > Recommend OP takes a look at the LeanPub series: Python Apprentice; > Journeyman; and Master. They also publish Mike Driscoll - few of which I > have read personally [hangs head in embarrassment/shame], but I do > follow his "Mouse Vs Python" web site... > > > I much prefer to learn from a dead-tree presentation - and likely gain > as much benefit from being able to 'look stuff up', thereafter. However, > YMMV! > > Accordingly, the OP might like to broaden his analysis beyond books > (paper or on-line) and take a look at MOOCs (on-line courses). Each > platform seems to offer something on Python (some good, some tedious, > some little more than puffery) [disclaimer: 'my' courses (non-Python) > are hosted on edX]. > > Just this morning I noted a veritable wave of free courses being > released on the Swayam platform (Indian universities) including: The > Joy of Computing using Python > ( > https://www.classcentral.com/course/swayam-the-joy-of-computing-using-python-14329). > > > > NB sadly I don't have time to attempt/review this myself, but would be > intrigued to hear from you, should you... > > > Last comment (to OP): you should be aware of the Python version > 'covered'. Am not convinced that v3.7 is that important - to a beginner. > Thus, maybe accept v3.5+, and make a practice of reviewing the Python > docs - especially the Release Notes if you think version differences are > important/worthy of particular concern. > > > On 16/07/19 9:24 AM, Andrew Z wrote: > > Gys - hats off. > > > > Basically what Dennis is saying- you dont need a book "about python ". > > Tutorials and general search online will get you further and faster than > > any book. > > > > Blah-blah about myself: > > my bookshelf has 2 technical books, just because i got them to prepare > for > > certifications. > > For my trading app, i had to figure out how to work with asyncio module, > > at the time -2017 , there were no semi- decent explanation for it, let > > alone books. By 2018 it became "the thing" with a ton of books. > > Blah-blah= off > > > > Good luck. > > > > P.s. and if you want to implement your idea really fast and easy - look > at > > the go (golang.org). In my humble opinion- it is super easy and > excellent > > all around. Doing their golangtour is all you need to write a working > app. > > P.p.s. i just started a holy war .. damn. > > > > On Mon, Jul 15, 2019, 17:03 Dennis Lee Bieber > wrote: > > > >> On Mon, 15 Jul 2019 22:17:34 +0200, Gys declaimed > >> the > >> following: > >> > >>> I also would like to have a good book, but have not yet decided which > >>> one. There is a 50$ book on learning Python; the language reference (?) > >>> There is a 50$ book for learning PyQt5 programming of a GUI. There is a > >>> 50$ book on using Python in Pandas for analysing tabular data. > >>> > >> > >> For the language and "batteries" -- every distribution should > >> provide > >> the language reference, and the standard library reference. If one has > a) > >> experience with other languages, the LRM should be sufficient for > learning > >> the syntax; b) skill at interpreting technical documents, one should > become > >> familiar with the contents of the SL reference (this does not mean > >> memorizing all of it -- critical would be the chapters on data types > [which > >> explains what one can do with lists, dictionaries, tuples...] and then > get > >> an idea of the contents of other chapters, so one can look up specifics > for > >> tasks. > >> > >> After that, one ends up with print books that tend to focus on > >> narrow > >> application domains: XML, WxPython, SQLAlchemy, MatPlotLib, Win32 (just > >> from scanning my bookshelf). > >> > >> If one lacks both A and B, one ends up with various editions of > >> "Learning Python", "Programming Python", and "Fluent Python" (among many > >> others). > >> > >> > >> -- > >> Wulfraed Dennis Lee Bieber AF6VN > >> wlfraed at ix.netcom.com > >> http://wlfraed.microdiversity.freeddns.org/ > >> > >> -- > >> https://mail.python.org/mailman/listinfo/python-list > >> > > -- > Regards =dn > -- > https://mail.python.org/mailman/listinfo/python-list > From rhodri at kynesim.co.uk Tue Jul 16 08:12:37 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 16 Jul 2019 13:12:37 +0100 Subject: super() in Python 3 In-Reply-To: References: Message-ID: <7a18d7ea-9c71-86d0-4789-8b02c871d3bc@kynesim.co.uk> Hi there! A lot of the answers to your questions are at least implied in the Fine Manual (https://docs.python.org/3/library/functions.html#super), but it's not very clear and written more for precision than comprehension. Here's my attempt at explaining :-) On 16/07/2019 11:08, ???? wrote: > Hi, > > 1. When we use super() in Python 3, we don't pass it the first argument > (self). Why? Actually the first argument to super() isn't self, it's the class that we want the superclass of. The *second* argument is self. In the normal course of using super() inside a class method, these arguments will almost always be the class itself and the instance the method was called on. Since that's almost always the case, the compiler offers us a short-cut: omit the "cls" and "self" and the compiler will fill them in for us. That way we don't have to repeat ourselves and risk mis-typing something. > What happens if the first argument is not self? The first argument of what? I'm not sure what you're getting at here. > I think it would make more sense to use something like > self.super().__init__(*args, **kwargs) or something like this. That would mean either forbidding classes to have a method named "super" or accepting that users' classes could completely screw up inheritance for their subclasses. > 2. I want to override a function called build_suite in an inherited class. > The function receives an argument "test_labels" which I want to change (I > define it if it's not defined), but I don't do anything with the argument > "extra_tests". Is it possible to include "extra_tests" in *args, **kwargs Yes. > and how? Don't list it in your parameters :-) def build_suite(self, test_labels=None, *args, **kwargs): ... return super().build_suite(test_labels=test_labels, *args, **kwargs) > I think maybe they will release another version in the future > without "extra_tests", or with additional arguments, and I don't want to > have to change my code then. With a shim layer like this, your chances of getting away with making no changes to your code when an API you use changes are rather small. -- Rhodri James *-* Kynesim Ltd From uri at speedy.net Tue Jul 16 11:43:20 2019 From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=) Date: Tue, 16 Jul 2019 18:43:20 +0300 Subject: super() in Python 3 In-Reply-To: <7a18d7ea-9c71-86d0-4789-8b02c871d3bc@kynesim.co.uk> References: <7a18d7ea-9c71-86d0-4789-8b02c871d3bc@kynesim.co.uk> Message-ID: Hi, Thanks for your explanation. But I tried your code and it doesn't work (with Django==1.11.22): File "<...>\site-packages\django\test\runner.py", line 600, in run_tests suite = self.build_suite(test_labels, extra_tests) File "<...>\speedy\core\base\test\models.py", line 35, in build_suite return super().build_suite(test_labels=test_labels, *args, **kwargs) TypeError: build_suite() got multiple values for argument 'test_labels' ???? uri at speedy.net On Tue, Jul 16, 2019 at 3:13 PM Rhodri James wrote: > Hi there! A lot of the answers to your questions are at least implied > in the Fine Manual > (https://docs.python.org/3/library/functions.html#super), but it's not > very clear and written more for precision than comprehension. Here's my > attempt at explaining :-) > > On 16/07/2019 11:08, ???? wrote: > > Hi, > > > > 1. When we use super() in Python 3, we don't pass it the first argument > > (self). Why? > > Actually the first argument to super() isn't self, it's the class that > we want the superclass of. The *second* argument is self. In the > normal course of using super() inside a class method, these arguments > will almost always be the class itself and the instance the method was > called on. Since that's almost always the case, the compiler offers us > a short-cut: omit the "cls" and "self" and the compiler will fill them > in for us. That way we don't have to repeat ourselves and risk > mis-typing something. > > > What happens if the first argument is not self? > > The first argument of what? I'm not sure what you're getting at here. > > > I think it would make more sense to use something like > > self.super().__init__(*args, **kwargs) or something like this. > > That would mean either forbidding classes to have a method named "super" > or accepting that users' classes could completely screw up inheritance > for their subclasses. > > > 2. I want to override a function called build_suite in an inherited > class. > > The function receives an argument "test_labels" which I want to change (I > > define it if it's not defined), but I don't do anything with the argument > > "extra_tests". Is it possible to include "extra_tests" in *args, **kwargs > > Yes. > > > and how? > > Don't list it in your parameters :-) > > def build_suite(self, test_labels=None, *args, **kwargs): > ... > return super().build_suite(test_labels=test_labels, *args, **kwargs) > > > I think maybe they will release another version in the future > > without "extra_tests", or with additional arguments, and I don't want to > > have to change my code then. > > With a shim layer like this, your chances of getting away with making no > changes to your code when an API you use changes are rather small. > > -- > Rhodri James *-* Kynesim Ltd > -- > https://mail.python.org/mailman/listinfo/python-list > From rhodri at kynesim.co.uk Tue Jul 16 12:38:41 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 16 Jul 2019 17:38:41 +0100 Subject: super() in Python 3 In-Reply-To: References: <7a18d7ea-9c71-86d0-4789-8b02c871d3bc@kynesim.co.uk> Message-ID: <84b535eb-860f-e4d5-9da4-3e72dc9d38ea@kynesim.co.uk> [Rearranged and snipped so this makes any kind of sense] On 16/07/2019 16:43, ???? wrote: > On Tue, Jul 16, 2019 at 3:13 PM Rhodri James wrote: >> On 16/07/2019 11:08, ???? wrote: >>> 2. I want to override a function called build_suite in an inherited >> class. >>> The function receives an argument "test_labels" which I want to change (I >>> define it if it's not defined), but I don't do anything with the argument >>> "extra_tests". Is it possible to include "extra_tests" in *args, **kwargs >> >> Yes. >> >>> and how? >> >> Don't list it in your parameters :-) >> >> def build_suite(self, test_labels=None, *args, **kwargs): >> ... >> return super().build_suite(test_labels=test_labels, *args, **kwargs) > > Thanks for your explanation. But I tried your code and it doesn't work > (with Django==1.11.22): > > File "<...>\site-packages\django\test\runner.py", line 600, in run_tests > suite = self.build_suite(test_labels, extra_tests) > File "<...>\speedy\core\base\test\models.py", line 35, in build_suite > return super().build_suite(test_labels=test_labels, *args, **kwargs) > TypeError: build_suite() got multiple values for argument 'test_labels' Ah, sorry, I was under the impression that you were calling the function with "test_labels" and "extra_tests" as keyword arguments. You can solve the immediate problem by omitting the "=test_labels" bit: return super().build_suite(test_labels, *args, **kwargs) assuming that "test_labels" is the first positional parameter. If it isn't, you've got trouble anyway :-) and you'll need to do some more involved parameter handling. -- Rhodri James *-* Kynesim Ltd From drsalists at gmail.com Tue Jul 16 13:35:13 2019 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 16 Jul 2019 10:35:13 -0700 Subject: Counting Python threads vs C/C++ threads Message-ID: I'm looking at a performance problem in a large CPython 2.x/3.x codebase with quite a few dependencies. I'm not sure what's causing the slowness yet. The CPU isn't getting hit hard, and I/O on the system appears to be low - but throughput is poor. I'm wondering if it could be CPU-bound Python threads causing the problem (because of the threading+GIL thing). The non-dependency Python portions don't Appear to have much in the way of threading going on based on a quick grep, but csysdig says a process running the code has around 32 threads running - the actual thread count varies, but that's the ballpark. I'm wondering if there's a good way to find two counts of those threads - how many are from CPython code that could run afoul of the GIL, and how many of them are from C/C++ extension modules that wouldn't be responsible for a GIL issue. Does anyone know how to find out these different thread counts, without mounting a huge effort to scrutinize all the code and its dependencies? I'd prefer a tool that can scrutinize the process(es) from the outside, but modifying the code is not out of the question. Please see https://stackoverflow.com/questions/56958009/how-can-i-tell-how-many-python-threads-a-process-has-from-the-outside for more information. Or perhaps https://python-forum.io/Thread-How-to-get-a-count-of-Python-threads-from-the-outside-or-via-code-instrumentation Thanks! From barry at barrys-emacs.org Tue Jul 16 13:37:20 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Tue, 16 Jul 2019 18:37:20 +0100 Subject: PyPi twine check does not like my README.rst files anymore In-Reply-To: <877e8imvw2.fsf@handshake.de> References: <719984C3-5731-4587-9DA3-F3D5B662C59A@barrys-emacs.org> <877e8imvw2.fsf@handshake.de> Message-ID: <3B50624D-B7B3-4D86-A1C6-85F5698F486E@barrys-emacs.org> > On 16 Jul 2019, at 05:43, dieter wrote: > > Barry Scott writes: >> I am update some PyPI projects and found that twine was refusing the upload. >> ... >> Failed >> The project's long_description has invalid markup which will not be rendered on PyPI. The following syntax errors were detected: >> line 1: Severe: Unexpected section title or transition. >> ... >> The other reason is the show stopper. >> >> It does not allow me to have sections in my README.rst. >> >> The README.rst starts with: >> >> Module namedstruct >> ------------------ >> >> namedstruct encapsulates struct.unpack() and struct.pack() with results accessed by name. >> ... > > It may not expect that your "README.rst" starts with a section: > PyPI will provide a title; typical descriptions then have some > introductory text and only then the section structuring begins. I did that and only the first line of the file is rendered. > > If this is not the cause, look at the source of the checking > implementation. It should tell you what is does not like. I was hoping to avoid reading the source that may well be a rabbit hole that keeps going down. Barry > > -- > https://mail.python.org/mailman/listinfo/python-list > From barry at barrys-emacs.org Tue Jul 16 13:53:18 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Tue, 16 Jul 2019 18:53:18 +0100 Subject: Class initialization with multiple inheritance In-Reply-To: References: Message-ID: <84050BC7-8143-4372-BE16-C320C163CE83@barrys-emacs.org> > On 16 Jul 2019, at 01:13, Joseph L. Casale wrote: > > I am trying to find explicit documentation on the initialization logic for a > Base class when multiple exist. For the example in the documentation at > https://docs.python.org/3/tutorial/classes.html#multiple-inheritance, > if Base1 and Base2 both themselves inherited from the same base class, > only Base1 would call __init__ on the subclass, Base2 would not. While > I know this from experience, I need to locate actual documentation, does > Anyone know if it exists? The docs are not clear on this. If you are using super().__init__() in all classes then the key to only calling __init__ once is that super() uses the MRO to find the next class's __init__ to call. The MRO algorithm takes any hierarchy of inheritance and turns it into a predictable ordered list of the classes. The __mro__ attribute allows you to see this. For example (m.py): class CommonBase(object): def __init__( self ): print( 'CommonBase.__init__()' ) super().__init__() class Left(CommonBase): def __init__( self ): print( 'Left.__init__()' ) super().__init__() class Right(CommonBase): def __init__( self ): print( 'Right.__init__()' ) super().__init__() class LeftAndRight(Left, Right): def __init__( self ): print( 'LeftAndRight.__init__()' ) super().__init__() instance = LeftAndRight() Outputs: LeftAndRight.__init__() Left.__init__() Right.__init__() CommonBase.__init__() And here is the MRO for LeftAndRight. >>> import m LeftAndRight.__init__() Left.__init__() Right.__init__() CommonBase.__init__() >>> m.LeftAndRight.__mro__ (, , , , ) >>> Notice that I explicitly derived from object that is the default. But here you see that class object is always there even if I omit it. >>> class Foo: ... pass ... >>> Foo.__mro__ (, ) Barry > > Thanks, > jlc > > -- > https://mail.python.org/mailman/listinfo/python-list > From ian.g.kelly at gmail.com Tue Jul 16 13:55:48 2019 From: ian.g.kelly at gmail.com (Ian Kelly) Date: Tue, 16 Jul 2019 11:55:48 -0600 Subject: super or not super? In-Reply-To: References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> Message-ID: On Tue, Jul 16, 2019 at 1:21 AM Chris Angelico wrote: > > On Tue, Jul 16, 2019 at 3:32 PM Ian Kelly wrote: > > > > Just using super() is not enough. You need to take steps if you want to > > ensure that you class plays nicely with MI. For example, consider the > > following: > > > > class C1: > > def __init__(self, name): > > self._name = name > > > > class C2(C1): > > def __init__(self, name, value): > > super().__init__(name) > > self._value = value > > > > This usage of super is just fine for the single-inheritance shown here. But > > there are two reasons why this cannot be neatly pulled into an MI > > hierarchy. Can you spot both of them? > > Well, obviously it's violating LSP by changing the signature of > __init__, which means that you have to be aware of its position in the > hierarchy. If you want things to move around smoothly, you HAVE to > maintain a constant signature (which might mean using *args and/or > **kwargs cooperatively). That's pretty close to what I had in mind. Many people treat __init__ as a constructor (I know, it's not) and so long as you're following that doctrine it's not really an LSP violation. But anything else that gets worked into an MI hierarchy has to somehow be compatible with both of these method signatures while also adding whatever new parameters it needs, which is a problem. The usual advice for working with this is to use **kwargs cooperatively as you suggested. *args doesn't work as well because every class needs to know the absolute index of every positional parameter it's interested in, and you can't just trim off the end as you go since you don't know if the later arguments have been consumed yet. With **kwargs you can just pop arguments off the dict as you go. On Tue, Jul 16, 2019 at 2:06 AM Antoon Pardon wrote: > > I guess the second problem is that C1 doesn't call super. Meaning that if > someone else uses this in a multiple heritance scheme, and the MRO reaches > C1, the call doesn't get propagated to the rest. That's it. With single inheritance it's both easy and common to assume that your base class is object, and since object.__init__ does nothing there's no point in calling it. But with MI, that assumption isn't valid. From barry at barrys-emacs.org Tue Jul 16 14:12:39 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Tue, 16 Jul 2019 19:12:39 +0100 Subject: Counting Python threads vs C/C++ threads In-Reply-To: References: Message-ID: I'm going to assume you are on linux. On 16 Jul 2019, at 18:35, Dan Stromberg wrote: > > I'm looking at a performance problem in a large CPython 2.x/3.x codebase > with quite a few dependencies. > > I'm not sure what's causing the slowness yet. The CPU isn't getting hit > hard, and I/O on the system appears to be low - but throughput is poor. > I'm wondering if it could be CPU-bound Python threads causing the problem > (because of the threading+GIL thing). Does top show the process using 100% CPU? > > The non-dependency Python portions don't Appear to have much in the way of > threading going on based on a quick grep, but csysdig says a process > running the code has around 32 threads running - the actual thread count > varies, but that's the ballpark. > > I'm wondering if there's a good way to find two counts of those threads - > how many are from CPython code that could run afoul of the GIL, and how > many of them are from C/C++ extension modules that wouldn't be responsible > for a GIL issue. From the docs on threading: threading.active_count() Return the number of Thread objects currently alive. The returned count is equal to the length of the list returned by enumerate() . Try running strace on the process to see what system calls its making. You could also connect gdb to the process and find out what code the threads are running. Barry > > Does anyone know how to find out these different thread counts, without > mounting a huge effort to scrutinize all the code and its dependencies? > I'd prefer a tool that can scrutinize the process(es) from the outside, but > modifying the code is not out of the question. > > Please see > https://stackoverflow.com/questions/56958009/how-can-i-tell-how-many-python-threads-a-process-has-from-the-outside > for more information. Or perhaps > https://python-forum.io/Thread-How-to-get-a-count-of-Python-threads-from-the-outside-or-via-code-instrumentation > > Thanks! > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Tue Jul 16 14:31:56 2019 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 17 Jul 2019 04:31:56 +1000 Subject: super or not super? In-Reply-To: References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> Message-ID: On Wed, Jul 17, 2019 at 3:58 AM Ian Kelly wrote: > > On Tue, Jul 16, 2019 at 1:21 AM Chris Angelico wrote: > > > > On Tue, Jul 16, 2019 at 3:32 PM Ian Kelly wrote: > > > > > > Just using super() is not enough. You need to take steps if you want to > > > ensure that you class plays nicely with MI. For example, consider the > > > following: > > > > > > class C1: > > > def __init__(self, name): > > > self._name = name > > > > > > class C2(C1): > > > def __init__(self, name, value): > > > super().__init__(name) > > > self._value = value > > > > > > This usage of super is just fine for the single-inheritance shown here. > But > > > there are two reasons why this cannot be neatly pulled into an MI > > > hierarchy. Can you spot both of them? > > > > Well, obviously it's violating LSP by changing the signature of > > __init__, which means that you have to be aware of its position in the > > hierarchy. If you want things to move around smoothly, you HAVE to > > maintain a constant signature (which might mean using *args and/or > > **kwargs cooperatively). > > That's pretty close to what I had in mind. Many people treat __init__ as a > constructor (I know, it's not) If you consider __new__ to be the constructor, and __init__ to be an implementation detail of how the constructor initializes the object, then it comes to the same thing anyway. > and so long as you're following that > doctrine it's not really an LSP violation. But anything else that gets > worked into an MI hierarchy has to somehow be compatible with both of these > method signatures while also adding whatever new parameters it needs, which > is a problem. The usual advice for working with this is to use **kwargs > cooperatively as you suggested. *args doesn't work as well because every > class needs to know the absolute index of every positional parameter it's > interested in, and you can't just trim off the end as you go since you > don't know if the later arguments have been consumed yet. With **kwargs you > can just pop arguments off the dict as you go. Yeah, makes sense. I'd advocate a slightly weakened form of LSP-style equivalence: the function signatures don't have to be *identical*, but they do have to be built with compatibility in mind. ChrisA From drsalists at gmail.com Tue Jul 16 15:48:33 2019 From: drsalists at gmail.com (Dan Stromberg) Date: Tue, 16 Jul 2019 12:48:33 -0700 Subject: Counting Python threads vs C/C++ threads In-Reply-To: References: Message-ID: On Tue, Jul 16, 2019 at 11:13 AM Barry Scott wrote: > I'm going to assume you are on linux. > Yes, I am. Ubuntu 16.04.6 LTS sometimes, Mint 19.1 other times. On 16 Jul 2019, at 18:35, Dan Stromberg wrote: > > > > I'm looking at a performance problem in a large CPython 2.x/3.x codebase > > with quite a few dependencies. > > > > I'm not sure what's causing the slowness yet. The CPU isn't getting hit > > hard, and I/O on the system appears to be low - but throughput is poor. > > I'm wondering if it could be CPU-bound Python threads causing the problem > > (because of the threading+GIL thing). > > Does top show the process using 100% CPU? > Nope. CPU utilization and disk use are both low. We've been going into top, and then hitting '1' to see things broken down by CPU core (there are 32 of them, probably counting hyperthreads as different cores), but the CPU use is in the teens or so. I've also tried dstat and csysdig. The hardware isn't breaking a sweat, but throughput is poor. > The non-dependency Python portions don't Appear to have much in the way of > > threading going on based on a quick grep, but csysdig says a process > > running the code has around 32 threads running - the actual thread count > > varies, but that's the ballpark. > > > > I'm wondering if there's a good way to find two counts of those threads - > > how many are from CPython code that could run afoul of the GIL, and how > > many of them are from C/C++ extension modules that wouldn't be > responsible > > for a GIL issue. > > From the docs on threading: > > threading.active_count() > > > Return the number of Thread > > objects currently alive. The returned count is equal to the length of the > list returned by enumerate() > . > Are you on a Mac? https://docs.python.org/2/library/threading.html appears to have some good info. I'll probably try logging threading.active_count() A question arises though: Does threading.active_count() only show Python threads created with the threading module? What about threads created with the thread module? Try running strace on the process to see what system calls its making. > I've tried it, but thank you. It's a good suggestion. I often find that when strace'ing a program, there's a bunch of mostly-irrelevant stuff at Initial Program Load (IPL), but then the main loop fails into a small cycle of system calls. Not with this program. Its main loop is busy and large. You could also connect gdb to the process and find out what code the > threads are running. > I used to use gdb, and wrappers for gdb, when I was doing C code, but I don't have much experience using it on a CPython interrpreter. Would I be doing a "thread apply all bt" or what? I'm guessing those backtraces could facilitate identifying the origin of a thread. Thanks a bunch. From scott at scolby.com Tue Jul 16 22:43:54 2019 From: scott at scolby.com (scott at scolby.com) Date: Tue, 16 Jul 2019 22:43:54 -0400 Subject: Difficulties configuring LTO on macOS with clang In-Reply-To: References: <76870224-4A91-48BF-B483-4EA9931BF293@barrys-emacs.org> Message-ID: Hmm, removing the path didn?t help either. I ended up defining the `LLVM_AR=/path/to/homebrew/llvm-ar` environment variable after installing clang from brew. I was worried that there would be problems using ar from a different version of the toolchain than the compiler, but things seem to have worked properly. I guess llvm-ar just isn?t packaged with macOS? In any case, this problem is solved for me now. Thanks for your help, Scott On Jul 10, 2019, 16:22 -0400, Barry Scott , wrote: > > > > On 10 Jul 2019, at 15:25, Scott Colby wrote: > > > > Unfortunately that path is absent on my system. Could the issue be that the llvm version of ar is not present at all on macOS? What remains confusing in that case is that by using CC=cc (and I assume just ar) the LTO compilation succeeds, despite cc being a symlink to clang anyway. > > Oh that may be because I have MacPorts installed on my mac > I guess you have macPorts installed as well? > > I see you are using brew openssl. Maybe the macPorts tools > and the brew tools are conflicting? > > What if you remove /opt/local/bin from your PATH does that help configure find the tools it needs? > > Barry > > > > > > > > Thanks, > > Scott > > > > On Tue, Jul 9, 2019, at 15:39, Barry Scott wrote: > > > > > > > > > > On 9 Jul 2019, at 19:28, Scott Colby wrote: > > > > > > > > Hello all, > > > > > > > > I am having difficulty building Python with `--with-lto` on macOS 10.14.5. With a clean checkout of the CPython source at the tagged release for 3.7.4, I ran: > > > > > > > > $ ./configure --prefix=/opt/python3.7.4 --enable-optimizations --enable-ipv6 --with-lto --with-openssl=$(brew --prefix openssl) CC=clang > > > > checking for git... found > > > > checking build system type... x86_64-apple-darwin18.6.0 > > > > checking host system type... x86_64-apple-darwin18.6.0 > > > > checking for python3.7... python3.7 > > > > checking for --enable-universalsdk... no > > > > checking for --with-universal-archs... no > > > > checking MACHDEP... checking for --without-gcc... no > > > > checking for --with-icc... no > > > > checking for gcc... clang > > > > checking whether the C compiler works... yes > > > > checking for C compiler default output file name... a.out > > > > checking for suffix of executables... > > > > checking whether we are cross compiling... no > > > > checking for suffix of object files... o > > > > checking whether we are using the GNU C compiler... yes > > > > checking whether clang accepts -g... yes > > > > checking for clang option to accept ISO C89... none needed > > > > checking how to run the C preprocessor... clang -E > > > > checking for grep that handles long lines and -e... /usr/bin/grep > > > > checking for a sed that does not truncate output... /usr/bin/sed > > > > checking for --with-cxx-main=... no > > > > checking for clang++... no > > > > configure: > > > > > > > >? By default, distutils will build C++ extension modules with "clang++". > > > >? If this is not intended, then set CXX on the configure command line. > > > > > > > > checking for the platform triplet based on compiler characteristics... darwin > > > > checking for -Wl,--no-as-needed... no > > > > checking for egrep... /usr/bin/grep -E > > > > checking for ANSI C header files... yes > > > > checking for sys/types.h... yes > > > > checking for sys/stat.h... yes > > > > checking for stdlib.h... yes > > > > checking for string.h... yes > > > > checking for memory.h... yes > > > > checking for strings.h... yes > > > > checking for inttypes.h... yes > > > > checking for stdint.h... yes > > > > checking for unistd.h... yes > > > > checking minix/config.h usability... no > > > > checking minix/config.h presence... no > > > > checking for minix/config.h... no > > > > checking whether it is safe to define __EXTENSIONS__... yes > > > > checking for the Android API level... not Android > > > > checking for --with-suffix... > > > > checking for case-insensitive build directory... yes > > > > checking LIBRARY... libpython$(VERSION)$(ABIFLAGS).a > > > > checking LINKCC... $(PURIFY) $(MAINCC) > > > > checking for GNU ld... no > > > > checking for --enable-shared... no > > > > checking for --enable-profiling... no > > > > checking LDLIBRARY... libpython$(VERSION)$(ABIFLAGS).a > > > > checking for ar... ar > > > > checking for readelf... no > > > > checking for a BSD-compatible install... /usr/local/bin/ginstall -c > > > > checking for a thread-safe mkdir -p... /usr/local/bin/gmkdir -p > > > > checking for --with-pydebug... no > > > > checking for --with-assertions... no > > > > checking for --enable-optimizations... yes > > > > checking for --with-lto... yes > > > > checking target system type... x86_64-apple-darwin18.6.0 > > > > checking for -llvm-ar... no > > > > checking for llvm-ar... '' > > > > configure: error: llvm-ar is required for a --with-lto build with clang but could not be found. > > > > > > I see /opt/local/bin/llvm-ar-mp-4.0 which ends up running the llvm-ar from > > > /opt/local/libexec/llvm-4.0/bin > > > > > > Just a guess but maybe you need to add /opt/local/libexec/llvm-4.0/bin/ to your path > > > as its where llvm-ar is. > > > > > > Barry > > > > > > > > > > > If I change the command to `... CC=cc` configuration appears to happen normally. > > > > > > > > On my version of macOS, cc is a symlink to clang. > > > > > > > > Why is this happening? What should I be doing differently to explicitly specify clang as my compiler? > > > > > > > > Thank you, > > > > Scott Colby > > > > -- > > > > https://mail.python.org/mailman/listinfo/python-list > > > > > From p_s_d_a_s_i_l_v_a_ns at netcabo.pt Wed Jul 17 00:38:50 2019 From: p_s_d_a_s_i_l_v_a_ns at netcabo.pt (Paulo da Silva) Date: Wed, 17 Jul 2019 05:38:50 +0100 Subject: super or not super? References: <849906fa-c70a-f431-20f7-27a7ee3d67c7@tjol.eu> Message-ID: ?s 02:11 de 15/07/19, Chris Angelico escreveu: > On Mon, Jul 15, 2019 at 10:51 AM Paulo da Silva > wrote: >> ... >> >> Thank you Jollans. I forgot multiple inheritance. I never needed it in >> python, so far. >> > > Something to consider is that super() becomes useful even if someone > else uses MI involving your class. Using super() ensures that your > class will play nicely in someone else's hierarchy, not just your own. > Yes, I see. Thanks. From barry at barrys-emacs.org Wed Jul 17 03:58:00 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Wed, 17 Jul 2019 08:58:00 +0100 Subject: Counting Python threads vs C/C++ threads In-Reply-To: References: Message-ID: > On 16 Jul 2019, at 20:48, Dan Stromberg wrote: > > > > On Tue, Jul 16, 2019 at 11:13 AM Barry Scott > wrote: > I'm going to assume you are on linux. > Yes, I am. Ubuntu 16.04.6 LTS sometimes, Mint 19.1 other times. > > On 16 Jul 2019, at 18:35, Dan Stromberg > wrote: > > > > I'm looking at a performance problem in a large CPython 2.x/3.x codebase > > with quite a few dependencies. > > > > I'm not sure what's causing the slowness yet. The CPU isn't getting hit > > hard, and I/O on the system appears to be low - but throughput is poor. > > I'm wondering if it could be CPU-bound Python threads causing the problem > > (because of the threading+GIL thing). > > Does top show the process using 100% CPU? > Nope. CPU utilization and disk use are both low. Then your problem is latency. You need to find the slow operation. > We've been going into top, and then hitting '1' to see things broken down by CPU core (there are 32 of them, probably counting hyperthreads as different cores), but the CPU use is in the teens or so. > > I've also tried dstat and csysdig. The hardware isn't breaking a sweat, but throughput is poor. > > The non-dependency Python portions don't Appear to have much in the way of > > threading going on based on a quick grep, but csysdig says a process > > running the code has around 32 threads running - the actual thread count > > varies, but that's the ballpark. > > > > I'm wondering if there's a good way to find two counts of those threads - > > how many are from CPython code that could run afoul of the GIL, and how > > many of them are from C/C++ extension modules that wouldn't be responsible > > for a GIL issue. > > >From the docs on threading: > > threading.active_count() > > Return the number of Thread objects currently alive. The returned count is equal to the length of the list returned by enumerate() . > > Are you on a Mac? Opss a file: link sorry should have search the online docs. I use many operating systems: Fedora, macOS, Windows, NetBSD, CentOS and others in the past. > > https://docs.python.org/2/library/threading.html appears to have some good info. I'll probably try logging threading.active_count() > > A question arises though: Does threading.active_count() only show Python threads created with the threading module? What about threads created with the thread module? Only pythons threads, if you think about it why would python care about threads it does not control? > > Try running strace on the process to see what system calls its making. > I've tried it, but thank you. It's a good suggestion. > > I often find that when strace'ing a program, there's a bunch of mostly-irrelevant stuff at Initial Program Load (IPL), but then the main loop fails into a small cycle of system calls. And what are thoses sys calls and what is the timing of them? If you are use select/poll how long before the call returns. If you in a read how long before it returns. > > Not with this program. Its main loop is busy and large. Does the code log any metrics or telemetry to help you? I work on a product that produces time-series data to show key information about the service. TPS, cache hit rates etc. Should have mention before you can run the code under python's cprofile. Do a test run against the process and then run analysis on the data that cprofile produces to find out elapse times and cpu times of the code. > > You could also connect gdb to the process and find out what code the threads are running. > > I used to use gdb, and wrappers for gdb, when I was doing C code, but I don't have much experience using it on a CPython interrpreter. > > Would I be doing a "thread apply all bt" or what? I'm guessing those backtraces could facilitate identifying the origin of a thread. Yes thread apply all bt works great on a python process. recent gdb releases knows how to format the stack and show you the python stack, forgot the command, but its easy to google for. Barry > > Thanks a bunch. > From tjol at tjol.eu Wed Jul 17 05:17:50 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Wed, 17 Jul 2019 11:17:50 +0200 Subject: Counting Python threads vs C/C++ threads In-Reply-To: References: Message-ID: On 17/07/2019 09.58, Barry Scott wrote: > >> On 16 Jul 2019, at 20:48, Dan Stromberg wrote: >> >> >> >> A question arises though: Does threading.active_count() only show Python threads created with the threading module? What about threads created with the thread module? > Only pythons threads, if you think about it why would python care about threads it does not control? As the docs say, this counts threading.Thread objects. It does not count all threads started from Python: threads started with the _thread module, for instance, are not included. What is more, threads started in external libraries can acquire the GIL and run Python code. A great example of this are QThreads in a PyQt5 application: QThreads are started by the Qt runtime, which calls a Qt slot. This Qt slot then might happen to be implemented in Python. I'm sure other libraries do similar things. Example with _thread just to check active_count behaviour: #!/usr/bin/env python3 import threading import _thread import time def thread_func(i): ??? print('Starting thread', i) ??? time.sleep(0.5) ??? print('Thread done', i) print('Using threading.Thread') t1 = threading.Thread(target=thread_func, args=(1,)) t1.start() time.sleep(0.1) print('active threads:', threading.active_count()) t1.join() print('Using threading & _thread') t1 = threading.Thread(target=thread_func, args=(1,)) t1.start() t2_id = _thread.start_new_thread(thread_func, (2,)) time.sleep(0.1) print('active threads:', threading.active_count()) time.sleep(0.6) print('Done, hopefully') From devel at jacobodevera.com Wed Jul 17 05:06:30 2019 From: devel at jacobodevera.com (Jacobo de Vera) Date: Wed, 17 Jul 2019 10:06:30 +0100 Subject: Why an InitVar pseudo field in dataclasses cannot have a default_factory? Message-ID: Hi all, I was surprised by an error when trying to set a default_factory for an InitVar pseudo-field in a dataclass. Inspecting the code in dataclasses.py led me to this: # Special restrictions for ClassVar and InitVar. if f._field_type in (_FIELD_CLASSVAR, _FIELD_INITVAR): if f.default_factory is not MISSING: raise TypeError(f'field {f.name} cannot have a ' 'default factory') # Should I check for other field settings? default_factory # seems the most serious to check for. Maybe add others. For # example, how about init=False (or really, # init=)? It makes no sense for # ClassVar and InitVar to specify init=. So this case is very explicitly prevented but I could not see why. Does anybody know what problem this is trying to prevent or what is the rationale behind this restriction? I asked in stackoverflow[1] and I was suggested to ask here. [1] https://stackoverflow.com/questions/57056029 Thanks, Jacobo de Vera @jovianjake From raju.mailinglists at gmail.com Wed Jul 17 08:49:31 2019 From: raju.mailinglists at gmail.com (kamaraju kusumanchi) Date: Wed, 17 Jul 2019 08:49:31 -0400 Subject: OT: Is there a name for this transformation? In-Reply-To: <20190710190556.auafngbtakbzig42@hjp.at> References: <20190710190556.auafngbtakbzig42@hjp.at> Message-ID: On Wed, Jul 10, 2019 at 3:08 PM Peter J. Holzer wrote: > > On 2019-07-10 08:57:29 -0400, kamaraju kusumanchi wrote: > > Given a csv file with the following contents > > > > 20180701, A > > 20180702, A, B > > 20180703, A, B, C > > 20180704, B, C > > 20180705, C > > > > I would like to transform the underlying data into a dataframe such as > > > > date, A, B, C > > 20180701, True, False, False > > 20180702, True, True, False > > 20180703, True, True, True > > 20180704, False, True, True > > 20180705, False, False, True > > > > the idea is that the first field in each line of the csv is the row > > index of the dataframe. The subsequent fields will be its column names > > and the values in the dataframe tell whether that element is present > > or not in the line. > > > > Is there a name for this transformation? > > This type of output is usually called a cross table, but I don't know > whether this specific transformation has a name (if you had only one of > A, B, and C per line it would be a kind of pivot operation). Thanks for telling me about cross table. I found out about cross-tabulation functionality in Pandas using pandas.crosstab() which is described in https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.crosstab.html As for my original problem, I solved it as follows: $cat data.csv 20180701, A 20180702, A, B 20180703, A, B, C 20180704, B, C 20180705, C import pandas as pd import numpy as np # expand the data into two numpy arrays such as # a = np.array(['20180701', '20180702', '20180702', '20180703', '20180703', '20180703', '20180704', '20180704', '20180705']) # b = np.array(['A', 'A', 'B', 'A', 'B', 'C', 'B', 'C', 'C']) rows = [] cols = [] with open('data.csv') as fo: for line in fo: line = line.strip() elem = line.split(',') N = len(elem) rows += elem[0:1] * (N-1) cols += elem[1:] a = np.array(rows) b = np.array(cols) df = pd.crosstab(a, b, rownames=['date']).astype('bool').reset_index() which gives print(df) col_0 date A B C 0 20180701 True False False 1 20180702 True True False 2 20180703 True True True 3 20180704 False True True 4 20180705 False False True -- Kamaraju S Kusumanchi | http://raju.shoutwiki.com/wiki/Blog From jesse.ibarra.1996 at gmail.com Wed Jul 17 11:57:00 2019 From: jesse.ibarra.1996 at gmail.com (jesse.ibarra.1996 at gmail.com) Date: Wed, 17 Jul 2019 08:57:00 -0700 (PDT) Subject: Embedding Python in C Message-ID: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> I am using Python3.6: [jibarra at redsky ~]$ python3.6 Python 3.6.8 (default, Apr 25 2019, 21:02:35) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux Type "help", "copyright", "credits" or "license" for more information. I am referencing:https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview Is there a way to call a shared C lib using PyObjects? Please advise. Thank you. From barry at barrys-emacs.org Wed Jul 17 13:54:33 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Wed, 17 Jul 2019 18:54:33 +0100 Subject: Embedding Python in C In-Reply-To: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> Message-ID: > On 17 Jul 2019, at 16:57, jesse.ibarra.1996 at gmail.com wrote: > > I am using Python3.6: > > [jibarra at redsky ~]$ python3.6 > Python 3.6.8 (default, Apr 25 2019, 21:02:35) > [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux > Type "help", "copyright", "credits" or "license" for more information. > > > I am referencing:https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview > > Is there a way to call a shared C lib using PyObjects? If what you want to call is simple enough then you can use the ctypes library that ships with python. If the code you want to call is more complex you will want to use one of a number of libraries to help you create a module that you can import. I use PyCXX for this purpose that allows me to write C++ code that can call C++ and C libs and interface easily with python. Home page http://cxx.sourceforge.net/ the source kit contains demo code that you shows how to cerate a module, a class and function etc. Example code: https://sourceforge.net/p/cxx/code/HEAD/tree/trunk/CXX/Demo/Python3/simple.cxx Barry PyCXX maintainer > > Please advise. > > Thank you. > -- > https://mail.python.org/mailman/listinfo/python-list > From jesse.ibarra.1996 at gmail.com Wed Jul 17 14:39:27 2019 From: jesse.ibarra.1996 at gmail.com (Jesse Ibarra) Date: Wed, 17 Jul 2019 11:39:27 -0700 (PDT) Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> Message-ID: <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> On Wednesday, July 17, 2019 at 11:55:28 AM UTC-6, Barry Scott wrote: > > On 17 Jul 2019, at 16:57, wrote: > > > > I am using Python3.6: > > > > [jibarra at redsky ~]$ python3.6 > > Python 3.6.8 (default, Apr 25 2019, 21:02:35) > > [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux > > Type "help", "copyright", "credits" or "license" for more information. > > > > > > I am referencing:https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview > > > > Is there a way to call a shared C lib using PyObjects? > > If what you want to call is simple enough then you can use the ctypes library > that ships with python. > > If the code you want to call is more complex you will want to use one of a number of libraries to help > you create a module that you can import. > > I use PyCXX for this purpose that allows me to write C++ code that can call C++ and C libs and interface > easily with python. Home page http://cxx.sourceforge.net/ the source kit contains demo code that you shows > how to cerate a module, a class and function etc. > > Example code: https://sourceforge.net/p/cxx/code/HEAD/tree/trunk/CXX/Demo/Python3/simple.cxx > > Barry > PyCXX maintainer > > > > > Please advise. > > > > Thank you. > > -- > > https://mail.python.org/mailman/listinfo/python-list > > My options seem rather limited, I need to make a Pipeline from (Smalltalk -> C -> Python) then go back (Smalltalk <- C <- Python). Since Smalltalk does not support Python directly I have to settle with the C/Python API (https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview). Any suggestions? From PythonList at DancesWithMice.info Wed Jul 17 16:08:04 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Thu, 18 Jul 2019 08:08:04 +1200 Subject: Embedding classes' names In-Reply-To: <20190716105738.GA83475@cskk.homeip.net> References: <20190716105738.GA83475@cskk.homeip.net> Message-ID: <46fb0acd-922d-6081-f941-4ae3417cdb1e@DancesWithMice.info> On 16/07/19 10:57 PM, Cameron Simpson wrote: > On 16Jul2019 10:20, Chris Angelico wrote: >> On Tue, Jul 16, 2019 at 10:17 AM DL Neil >> wrote: >>> When used, do you embed a class's name within its own code, as a >>> literal? > [...] >>> So, what about other situations where one might need to access the >>> class's own name or that of its/a super-class? eg >>> >>> class C2(C1): >>> ??????? def __init__(self, fred, barney ): >>> ??????????????? super().__init__( fred ) >>> ??????????????? self.barney = barney >>> >>> ??????? def __repr__( self ): >>> ??????????????? return f"C2( { self.fred }, { self.barney }" >>> ??????????????? ### note: 'common practice' of "C2" embedded as constant >>> >>> >>> How 'purist' do you go, cf YAGNI? >> >> In the case of __repr__, I would most definitely use >> self.__class__.__name__, because that way, a subclass can leave repr >> untouched and still get decent behaviour. > > Yeah, me too, though I spell it "type(self).__name__" for no totally > rational reason. +1 (looking through my code - perhaps at one time it was 'the old way'?) -- Regards =dn From auriocus at gmx.de Wed Jul 17 16:20:37 2019 From: auriocus at gmx.de (Christian Gollwitzer) Date: Wed, 17 Jul 2019 22:20:37 +0200 Subject: Embedding Python in C In-Reply-To: <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> Message-ID: Am 17.07.19 um 20:39 schrieb Jesse Ibarra: > My options seem rather limited, I need to make a Pipeline from (Smalltalk -> C -> Python) then go back (Smalltalk <- C <- Python). Since Smalltalk does not support Python directly I have to settle with the C/Python API (https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview). Any suggestions? > Ah, now you finally tell us your problem! Depending on, how complete / advanced / efficient the bridge needs to be, it can be easy or hard. What level of integration do you want to achieve? Do you want a) to call Python functions from Smalltalk b) call Smalltalk functions from Python c) pass callbacks around, e.g. use a Smalltalk function within a Python list comprehension, and if so, which way d) integrate the class systems - derive a Python class from a Smalltalk base or the other way round e) ? The most basic thing is a), but even getting that right might be non-trivial, since both C APIs will have different type systems which you need to match. I don't speak Smalltalk, so can't comment in detail on this - but in practice it will also depend on the implementation you are using. Best regards, Christian From PythonList at DancesWithMice.info Wed Jul 17 16:53:15 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Thu, 18 Jul 2019 08:53:15 +1200 Subject: super() in Python 3 In-Reply-To: References: Message-ID: On 16/07/19 10:08 PM, ???? wrote: > Hi, > > 1. When we use super() in Python 3, we don't pass it the first argument > (self). Why? > > What happens if the first argument is not self? > > def __init__(self, *args, **kwargs): > super().__init__(*args, **kwargs) > > I think it would make more sense to use something like > self.super().__init__(*args, **kwargs) or something like this. NB folk coming to Python from other (programming) languages are often surprised to discover that a sub-class does not automatically execute the initialiser/constructor of its parent. The extra control offered is (IMHO) both subtle and powerful! I'm not sure about this (and perhaps better minds will clarify): isn't self about an instance, whereas super() is about a class? Another way to look at it is to refer to the super-class as the 'parent'. Thinking of yourself, do?did you address him as "my father" (self.father) or her as "my mother", or is the possessive description (the "my"), unnecessary? Whilst it wouldn't be at all wrong to address him as "my father", in English (and/or in other tongues), what happened when you tried it in Python? Perhaps then, we only use the "my" descriptor when we need to distinguish between multiple parents, eg yours cf mine? Even with multiple-inheritance*, Python's "MRO" saves us from needing to do that! * yes Python can, even if xyz-other-language couldn't! (Python's ability to track all this sounds like something from the xkcd Comic: with Python you will never again cry, "I've lost my mummy"...) -- Regards =dn From Irv at furrypants.com Wed Jul 17 17:02:58 2019 From: Irv at furrypants.com (Irv Kalb) Date: Wed, 17 Jul 2019 14:02:58 -0700 Subject: join and split with empty delimiter Message-ID: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> I have always thought that split and join are opposite functions. For example, you can use a comma as a delimiter: >>> myList = ['a', 'b', 'c', 'd', 'e'] >>> myString = ','.join(myList) >>> print(myString) a,b,c,d,e >>> myList = myString.split(',') >>> print(myList) ['a', 'b', 'c', 'd', 'e'] Works great. But i've found a case where they don't work that way. If I join the list with the empty string as the delimiter: >>> myList = ['a', 'b', 'c', 'd'] >>> myString = ''.join(myList) >>> print(myString) abcd That works great. But attempting to split using the empty string generates an error: >>> myString.split('') Traceback (most recent call last): File "", line 1, in myString.split('') ValueError: empty separator I know that this can be accomplished using the list function: >>> myString = list(myString) >>> print(myString) ['a', 'b', 'c', 'd'] But my question is: Is there any good reason why the split function should give an "empty separator" error? I think the meaning of trying to split a string into a list using the empty string as a delimiter is unambiguous - it should just create a list of single characters strings like the list function does here. My guess is that by definition, the split function attempts to separate the string wherever it finds the delimiter between characters, and because in this case its the empty string, it gives an error. But if it's going to check for the empty string anyway, it could just call the list function and return a list of characters. Irv From rosuav at gmail.com Wed Jul 17 17:24:36 2019 From: rosuav at gmail.com (Chris Angelico) Date: Thu, 18 Jul 2019 07:24:36 +1000 Subject: join and split with empty delimiter In-Reply-To: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> References: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> Message-ID: On Thu, Jul 18, 2019 at 7:06 AM Irv Kalb wrote: > If I join the list with the empty string as the delimiter: > > >>> myList = ['a', 'b', 'c', 'd'] > >>> myString = ''.join(myList) > >>> print(myString) > abcd > > That works great. But attempting to split using the empty string generates an error: > > >>> myString.split('') > Traceback (most recent call last): > File "", line 1, in > myString.split('') > ValueError: empty separator > > But my question is: Is there any good reason why the split function should give an "empty separator" error? I think the meaning of trying to split a string into a list using the empty string as a delimiter is unambiguous - it should just create a list of single characters strings like the list function does here. > Agreed. There are a number of other languages where splitting on an empty delimiter simply fractures the string into characters (I checked Pike, JavaScript, Tcl, and Ruby), and it's a practical and useful feature. +1. ChrisA From info at tundraware.com Wed Jul 17 19:24:17 2019 From: info at tundraware.com (Tim Daneliuk) Date: Wed, 17 Jul 2019 18:24:17 -0500 Subject: join and split with empty delimiter In-Reply-To: References: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> Message-ID: <1an40g-8813.ln1@oceanview.tundraware.com> On 7/17/19 4:24 PM, Chris Angelico wrote: > Agreed. There are a number of other languages where splitting on an > empty delimiter simply fractures the string into characters (I checked > Pike, JavaScript, Tcl, and Ruby), and it's a practical and useful > feature. +1. Not only that, it makes the language more symmetric/consistent. Put me down for +1 as well. From python at mrabarnett.plus.com Wed Jul 17 20:35:38 2019 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 18 Jul 2019 01:35:38 +0100 Subject: join and split with empty delimiter In-Reply-To: <1an40g-8813.ln1@oceanview.tundraware.com> References: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> <1an40g-8813.ln1@oceanview.tundraware.com> Message-ID: On 2019-07-18 00:24, Tim Daneliuk wrote: > On 7/17/19 4:24 PM, Chris Angelico wrote: >> Agreed. There are a number of other languages where splitting on an >> empty delimiter simply fractures the string into characters (I checked >> Pike, JavaScript, Tcl, and Ruby), and it's a practical and useful >> feature. +1. > > Not only that, it makes the language more symmetric/consistent. Put > me down for +1 as well. > Since the fix in the re module in Python 3.7, it can split on an empty string: >>> import re >>> re.split('', 'abc') ['', 'a', 'b', 'c', ''] which gives us the chance to bikeshed on whether str.split should do the same. (In case you're wondering, there _is_ an empty string (the delimiter) before the first character and after the last character.) From dieter at handshake.de Thu Jul 18 00:41:32 2019 From: dieter at handshake.de (dieter) Date: Thu, 18 Jul 2019 06:41:32 +0200 Subject: Embedding Python in C References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> Message-ID: <87ef2okl8j.fsf@handshake.de> Jesse Ibarra writes: > ... > My options seem rather limited, I need to make a Pipeline from (Smalltalk -> C -> Python) then go back (Smalltalk <- C <- Python). Since Smalltalk does not support Python directly I have to settle with the C/Python API (https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview). Any suggestions? Decades ago, I implemented something similar (Chomsky <-> C/C++ <-> Python). Thus, it is possible. Ensure that you acquire the GIL when you enter the Python world (almost all Python API function have the implicit precondition that the GIL is hold); and release it when you leave the Python world. I forgot whether it was necessary to initialize Python as a whole (likely, it is). Maybe, you consider also a looser coupling. In my scenario above, the various components could be integrated via CORBA (= "Common Object Request Broker Architecture"). From barry at barrys-emacs.org Thu Jul 18 03:16:31 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Thu, 18 Jul 2019 08:16:31 +0100 Subject: Embedding Python in C In-Reply-To: <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> Message-ID: > On 17 Jul 2019, at 19:39, Jesse Ibarra wrote: > > On Wednesday, July 17, 2019 at 11:55:28 AM UTC-6, Barry Scott wrote: >>> On 17 Jul 2019, at 16:57, wrote: >>> >>> I am using Python3.6: >>> >>> [jibarra at redsky ~]$ python3.6 >>> Python 3.6.8 (default, Apr 25 2019, 21:02:35) >>> [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux >>> Type "help", "copyright", "credits" or "license" for more information. >>> >>> >>> I am referencing:https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview >>> >>> Is there a way to call a shared C lib using PyObjects? >> >> If what you want to call is simple enough then you can use the ctypes library >> that ships with python. >> >> If the code you want to call is more complex you will want to use one of a number of libraries to help >> you create a module that you can import. >> >> I use PyCXX for this purpose that allows me to write C++ code that can call C++ and C libs and interface >> easily with python. Home page http://cxx.sourceforge.net/ the source kit contains demo code that you shows >> how to cerate a module, a class and function etc. >> >> Example code: https://sourceforge.net/p/cxx/code/HEAD/tree/trunk/CXX/Demo/Python3/simple.cxx >> >> Barry >> PyCXX maintainer >> >>> >>> Please advise. >>> >>> Thank you. >>> -- >>> https://mail.python.org/mailman/listinfo/python-list >>> > > My options seem rather limited, I need to make a Pipeline from (Smalltalk -> C -> Python) then go back (Smalltalk <- C <- Python). Since Smalltalk does not support Python directly I have to settle with the C/Python API (https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview ). Any suggestions? 1. Run a new python process for each "call" you need to make. 2. Start a subprocess running python that talks via pipes to smalltalk and send messages back and forth to get the work done. 3. Write a C (I'd use C++ myself and PyCXX to avoid the complexity of the Python C API) extension to Smalltalk that initialises a python interpreter and bridge calls from Smalltalk into Python on that intepreter. Depending on the performance you need and the amount of data involved will help decide what is a reasonable design to choose. Barry > -- > https://mail.python.org/mailman/listinfo/python-list From ben.usenet at bsb.me.uk Thu Jul 18 06:27:48 2019 From: ben.usenet at bsb.me.uk (Ben Bacarisse) Date: Thu, 18 Jul 2019 11:27:48 +0100 Subject: join and split with empty delimiter References: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> Message-ID: <878ssvzlgb.fsf@bsb.me.uk> Irv Kalb writes: > I have always thought that split and join are opposite functions. For > example, you can use a comma as a delimiter: > >>>> myList = ['a', 'b', 'c', 'd', 'e'] >>>> myString = ','.join(myList) >>>> print(myString) > a,b,c,d,e > >>>> myList = myString.split(',') >>>> print(myList) > ['a', 'b', 'c', 'd', 'e'] > > Works great. Note that join and split do not always recover the same list: >>> ','.join(['a', 'b,c', 'd']).split(',') ['a', 'b', 'c', 'd'] You don't even have to have the delimiter in one of the strings: >>> '//'.join(['a', 'b/', 'c']).split('//') ['a', 'b', '/c'] > But i've found a case where they don't work that way. If > I join the list with the empty string as the delimiter: > >>>> myList = ['a', 'b', 'c', 'd'] >>>> myString = ''.join(myList) >>>> print(myString) > abcd > > That works great. But attempting to split using the empty string > generates an error: > >>>> myString.split('') > Traceback (most recent call last): > File "", line 1, in > myString.split('') > ValueError: empty separator > > I know that this can be accomplished using the list function: > >>>> myString = list(myString) >>>> print(myString) > ['a', 'b', 'c', 'd'] > > But my question is: Is there any good reason why the split function > should give an "empty separator" error? I think the meaning of trying > to split a string into a list using the empty string as a delimiter is > unambiguous - it should just create a list of single characters > strings like the list function does here. One reason might be that str.split('') is not unambiguous. For example, there's a case to be made that there is a '' delimiter at the start and the end of the string as well as between letters. '' is a very special delimiter because every string that gets joined using it includes it! It's a wild version of ','.join(['a', 'b,c', 'd']).split(','). Of course str.split('') could be defined to work the way you expect, but it's possible that the error is there to prompt the programmer to be more explicit. -- Ben. From Richard at Damon-Family.org Thu Jul 18 07:46:01 2019 From: Richard at Damon-Family.org (Richard Damon) Date: Thu, 18 Jul 2019 07:46:01 -0400 Subject: join and split with empty delimiter In-Reply-To: <878ssvzlgb.fsf@bsb.me.uk> References: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> <878ssvzlgb.fsf@bsb.me.uk> Message-ID: <26e077df-011f-58cd-5158-86707036221b@Damon-Family.org> On 7/18/19 6:27 AM, Ben Bacarisse wrote: > One reason might be that str.split('') is not unambiguous. For example, > there's a case to be made that there is a '' delimiter at the start and > the end of the string as well as between letters. '' is a very special > delimiter because every string that gets joined using it includes it! > It's a wild version of ','.join(['a', 'b,c', 'd']).split(','). > > Of course str.split('') could be defined to work the way you expect, but > it's possible that the error is there to prompt the programmer to be > more explicit. One thought I am having is that if the split is explicitly on '', then it is easy enough to use list(), but if it isn't a hard coded '', then the gotchas described above? are significant, that unless the join list was built of exactly 1 character strings, the split won't match, and that seems somewhat special case for a variable delimiter. -- Richard Damon From antoon.pardon at vub.be Thu Jul 18 08:52:06 2019 From: antoon.pardon at vub.be (Antoon Pardon) Date: Thu, 18 Jul 2019 14:52:06 +0200 Subject: Extending an Enum type Message-ID: <1b75db59-87bf-8861-d0a4-b92e1ad4cdcd@vub.be> Hi, I am experimenting with writing an Earley Parser. Now I would like to have the non-terminals from the grammer I am reading in, be represented bye an enum like type. So that if the grammer contains the following production: Term -> Term '+' Factor I can reprensent the right hand side with a list that gets printed something like the following: [, '+', ] I am a bit at a loss right now on how to start. Can someone point me in the right direction? -- Antoon Pardon From antoon.pardon at vub.be Thu Jul 18 09:04:24 2019 From: antoon.pardon at vub.be (Antoon Pardon) Date: Thu, 18 Jul 2019 15:04:24 +0200 Subject: Extendable Enum like Type? Message-ID: <3148ba72-a667-cbd4-6123-fabc886e84e9@vub.be> Something seems to have gone wrong with the formatting of my latest contribution, so let me try again. I am experimenting with writing an Earley Parser. Now I would like to have the non-terminals from the grammer I am reading in, be represented bye an enum like type. So that if the grammer contains the following production: Term -> Term '+' Factor I can reprensent the right hand side with a list that gets printed something like the following: [, '+', ] I am a bit at a loss right now on how to start. Can someone point me in the right direction? -- Antoon Pardon From jesse.ibarra.1996 at gmail.com Thu Jul 18 10:18:16 2019 From: jesse.ibarra.1996 at gmail.com (Jesse Ibarra) Date: Thu, 18 Jul 2019 07:18:16 -0700 (PDT) Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> Message-ID: <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> On Wednesday, July 17, 2019 at 2:20:51 PM UTC-6, Christian Gollwitzer wrote: > Am 17.07.19 um 20:39 schrieb Jesse Ibarra: > > My options seem rather limited, I need to make a Pipeline from (Smalltalk -> C -> Python) then go back (Smalltalk <- C <- Python). Since Smalltalk does not support Python directly I have to settle with the C/Python API (https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview). Any suggestions? > > > > Ah, now you finally tell us your problem! > > Depending on, how complete / advanced / efficient the bridge needs to > be, it can be easy or hard. > > What level of integration do you want to achieve? Do you want > > a) to call Python functions from Smalltalk > b) call Smalltalk functions from Python > c) pass callbacks around, e.g. use a Smalltalk function within a Python > list comprehension, and if so, which way > d) integrate the class systems - derive a Python class from a Smalltalk > base or the other way round > > e) ? > > > The most basic thing is a), but even getting that right might be > non-trivial, since both C APIs will have different type systems which > you need to match. I don't speak Smalltalk, so can't comment in detail > on this - but in practice it will also depend on the implementation you > are using. > > Best regards, > > Christian For right now, I need to call a .so file from Smalltalk. I can't explicitly use Python libraries since Smalltalk does not support Python. I need to use the C/Python API (in Smalltalk) to create a bridge where I can call a .so and a function in that .so file with a PyObject (This will be called back to Smalltalk from the .so file). I can't figure out a way to do that since the C/API can't call C or .so files. sorry for the confusion on my part From ethan at stoneleaf.us Thu Jul 18 12:14:09 2019 From: ethan at stoneleaf.us (Ethan Furman) Date: Thu, 18 Jul 2019 09:14:09 -0700 Subject: Extendable Enum like Type? In-Reply-To: <3148ba72-a667-cbd4-6123-fabc886e84e9@vub.be> References: <3148ba72-a667-cbd4-6123-fabc886e84e9@vub.be> Message-ID: On 07/18/2019 06:04 AM, Antoon Pardon wrote: > I am experimenting with writing an Earley Parser. Now I would like to > have the non-terminals from the grammer I am reading in, be represented > bye an enum like type. So that if the grammer contains the following > production: Term -> Term '+' Factor I can reprensent the right hand side > with a list that gets printed something like the following: > [, '+', ] I am a bit at a > loss right now on how to start. Can someone point me in the right > direction? The basic method is: from enum import Enum # `from aenum` [1][2] if less than Python 3.4 Class NonTerminal(Enum): Term = 1 Factor = 2 ... The docs [3] also have a lot of information. Does that answer your question? -- ~Ethan~ [1] https://pypi.org/project/aenum/ [2] I am the author of Enum, aenum, and the enum34 backport. [3] https://docs.python.org/3/library/enum.html From daniloco at acm.org Thu Jul 18 15:44:27 2019 From: daniloco at acm.org (Danilo Coccia) Date: Thu, 18 Jul 2019 21:44:27 +0200 Subject: join and split with empty delimiter In-Reply-To: <878ssvzlgb.fsf@bsb.me.uk> References: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> <878ssvzlgb.fsf@bsb.me.uk> Message-ID: Il 18/07/2019 12:27, Ben Bacarisse ha scritto: > Irv Kalb writes: > >> I have always thought that split and join are opposite functions. For >> example, you can use a comma as a delimiter: >> >>>>> myList = ['a', 'b', 'c', 'd', 'e'] >>>>> myString = ','.join(myList) >>>>> print(myString) >> a,b,c,d,e >> >>>>> myList = myString.split(',') >>>>> print(myList) >> ['a', 'b', 'c', 'd', 'e'] >> >> Works great. > > Note that join and split do not always recover the same list: > >>>> ','.join(['a', 'b,c', 'd']).split(',') > ['a', 'b', 'c', 'd'] > > You don't even have to have the delimiter in one of the strings: > >>>> '//'.join(['a', 'b/', 'c']).split('//') > ['a', 'b', '/c'] > >> But i've found a case where they don't work that way. If >> I join the list with the empty string as the delimiter: >> >>>>> myList = ['a', 'b', 'c', 'd'] >>>>> myString = ''.join(myList) >>>>> print(myString) >> abcd >> >> That works great. But attempting to split using the empty string >> generates an error: >> >>>>> myString.split('') >> Traceback (most recent call last): >> File "", line 1, in >> myString.split('') >> ValueError: empty separator >> >> I know that this can be accomplished using the list function: >> >>>>> myString = list(myString) >>>>> print(myString) >> ['a', 'b', 'c', 'd'] >> >> But my question is: Is there any good reason why the split function >> should give an "empty separator" error? I think the meaning of trying >> to split a string into a list using the empty string as a delimiter is >> unambiguous - it should just create a list of single characters >> strings like the list function does here. > > One reason might be that str.split('') is not unambiguous. For example, > there's a case to be made that there is a '' delimiter at the start and > the end of the string as well as between letters. '' is a very special > delimiter because every string that gets joined using it includes it! > It's a wild version of ','.join(['a', 'b,c', 'd']).split(','). > > Of course str.split('') could be defined to work the way you expect, but > it's possible that the error is there to prompt the programmer to be > more explicit. > It is even more ambiguous if you consider that any string starts with an infinite number of empty strings, followed by a character, followed by an infinite number of empty strings, followed by ... The result wouldn't fit on screen, or in memory for that! From auriocus at gmx.de Thu Jul 18 15:45:50 2019 From: auriocus at gmx.de (Christian Gollwitzer) Date: Thu, 18 Jul 2019 21:45:50 +0200 Subject: Embedding Python in C In-Reply-To: <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> Message-ID: Am 18.07.19 um 16:18 schrieb Jesse Ibarra: > On Wednesday, July 17, 2019 at 2:20:51 PM UTC-6, Christian Gollwitzer wrote: >> What level of integration do you want to achieve? Do you want >> >> a) to call Python functions from Smalltalk >> b) call Smalltalk functions from Python >> c) pass callbacks around, e.g. use a Smalltalk function within a Python >> list comprehension, and if so, which way >> d) integrate the class systems - derive a Python class from a Smalltalk >> base or the other way round >> > > For right now, I need to call a .so file from Smalltalk. I can't explicitly use Python libraries since Smalltalk does not support Python. I need to use the C/Python API (in Smalltalk) to create a bridge where I can call a .so and a function in that .so file with a PyObject (This will be called back to Smalltalk from the .so file). I can't figure out a way to do that since the C/API can't call C or .so files. > > sorry for the confusion on my part > I still don't get it, sorry. To me it is unclear which part of the integration you manage to do so far, and which part is the problem. Which Smalltalk interpreter are you using? The answer to the following will heavily depend on that. Suppose I'd give you a C file with the following simple function: double add(double a, double b) { return a+b; } Do you know how to compile this code and make it usable from Smalltalk? One level up, consider a C function working on an array: double arrsum(int n, double *arr) { double sum=0.0; for (int i=0; i <878ssvzlgb.fsf@bsb.me.uk> Message-ID: <87r26nxgqm.fsf@bsb.me.uk> Danilo Coccia writes: > Il 18/07/2019 12:27, Ben Bacarisse ha scritto: >> Irv Kalb writes: >> >>> I have always thought that split and join are opposite functions. For >>> example, you can use a comma as a delimiter: >>> >>>>>> myList = ['a', 'b', 'c', 'd', 'e'] >>>>>> myString = ','.join(myList) >>>>>> print(myString) >>> a,b,c,d,e >>> >>>>>> myList = myString.split(',') >>>>>> print(myList) >>> ['a', 'b', 'c', 'd', 'e'] >>> >>> Works great. >> >> Note that join and split do not always recover the same list: >> >>>>> ','.join(['a', 'b,c', 'd']).split(',') >> ['a', 'b', 'c', 'd'] >> >> You don't even have to have the delimiter in one of the strings: >> >>>>> '//'.join(['a', 'b/', 'c']).split('//') >> ['a', 'b', '/c'] >> >>> But i've found a case where they don't work that way. If >>> I join the list with the empty string as the delimiter: >>> >>>>>> myList = ['a', 'b', 'c', 'd'] >>>>>> myString = ''.join(myList) >>>>>> print(myString) >>> abcd >>> >>> That works great. But attempting to split using the empty string >>> generates an error: >>> >>>>>> myString.split('') >>> Traceback (most recent call last): >>> File "", line 1, in >>> myString.split('') >>> ValueError: empty separator >>> >>> I know that this can be accomplished using the list function: >>> >>>>>> myString = list(myString) >>>>>> print(myString) >>> ['a', 'b', 'c', 'd'] >>> >>> But my question is: Is there any good reason why the split function >>> should give an "empty separator" error? I think the meaning of trying >>> to split a string into a list using the empty string as a delimiter is >>> unambiguous - it should just create a list of single characters >>> strings like the list function does here. >> >> One reason might be that str.split('') is not unambiguous. For example, >> there's a case to be made that there is a '' delimiter at the start and >> the end of the string as well as between letters. '' is a very special >> delimiter because every string that gets joined using it includes it! >> It's a wild version of ','.join(['a', 'b,c', 'd']).split(','). >> >> Of course str.split('') could be defined to work the way you expect, but >> it's possible that the error is there to prompt the programmer to be >> more explicit. > > It is even more ambiguous if you consider that any string starts with an > infinite number of empty strings, followed by a character, followed by > an infinite number of empty strings, followed by ... > The result wouldn't fit on screen, or in memory for that! Right, but that can be finessed by saying that two delimiters can't overlap, which is the usual rule. A reasonable interpretation of "not overlapping" might well exclude having more the one delimiter in the same place. -- Ben. From rosuav at gmail.com Thu Jul 18 16:01:10 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 19 Jul 2019 06:01:10 +1000 Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> Message-ID: On Fri, Jul 19, 2019 at 5:51 AM Christian Gollwitzer wrote: > Once you can do this, you can proceed to call a Python function, which > in C means that you invoke the function PyObject_CallObject(). A basic > example is shown here: > > https://docs.python.org/2/extending/embedding.html#pure-embedding > Or, better: https://docs.python.org/3/extending/embedding.html#pure-embedding ChrisA From python at mrabarnett.plus.com Thu Jul 18 21:42:26 2019 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 19 Jul 2019 02:42:26 +0100 Subject: join and split with empty delimiter In-Reply-To: <87r26nxgqm.fsf@bsb.me.uk> References: <5DD0A866-07A6-447D-902A-DADAC6870CCB@furrypants.com> <878ssvzlgb.fsf@bsb.me.uk> <87r26nxgqm.fsf@bsb.me.uk> Message-ID: <6964a90c-5169-0f9e-2e3c-9946efcb5900@mrabarnett.plus.com> On 2019-07-18 20:52, Ben Bacarisse wrote: > Danilo Coccia writes: > >> Il 18/07/2019 12:27, Ben Bacarisse ha scritto: [snip] >>> Of course str.split('') could be defined to work the way you expect, but >>> it's possible that the error is there to prompt the programmer to be >>> more explicit. >> >> It is even more ambiguous if you consider that any string starts with an >> infinite number of empty strings, followed by a character, followed by >> an infinite number of empty strings, followed by ... >> The result wouldn't fit on screen, or in memory for that! > > Right, but that can be finessed by saying that two delimiters can't > overlap, which is the usual rule. A reasonable interpretation of "not > overlapping" might well exclude having more the one delimiter in the > same place. > The delimiters wouldn't be overlapping, they'd be adjacent, but it does seem reasonable not to split on an empty delimiter more than once at a certain position. That's what a regex split (usually) does (in the re module since Python 3.7, and in other regex implementations). From antoon.pardon at vub.be Fri Jul 19 04:23:20 2019 From: antoon.pardon at vub.be (Antoon Pardon) Date: Fri, 19 Jul 2019 10:23:20 +0200 Subject: Extendable Enum like Type? In-Reply-To: References: <3148ba72-a667-cbd4-6123-fabc886e84e9@vub.be> Message-ID: On 18/07/19 18:14, Ethan Furman wrote: > On 07/18/2019 06:04 AM, Antoon Pardon wrote: > >> I am experimenting with writing an Earley Parser. Now I would like to >> have the non-terminals from the grammer I am reading in, be represented >> bye an enum like type. So that if the grammer contains the following >> production: Term -> Term '+' Factor I can reprensent the right hand side >> with a list that gets printed something like the following: >> [, '+', ] I am a bit at a >> loss right now on how to start. Can someone point me in the right >> direction? > > The basic method is: > > from enum import Enum???? # `from aenum` [1][2] if less than Python 3.4 > > Class NonTerminal(Enum): > ??? Term = 1 > ??? Factor = 2 > ??? ... > > The docs [3] also have a lot of information. > > Does that answer your question? I don't seem to have made myself clear. The grammar with its Terminals and NonTerminals is read in from a file. The program doesn't know what they will be. For the moment what I am thinking about is something as follows: grammar = LoadGrammer(GrammarFile) EnumList = ['class NonTerminal(Enum):\n'] for Index, NonTerminal in enumerate(grammar.NonTerminals): EnumList.append(' %s = %d\n' % (NonTerminal, Index)) exec(''.join(EnumList), ..., ...) The problem with this approach is that I can't use these values until the whole grammer is loaded. I would prefer some way to add values during the loading of the grammar. From jesse.ibarra.1996 at gmail.com Fri Jul 19 10:12:06 2019 From: jesse.ibarra.1996 at gmail.com (Jesse Ibarra) Date: Fri, 19 Jul 2019 07:12:06 -0700 (PDT) Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> Message-ID: <808e7c7b-a405-48ef-a465-e0a4663fbdbf@googlegroups.com> On Thursday, July 18, 2019 at 2:01:39 PM UTC-6, Chris Angelico wrote: > On Fri, Jul 19, 2019 at 5:51 AM Christian Gollwitzer wrote: > > Once you can do this, you can proceed to call a Python function, which > > in C means that you invoke the function PyObject_CallObject(). A basic > > example is shown here: > > > > https://docs.python.org/2/extending/embedding.html#pure-embedding > > > > Or, better: > > https://docs.python.org/3/extending/embedding.html#pure-embedding > > ChrisA I need to call a .so file, but I don't know I way to do that with PyObject. I can only seem to bring in .py files From rosuav at gmail.com Fri Jul 19 10:17:18 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 20 Jul 2019 00:17:18 +1000 Subject: Embedding Python in C In-Reply-To: <808e7c7b-a405-48ef-a465-e0a4663fbdbf@googlegroups.com> References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> <808e7c7b-a405-48ef-a465-e0a4663fbdbf@googlegroups.com> Message-ID: On Sat, Jul 20, 2019 at 12:16 AM Jesse Ibarra wrote: > > On Thursday, July 18, 2019 at 2:01:39 PM UTC-6, Chris Angelico wrote: > > On Fri, Jul 19, 2019 at 5:51 AM Christian Gollwitzer wrote: > > > Once you can do this, you can proceed to call a Python function, which > > > in C means that you invoke the function PyObject_CallObject(). A basic > > > example is shown here: > > > > > > https://docs.python.org/2/extending/embedding.html#pure-embedding > > > > > > > Or, better: > > > > https://docs.python.org/3/extending/embedding.html#pure-embedding > > > > ChrisA > > I need to call a .so file, but I don't know I way to do that with PyObject. I can only seem to bring in .py files I have vastly insufficient context to be able to even attempt to answer this. ChrisA From jesse.ibarra.1996 at gmail.com Fri Jul 19 10:24:00 2019 From: jesse.ibarra.1996 at gmail.com (Jesse Ibarra) Date: Fri, 19 Jul 2019 07:24:00 -0700 (PDT) Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> Message-ID: <05299e2d-c491-4f6b-9e7a-196af5e9d112@googlegroups.com> On Thursday, July 18, 2019 at 1:46:05 PM UTC-6, Christian Gollwitzer wrote: > Am 18.07.19 um 16:18 schrieb Jesse Ibarra: > > On Wednesday, July 17, 2019 at 2:20:51 PM UTC-6, Christian Gollwitzer wrote: > >> What level of integration do you want to achieve? Do you want > >> > >> a) to call Python functions from Smalltalk > >> b) call Smalltalk functions from Python > >> c) pass callbacks around, e.g. use a Smalltalk function within a Python > >> list comprehension, and if so, which way > >> d) integrate the class systems - derive a Python class from a Smalltalk > >> base or the other way round > >> > > > > > For right now, I need to call a .so file from Smalltalk. I can't explicitly use Python libraries since Smalltalk does not support Python. I need to use the C/Python API (in Smalltalk) to create a bridge where I can call a .so and a function in that .so file with a PyObject (This will be called back to Smalltalk from the .so file). I can't figure out a way to do that since the C/API can't call C or .so files. > > > > sorry for the confusion on my part > > > > I still don't get it, sorry. To me it is unclear which part of the > integration you manage to do so far, and which part is the problem. > > Which Smalltalk interpreter are you using? The answer to the following > will heavily depend on that. > > > Suppose I'd give you a C file with the following simple function: > > > double add(double a, double b) { > return a+b; > } > > Do you know how to compile this code and make it usable from Smalltalk? > > One level up, consider a C function working on an array: > > double arrsum(int n, double *arr) { > double sum=0.0; > for (int i=0; i return sum; > } > > > How would you compile and link this with your Smalltalk implementation, > such that you can pass it a Smalltalk array? > > Once you can do this, you can proceed to call a Python function, which > in C means that you invoke the function PyObject_CallObject(). A basic > example is shown here: > > https://docs.python.org/2/extending/embedding.html#pure-embedding > > Christian I am using Visualworks 8.3. I can write that double arrsum function in Smalltalk. No Problem. I can also make shared library and bring those function into Smalltalk with C (Since VW 8.3 supports C and C types). No Problem. Smalltalk does not support Python. So I can use C/API to bring in Python libs,types and files. No Problem Now ,I need to bring in shared libraries using C/Python API using Smalltalk. It seems like I can't directly bring in C shared libraries (.so files). PROBLEM. HOW CAN I BRING THESE IN DIRECTLY? Why do I want to call a C shared lib (.so file) using C/Python API from Smalltalk? Because I want to make C shared libs that bring in individual Python libraries into Smalltalk, such as as (Scipy,NumPy,etc). That can be called using that C/Python API in Smalltalk From jesse.ibarra.1996 at gmail.com Fri Jul 19 10:26:19 2019 From: jesse.ibarra.1996 at gmail.com (Jesse Ibarra) Date: Fri, 19 Jul 2019 07:26:19 -0700 (PDT) Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> <808e7c7b-a405-48ef-a465-e0a4663fbdbf@googlegroups.com> Message-ID: <2ce496d6-2ee5-4cf7-bf0c-764b9d55877f@googlegroups.com> On Friday, July 19, 2019 at 8:17:43 AM UTC-6, Chris Angelico wrote: > On Sat, Jul 20, 2019 at 12:16 AM Jesse Ibarra > wrote: > > > > On Thursday, July 18, 2019 at 2:01:39 PM UTC-6, Chris Angelico wrote: > > > On Fri, Jul 19, 2019 at 5:51 AM Christian Gollwitzer wrote: > > > > Once you can do this, you can proceed to call a Python function, which > > > > in C means that you invoke the function PyObject_CallObject(). A basic > > > > example is shown here: > > > > > > > > https://docs.python.org/2/extending/embedding.html#pure-embedding > > > > > > > > > > Or, better: > > > > > > https://docs.python.org/3/extending/embedding.html#pure-embedding > > > > > > ChrisA > > > > I need to call a .so file, but I don't know I way to do that with PyObject. I can only seem to bring in .py files > > I have vastly insufficient context to be able to even attempt to answer this. > > ChrisA Thank you for your help. If this helps. I need PyImport_Import to bring in a C shared lib. (.so file) From auriocus at gmx.de Fri Jul 19 12:48:11 2019 From: auriocus at gmx.de (Christian Gollwitzer) Date: Fri, 19 Jul 2019 18:48:11 +0200 Subject: Embedding Python in C In-Reply-To: <2ce496d6-2ee5-4cf7-bf0c-764b9d55877f@googlegroups.com> References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> <808e7c7b-a405-48ef-a465-e0a4663fbdbf@googlegroups.com> <2ce496d6-2ee5-4cf7-bf0c-764b9d55877f@googlegroups.com> Message-ID: Am 19.07.19 um 16:26 schrieb Jesse Ibarra: > On Friday, July 19, 2019 at 8:17:43 AM UTC-6, Chris Angelico wrote: >> On Sat, Jul 20, 2019 at 12:16 AM Jesse Ibarra >> wrote: >>> >>> On Thursday, July 18, 2019 at 2:01:39 PM UTC-6, Chris Angelico wrote: >>>> On Fri, Jul 19, 2019 at 5:51 AM Christian Gollwitzer wrote: >>>>> Once you can do this, you can proceed to call a Python function, which >>>>> in C means that you invoke the function PyObject_CallObject(). A basic >>>>> example is shown here: >>>>> >>>>> https://docs.python.org/2/extending/embedding.html#pure-embedding >>>>> >>>> >>>> Or, better: >>>> >>>> https://docs.python.org/3/extending/embedding.html#pure-embedding >>>> >>>> ChrisA >>> >>> I need to call a .so file, but I don't know I way to do that with PyObject. I can only seem to bring in .py files >> >> I have vastly insufficient context to be able to even attempt to answer this. >> >> ChrisA > > Thank you for your help. > > If this helps. I need PyImport_Import to bring in a C shared lib. (.so file) So your question is which CPython function is used to load a compiled Python extension (.so file). I haven't done this, but it will also be not sufficient to import numpy or other packages. These larger packages typically have a layer of Python code which acts as a loader and upon import loads the corresponding .so files at runtime. After browsing the C-API, I would try either: PyImport_ImportModule("numpy") or even PyRun_SimpleString("import numpy") If this does not work, then maybe you need to setup the correct PYTHONPATH to point to the places where these packages are installed? Christian From stefan_ml at behnel.de Fri Jul 19 14:32:33 2019 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 19 Jul 2019 20:32:33 +0200 Subject: Embedding Python in C In-Reply-To: <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> Message-ID: Jesse Ibarra schrieb am 17.07.19 um 20:39: > My options seem rather limited, I need to make a Pipeline from > (Smalltalk -> C -> Python) then go back (Smalltalk <- C <- Python). > Since Smalltalk does not support Python directly I have to settle with > the C/Python API > (https://docs.python.org/3.6/extending/embedding.html#beyond-very-high-level-embedding-an-overview). > Any suggestions? First of all: don't use the C-API! :-) Use Cython instead. It's a Python-to-C compiler that covers up all the ugly little details of talking to Python from C (importing a module is just "import module", and it even adapts to different Python versions automatically). You can keep writing Python code, and at the same time trivially use external C code. https://cython.org/ http://docs.cython.org/en/latest/src/tutorial/ For embedding Python in an external program (in case you really need to do that and can't live with starting Python instead of Smalltalk), here's an example: https://github.com/cython/cython/tree/master/Demos/embed It uses the "--embed" argument to Cython that generates a C (main) function to start up the CPython runtime. Stefan From jesse.ibarra.1996 at gmail.com Fri Jul 19 22:12:40 2019 From: jesse.ibarra.1996 at gmail.com (Jesse Ibarra) Date: Fri, 19 Jul 2019 19:12:40 -0700 (PDT) Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> Message-ID: <21ada95f-9f12-4b49-a2f4-f8b5f8515453@googlegroups.com> Sorry, I am not understanding. Smalltlak VW 8.3 does not support Python. I can only call Pyhton code through C/Python API. From dieter at handshake.de Sat Jul 20 00:33:53 2019 From: dieter at handshake.de (dieter) Date: Sat, 20 Jul 2019 06:33:53 +0200 Subject: Embedding Python in C References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <604e9337-6100-4b98-add8-b0f72f35ebca@googlegroups.com> <05299e2d-c491-4f6b-9e7a-196af5e9d112@googlegroups.com> Message-ID: <87d0i5l3ym.fsf@handshake.de> Jesse Ibarra writes: > ... > Now ,I need to bring in shared libraries using C/Python API using Smalltalk. It seems like I can't directly bring in C shared libraries (.so files). PROBLEM. With Python, you typically do not load ("bring in") shared libraries explicitly; instead, you simply import a module; should this module be implemented by a shared library, it will be loaded implicitly. Thus, once you are in the Python world, shared libraries are no longer a problem. You told us that you already know how to build shared libraries and call them from Smalltalk. Thus, do this: ensure that the activated shared library initializes the [C]Python runtime (Stefan posted a link for this), use "Cython" to interface Python with C. From jcasale at activenetwerx.com Sat Jul 20 09:23:51 2019 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Sat, 20 Jul 2019 13:23:51 +0000 Subject: Class initialization with multiple inheritance In-Reply-To: <84050BC7-8143-4372-BE16-C320C163CE83@barrys-emacs.org> References: <84050BC7-8143-4372-BE16-C320C163CE83@barrys-emacs.org> Message-ID: -----Original Message----- From: Barry Scott Sent: Tuesday, July 16, 2019 11:53 AM To: Joseph L. Casale Cc: python-list at python.org Subject: Re: Class initialization with multiple inheritance > And here is the MRO for LeftAndRight. > > >>> import m > LeftAndRight.__init__() > Left.__init__() > Right.__init__() > CommonBase.__init__() > >>> m.LeftAndRight.__mro__ > (, > , > , > , > ) > >>> Thank you DL and Barry, that was very helpful. It seems that if Base1, 2, or 3 creates its own arguments for the common base class, this becomes notable and this one case becomes the arguments preserved and sent to the base regardless of the order of Base1, 2 or 3 inside DerivedClassName. Now if all three of Base1, 2, or 3 produce their own arguments (discarding those intended from DerivedClassName, only one is chosen regardless of the order, indicating some deterministic approach on the name possibly. Presumably a choice had to be made as to how to handle this case (though the runtime could have thrown), it would be enlightening to know what logic is utilized in this case. Thanks a lot! jlc From blmadhavan at gmail.com Sat Jul 20 09:38:28 2019 From: blmadhavan at gmail.com (Madhavan Bomidi) Date: Sat, 20 Jul 2019 06:38:28 -0700 (PDT) Subject: How to execute shell command in Python program? Message-ID: <946b629e-59b6-4a61-bf1d-cf1419e31932@googlegroups.com> Hi, I have the following shell commands to run my executable created with gfortran: ----------------------------------------------------------------------------- $ pwd /OPAC/opac31a/opac31 $ ls extback.dat opac opac.f result terr.dat input opac.cfg pyopac3.py solar.dat wel.dat $ ./opac <====== ******************************************************** * Optical Properties of Aerosols and Clouds OPAC 3.1 * * -------------------------------------------------- * * * * The contents of OPAC has been described in: * * M. Hess, P. Koepke and I. Schult (1997): * * "Optical Properties of Aerosols and Clouds: * * The software package OPAC" * * submitted to Bull. Am. Meteor. Soc. * * * * last update: 14.11.97 M. Hess * ******************************************************** name of input file (without extension .inp), (max. 8 characters) default: opac.inp opac <====== input file input/opac.inp used READY. Note: The following floating-point exceptions are signalling: IEEE_DENORMAL ----------------------------------------------------------------------------------- Now my concern is the following commands (pointed by <======) to be run within the Python program. I tried to use the following lines in the Python program: --------------------- import subprocess subprocess.call(['./opac'],shell=True) Out[21]: 2 I am not able to know how I input the 'opac' (which is the input filename) Please help me. From hello at ezaquarii.com Sat Jul 20 10:39:58 2019 From: hello at ezaquarii.com (Chris Narkiewicz) Date: Sat, 20 Jul 2019 15:39:58 +0100 Subject: How to execute shell command in Python program? In-Reply-To: <946b629e-59b6-4a61-bf1d-cf1419e31932@googlegroups.com> References: <946b629e-59b6-4a61-bf1d-cf1419e31932@googlegroups.com> Message-ID: Madhavan Bomidi wrote: > import subprocess > subprocess.call(['./opac'],shell=True) subprocess.call(['./opac', "my-input.inp"], shell=True) The array takes command with a list of arguments. This way you don't need to do space escaping and other Jujitsu gimmicks. If you want to feed the command from stdin, this becomes a bit more complicated as you need to start a subprocess and use PIPE to feed it. Cheers, Chris Narkiewicz -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From hjp-python at hjp.at Sat Jul 20 13:07:22 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 20 Jul 2019 19:07:22 +0200 Subject: How to execute shell command in Python program? In-Reply-To: References: <946b629e-59b6-4a61-bf1d-cf1419e31932@googlegroups.com> Message-ID: <20190720170722.i47phum3zyg2p6vz@hjp.at> On 2019-07-20 15:39:58 +0100, Chris Narkiewicz via Python-list wrote: > Madhavan Bomidi wrote: > > import subprocess > > subprocess.call(['./opac'],shell=True) There may be an os.chdir() missing here. > subprocess.call(['./opac', "my-input.inp"], shell=True) We don't know whether the OP's program accepts command line arguments. But I agree that it probably does and then this is the best way. > The array takes command with a list of arguments. This > way you don't need to do space escaping and other > Jujitsu gimmicks. In this case you should set shell=False. Either invoke the command directly with an argument vector (almost always preferrable), or invoke the shell with a shell command. Don't mix them. hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From ml_news at posteo.de Sat Jul 20 13:35:43 2019 From: ml_news at posteo.de (Manfred Lotz) Date: Sat, 20 Jul 2019 19:35:43 +0200 Subject: Proper shebang for python3 Message-ID: <20190720193543.3b457019@arcor.com> Hi there, Pretty new to python I've got a question regarding the proper shebang for Python 3. I use #!/usr/bin/python3 which works fine. Today I saw #!/usr/bin/python3 -tt and was wondering what -tt means. Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't even mention -t. python 2 man page mentions -t Issue a warning when a source file mixes tabs and spaces for indentation in a way that makes it depend on the worth of a tab expressed in spaces. Issue an error when the option is given twice. I guess that -t has the same meaning with python 3.7.3. My questions: 1. Is my guess correct? 2. Is it a bug that it is not mentioned? python3 --help doesn't mention it either. -- Manfred From rosuav at gmail.com Sat Jul 20 13:44:24 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jul 2019 03:44:24 +1000 Subject: Proper shebang for python3 In-Reply-To: <20190720193543.3b457019@arcor.com> References: <20190720193543.3b457019@arcor.com> Message-ID: On Sun, Jul 21, 2019 at 3:41 AM Manfred Lotz wrote: > > Hi there, > Pretty new to python I've got a question regarding the proper shebang > for Python 3. > > I use > #!/usr/bin/python3 > > which works fine. > > Today I saw > #!/usr/bin/python3 -tt > > and was wondering what -tt means. > > Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't even > mention -t. > > python 2 man page mentions > > -t Issue a warning when a source file mixes tabs and spaces > for indentation in a way that makes it depend on the worth > of a tab expressed in spaces. Issue an error when the option is > given twice. > > I guess that -t has the same meaning with python 3.7.3. Far as I know, the -tt option to Python 3 is accepted for 2/3 compatibility, but does absolutely nothing. Having it on the shebang probably means that it used to be /usr/bin/python -tt, and got migrated to python3 without removing it. ChrisA From ml_news at posteo.de Sat Jul 20 13:53:18 2019 From: ml_news at posteo.de (Manfred Lotz) Date: Sat, 20 Jul 2019 19:53:18 +0200 Subject: Proper shebang for python3 References: <20190720193543.3b457019@arcor.com> Message-ID: <20190720195318.2c7cb822@arcor.com> On Sun, 21 Jul 2019 03:44:24 +1000 Chris Angelico wrote: > On Sun, Jul 21, 2019 at 3:41 AM Manfred Lotz > wrote: > > > > Hi there, > > Pretty new to python I've got a question regarding the proper > > shebang for Python 3. > > > > I use > > #!/usr/bin/python3 > > > > which works fine. > > > > Today I saw > > #!/usr/bin/python3 -tt > > > > and was wondering what -tt means. > > > > Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't > > even mention -t. > > > > python 2 man page mentions > > > > -t Issue a warning when a source file mixes tabs and > > spaces for indentation in a way that makes it depend on the > > worth of a tab expressed in spaces. Issue an error when the option > > is given twice. > > > > I guess that -t has the same meaning with python 3.7.3. > > Far as I know, the -tt option to Python 3 is accepted for 2/3 > compatibility, but does absolutely nothing. Having it on the shebang > probably means that it used to be /usr/bin/python -tt, and got > migrated to python3 without removing it. > This is a plausible explanation. Thanks, Manfred From knomenet at gmail.com Sat Jul 20 14:11:21 2019 From: knomenet at gmail.com (Michael Speer) Date: Sat, 20 Jul 2019 14:11:21 -0400 Subject: Proper shebang for python3 In-Reply-To: <20190720193543.3b457019@arcor.com> References: <20190720193543.3b457019@arcor.com> Message-ID: You may want to use `#!/usr/bin/env python3` instead. There is a concept in python called the virtual environment. This used to be done with a tool called virtualenv in python2, and is now done mainly through a venv module in python3. A virtual environment goes into a directory of your choosing and will have its own python3 executable, and pip3 executable, and when you add dependencies, they are also placed into the directory structure under your chosen directory. When you do a `. /bin/activate` the included source will places the virtual environment's bin/ folder at the beginning of your PATH environment variable, making it the default python3 when you type it without a full path. This allows you to run scripts that need different, or even conflicting, sets of dependencies without bothering with the underlying linux distribution's python installation's modules. If you use `#!/usr/bin/python3`, it will always use exactly the system version that is installed, and the system's installed modules. Your scripts will still default to the system installation if a virtual environment is not activated. So you lose nothing by doing it this way, but gain a little control from it. On Sat, Jul 20, 2019 at 1:41 PM Manfred Lotz wrote: > Hi there, > Pretty new to python I've got a question regarding the proper shebang > for Python 3. > > I use > #!/usr/bin/python3 > > which works fine. > > Today I saw > #!/usr/bin/python3 -tt > > and was wondering what -tt means. > > Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't even > mention -t. > > python 2 man page mentions > > -t Issue a warning when a source file mixes tabs and spaces > for indentation in a way that makes it depend on the worth > of a tab expressed in spaces. Issue an error when the option is > given twice. > > I guess that -t has the same meaning with python 3.7.3. > > > My questions: > > 1. Is my guess correct? > > 2. Is it a bug that it is not mentioned? python3 --help doesn't mention > it either. > > > -- > Manfred > > > -- > https://mail.python.org/mailman/listinfo/python-list > From rosuav at gmail.com Sat Jul 20 14:20:34 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jul 2019 04:20:34 +1000 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: On Sun, Jul 21, 2019 at 4:13 AM Michael Speer wrote: > > You may want to use `#!/usr/bin/env python3` instead. > > There is a concept in python called the virtual environment. This used to > be done with a tool called virtualenv in python2, and is now done mainly > through a venv module in python3. > > A virtual environment goes into a directory of your choosing and will have > its own python3 executable, and pip3 executable, and when you add > dependencies, they are also placed into the directory structure under your > chosen directory. > > When you do a `. /bin/activate` the included source will places > the virtual environment's bin/ folder at the beginning of your PATH > environment variable, making it the default python3 when you type it > without a full path. > > This allows you to run scripts that need different, or even conflicting, > sets of dependencies without bothering with the underlying linux > distribution's python installation's modules. > > If you use `#!/usr/bin/python3`, it will always use exactly the system > version that is installed, and the system's installed modules. > > Your scripts will still default to the system installation if a virtual > environment is not activated. So you lose nothing by doing it this way, but > gain a little control from it. > ONLY if you actually want this script to behave differently inside a venv. When you're setting a shebang on a script, you often do not want that. You potentially LOSE a lot of control. ChrisA From stefan_ml at behnel.de Sat Jul 20 15:11:24 2019 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 20 Jul 2019 21:11:24 +0200 Subject: Embedding Python in C In-Reply-To: <21ada95f-9f12-4b49-a2f4-f8b5f8515453@googlegroups.com> References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <21ada95f-9f12-4b49-a2f4-f8b5f8515453@googlegroups.com> Message-ID: Jesse Ibarra schrieb am 20.07.19 um 04:12: > Sorry, I am not understanding. Smalltlak VW 8.3 does not support Python. > I can only call Pyhton code through C/Python API. Ok, but that doesn't mean you need to write code that uses the C-API of Python. All you need to do is: 1) Start up a CPython runtime from Smalltalk (see the embedding example I posted) and make it import an extension module that you write (e.g. using the "inittab" mechanism [1]). 2) Use Cython to implement this extension module to provide an interface between your Smalltalk code and your Python code. Use the Smalltalk C-API from your Cython code to call into Smalltalk and exchange data with it. Now you can execute Python code inside of Python and make it call back and forth into your Smalltalk code, through the interface module. And there is no need to use the Python C-API for anything beyond step 1), which is about 5 lines of Python C-API code if you write it yourself. Everything else can be implemented in Cython and Python. Stefan [1] https://docs.python.org/3/extending/embedding.html?highlight=PyImport_appendinittab#extending-embedded-python From info at tundraware.com Sat Jul 20 15:11:44 2019 From: info at tundraware.com (Tim Daneliuk) Date: Sat, 20 Jul 2019 14:11:44 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: On 7/20/19 1:20 PM, Chris Angelico wrote: > On Sun, Jul 21, 2019 at 4:13 AM Michael Speer wrote: >> >> You may want to use `#!/usr/bin/env python3` instead. >> >> There is a concept in python called the virtual environment. This used to >> be done with a tool called virtualenv in python2, and is now done mainly >> through a venv module in python3. >> >> A virtual environment goes into a directory of your choosing and will have >> its own python3 executable, and pip3 executable, and when you add >> dependencies, they are also placed into the directory structure under your >> chosen directory. >> >> When you do a `. /bin/activate` the included source will places >> the virtual environment's bin/ folder at the beginning of your PATH >> environment variable, making it the default python3 when you type it >> without a full path. >> >> This allows you to run scripts that need different, or even conflicting, >> sets of dependencies without bothering with the underlying linux >> distribution's python installation's modules. >> >> If you use `#!/usr/bin/python3`, it will always use exactly the system >> version that is installed, and the system's installed modules. >> >> Your scripts will still default to the system installation if a virtual >> environment is not activated. So you lose nothing by doing it this way, but >> gain a little control from it. >> > > ONLY if you actually want this script to behave differently inside a > venv. When you're setting a shebang on a script, you often do not want > that. You potentially LOSE a lot of control. > > ChrisA Disagree strongly. The environment should always define where you want to find key binaries, whether in a venv or not. There are many use cases where you want to override system binaries. For example, you may be running on an older OS release and want to point to a newer one installed elsewhere. Similarly, the DevOps workflow may demand particular configurations for the pipelines to run properly. I have spent way too much time in my career trying to undo this kind of imperious programming that thinks the coder knows best, when in the actual runtime environment, they actually don't. Most recently, I have been working to produce a configuration for linuxbrew that can be installed by an unprivileged user at any location they like. It is maddening to spend hours getting things working only to find out that some genius decided that the only version of, say, perl or autoconf, is to be found exclusively in /usr/bin. So, no, do NOT encode the hard location - ever. Always use env to discover the one that the user has specified. The only exception is /bin/sh which - for a variety of reasons - can reliably counted upon. We don't need to bikeshed this. All we need is people who disagree with this view to spend a year in software packaging, operations, deployment and DevOps ... then get back to us... Grrrrrrrrrr...... From ml_news at posteo.de Sat Jul 20 15:40:03 2019 From: ml_news at posteo.de (Manfred Lotz) Date: Sat, 20 Jul 2019 21:40:03 +0200 Subject: Proper shebang for python3 References: <20190720193543.3b457019@arcor.com> Message-ID: <20190720214003.2539227c@arcor.com> On Sat, 20 Jul 2019 14:11:21 -0400 Michael Speer wrote: > You may want to use `#!/usr/bin/env python3` instead. > In my case it doesn't matter. However, I agree that your suggestion is usually preferable. > There is a concept in python called the virtual environment. This > used to be done with a tool called virtualenv in python2, and is now > done mainly through a venv module in python3. > > A virtual environment goes into a directory of your choosing and will > have its own python3 executable, and pip3 executable, and when you add > dependencies, they are also placed into the directory structure under > your chosen directory. > > When you do a `. /bin/activate` the included source will > places the virtual environment's bin/ folder at the beginning of your > PATH environment variable, making it the default python3 when you > type it without a full path. > > This allows you to run scripts that need different, or even > conflicting, sets of dependencies without bothering with the > underlying linux distribution's python installation's modules. > New to me. Interesting. > If you use `#!/usr/bin/python3`, it will always use exactly the system > version that is installed, and the system's installed modules. > > Your scripts will still default to the system installation if a > virtual environment is not activated. So you lose nothing by doing it > this way, but gain a little control from it. > Ok, understood. -- Manfred From hjp-python at hjp.at Sat Jul 20 15:56:55 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sat, 20 Jul 2019 21:56:55 +0200 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: <20190720195655.2y66jjeqdjckyimq@hjp.at> On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote: > So, no, do NOT encode the hard location - ever. Always use env to > discover the one that the user has specified. The only exception is > /bin/sh which - for a variety of reasons - can reliably counted upon. > > We don't need to bikeshed this. All we need is people who disagree > with this view to spend a year in software packaging, operations, > deployment and DevOps ... then get back to us... After 25 years in software packaging, operations, deployment and DevOps I disagree: A program should not behave differently because of a different path, #!/usr/bin/env is a total no-no. There is a nice way to achieve this: Just use the interpreter of the virtual environment in the shebang. (That requires rewriting the shebang during installation, but that's a minor inconvenience) hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From info at tundraware.com Sat Jul 20 16:26:46 2019 From: info at tundraware.com (Tim Daneliuk) Date: Sat, 20 Jul 2019 15:26:46 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> <20190720195655.2y66jjeqdjckyimq@hjp.at> Message-ID: <61ac0g-4m2.ln1@oceanview.tundraware.com> On 7/20/19 2:56 PM, Peter J. Holzer wrote: > On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote: >> So, no, do NOT encode the hard location - ever. Always use env to >> discover the one that the user has specified. The only exception is >> /bin/sh which - for a variety of reasons - can reliably counted upon. >> >> We don't need to bikeshed this. All we need is people who disagree >> with this view to spend a year in software packaging, operations, >> deployment and DevOps ... then get back to us... > > After 25 years in software packaging, operations, deployment and DevOps > I disagree: A program should not behave differently because of a > different path, #!/usr/bin/env is a total no-no. > > There is a nice way to achieve this: Just use the interpreter of the > virtual environment in the shebang. > (That requires rewriting the shebang during installation, but that's a > minor inconvenience) > > hp > And what happens with programs that have no virtenv equivalent? perl, go, ruby, awk, sed, grep, etc. have no simple way to get installed virtual short of insisting the everything live in a docker container or VM? The fact is that most large compute environments are slow to upgrade the OS. That means core tools also lag considerably behind as well. Being able to install newer versions along side the OS' own and then use them by default is manifestly necessary. That's why users have the ability to modify $PATH to suit their own needs. All /usr/bin/env does is to tell the interpreter, "honor the intent of the spawning shell". This shouldn't even be a question ... and it's why so much garbage continues to live forever. Having to constantly code around old systems versions of tools with not other recourse is just crazy. In actual fact, the very best way to write portable, reliable, and operable systems of code is to divorce them entirely (or as a nearly as possible) for the OS tools as you can. That's why docker works so well. That's why go avoids dynamic linking. In my case, that's why I compile my own version of languages and daily use utilities to live outside the OS. I get absolutely predicable behavior irrespective of my distro and whatever backleveled cruft it has laying around. I _know_ every version of every important tool my code will be using ... all by setting up $PATH properly and using /usr/bin/env in my interpreters. If you want really big fun, try going into an older CentOS or RedHat instances and, say, upgrading system python to python3. It's super fun. Yes, in that case, you COULD use a venv. But there are tons of other tools for which this is not an option - gcc, autoconf, perl, go awk, sed, bash, ad infinitum, ad nauseum are invariably backleveled on production OS instances. My way fixes that problem. Your way forces me to code around it ... which is a way worse solution. You may have 25 years at this but I have 40 - does that make me nearly twice as right? Arguments from authority are silly. P.S. https://www.tundraware.com/TechnicalNotes/Divorce-Your-Linux-Admin/ From brian.j.oney at googlemail.com Sat Jul 20 17:28:35 2019 From: brian.j.oney at googlemail.com (Brian Oney) Date: Sat, 20 Jul 2019 23:28:35 +0200 Subject: Proper shebang for python3 In-Reply-To: <61ac0g-4m2.ln1@oceanview.tundraware.com> References: <20190720193543.3b457019@arcor.com> <20190720195655.2y66jjeqdjckyimq@hjp.at> <61ac0g-4m2.ln1@oceanview.tundraware.com> Message-ID: On Sat, 2019-07-20 at 15:26 -0500, Tim Daneliuk wrote: > On 7/20/19 2:56 PM, Peter J. Holzer wrote: > > On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote: > > > So, no, do NOT encode the hard location - ever. Always use env to > > > discover the one that the user has specified. The only exception is > > > /bin/sh which - for a variety of reasons - can reliably counted upon. > > > > > > We don't need to bikeshed this. All we need is people who disagree > > > with this view to spend a year in software packaging, operations, > > > deployment and DevOps ... then get back to us... > > > > After 25 years in software packaging, operations, deployment and DevOps > > I disagree: A program should not behave differently because of a > > different path, #!/usr/bin/env is a total no-no. > > > > There is a nice way to achieve this: Just use the interpreter of the > > virtual environment in the shebang. > > (That requires rewriting the shebang during installation, but that's a > > minor inconvenience) > > > > hp > > > > > And what happens with programs that have no virtenv equivalent? > perl, go, ruby, awk, sed, grep, etc. have no simple way to > get installed virtual short of insisting the everything live in a > docker container or VM? > > The fact is that most large compute environments are slow to upgrade > the OS. That means core tools also lag considerably behind as well. > Being able to install newer versions along side the OS' own and then > use them by default is manifestly necessary. That's why users have > the ability to modify $PATH to suit their own needs. All /usr/bin/env > does is to tell the interpreter, "honor the intent of the spawning shell". > This shouldn't even be a question ... and it's why so much garbage continues > to live forever. Having to constantly code around old systems versions of > tools with not other recourse is just crazy. > > In actual fact, the very best way to write portable, reliable, and operable > systems of code is to divorce them entirely (or as a nearly as possible) for > the OS tools as you can. That's why docker works so well. That's why go > avoids dynamic linking. > > In my case, that's why I compile my own version of > languages and daily use utilities to live outside the OS. I get absolutely > predicable behavior irrespective of my distro and whatever backleveled cruft > it has laying around. I _know_ every version of every important tool my code > will be using ... all by setting up $PATH properly and using /usr/bin/env in > my interpreters. > > If you want really big fun, try going into an older CentOS or RedHat instances and, say, > upgrading system python to python3. It's super fun. Yes, in that case, you COULD > use a venv. But there are tons of other tools for which this is not an option - > gcc, autoconf, perl, go awk, sed, bash, ad infinitum, ad nauseum are invariably > backleveled on production OS instances. My way fixes that problem. Your way > forces me to code around it ... which is a way worse solution. > > You may have 25 years at this but I have 40 - does that make me nearly twice > as right? Arguments from authority are silly. > > P.S. https://www.tundraware.com/TechnicalNotes/Divorce-Your-Linux-Admin/ Nice. Emacs (well spacemacs) is my authority. The key sequence 'SPC i !' inserts #!/usr/bin/env python as specified in the insert-shebang package https://github.com/psachin/insert-shebang My limited experience tells me to expect the user (me) to know what they're doing, hence env. Why not make a compromise? What would be a potential pitfall of the following spitbang? #!python From PythonList at DancesWithMice.info Sat Jul 20 17:59:54 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Sun, 21 Jul 2019 09:59:54 +1200 Subject: How to execute shell command in Python program? In-Reply-To: <20190720170722.i47phum3zyg2p6vz@hjp.at> References: <946b629e-59b6-4a61-bf1d-cf1419e31932@googlegroups.com> <20190720170722.i47phum3zyg2p6vz@hjp.at> Message-ID: <4a5e9ac5-45ac-79e1-c990-60f465073ce6@DancesWithMice.info> On 21/07/19 5:07 AM, Peter J. Holzer wrote: > On 2019-07-20 15:39:58 +0100, Chris Narkiewicz via Python-list wrote: >> Madhavan Bomidi wrote: >>> import subprocess >>> subprocess.call(['./opac'],shell=True) > > There may be an os.chdir() missing here. > >> subprocess.call(['./opac', "my-input.inp"], shell=True) > > We don't know whether the OP's program accepts command line arguments. > But I agree that it probably does and then this is the best way. ... > In this case you should set shell=False. Either invoke the command > directly with an argument vector (almost always preferrable), or invoke > the shell with a shell command. Don't mix them. +1 for simplicity subprocess.call() is now described as "older" (since v3.5). The manual says: <<>> Note that if the existing Fortran s/w puts its output to stdout (which I think ALL of 'mine' do), then you will need to "capture" that output from the subprocess (not necessary if the Fortran writes to a report-file on disk). If you are using v3.7+, use: subprocess.run( [ ...etc... ], capture_output=True ) Older versions 3.5+: subprocess.run( [ ...etc... ], stdout=subprocess.PIPE ) run() returns a subprocess.CompletedProcess class. To check that opac executed correctly, inspect the CompletedProcess.returncode. To access any "captured" output from opac, use CompletedProcess.stdout (similarly stderr, if execution was unsuccessful). Be aware that the returned data may be string or bytes, and may have different line-separators from the one you/your Operating System normally uses! Please study the first sections of subprocess ? Subprocess management for details, (https://docs.python.org/3/library/subprocess.html), experiment, and come back to us with any questions... -- Regards =dn From rosuav at gmail.com Sat Jul 20 18:14:12 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jul 2019 08:14:12 +1000 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: On Sun, Jul 21, 2019 at 5:26 AM Tim Daneliuk wrote: > So, no, do NOT encode the hard location - ever. Always use env to discover the one that > the user has specified. The only exception is /bin/sh which - for a variety of reasons - > can reliably counted upon. A quick grep through my $PATH shows that there are a number of executable Python scripts there, including add-apt-repository, calibre, some lilypond stuff, trash-can management, samba-tool, iotop, and youtube-dl. If I have a venv active with, say, Python 3.9, then `/usr/bin/env python` is going to point to Python 3.9. What are the odds that all those scripts will work with Python 3.9 with no libraries installed? Why should typing "youtube-dl B7xai5u_tnk" be affected by a virtual environment, when typing "man youtube-dl" wouldn't be?? Using env for everything is a terrible idea and one that will basically make virtual environments useless. ChrisA From info at tundraware.com Sat Jul 20 18:47:54 2019 From: info at tundraware.com (Tim Daneliuk) Date: Sat, 20 Jul 2019 17:47:54 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: On 7/20/19 5:14 PM, Chris Angelico wrote: > Using env for everything is a terrible idea and one that > will basically make virtual environments useless. Not if you manage them properly. Everyone's mileage is different, but when I enter a venv, I ensure everything I do there is packaged to work together. I pip install things in there and ensure that the tools I invoke will work in that environment. In the example you cite, I would simply run the tools in question outside any environment where I know they don't work. From rosuav at gmail.com Sat Jul 20 19:04:30 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jul 2019 09:04:30 +1000 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: On Sun, Jul 21, 2019 at 8:56 AM Tim Daneliuk wrote: > > On 7/20/19 5:14 PM, Chris Angelico wrote: > > Using env for everything is a terrible idea and one that > > will basically make virtual environments useless. > > Not if you manage them properly. > > Everyone's mileage is different, but when I enter a venv, I ensure everything I do > there is packaged to work together. I pip install things in there and ensure > that the tools I invoke will work in that environment. > > In the example you cite, I would simply run the tools in question outside any > environment where I know they don't work. Are you aware of every systemwide command that happens to be implemented in Python, such that you won't execute any of them while you have the venv active? The ones I mentioned are all installed into /usr/local/bin or /usr/bin or some other directory on my main $PATH. I only know they're Python scripts because I just ran "file /usr/local/bin/*" and had it check them all. If they all reacted to the activation of a venv, I would have to be extremely careful of which commands I ever run while a venv is active. Which basically means I would never activate a venv ever again - not worth it. ChrisA From info at tundraware.com Sat Jul 20 19:02:12 2019 From: info at tundraware.com (Tim Daneliuk) Date: Sat, 20 Jul 2019 18:02:12 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: On 7/20/19 5:47 PM, Tim Daneliuk wrote: > On 7/20/19 5:14 PM, Chris Angelico wrote: >> Using env for everything is a terrible idea and one that >> will basically make virtual environments useless. > > Not if you manage them properly. > > Everyone's mileage is different, but when I enter a venv, I ensure everything I do > there is packaged to work together. I pip install things in there and ensure > that the tools I invoke will work in that environment. > > In the example you cite, I would simply run the tools in question outside any > environment where I know they don't work. > I guess I should clarify something: 1) I pip install everything locally inside my own $HOME for that set of things I need generally and will not be using in a venv. 2) I pip install everything I need in each venv, even if it already exists under 1) above. I effectively treat venvs as fully self-contained environments to the degree possible. From cs at cskk.id.au Sat Jul 20 19:04:43 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 21 Jul 2019 09:04:43 +1000 Subject: Proper shebang for python3 In-Reply-To: References: Message-ID: <20190720230443.GA53640@cskk.homeip.net> On 21Jul2019 08:14, Chris Angelico wrote: >On Sun, Jul 21, 2019 at 5:26 AM Tim Daneliuk wrote: >> So, no, do NOT encode the hard location - ever. Always use env to discover the one that >> the user has specified. The only exception is /bin/sh which - for a variety of reasons - >> can reliably counted upon. > >A quick grep through my $PATH shows that there are a number of >executable Python scripts there, including add-apt-repository, >calibre, some lilypond stuff, trash-can management, samba-tool, iotop, >and youtube-dl. If I have a venv active with, say, Python 3.9, then >`/usr/bin/env python` is going to point to Python 3.9. What are the >odds that all those scripts will work with Python 3.9 with no >libraries installed? Why should typing "youtube-dl B7xai5u_tnk" be >affected by a virtual environment, when typing "man youtube-dl" >wouldn't be?? Using env for everything is a terrible idea and one that >will basically make virtual environments useless. I'm with Tim Daneliuk. The environment matters and should be honoured except in extremely weird cases. If you require a specific outcoming, set a specific environment. It is under your control. Control it. For your specific example, "man youtube-dl" _is_ affected by the environment. It honours the $MANPATH variable. For Peter J. Holzer, if we must play the "I've been doing this forever" game: I've been sysadmining etc longer than your 25 years and disagree with you. If your tools are strongly affected by version, run them with an environment that finds the right version. Installers, particularly those run as root or in other low level setup situations, should have a WELL DEFINED environment. If you have a tool that isn't portable, put it in a wrapper that coddles its behaviour. Cheers, Cameron Simpson From rosuav at gmail.com Sat Jul 20 19:31:07 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jul 2019 09:31:07 +1000 Subject: Proper shebang for python3 In-Reply-To: <20190720230443.GA53640@cskk.homeip.net> References: <20190720230443.GA53640@cskk.homeip.net> Message-ID: On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson wrote: > > On 21Jul2019 08:14, Chris Angelico wrote: > >On Sun, Jul 21, 2019 at 5:26 AM Tim Daneliuk wrote: > >> So, no, do NOT encode the hard location - ever. Always use env to discover the one that > >> the user has specified. The only exception is /bin/sh which - for a variety of reasons - > >> can reliably counted upon. > > > >A quick grep through my $PATH shows that there are a number of > >executable Python scripts there, including add-apt-repository, > >calibre, some lilypond stuff, trash-can management, samba-tool, iotop, > >and youtube-dl. If I have a venv active with, say, Python 3.9, then > >`/usr/bin/env python` is going to point to Python 3.9. What are the > >odds that all those scripts will work with Python 3.9 with no > >libraries installed? Why should typing "youtube-dl B7xai5u_tnk" be > >affected by a virtual environment, when typing "man youtube-dl" > >wouldn't be?? Using env for everything is a terrible idea and one that > >will basically make virtual environments useless. > > I'm with Tim Daneliuk. The environment matters and should be honoured > except in extremely weird cases. > > If you require a specific outcoming, set a specific environment. It is > under your control. Control it. > > For your specific example, "man youtube-dl" _is_ affected by the > environment. It honours the $MANPATH variable. > > For Peter J. Holzer, if we must play the "I've been doing this forever" > game: I've been sysadmining etc longer than your 25 years and disagree > with you. If your tools are strongly affected by version, run them with > an environment that finds the right version. > > Installers, particularly those run as root or in other low level setup > situations, should have a WELL DEFINED environment. > > If you have a tool that isn't portable, put it in a wrapper that coddles > its behaviour. So.... you mean that a tool that depends on running on a consistent environment, it should use a shebang of "/usr/bin/python3.6" instead of "/usr/bin/env python3"? Because, wow, that would be exactly what is already happening on my system. Why use /usr/bin/env and then wrap something around it to force the environment, when you could just set a correct shebang so it properly defines its execution environment? ChrisA From PythonList at DancesWithMice.info Sat Jul 20 20:02:27 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Sun, 21 Jul 2019 12:02:27 +1200 Subject: Namespaces: memory vs 'pollution' Message-ID: How do you remember to from-import- 'everything' that is needed? I have a 'utility module' which contains a bunch of classes which examine/check/log aspects of the execution environment. One of which is PythonEnvironment, another relates to the HostSystem (as examples). They are most-frequently used to ensure that an application runs within an appropriate environment, eg don't use Posix code on MS-Windows and don't call for v3.7 features from Python3.5. Today, I 'grabbed' the module, imported the PythonEnvironment, and PyTested the application code's Python version - as much for my amusement as anything else I asked for an impossible versionNR. Of course, the test failed. Upon closer inspection, I realised it didn't just fail; it failed badly! Some silly, little, boy had imported the PythonEnvironment class but failed to ALSO import PythonVersionError. So, the reported error was not the expected exception! Yes, the two classes appear in the module one-above-the-other, and yes, in the module's own pytest import-ing both classes is 'right there' in front of my nose. So, not my finest hour! Is there some 'easy way' to make sure that one doesn't just import the desired class, but also remembers to import 'everything else' that might be necessary. In this case, it'd be rather nice to: from environment_module import Python* NB this is a syntax error, and won't import both the PythonEnvironment and PythonVersionError classes. NBB I could import the module, fish-around in its __dict__, and do something like update-ing locals() with every key containing "Python" - but that is surely a truly, ugly, hack. The 'lazy' answer is (purists, look away now!): from environment_module import * but that import-s "everything, including the kitchen sink"! After you've recovered from the shock-horror of thinking about such blatant "namespace pollution", if the module only contains a bunch of classes, is it really such an evil act? (and if the module's import-as-abbreviations are moved 'inside' the relevant class, those 'disappear' from the global namespace reducing the risk of name-clashes down to the names of the custom-classes and the PSL imports, eg os and platform) What do you do to (respecting purism) ensure 'everything' (necessary) is imported (and nothing more), preferably without relying upon (faulty, in my case) human-memory or reading through volumes of code/documentation? -- Regards, =dn From cs at cskk.id.au Sat Jul 20 20:21:55 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 21 Jul 2019 10:21:55 +1000 Subject: Proper shebang for python3 In-Reply-To: References: Message-ID: <20190721002155.GA51694@cskk.homeip.net> On 21Jul2019 09:31, Chris Angelico wrote: >On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson wrote: >So.... you mean that a tool that depends on running on a consistent >environment, it should use a shebang of "/usr/bin/python3.6" instead >of "/usr/bin/env python3"? Jeez. No. That is the _opposite_ of what I'm saying. >Because, wow, that would be exactly what is >already happening on my system. Why use /usr/bin/env and then wrap >something around it to force the environment, when you could just set >a correct shebang so it properly defines its execution environment? Because the shebang is hardwired and inflexible. Because it means hand patching (even scripted) a bazillion scripts to that they know their physical install. Because it presumes detailed hardwired knowledge of the target system in a script which should work anywhere. Instead a tiny _common_ shell script resembling this: #!/bin/sh # Run command in the official environment. exec env - PATH=/path/to/3.6venv/bin:/usr/sbin:/bin exec ${1+"$@"} arranges things. The "env -" is aimed at "clean" daemon or install environments. You can do subtler or less intrusive things in other settings. There are any number of variations available here that don't hardwire the environment into the various _scripts_, but into the invocation setting. Or just start an environment (eg a shell) with the desired environment. Cheers, Cameron Simpson From suabiut at gmail.com Sat Jul 20 21:04:30 2019 From: suabiut at gmail.com (sum abiut) Date: Sun, 21 Jul 2019 13:04:30 +1200 Subject: How to print out html tags excluding the attributes Message-ID: I want to use regular expression to print out the HTML tags excluding the attributes. for example: import re html = '

Hi

test test

' tags = re.findall(r'<[^>]+>', html) for a in tags: print(a) the output is :

But I just want the tag, not the attributes

From rosuav at gmail.com Sat Jul 20 21:13:25 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jul 2019 11:13:25 +1000 Subject: How to print out html tags excluding the attributes In-Reply-To: References: Message-ID: On Sun, Jul 21, 2019 at 11:06 AM sum abiut wrote: > > I want to use regular expression to print out the HTML tags excluding the > attributes. I'll just leave this here... https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags No, I won't be that cruel. I'll also suggest that Beautiful Soup is an excellent non-regex way to parse HTML. ChrisA From bill at baddogconsulting.com Sat Jul 20 22:40:08 2019 From: bill at baddogconsulting.com (Bill Deegan) Date: Sat, 20 Jul 2019 19:40:08 -0700 Subject: SCons 3.1.0 Released Message-ID: A new SCons checkpoint release, 3.1.0, is now available on the SCons download page: https://scons.org/pages/download.html SCons is an Open Source software construction tool?that is, a next-generation build tool. Think of SCons as an improved, cross-platform substitute for the classic Make utility with integrated functionality similar to autoconf/automake and compiler caches such as ccache. In short, SCons is an easier, more reliable and faster way to build software. Here is a summary of the changes since 3.0.5: NEW FUNCTIONALITY - Added variable TEMPFILEARGJOIN to specify how to join arguments written to temp files used when command lines exceed MAXLINELENGTH when the command uses $TEMPFILE{...} - Support for MSVC 2019 - Upgraded and improved Visual Studio solution/project generation code using the MSVSProject builder. - Added support for Visual Studio 2017 and 2019. - Added support for the following per-variant parameters to the builder: - cpppaths: Provides per-variant include paths. - cppdefines: Provides per-variant preprocessor definitions. CHANGED/ENHANCED EXISTING FUNCTIONALITY - Fix performance degradation for MD5-timestamp decider. NOTE: This changes the Decider() function arguments. From: def my_decider(dependency, target, prev_ni): To: def my_decider(dependency, target, prev_ni, repo_node): Where repo_node is the repository (or other) node to use to check if the node is out of date instead of dependency. - Enhanced --debug=explain output. Now the separate components of the dependency list are split up as follows: scons: rebuilding `file3' because: the dependency order changed: ->Sources Old:xxx New:zzz Old:yyy New:yyy Old:zzz New:xxx ->Depends ->Implicit Old:/usr/bin/python New:/usr/bin/python - Changed: Pseudo-builders now inherit OverrideEnvironments. For example when calling a pseudo-builder from another pseudo-builder the override variables passed to the first pseudo-builder call had to be explicitly passed on to the internal pseudo-builder call. Now the second pseudo-builder call will automatically inherit these override values. FIXES - Fix Issue #3350 - SCons Exception EnvironmentError is conflicting with Python's EnvironmentError. - Fix spurious rebuilds on second build for cases where builder has > 1 target and the source file is generated. This was causing the > 1th target to not have it's implicit list cleared when the source file was actually built, leaving an implicit list similar to follows for 2nd and higher target ['/usr/bin/python', 'xxx', 'yyy', 'zzz'] This was getting persisted to SConsign and on rebuild it would be corrected to be similar to this ['zzz', 'yyy', 'xxx', '/usr/bin/python'] Which would trigger a rebuild because the order changed. The fix involved added logic to mark all shared targets as peers and then ensure they're implicit list is all cleared together. - Fix Issue #3349 - SCons Exception EnvironmentError is conflicting with Python's EnvironmentError. Renamed to SConsEnvironmentError - Fix Issue #3350 - mslink failing when too many objects. This is resolved by adding TEMPFILEARGJOIN variable which specifies what character to join all the argements output into the tempfile. The default remains a space when mslink, msvc, or mslib tools are loaded they change the TEMPFILEARGJOIN to be a line separator (\r\n on win32) - Additional fix to issue #3135 - Also handle 'pure' and 'elemental' type bound procedures - Fix handling of Visual Studio Compilers to properly reject any unknown HOST_PLATFORM or TARGET_PLATFORM - Enable LaTeX scanner to find more than one include per line under which they would be observed), or major code cleanups git shortlog --no-merges -ns 3.0.5..HEAD 64 William Deegan 56 Mats Wichmann 10 Adam Gross 4 Mathew Robinson 4 Peter Diener 3 Lukas Schrangl 1 Daniel Holth 1 bdbaddog From ml_news at posteo.de Sun Jul 21 00:03:38 2019 From: ml_news at posteo.de (Manfred Lotz) Date: Sun, 21 Jul 2019 06:03:38 +0200 Subject: Proper shebang for python3 References: <20190720193543.3b457019@arcor.com> <20190720195655.2y66jjeqdjckyimq@hjp.at> <61ac0g-4m2.ln1@oceanview.tundraware.com> Message-ID: <20190721060338.0aa6c2ca@arcor.com> On Sat, 20 Jul 2019 23:28:35 +0200 Brian Oney wrote: > Why not make a compromise? What would be a potential pitfall of the > following spitbang? > > #!python I think that per definition a path in a shebang has to be absolute. Actually, your suggestion won't work for people who use the fish shell (just for the given reason). From info at tundraware.com Sun Jul 21 01:31:50 2019 From: info at tundraware.com (Tim Daneliuk) Date: Sun, 21 Jul 2019 00:31:50 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: <6v9d0g-6qa.ln1@oceanview.tundraware.com> On 7/20/19 6:04 PM, Chris Angelico wrote: > Are you aware of every systemwide command that happens to be > implemented in Python, such that you won't execute any of them while > you have the venv active? No, but this has never been a problem because the newer versions of python tend to be pretty good - within a major release tree - of being backward compatible. Certainly, if I were running, say, RedHat 4 with a very new version of python in my path first, this could theoretically be an issue. I've just not run into it. From info at tundraware.com Sun Jul 21 01:39:13 2019 From: info at tundraware.com (Tim Daneliuk) Date: Sun, 21 Jul 2019 00:39:13 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720230443.GA53640@cskk.homeip.net> Message-ID: <1dad0g-esa.ln1@oceanview.tundraware.com> On 7/20/19 6:04 PM, Cameron Simpson wrote: > If you require a specific outcoming, set a specific environment. It is under?your?control.?Control?it. Exactly right. I have just had the REALLY irritating experience of trying to bootstrap a location insensitive version of linuxbrew that mostly works, but is crippled by brain damage that insists that the best version of perl will always be found in /usr/bin. I have been a hardware engineer (analog and digital), software implementor, systems designer, devops person, and large scale (physical data center) platform engineer and this kind of bad thinking just kills site reliability. I stipulate that there are corner cases where Chris A. is correct - it is certainly possible to clobber a system with incorrect /usr/bin/env findings. But I'd argue that there is a simple work around - run another terminal session (terminator, tmux, screen, ... whatever floats your boat) that has $PATH set up to find things in /usr/bin first. Voila! Problem solved and everyone can use env the way they want to. In some respects, this problem does get a little simpler when you docker-ize your software distributions until ... you need newer versions of tools than even the latest docker OS implementations support. Then you're back to the same old noise ... From rosuav at gmail.com Sun Jul 21 01:53:57 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 21 Jul 2019 15:53:57 +1000 Subject: Proper shebang for python3 In-Reply-To: <6v9d0g-6qa.ln1@oceanview.tundraware.com> References: <20190720193543.3b457019@arcor.com> <6v9d0g-6qa.ln1@oceanview.tundraware.com> Message-ID: On Sun, Jul 21, 2019 at 3:41 PM Tim Daneliuk wrote: > > On 7/20/19 6:04 PM, Chris Angelico wrote: > > Are you aware of every systemwide command that happens to be > > implemented in Python, such that you won't execute any of them while > > you have the venv active? > > No, but this has never been a problem because the newer versions of > python tend to be pretty good - within a major release tree - of being > backward compatible. Certainly, if I were running, say, RedHat 4 with > a very new version of python in my path first, this could theoretically > be an issue. I've just not run into it. > Many Python packages are incompatible with CPython 3.9 at the moment. MANY. ChrisA From auriocus at gmx.de Sun Jul 21 03:29:42 2019 From: auriocus at gmx.de (Christian Gollwitzer) Date: Sun, 21 Jul 2019 09:29:42 +0200 Subject: Proper shebang for python3 In-Reply-To: <6v9d0g-6qa.ln1@oceanview.tundraware.com> References: <20190720193543.3b457019@arcor.com> <6v9d0g-6qa.ln1@oceanview.tundraware.com> Message-ID: Am 21.07.19 um 07:31 schrieb Tim Daneliuk: > On 7/20/19 6:04 PM, Chris Angelico wrote: >> Are you aware of every systemwide command that happens to be >> implemented in Python, such that you won't execute any of them while >> you have the venv active? > > No, but this has never been a problem because the newer versions of > python tend to be pretty good - within a major release tree - of being > backward compatible. Certainly, if I were running, say, RedHat 4 with > a very new version of python in my path first, this could theoretically > be an issue. I've just not run into it. > It's not about the core language, but these system tools depend on packages, sometimes specialist packages, which might not be available in your venv. Christian From barry at barrys-emacs.org Sun Jul 21 04:02:44 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Sun, 21 Jul 2019 09:02:44 +0100 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> <6v9d0g-6qa.ln1@oceanview.tundraware.com> Message-ID: > On 21 Jul 2019, at 08:29, Christian Gollwitzer wrote: > > Am 21.07.19 um 07:31 schrieb Tim Daneliuk: >> On 7/20/19 6:04 PM, Chris Angelico wrote: >>> Are you aware of every systemwide command that happens to be >>> implemented in Python, such that you won't execute any of them while >>> you have the venv active? >> No, but this has never been a problem because the newer versions of >> python tend to be pretty good - within a major release tree - of being >> backward compatible. Certainly, if I were running, say, RedHat 4 with >> a very new version of python in my path first, this could theoretically >> be an issue. I've just not run into it. > > It's not about the core language, but these system tools depend on packages, sometimes specialist packages, which might not be available in your venv. Well said. The fedora packaging system replaces all shebang lines with the full path to either python2 or python3 on the system to make sure that scripts installed in the system run as intended and do not break because of the users PATH. How you think about the shebang lines depends on where the scripts will be used. If its as an installed system component the exact path is usually the right thing to do. If it you personal scripts then you may well find /usr/bin/env python3 makes you life simpler. For code I'm testing I usual do not depend no shebang lines at all. Often I'm testing on many python versions. And end up with something like this to test on all interesting python versions. for PTYHON in python3.6 python3.7 python3.8 do $PYTHON my_script.py done Barry From ml_news at posteo.de Sun Jul 21 04:04:47 2019 From: ml_news at posteo.de (Manfred Lotz) Date: Sun, 21 Jul 2019 10:04:47 +0200 Subject: Proper shebang for python3 References: <20190721002155.GA51694@cskk.homeip.net> Message-ID: <20190721100447.3c0339c9@arcor.com> On Sun, 21 Jul 2019 10:21:55 +1000 Cameron Simpson wrote: > On 21Jul2019 09:31, Chris Angelico wrote: > >On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson > >wrote: So.... you mean that a tool that depends on running on a > >consistent environment, it should use a shebang of > >"/usr/bin/python3.6" instead of "/usr/bin/env python3"? > > Jeez. No. That is the _opposite_ of what I'm saying. > > >Because, wow, that would be exactly what is > >already happening on my system. Why use /usr/bin/env and then wrap > >something around it to force the environment, when you could just set > >a correct shebang so it properly defines its execution environment? > > Because the shebang is hardwired and inflexible. > > Because it means hand patching (even scripted) a bazillion scripts to > that they know their physical install. > > Because it presumes detailed hardwired knowledge of the target system > in a script which should work anywhere. > > Instead a tiny _common_ shell script resembling this: > > #!/bin/sh > # Run command in the official environment. > exec env - PATH=/path/to/3.6venv/bin:/usr/sbin:/bin exec ${1+"$@"} > > arranges things. The "env -" is aimed at "clean" daemon or install > environments. You can do subtler or less intrusive things in other > settings. > I took a look and found that Fedora 30 and Debian Jessie both use hard-wired paths for python in the rpm resp. deb packages. I'm being new to Python and I am not acquainted in any way with virtualenv resp. venv so cannot currently judge its pro and cons. So I will stick to: #!/usr/bin/env python3 as shebang for my scripts. -- Manfred From brian.j.oney at googlemail.com Sun Jul 21 04:50:06 2019 From: brian.j.oney at googlemail.com (Brian Oney) Date: Sun, 21 Jul 2019 10:50:06 +0200 Subject: Proper shebang for python3 In-Reply-To: <20190721100447.3c0339c9@arcor.com> References: <20190721002155.GA51694@cskk.homeip.net> <20190721100447.3c0339c9@arcor.com> Message-ID: On July 21, 2019 10:04:47 AM GMT+02:00, Manfred Lotz wrote: >On Sun, 21 Jul 2019 10:21:55 +1000 >Cameron Simpson wrote: > >> On 21Jul2019 09:31, Chris Angelico wrote: >> >On Sun, Jul 21, 2019 at 9:15 AM Cameron Simpson >> >wrote: So.... you mean that a tool that depends on running on a >> >consistent environment, it should use a shebang of >> >"/usr/bin/python3.6" instead of "/usr/bin/env python3"? >> >> Jeez. No. That is the _opposite_ of what I'm saying. >> >> >Because, wow, that would be exactly what is >> >already happening on my system. Why use /usr/bin/env and then wrap >> >something around it to force the environment, when you could just >set >> >a correct shebang so it properly defines its execution environment? > >> >> Because the shebang is hardwired and inflexible. >> >> Because it means hand patching (even scripted) a bazillion scripts to > >> that they know their physical install. >> >> Because it presumes detailed hardwired knowledge of the target system >> in a script which should work anywhere. >> >> Instead a tiny _common_ shell script resembling this: >> >> #!/bin/sh >> # Run command in the official environment. >> exec env - PATH=/path/to/3.6venv/bin:/usr/sbin:/bin exec ${1+"$@"} >> >> arranges things. The "env -" is aimed at "clean" daemon or install >> environments. You can do subtler or less intrusive things in other >> settings. >> > >I took a look and found that Fedora 30 and Debian Jessie both use >hard-wired paths for python in the rpm resp. deb packages. > >I'm being new to Python and I am not acquainted in any way with >virtualenv resp. venv so cannot currently judge its pro and cons. > >So I will stick to: > #!/usr/bin/env python3 > >as shebang for my scripts. I think that's a good decision. Most of the conversation applies to sysadmins concerned with answering your question in absolute terms. When you start writing scripts which are restricted to a specific environment and are intended to be distributed, you may revisit this thread. Be blissful until then :). Chris et al have "fixed" things for you. From eryksun at gmail.com Sun Jul 21 07:54:26 2019 From: eryksun at gmail.com (eryk sun) Date: Sun, 21 Jul 2019 06:54:26 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: On 7/20/19, Michael Speer wrote: > > You may want to use `#!/usr/bin/env python3` instead. This is partially supported in Windows, but only if .py files are associated with the py.exe launcher and the shebang runs "python" instead of "python3". py.exe supports four builtin virtual commands: "/usr/bin/env python", "/usr/bin/python", "/usr/local/bin/python", and "python". The "python" command can be qualified with a version specification of the form "X[.Y][-32|-64]", such as "python3". The "/usr/bin/env python" virtual command searches PATH, but only if the command matches "python" exactly. If there's a version specified, or if it's not an "env" virtual command, the launcher looks for a registered Python installation instead of searching PATH. For all other commands, the launcher skips a virtual Unix prefix (i.e. "/usr/bin/env", "/usr/bin", and "/usr/local/bin"), if present, and searches PATH. For example, given "/usr/bin/pypy", it searches PATH for "pypy" plus the file extensions in PATHEXT (i.e. .COM, .EXE, etc). If the command isn't found in PATH, it checks in the "[commands]" section of "%LocalAppData%\py.ini". For example, py.ini could define "pypy=C:\PyPy71\pypy.exe". A virtual environment should have a "python" executable, so shebangs that use "/usr/bin/env python" will prefer an active virtual environment. But a lot of scripts use "python3" instead of "python". This is a stumbling block in Windows since we don't install versioned "pythonX[.Y].exe" binaries. Thus the "env" virtual command is special cased to find a qualified Python version in the list of registered Python installations instead of searching PATH. (The new store app distribution does install versioned app links, but, even in this case, virtual environments lack versioned binaries in the "Scripts" directory.) That said, some developers manually create versioned binaries as symlinks, hardlinks, or copies. Also, maybe a future version of the full (non-app) Python installation will do the same. Thus I think the launcher should search PATH for all "/usr/bin/env python*" shebangs, but without the binary-type specification (i.e. "-32" or "-64") if present. In this case, instead of a regular search based on WINAPI SearchPath, we'll need a custom search that tokenizes and walks PATH and checks the binary type via GetBinaryTypeW. If no matching version is found in PATH, the "env" search should fall back on the list of registered versions. From hjp-python at hjp.at Sun Jul 21 09:34:31 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 21 Jul 2019 15:34:31 +0200 Subject: Proper shebang for python3 In-Reply-To: <61ac0g-4m2.ln1@oceanview.tundraware.com> References: <20190720193543.3b457019@arcor.com> <20190720195655.2y66jjeqdjckyimq@hjp.at> <61ac0g-4m2.ln1@oceanview.tundraware.com> Message-ID: <20190721133431.2njg6me6ftadhngw@hjp.at> On 2019-07-20 15:26:46 -0500, Tim Daneliuk wrote: > On 7/20/19 2:56 PM, Peter J. Holzer wrote: > > On 2019-07-20 14:11:44 -0500, Tim Daneliuk wrote: > >> So, no, do NOT encode the hard location - ever. Always use env to > >> discover the one that the user has specified. The only exception is > >> /bin/sh which - for a variety of reasons - can reliably counted upon. > >> > >> We don't need to bikeshed this. All we need is people who disagree > >> with this view to spend a year in software packaging, operations, > >> deployment and DevOps ... then get back to us... > > > > After 25 years in software packaging, operations, deployment and DevOps > > I disagree: A program should not behave differently because of a > > different path, #!/usr/bin/env is a total no-no. > > > > There is a nice way to achieve this: Just use the interpreter of the > > virtual environment in the shebang. > > (That requires rewriting the shebang during installation, but that's a > > minor inconvenience) > > > And what happens with programs that have no virtenv equivalent? > perl, go, ruby, awk, sed, grep, etc. have no simple way to > get installed virtual short of insisting the everything live in a > docker container or VM? Perl is an excellent example: When you install a Perl script which has been packaged with one of the usual tools (e.g., Makemaker or Module::Build), the shebang is set to the interpreter used for installation. The user of the script doesn't have to know what environment to use. (Perl also has perlbrew which is similar to virtual environments, but I haven't used that much). Go is a compiled language: The executables are binaries (no shebang) and even statically linked. Again, the user doesn't have to care about a language environment. awk, grep, sed, etc. are often used in shell scripts, and they were probably the reason why I first developed a "a program's behaviour must not depend on the environment /except as documented/" policy: Back then in the 1990s one usually wanted to have GNU awk, sed, grep etc. in addition to the system utilities. Having scripts randomly fail or (worse) produce wrong results depending on the order of /usr/bin and /usr/local/bin in the user's PATH and the exact contents of /usr/local/bin wasn't fun (oh, and don't forget cron). > The fact is that most large compute environments are slow to upgrade > the OS. That means core tools also lag considerably behind as well. > Being able to install newer versions along side the OS' own and then > use them by default is manifestly necessary. I agree. But (for me at least) it is important to do that robustly, and #!/usr/bin/env is not robust. > That's why users have the ability to modify $PATH to suit their own > needs. All /usr/bin/env does is to tell the interpreter, "honor the > intent of the spawning shell". You probably just phrased that badly, but as written this is completely wrong: /usr/bin/env doesn't tell the interpreter anything. It *chooses* the interpreter to invoke. > If you want really big fun, try going into an older CentOS or RedHat instances and, say, > upgrading system python to python3. Don't do that. Install python3 somewhere else, e.g. into /usr/local. > It's super fun. Yes, in that case, you COULD use a venv. But there > are tons of other tools for which this is not an option - gcc, > autoconf, perl, go awk, sed, bash, ad infinitum, ad nauseum are > invariably backleveled on production OS instances. My way fixes that > problem. Your way only fixes your problem only if environments where your script may be called. You may know that /usr/local/bin/foo is a python script that requires /usr/local/python3.8/bin/python to be in the path before any other python interpreter and you may rememmber that in every instance where that script is called directly or indirectly. But do your colleagues? Especially if they aren't programmers? And what happens if /usr/local/bin/bar requires /usr/local/python3.4/bin/python? Do you explicitely set the PATH before invoking each script? By putting #!/usr/local/python3.8/bin/python at the top of /usr/local/bin/foo and #!/usr/local/python3.4/bin/python at the top of /usr/local/bin/bar I can just invoke both scripts. Even better, my users can invoke both scripts and they don't even have to know they are written in Python. > You may have 25 years at this but I have 40 - does that make me nearly twice > as right? Arguments from authority are silly. No, but it invalidates your notion that anybody with a bit of experience would obviously use #!/usr/bin/env. I do have a bit more than one year of experience and I think #!/usr/bin/env is a terrible idea. (I use it for throwaway scripts which need a venv, but not for anything in production.) hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sun Jul 21 09:47:47 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 21 Jul 2019 15:47:47 +0200 Subject: Proper shebang for python3 In-Reply-To: <20190720230443.GA53640@cskk.homeip.net> References: <20190720230443.GA53640@cskk.homeip.net> Message-ID: <20190721134747.3h3ury7tcbwlppwk@hjp.at> On 2019-07-21 09:04:43 +1000, Cameron Simpson wrote: > On 21Jul2019 08:14, Chris Angelico wrote: > > On Sun, Jul 21, 2019 at 5:26 AM Tim Daneliuk wrote: > > > So, no, do NOT encode the hard location - ever. Always use env to > > > discover the one that the user has specified. The only exception > > > is /bin/sh which - for a variety of reasons - can reliably counted > > > upon. > > > > A quick grep through my $PATH shows that there are a number of > > executable Python scripts there, including add-apt-repository, > > calibre, some lilypond stuff, trash-can management, samba-tool, iotop, > > and youtube-dl. If I have a venv active with, say, Python 3.9, then > > `/usr/bin/env python` is going to point to Python 3.9. What are the > > odds that all those scripts will work with Python 3.9 with no > > libraries installed? Why should typing "youtube-dl B7xai5u_tnk" be > > affected by a virtual environment, when typing "man youtube-dl" > > wouldn't be?? Using env for everything is a terrible idea and one that > > will basically make virtual environments useless. > > I'm with Tim Daneliuk. The environment matters and should be honoured except > in extremely weird cases. I don't think that all the scripts in /usr/bin are extremely weird cases. > If you require a specific outcoming, set a specific environment. It is under > your control. Control it. > > For your specific example, "man youtube-dl" _is_ affected by the > environment. It honours the $MANPATH variable. MANPATH is explicitely intended to control man. But man doesn't fail if you set your PATH to something weird. It will still invoke /usr/bin/groff even if that isn't in the PATH. (I expected that there is also an environment variable to control that but the manpage doesn't mention one). > For Peter J. Holzer, if we must play the "I've been doing this forever" > game: I've been sysadmining etc longer than your 25 years and disagree with > you. That's fine. Unlike Tim I don't claim that anybody who disagrees with me must be a newbie. hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From hjp-python at hjp.at Sun Jul 21 09:55:59 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 21 Jul 2019 15:55:59 +0200 Subject: Namespaces: memory vs 'pollution' In-Reply-To: References: Message-ID: <20190721135559.s5rukmcgqr22vy32@hjp.at> On 2019-07-21 12:02:27 +1200, DL Neil wrote: > What do you do to (respecting purism) ensure 'everything' (necessary) is > imported (and nothing more), preferably without relying upon (faulty, in my > case) human-memory or reading through volumes of code/documentation? I write tests (not as consistently as I would like). I don't think there is much more you can do. The problem is that Python doesn't have variable declarations, so there is no reliable way to discover an undefined symbol except by executing the line where it is used. An IDE might help. Even though the problem is in general not solvable, most uses of symbols fall into a few common categories which can be discovered by static analysis. So an IDE (or even a syntax-highlighting editor) could flag all symbols where it can't find the definition. hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From info at tundraware.com Sun Jul 21 11:26:17 2019 From: info at tundraware.com (Tim Daneliuk) Date: Sun, 21 Jul 2019 10:26:17 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720230443.GA53640@cskk.homeip.net> <20190721134747.3h3ury7tcbwlppwk@hjp.at> Message-ID: On 7/21/19 8:47 AM, Peter J. Holzer wrote: > That's fine. Unlike Tim I don't claim that anybody who disagrees with me > must be a newbie. Peter, that's ad hominem and unfair. I never said anything close to that. What I said is that if someone were to spend an extended period of time in devops and systems engineering at large scale, they'd quickly come to hate hardwired paths. The truth is that there is no single answer to this problem until you hermetically seal an environment. Docker comes close, a freestanding VM does this most completely. But short of that, the ability to decide to use something other than system-provided tools absolutely IS a requirement in systems of any scale. From rosuav at gmail.com Sun Jul 21 12:09:27 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 22 Jul 2019 02:09:27 +1000 Subject: Proper shebang for python3 In-Reply-To: References: <20190720230443.GA53640@cskk.homeip.net> <20190721134747.3h3ury7tcbwlppwk@hjp.at> Message-ID: On Mon, Jul 22, 2019 at 1:36 AM Tim Daneliuk wrote: > > On 7/21/19 8:47 AM, Peter J. Holzer wrote: > > That's fine. Unlike Tim I don't claim that anybody who disagrees with me > > must be a newbie. > > Peter, that's ad hominem and unfair. I never said anything close to that. > What I said is that if someone were to spend an extended period of time > in devops and systems engineering at large scale, they'd quickly come to > hate hardwired paths. > > The truth is that there is no single answer to this problem until you > hermetically seal an environment. Docker comes close, a freestanding > VM does this most completely. But short of that, the ability to decide > to use something other than system-provided tools absolutely IS a requirement > in systems of any scale. Your final paragraph does not justify your preceding. Yes, you need the ability to decide to use something other than the system-provided tool; that's why "/usr/bin/env python3" is a valid shebang. No, that does NOT mean that people with any decent experience will "hate hardwired paths". The hardwired path ("#!/usr/bin/python3") has its own value. Thanks to an absolute path in its shebang, I can run "sudo iotop" regardless of whether I have a venv active. I cannot run "sudo python3 `which iotop`" reliably, because that command uses the environment to look things up. You're absolutely right that there's no single answer to the problem. That's why script authors have the option to use any path they like, including /usr/bin/env. ChrisA From hjp-python at hjp.at Sun Jul 21 13:13:49 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 21 Jul 2019 19:13:49 +0200 Subject: Proper shebang for python3 In-Reply-To: References: <20190720230443.GA53640@cskk.homeip.net> <20190721134747.3h3ury7tcbwlppwk@hjp.at> Message-ID: <20190721171349.35deamhzd2cpkuta@hjp.at> On 2019-07-21 10:26:17 -0500, Tim Daneliuk wrote: > On 7/21/19 8:47 AM, Peter J. Holzer wrote: > > That's fine. Unlike Tim I don't claim that anybody who disagrees with me > > must be a newbie. > > Peter, that's ad hominem and unfair. No, it isn't. Please read the first paragraph of https://en.wikipedia.org/wiki/Ad_hominem > I never said anything close to that. Well, let's see what you wrote: | We don't need to bikeshed this. Translation: This is not a matter of opinion or taste. There is one objectively true answer. | All we need is people who disagree with this view to spend a year in | software packaging, operations, deployment and DevOps ... then get back | to us... Translation: People who disagree with your opinion obviously haven't worked even a single year in this field because otherwise they would have discovered the one objectively true answer. | Grrrrrrrrrr...... I don't have to translate that :-) > What I said is that if someone were to spend an extended period of time > in devops and systems engineering at large scale, they'd quickly come to > hate hardwired paths. You are just stating the same point again: Everybody who has experience in the field agrees with you, therefore anybody who disagrees can't have experience. > The truth is that there is no single answer to this problem until you > hermetically seal an environment. If you can hermetically seal the environment, the argument is moot. I thought (and still think) we are talking about the case where you can't. Therefore I took exception to your insistence that there is in fact one single answer and that this single answer is one which I know from bitter experience to be quite fragile. > But short of that, the ability to decide to use something other than > system-provided tools absolutely IS a requirement in systems of any > scale. I completely agree with that. I just don't think that using #!/usr/bin/env is a good way to achieve that. hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From roel at roelschroeven.net Sun Jul 21 13:30:22 2019 From: roel at roelschroeven.net (Roel Schroeven) Date: Sun, 21 Jul 2019 19:30:22 +0200 Subject: Namespaces: memory vs 'pollution' In-Reply-To: References: Message-ID: DL Neil schreef op 21/07/2019 om 2:02: > How do you remember to from-import- 'everything' that is needed? > ... > Upon closer inspection, I realised it didn't just fail; it failed badly! > Some silly, little, boy had imported the PythonEnvironment class but > failed to ALSO import PythonVersionError. So, the reported error was not > the expected exception! > ... > Is there some 'easy way' to make sure that one doesn't just import the > desired class, but also remembers to import 'everything else' that might > be necessary. In this case, it'd be rather nice to: > > from environment_module import Python* > > NB this is a syntax error, and won't import both the PythonEnvironment > and PythonVersionError classes. > ... > What do you do to (respecting purism) ensure 'everything' (necessary) is > imported (and nothing more), preferably without relying upon (faulty, in > my case) human-memory or reading through volumes of code/documentation? This is one of the advantages of using import instead of from-import. import environment_module ... try: .... # Just an example, since I don't know PythonEnvironment env = environment_module.PythonEnvironment() ... except environment_module.PythonVersionError: # Handle the exception In this case you have a pretty long module name (by the way, you could probably shorten it by removing _module; there's normally no need to repeat it in the name of a module that it's a module), making repeating it everywhere somewhat awkward. You can import the module using another name to work around that: import environment_module as envmod ... try: .... # Just an example, since I don't know PythonEnvironment env = envmod.PythonEnvironment() ... except envmod.PythonVersionError: # Handle the exception -- "Honest criticism is hard to take, particularly from a relative, a friend, an acquaintance, or a stranger." -- Franklin P. Jones Roel Schroeven From hjp-python at hjp.at Sun Jul 21 16:09:38 2019 From: hjp-python at hjp.at (Peter J. Holzer) Date: Sun, 21 Jul 2019 22:09:38 +0200 Subject: Counting Python threads vs C/C++ threads In-Reply-To: References: Message-ID: <20190721200938.cxef364orwhpywov@hjp.at> On 2019-07-16 12:48:33 -0700, Dan Stromberg wrote: > On Tue, Jul 16, 2019 at 11:13 AM Barry Scott wrote: > > Does top show the process using 100% CPU? > > > Nope. CPU utilization and disk use are both low. > > We've been going into top, and then hitting '1' to see things broken down > by CPU core (there are 32 of them, probably counting hyperthreads as > different cores), but the CPU use is in the teens or so. If you had many CPU-bound Python threads, then with 32 cores each core might show as 3 % busy (the sum of the threads can't use more then 100 % because of the GIL and 100 / 32 = 3.125). hp -- _ | Peter J. Holzer | we build much bigger, better disasters now |_|_) | | because we have much more sophisticated | | | hjp at hjp.at | management tools. __/ | http://www.hjp.at/ | -- Ross Anderson -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From michael.stemper at gmail.com Sun Jul 21 16:41:45 2019 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Sun, 21 Jul 2019 15:41:45 -0500 Subject: How to print out html tags excluding the attributes In-Reply-To: References: Message-ID: On 20/07/2019 20.04, sum abiut wrote: > I want to use regular expression to print out the HTML tags excluding the > attributes. > > for example: > > import re > html = '

Hi

test test

' > tags = re.findall(r'<[^>]+>', html) > for a in tags: > print(a) > > > the output is : > >

>

>

> > >

> > But I just want the tag, not the attributes Try this: for a in tags: a = re.sub( " .*>", ">", a ) print(a) (The two statements could be combined.) -- Michael F. Stemper Galatians 3:28 From cs at cskk.id.au Sun Jul 21 21:46:56 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Mon, 22 Jul 2019 11:46:56 +1000 Subject: Proper shebang for python3 In-Reply-To: <20190721134747.3h3ury7tcbwlppwk@hjp.at> References: <20190721134747.3h3ury7tcbwlppwk@hjp.at> Message-ID: <20190722014656.GA78297@cskk.homeip.net> On 21Jul2019 15:47, Peter J. Holzer wrote: >On 2019-07-21 09:04:43 +1000, Cameron Simpson wrote: >> I'm with Tim Daneliuk. The environment matters and should be honoured >> except >> in extremely weird cases. > >I don't think that all the scripts in /usr/bin are extremely weird >cases. I think I'd better add some nuance to my stance. I've no problem with all the scripts shipped from an OS vendor having #!/usr/bin/python (or whatever fixed path) in them. They have been released tested against the system python and should _expect_ to run against it. My position here is that the entire OS distribution constitutes a working (and via the #! controlled) environment. But consider third party tools. Including personal tools, but basicly anything from _outside_ the local system in terms of authorship. Particularly with a language like Python which is strongly backwards compatible, they should generally use "#!/usr/bin/env python" (or python2 or python3 as appropriate, when that matters) so that they can run in the environment they find themselves in. 1: You can can always execute a script via a specific interpreter explicitly. 2: If you want to test a script it is easier to provide an environment that exercises the script in a particular way than to hand patch the shebang lines on every run/reconfig. 3: If the script (per one of Chris' examples) requires a specific python such as 3.6, you can always go "#!usr/bin/env python3.6" in the script to express the target version and provide a executable "python3.6" name in you environment. I keep a personal ~/bin-local directory for just this kind of per-host stuff myself, and of course one can do the same thing in places like venvs or /usr/local/bin etc. And thus _still_ leave the script itself without a hardwired path. [...] >> If you require a specific outcoming, set a specific environment. It >> is under >> your control. Control it. >> >> For your specific example, "man youtube-dl" _is_ affected by the >> environment. It honours the $MANPATH variable. > >MANPATH is explicitely intended to control man. > >But man doesn't fail if you set your PATH to something weird. It will >still invoke /usr/bin/groff even if that isn't in the PATH. >(I expected that there is also an environment variable to control that >but the manpage doesn't mention one). Heh. I wrote my own "man" yonks ago for various reasons. Guess what? I expect to type "man" and get mine most of the time, but type "man" when not me and get /usr/bin/man (absent weirdness). That applies interactively and also in scripts. Same philosophy. Use the command name to express intent and the environment to choose the implementation of the intent. And so also in the shebang lines. >> For Peter J. Holzer, if we must play the "I've been doing this >> forever" game: I've been sysadmining etc longer than your 25 years and disagree with >> you. > >That's fine. Unlike Tim I don't claim that anybody who disagrees with me >must be a newbie. Aye; sorry for the snarkiness. Which is why I'm disagreeing on some things instead of asserting that you're wrong, because you're not "wrong". Cheers, Cameron Simpson From PythonList at DancesWithMice.info Mon Jul 22 01:06:30 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Mon, 22 Jul 2019 17:06:30 +1200 Subject: Namespaces: memory vs 'pollution' In-Reply-To: References: Message-ID: <7da372db-b7ff-294f-e748-a4690fd891f9@DancesWithMice.info> On 22/07/19 5:30 AM, Roel Schroeven wrote: > DL Neil schreef op 21/07/2019 om 2:02: >> How do you remember to from-import- 'everything' that is needed? >> ... > Upon closer inspection, I realised it didn't just fail; it >> failed badly! >> Some silly, little, boy had imported the PythonEnvironment class but >> failed to ALSO import PythonVersionError. So, the reported error was not >> the expected exception! >> ... >> Is there some 'easy way' to make sure that one doesn't just import the >> desired class, but also remembers to import 'everything else' that might >> be necessary. In this case, it'd be rather nice to: >> >> ????from environment_module import Python* >> >> NB this is a syntax error, and won't import both the PythonEnvironment >> and PythonVersionError classes. > > ... >> What do you do to (respecting purism) ensure 'everything' (necessary) is >> imported (and nothing more), preferably without relying upon (faulty, in >> my case) human-memory or reading through volumes of code/documentation? > > This is one of the advantages of using import instead of from-import. > > import environment_module > > ... > try: > ??? .... > ??? # Just an example, since I don't know PythonEnvironment > ??? env = environment_module.PythonEnvironment() > ??? ... > except environment_module.PythonVersionError: > ??? # Handle the exception > > In this case you have a pretty long module name (by the way, you could > probably shorten it by removing _module; there's normally no need to > repeat it in the name of a module that it's a module), making repeating > it everywhere somewhat awkward. You can import the module using another > name to work around that: > > import environment_module as envmod > > ... > try: > ??? .... > ??? # Just an example, since I don't know PythonEnvironment > ??? env = envmod.PythonEnvironment() > ??? ... > except envmod.PythonVersionError: > ??? # Handle the exception Greetings to Belgians, and thanks! Yes, I like this (interestingly, I recall posing a question some months back, asking how many of us bother to check for import exceptions). Long names: agreed, although I don't worry about it too much because competent editors 'pop-up' suggestions as one types. (incidentally, you are correct - I wouldn't really use that naming system, but inserted the word "module" in a bid to make the example illustrative) The opposite is even worse (and sometimes working with statisticians I'm often given 'algebra' with one-letter 'variable names') - in the general case, I criticise non-obvious abbreviations in code-review. Yet... Yesterday, I went 'full-circle' around the options for import statements (and some importlib facilities), looking at the effects on 'namespace pollution' and trying to find some sort of "wild-card" method. In the end, my conclusions were close-to, but not as well-developed as the above. Current thoughts: import environment_module as em - so, even more of an abbreviation than suggested!? - I rarely need to write a long list of import statements, so there won't be many. - not normally using such abbreviations in my code, they will stand-out. - considered using upper-case, eg "EM" - it is a form of constant after-all - considered adding a single under(-line) suffix, eg "em_" (on the basis of "made you think"). No, don't make me think (too much)! - so, perhaps a two-letter abbreviation with a single under(-line), eg "e_m", won't be confused with other naming conventions and yet stands-out. (sadly, that is "stands-out" to me, but 'everyone else' won't know its significance...) The try...except construct is a brilliant idea because it does exactly what I asked - requires both the class (what I wanted to include) AND the custom-exception class (what it needs included). If the class does not have any related error classes, then the try...except can simply check for 'exists?'... It's a habit I'm about to adopt. Many thanks! -- Regards =dn From rhodri at kynesim.co.uk Mon Jul 22 06:29:56 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Mon, 22 Jul 2019 11:29:56 +0100 Subject: How to print out html tags excluding the attributes In-Reply-To: References: Message-ID: <309d9d4d-653c-aad3-7004-4631e1765c8e@kynesim.co.uk> On 21/07/2019 02:04, sum abiut wrote: > I want to use regular expression to print out the HTML tags excluding the > attributes. That's a very good way of creating hard-to-read code and introducing subtle bugs and unexpected behaviours. Try using an HTML parser like BeautifulSoup instead. -- Rhodri James *-* Kynesim Ltd From arj.python at gmail.com Sun Jul 21 14:22:34 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Sun, 21 Jul 2019 22:22:34 +0400 Subject: Cool Mailing List For Article Writers Message-ID: Greetings all, There is a really nice list for people writing about Python. Share your latest articles and get reviews ^^_ Link: https://mail.python.org/mailman/listinfo/python-authors Abdur-Rahmaan Janhangeer Mauritius From hgs at dmu.ac.uk Fri Jul 19 05:53:05 2019 From: hgs at dmu.ac.uk (Hugh Sasse) Date: Fri, 19 Jul 2019 10:53:05 +0100 Subject: asyncio, transports, protocols, etc. Message-ID: <2086e26c-b39b-a5f0-4495-b1129a0265ce@dmu.ac.uk> Hello, I'm trying to get my head around asyncio, and I think I'm mostly there now, (but expect to be proved wrong :-)!). It appears to be about the newest of the PEPs according to my searches, including PEP 0, so I don't expect a huge amount of supporting documentation out there yet. Looking at: https://docs.python.org/3/library/asyncio-protocol.html#tcp-echo-server I know the page introduces the relationships between protocols, transports and the servers/clients that use them. When I read this code it took me longer than I would wish :-) to realise that the example defining the Echo server was doing this through the protocol, (yes, it says it *right there* :-)!) and not a Server class, so for some time I was puzzled about where 'transport' was being defined. Given these concepts are new on this page, at least ostensibly, would it make sense to introduce this example with something like: """ Create a TCP echo server using the loop.create_server() method, send back received data. This is done by defining the Protocol, which gets the transport from the server when it is created by asyncio. """ just to refresh the connection between the things in the mind of the reader? Also, according to my reading of RFC 862, I don't think the Echo server should be closing the connection just after echoing: """ TCP Based Echo Service One echo service is defined as a connection based application on TCP. A server listens for TCP connections on TCP port 7. Once a connection is established any data received is sent back. This continues until the calling user terminates the connection. """ I could have missed something here though. (If it is this protocol that is under discussion here, then a link might be useful, as well?) Why does this matter? Because it was not clear to me whether the closing of the connection was something that had to happen, as part of the functionality of the echo service or of asyncio itself. If I'm implementing something that does something based on the cmd or cmd2 module, interpreting commands, then having some kind of dialogue with the server matters. Writing documentation so that everyone will understand it and NOT get the wrong end of the stick is difficult. I don't know if this would make things more confusing for others, or even if it might have helped me. At the moment, I think it might be in the right direction, though. Thank you, Hugh -- -- Dr. Hugh Sasse, BSc(Hons), PhD Computer Systems Electronic Engineer School of Engineering and Sustainable Development DE MONTFORT UNIVERSITY From santiago.basulto at gmail.com Sun Jul 21 18:50:05 2019 From: santiago.basulto at gmail.com (Santiago Basulto) Date: Sun, 21 Jul 2019 19:50:05 -0300 Subject: Google Search results pointing to Python 2 (most of the time) Message-ID: Hello community, this is my first email to this list. Sorry if this sounds dumb, but anytime I do a google search I notice that the first result is from Python 2 docs. I always have to remind my students about it, and sometimes I even trip myself. [image: image.png] With the deprecation of Py2 so close, wouldn't it be preferable to try having Python 3 ranking first? I'm not a SEO expert, by maybe adding a warning title with a link to the corresponding Py3 document might help. Here's an example: [image: image.png] This might require a little bit more thought and some research with someone that knows about SEO. I just want to start the discussion. Thanks! -- Santiago Basulto.- Up! From info at tundraware.com Sat Jul 20 18:48:59 2019 From: info at tundraware.com (Tim Daneliuk) Date: Sat, 20 Jul 2019 17:48:59 -0500 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> <20190720195655.2y66jjeqdjckyimq@hjp.at> <61ac0g-4m2.ln1@oceanview.tundraware.com> Message-ID: On 7/20/19 4:28 PM, Brian Oney wrote: > Why not make a compromise? What would be a potential pitfall of the > following spitbang? > > #!python Not sure this really changes the discussion. From nicholas.cole at gmail.com Sun Jul 21 18:01:09 2019 From: nicholas.cole at gmail.com (Nicholas Cole) Date: Sun, 21 Jul 2019 23:01:09 +0100 Subject: List comprehension strangeness Message-ID: I was profiling a slow function in an application last week, and came across something that I still can?t explain. Inside a loop that was being called 4 times, inside a for loop that ran for a few dozen times there was a list compression of the form: [x.id for x in some_function()] According to the profiler, some_function was being called 52,000 times ? many, many times more than the few dozen it should have been. And sure enough, removing the comprehension did indeed speed up the function. It is very strange and feels like a possible bug in python. Has anyone ever encountered anything similar? Best wishes, Nicholas From cspealma at redhat.com Mon Jul 22 09:48:01 2019 From: cspealma at redhat.com (Calvin Spealman) Date: Mon, 22 Jul 2019 09:48:01 -0400 Subject: List comprehension strangeness In-Reply-To: References: Message-ID: It is impossible to diagnose without seeing more context. Specifically, you'll need to share the code around this line. The whole function, preferably. On Mon, Jul 22, 2019 at 9:31 AM Nicholas Cole wrote: > I was profiling a slow function in an application last week, and came > across something that I still can?t explain. Inside a loop that was being > called 4 times, inside a for loop that ran for a few dozen times there was > a list compression of the form: > > [x.id for x in some_function()] > > According to the profiler, some_function was being called 52,000 times ? > many, many times more than the few dozen it should have been. And sure > enough, removing the comprehension did indeed speed up the function. > > It is very strange and feels like a possible bug in python. Has anyone ever > encountered anything similar? > > Best wishes, > > Nicholas > -- > https://mail.python.org/mailman/listinfo/python-list > -- CALVIN SPEALMAN SENIOR QUALITY ENGINEER cspealma at redhat.com M: +1.336.210.5107 [image: https://red.ht/sig] TRIED. TESTED. TRUSTED. From bgailer at gmail.com Mon Jul 22 11:41:36 2019 From: bgailer at gmail.com (Bob Gailer) Date: Mon, 22 Jul 2019 11:41:36 -0400 Subject: List comprehension strangeness In-Reply-To: References: Message-ID: The function IMHO must be returning a generator. I would look for a problem in the generator code. From rosuav at gmail.com Mon Jul 22 11:48:58 2019 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Jul 2019 01:48:58 +1000 Subject: List comprehension strangeness In-Reply-To: References: Message-ID: On Mon, Jul 22, 2019 at 11:33 PM Nicholas Cole wrote: > > I was profiling a slow function in an application last week, and came > across something that I still can?t explain. Inside a loop that was being > called 4 times, inside a for loop that ran for a few dozen times there was > a list compression of the form: > > [x.id for x in some_function()] > > According to the profiler, some_function was being called 52,000 times ? > many, many times more than the few dozen it should have been. And sure > enough, removing the comprehension did indeed speed up the function. > Was it actually being CALLED that many times, or was some part of it being RUN that many times? Try adding an actual counter to the top of some_function to see how many times it's being invoked. ChrisA From jesse.ibarra.1996 at gmail.com Mon Jul 22 12:12:11 2019 From: jesse.ibarra.1996 at gmail.com (Jesse Ibarra) Date: Mon, 22 Jul 2019 09:12:11 -0700 (PDT) Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <21ada95f-9f12-4b49-a2f4-f8b5f8515453@googlegroups.com> Message-ID: On Saturday, July 20, 2019 at 1:11:51 PM UTC-6, Stefan Behnel wrote: > Jesse Ibarra schrieb am 20.07.19 um 04:12: > > Sorry, I am not understanding. Smalltlak VW 8.3 does not support Python. > > I can only call Pyhton code through C/Python API. > > Ok, but that doesn't mean you need to write code that uses the C-API of > Python. All you need to do is: > > 1) Start up a CPython runtime from Smalltalk (see the embedding example I > posted) and make it import an extension module that you write (e.g. using > the "inittab" mechanism [1]). > > 2) Use Cython to implement this extension module to provide an interface > between your Smalltalk code and your Python code. Use the Smalltalk C-API > from your Cython code to call into Smalltalk and exchange data with it. > > Now you can execute Python code inside of Python and make it call back and > forth into your Smalltalk code, through the interface module. And there is > no need to use the Python C-API for anything beyond step 1), which is about > 5 lines of Python C-API code if you write it yourself. Everything else can > be implemented in Cython and Python. > > Stefan > > > [1] > https://docs.python.org/3/extending/embedding.html?highlight=PyImport_appendinittab#extending-embedded-python This cleared so much @Stefan, thank you. I just need some clarification if you don't mind. In (1), when you say "import an extension module that you write", do you mean the Python library that was created "import emb"? Is that gonna be written in Cython or standalone .C file? in (2), what do to mean when you said "Use the Smalltalk C-API from your Cython code to call into Smalltalk and exchange data with it."? Please advise, Thank you so much for your help. From bgailer at gmail.com Mon Jul 22 12:58:43 2019 From: bgailer at gmail.com (Bob Gailer) Date: Mon, 22 Jul 2019 12:58:43 -0400 Subject: List comprehension strangeness In-Reply-To: References: Message-ID: The length of the list produced by the comprehension also give you good information. From rkartunova at yahoo.com Mon Jul 22 13:07:13 2019 From: rkartunova at yahoo.com (rkartunova at yahoo.com) Date: Mon, 22 Jul 2019 10:07:13 -0700 (PDT) Subject: Transfer Image from Raspberry Pi (Python) to Android app (Java) Message-ID: <16df6a0b-caf1-49d1-84aa-8c48f9a547ed@googlegroups.com> Hi everyone! I need help transferring an image via TCP from a python program on my raspberry pi to an android application. I have set up a client-server architecture such that my raspberry pi 3 records audio, performs some analysis on it, and then sends the data (via TCP) to the android app to display on the app screen. The recording and analysis is done and I am able to make the connection and transfer string data that displays on the app with no problem. However, I have been unsuccessful in transferring an image from rpi to android app. So basically, the image is stored on the rpi and I an attempting to transfer the image to the app to display it. I have been working on this for over a week with no luck so any help would be greatly appreciated! My current implementation (code snippets provided below): On rpi (python): Like I said, sending strings and displaying them on the android app is done without any problem. When I am sending the image portion of the audio analysis, I send a string first that says "?start" so that the android side knows that an image instead of a string is about to be sent (and will wait to update the GUI until it receives the entire image). Then, I open the image stored on rpi and read the entire image as a byte array (typically about 40-50k bytes). I get the length of the byte array and send that as a string to android app. Finally, I send the byte array to the android and it waits for an OK message from the app. All of this works without reporting any errors. On android app (java): When the app receives the "?start" string, it then uses a Buffered Reader (which is what I used to read the string data I had transferred to the app successfully earlier) to read the size of the image byte array. Then, I create 2 buffers, msg_buff and img_buff. msg_buff will read in 1024 bytes at a time while img_buff will hold the entire byte array of the image. In the infinite while loop, I have a DataInputStream, called in, read bytes into msg_buff and returns the number of bytes read. Then, I concatenate the copied contents of msg_buff into img_buff. Once the bytes read from in is -1 or the img_offset (which is just the total number of bytes read) is greater than or equal to the size of the image bytes array, the while loop is broken. Then, I would attempt to save the image to android internal storage and then load it later to an imageView to display it. This code does successfully read in the bytes until there are around 2000-3000 bytes left to be read and then it seems to freeze on the int bytes_read = in.read(msg_buff, 0, msg_buff.length) line. I have not been able to get past that point so I do not know if saving the image to internal storage and then loading it to imageview that way will work either. I have also tried using base64 encoding/decoding but that also kept producing errors. I have tried rpi only sending 1024 bytes of the image at a time but that also did not work. I have tried several implementations of this approach but nothing has worked so far. If anyone sees anything wrong or has another approach, I am all ears! Android Studio (app side): //receives the message which the server sends back InputStream sin = socket.getInputStream(); OutputStream sout = socket.getOutputStream(); DataInputStream in = new DataInputStream(sin); mBufferIn = new BufferedReader(new InputStreamReader(socket.getInputStream())); //in this while the client listens for the messages sent by the server while (mRun) { mServerMessage = mBufferIn.readLine(); if (mServerMessage != null && mMessageListener != null) { //Check if data is image if(mServerMessage.equals("?start")) { // Get length of image byte array int size = Integer.parseInt(mBufferIn.readLine()); // Create buffers byte[] msg_buff = new byte[1024]; byte[] img_buff = new byte[size]; int img_offset = 0; while(true){ int bytes_read = in.read(msg_buff, 0, msg_buff.length); if(bytes_read == -1){ break; } //copy bytes into img_buff System.arraycopy(msg_buff, 0, img_buff, img_offset, bytes_read); img_offset += bytes_read; if( img_offset >= size) { break; } } ContextWrapper cw = new ContextWrapper(ApplicationContextProvider.getContext()); File directory = cw.getDir("imageDir", Context.MODE_PRIVATE); File mypath = new File(directory, "signal.jpeg"); Bitmap bitmap = BitmapFactory.decodeByteArray(img_buff, 0, img_buff.length); FileOutputStream fos = null; try{ fos = new FileOutputStream(mypath); //Use compress method on Bitmap object to write image to OutputStream bitmap.compress(Bitmap.CompressFormat.JPEG, 100, fos); //Send OK byte[] OK = new byte[] {0x4F, 0x4B}; sout.write(OK); } catch (Exception e) { e.printStackTrace(); } } } Raspberry Pi (python): def image_to_byte_array(image_file, conn): with open(image_file, 'rb') as imageFile: content = imageFile.read() conn.sendall("?start\n".encode('utf-8')) size = len(content) strSize = str(size) + "\n" conn.sendall(strSize.encode('utf-8')) conn.sendall(content) Note that conn is the connection between the app and the rpi and the images are PNG. If anyone knows why this isn't working or has a better way for me to do this, I would greatly appreciate it!! Thank you in advance!! From ethan at stoneleaf.us Mon Jul 22 09:41:33 2019 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 22 Jul 2019 06:41:33 -0700 Subject: Extendable Enum like Type? In-Reply-To: References: <3148ba72-a667-cbd4-6123-fabc886e84e9@vub.be> Message-ID: <638149cb-b56d-ca7c-6337-0ce2d155d6c7@stoneleaf.us> On 07/19/2019 01:23 AM, Antoon Pardon wrote: > I don't seem to have made myself clear. The grammar with its Terminals > and NonTerminals is read in from a file. The program doesn't know what > they will be. > > For the moment what I am thinking about is something as follows: > > grammar = LoadGrammer(GrammarFile) > EnumList = ['class NonTerminal(Enum):\n'] > for Index, NonTerminal in enumerate(grammar.NonTerminals): > EnumList.append(' %s = %d\n' % (NonTerminal, Index)) > exec(''.join(EnumList), ..., ...) > > The problem with this approach is that I can't use these values until > the whole grammer is loaded. I would prefer some way to add values > during the loading of the grammar. Ah, you want `extend_enum` from the aenum library: class NonTerminal(Enum): pass while reading_grammar: non_terminal = .... index = .... extend_enum(NonTerminal, non_terminal, index) See also: - https://stackoverflow.com/a/33680929/208880 - https://stackoverflow.com/a/35899963/208880 - https://stackoverflow.com/a/36918171/208880 - https://stackoverflow.com/a/43004033/208880 -- ~Ethan~ From michael.stemper at gmail.com Mon Jul 22 16:01:29 2019 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 22 Jul 2019 15:01:29 -0500 Subject: Creating time stamps Message-ID: I have some code that generates a time-stamp as follows: from datetime import datetime tt = datetime.now() timestamp = "%4d-%02d-%02d %02d:%02d" % \ (tt.year, tt.month, tt.day, tt.hour, tt.minute) I later realized that I could have written it as: from datetime import datetime from time import strftime timestamp = datetime.now().strftime( "%Y-%m-%d %H:%M" ) The first seems a little clunky with its accessing of multiple attributes, but the second has an additional import. Is there any reason to prefer one over the other? -- Michael F. Stemper There's no "me" in "team". There's no "us" in "team", either. From rosuav at gmail.com Mon Jul 22 16:58:16 2019 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 23 Jul 2019 06:58:16 +1000 Subject: Creating time stamps In-Reply-To: References: Message-ID: On Tue, Jul 23, 2019 at 6:34 AM Michael F. Stemper wrote: > > I have some code that generates a time-stamp as follows: > > from datetime import datetime > tt = datetime.now() > timestamp = "%4d-%02d-%02d %02d:%02d" % \ > (tt.year, tt.month, tt.day, tt.hour, tt.minute) > > I later realized that I could have written it as: > > from datetime import datetime > from time import strftime > timestamp = datetime.now().strftime( "%Y-%m-%d %H:%M" ) > > The first seems a little clunky with its accessing of multiple > attributes, but the second has an additional import. Is there > any reason to prefer one over the other? > What's the second import doing though? You never use strftime. I'd go with the second one, but with just a single import. ChrisA From skip.montanaro at gmail.com Mon Jul 22 17:12:11 2019 From: skip.montanaro at gmail.com (Skip Montanaro) Date: Mon, 22 Jul 2019 16:12:11 -0500 Subject: Creating time stamps In-Reply-To: References: Message-ID: Assuming you're using Python 3, why not use an f-string? >>> dt = datetime.datetime.now() >>> dt.strftime("%Y-%m-%d %H:%M") '2019-07-22 16:10' >>> f"{dt:%Y-%m-%d %H:%M}" '2019-07-22 16:10' Skip From * at eli.users.panix.com Mon Jul 22 17:14:57 2019 From: * at eli.users.panix.com (Eli the Bearded) Date: Mon, 22 Jul 2019 21:14:57 +0000 (UTC) Subject: Proper shebang for python3 References: <20190720193543.3b457019@arcor.com> Message-ID: In comp.lang.python, Tim Daneliuk wrote: > On 7/20/19 1:20 PM, Chris Angelico wrote: > > On Sun, Jul 21, 2019 at 4:13 AM Michael Speer wrote: > >> You may want to use `#!/usr/bin/env python3` instead. I no longer have one to verify, but I recall Solaris boxen used /bin/env not /usr/bin/env. > So, no, do NOT encode the hard location - ever. Always use env to > discover the one that the user has specified. But wait, you just hard coded the location of env... > The only exception is > /bin/sh which - for a variety of reasons - can reliably counted upon. BZZZZ! Fully half of my work porting trn4 to my cellphone was fixing all the places that ancient build system believed /bin/sh was the name of sh. In that environment (Termux shell on an Android phone) the location is /data/data/com.termux/files/usr/bin/sh (and env is also in /data/data/com.termux/files/usr/bin hahaha). Even on more traditional environments -cough-Solaris-cough- /bin/sh may exist but be so ancient as to break things that work elsewhere. "^" as a synonym for "|", is a noteworthy gotcha. Figuring out where things are on the user's path is a laudable goal, but do it only at install time, not run time, for consistent runs. Elijah ------ pathological edge cases -r- us From uri at speedy.net Mon Jul 22 17:19:26 2019 From: uri at speedy.net (=?UTF-8?B?15DXldeo15k=?=) Date: Tue, 23 Jul 2019 00:19:26 +0300 Subject: Proper shebang for python3 In-Reply-To: References: <20190720193543.3b457019@arcor.com> Message-ID: We are using `#!/usr/bin/env python`, for example on https://github.com/speedy-net/speedy-net/blob/master/speedy/core/manage.py For bash we are using `#!/usr/bin/env bash`. I don't know if those are the best but they work. ???? uri at speedy.net On Sat, Jul 20, 2019 at 9:12 PM Michael Speer wrote: > You may want to use `#!/usr/bin/env python3` instead. > > There is a concept in python called the virtual environment. This used to > be done with a tool called virtualenv in python2, and is now done mainly > through a venv module in python3. > > A virtual environment goes into a directory of your choosing and will have > its own python3 executable, and pip3 executable, and when you add > dependencies, they are also placed into the directory structure under your > chosen directory. > > When you do a `. /bin/activate` the included source will places > the virtual environment's bin/ folder at the beginning of your PATH > environment variable, making it the default python3 when you type it > without a full path. > > This allows you to run scripts that need different, or even conflicting, > sets of dependencies without bothering with the underlying linux > distribution's python installation's modules. > > If you use `#!/usr/bin/python3`, it will always use exactly the system > version that is installed, and the system's installed modules. > > Your scripts will still default to the system installation if a virtual > environment is not activated. So you lose nothing by doing it this way, but > gain a little control from it. > > > On Sat, Jul 20, 2019 at 1:41 PM Manfred Lotz wrote: > > > Hi there, > > Pretty new to python I've got a question regarding the proper shebang > > for Python 3. > > > > I use > > #!/usr/bin/python3 > > > > which works fine. > > > > Today I saw > > #!/usr/bin/python3 -tt > > > > and was wondering what -tt means. > > > > Being on Fedora 30, Python 3.7.3 the man page of python3 doesn't even > > mention -t. > > > > python 2 man page mentions > > > > -t Issue a warning when a source file mixes tabs and spaces > > for indentation in a way that makes it depend on the worth > > of a tab expressed in spaces. Issue an error when the option is > > given twice. > > > > I guess that -t has the same meaning with python 3.7.3. > > > > > > My questions: > > > > 1. Is my guess correct? > > > > 2. Is it a bug that it is not mentioned? python3 --help doesn't mention > > it either. > > > > > > -- > > Manfred > > > > > > -- > > https://mail.python.org/mailman/listinfo/python-list > > > -- > https://mail.python.org/mailman/listinfo/python-list > From sjeik_appie at hotmail.com Mon Jul 22 17:27:38 2019 From: sjeik_appie at hotmail.com (Albert-Jan Roskam) Date: Mon, 22 Jul 2019 21:27:38 +0000 Subject: Creating time stamps In-Reply-To: Message-ID: On 22 Jul 2019 23:12, Skip Montanaro wrote: Assuming you're using Python 3, why not use an f-string? >>> dt = datetime.datetime.now() >>> dt.strftime("%Y-%m-%d %H:%M") '2019-07-22 16:10' >>> f"{dt:%Y-%m-%d %H:%M}" '2019-07-22 16:10' ===?? Or if you're running < Python 3.6 (no f strings): format(datetime.now(), "%Y-%m-%d %H:%M") From michael.stemper at gmail.com Mon Jul 22 17:25:32 2019 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 22 Jul 2019 16:25:32 -0500 Subject: Creating time stamps In-Reply-To: References: Message-ID: On 22/07/2019 15.58, Chris Angelico wrote: > On Tue, Jul 23, 2019 at 6:34 AM Michael F. Stemper > wrote: >> >> I have some code that generates a time-stamp as follows: >> >> from datetime import datetime >> tt = datetime.now() >> timestamp = "%4d-%02d-%02d %02d:%02d" % \ >> (tt.year, tt.month, tt.day, tt.hour, tt.minute) >> >> I later realized that I could have written it as: >> >> from datetime import datetime >> from time import strftime >> timestamp = datetime.now().strftime( "%Y-%m-%d %H:%M" ) >> >> The first seems a little clunky with its accessing of multiple >> attributes, but the second has an additional import. Is there >> any reason to prefer one over the other? > > What's the second import doing though? You never use strftime. Cleaned my contacts, cleaned my readers. Still see strftime() in the third line. Tried to run the code without the second line. It ran without complaint. Apparently, the strftime() in that last line is not the one that I explicitly imported, but a method of datetime.now(). Did I get that right? > I'd go > with the second one, but with just a single import. Sounds like a winner to me. Thanks. -- Michael F. Stemper 87.3% of all statistics are made up by the person giving them. From grant.b.edwards at gmail.com Mon Jul 22 17:41:32 2019 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 22 Jul 2019 21:41:32 -0000 (UTC) Subject: Creating time stamps References: Message-ID: On 2019-07-22, Michael F. Stemper wrote: >>> from datetime import datetime >>> from time import strftime >>> timestamp = datetime.now().strftime( "%Y-%m-%d %H:%M" ) [...] > Apparently, the strftime() in that last line is not the one that I > explicitly imported, but a method of datetime.now(). Did I get that > right? Exactly. -- Grant Edwards grant.b.edwards Yow! TONY RANDALL! Is YOUR at life a PATIO of FUN?? gmail.com From python at mrabarnett.plus.com Mon Jul 22 18:38:51 2019 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 22 Jul 2019 23:38:51 +0100 Subject: Creating time stamps In-Reply-To: References: Message-ID: On 2019-07-22 22:41, Grant Edwards wrote: > On 2019-07-22, Michael F. Stemper wrote: > >>>> from datetime import datetime >>>> from time import strftime >>>> timestamp = datetime.now().strftime( "%Y-%m-%d %H:%M" ) > [...] >> Apparently, the strftime() in that last line is not the one that I >> explicitly imported, but a method of datetime.now(). Did I get that >> right? > > Exactly. > You can tell that it's a method because it's: something.strftime(...) If it was using the imported 'strftime' then it would be: timestamp = strftime(...) From michael.stemper at gmail.com Mon Jul 22 18:58:29 2019 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 22 Jul 2019 17:58:29 -0500 Subject: Creating time stamps In-Reply-To: References: Message-ID: On 22/07/2019 16.00, Stefan Ram wrote: > "Michael F. Stemper" writes: >> The first seems a little clunky with its accessing of multiple >> attributes, but the second has an additional import. Is there >> any reason to prefer one over the other? > > |>>> import datetime > |>>> datetime.datetime.now().replace(microsecond=0).isoformat() > |'2019-07-22T21:59:58' Yeah, if I'd wanted that format, isoformat() would have been simpler. Since I did not want seconds, or the letter "T", I had to go for a home-brewed solution. -- Michael F. Stemper No animals were harmed in the composition of this message. From greg.ewing at canterbury.ac.nz Mon Jul 22 19:04:50 2019 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 23 Jul 2019 11:04:50 +1200 Subject: List comprehension strangeness In-Reply-To: References: Message-ID: Nicholas Cole wrote: [x.id for x in some_function()] > > According to the profiler, some_function was being called 52,000 times Is some_function recursive, by any chance? -- Greg From ethan at stoneleaf.us Mon Jul 22 19:00:36 2019 From: ethan at stoneleaf.us (Ethan Furman) Date: Mon, 22 Jul 2019 16:00:36 -0700 Subject: Namespaces: memory vs 'pollution' In-Reply-To: References: Message-ID: <74534bdd-8815-ee86-5a07-d9be25e6ffe1@stoneleaf.us> On 07/20/2019 05:02 PM, DL Neil wrote: > Upon closer inspection, I realised it didn't just fail; it failed badly! Some silly, little, boy had imported the PythonEnvironment class but failed to ALSO import PythonVersionError. So, the reported error was not the expected exception! I don't understand the significance of not importing PythonVersionError: - PythonEnvironment does not need it to be imported - when PythonEnvironment raises PythonVersionError you still get PythonVersionError - if your code says `except PythonVersionError` and you haven't imported it you'll get a NameError So, what's the issue? -- ~Ethan~ From cs at cskk.id.au Mon Jul 22 19:13:58 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 23 Jul 2019 09:13:58 +1000 Subject: Proper shebang for python3 In-Reply-To: References: Message-ID: <20190722231358.GA83700@cskk.homeip.net> On 22Jul2019 21:14, Eli the Bearded <*@eli.users.panix.com> wrote: >In comp.lang.python, Tim Daneliuk wrote: >> On 7/20/19 1:20 PM, Chris Angelico wrote: >> > On Sun, Jul 21, 2019 at 4:13 AM Michael Speer wrote: >> >> You may want to use `#!/usr/bin/env python3` instead. > >I no longer have one to verify, but I recall Solaris boxen used /bin/env >not /usr/bin/env. That is my recollection too. A pain point. To the point that I'd hand make a /usr/bin/env symlink. Why _any_ modern system has anything other than /bin in the base install escapes me. In the distant past /sbin and a distinct /usr with its own bin had their values, but these days? Bah! (I'm not complaining about /usr/local/bin here - keeping added stuff distinct from the vendor/distributor stuff is very valuable.) >> So, no, do NOT encode the hard location - ever. Always use env to >> discover the one that the user has specified. > >But wait, you just hard coded the location of env... Yeah. Too many boots, too many straps. >> The only exception is >> /bin/sh which - for a variety of reasons - can reliably counted upon. > >BZZZZ! Fully half of my work porting trn4 to my cellphone was fixing all >the places that ancient build system believed /bin/sh was the name of >sh. In that environment (Termux shell on an Android phone) the location >is /data/data/com.termux/files/usr/bin/sh (and env is also in >/data/data/com.termux/files/usr/bin hahaha). I'd do the symlink thing there too, if feasible. (Counterpoint: I've a DVR here where changes to / don't survive a reboot and there's no /etc/rc.local like file which survives a reboot either.) POSIX systems have a /bin/sh. I would move heaven itself to ensure this. >Even on more traditional environments -cough-Solaris-cough- /bin/sh may >exist but be so ancient as to break things that work elsewhere. "^" as >a synonym for "|", is a noteworthy gotcha. Aye. I still quote ^ in my scripts to this day for this reason. But then, I learnt shell programming on V7 UNIX, well older than Solaris. >Figuring out where things are on the user's path is a laudable goal, but >do it only at install time, not run time, for consistent runs. Perhaps. Consistent runs require consistent environments. That often includes executables. Cheers, Cameron Simpson From cs at cskk.id.au Mon Jul 22 19:16:18 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 23 Jul 2019 09:16:18 +1000 Subject: Proper shebang for python3 In-Reply-To: References: Message-ID: <20190722231618.GA9387@cskk.homeip.net> On 23Jul2019 00:19, ???? wrote: >We are using `#!/usr/bin/env python`, for example on >https://github.com/speedy-net/speedy-net/blob/master/speedy/core/manage.py > >For bash we are using `#!/usr/bin/env bash`. I don't know if those are the >best but they work. Worthwhile. Plenty of platforms do not install bash in /bin. Though personally I only VERY RARELY want bash for a shell script. /bin/sh is portable and the extras in bash have relatively little value in scripting; by the time your script really wants them it is usually worth moving to a more expressive language. Like Python. Cheers, Cameron Simpson From greg.ewing at canterbury.ac.nz Mon Jul 22 19:20:24 2019 From: greg.ewing at canterbury.ac.nz (Gregory Ewing) Date: Tue, 23 Jul 2019 11:20:24 +1200 Subject: Transfer Image from Raspberry Pi (Python) to Android app (Java) In-Reply-To: <16df6a0b-caf1-49d1-84aa-8c48f9a547ed@googlegroups.com> References: <16df6a0b-caf1-49d1-84aa-8c48f9a547ed@googlegroups.com> Message-ID: rkartunova at yahoo.com wrote: > This > code does successfully read in the bytes until there are around 2000-3000 > bytes left to be read and then it seems to freeze on the int bytes_read = > in.read(msg_buff, 0, msg_buff.length) line. This happens because you're trying to read more bytes than the sender is sending. The receiver is trying to read a whole bufferful and has no way to know that there aren't any more bytes coming, so it waits forever. Your receiver needs to read exactly the right number of bytes. Keep track of the number of bytes left to read, and when it's less than the buffer size, issue a read for just that number instead of a whole bufferful. (The code you have would work if the sender closed its end of the connection after sending the image, but you can't do that if you want to get an OK message back.) -- Greg From tjol at tjol.eu Mon Jul 22 05:40:49 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Mon, 22 Jul 2019 11:40:49 +0200 Subject: Namespaces: memory vs 'pollution' In-Reply-To: <7da372db-b7ff-294f-e748-a4690fd891f9@DancesWithMice.info> References: <7da372db-b7ff-294f-e748-a4690fd891f9@DancesWithMice.info> Message-ID: On 22/07/2019 07.06, DL Neil wrote: > > Current thoughts: > > ????import environment_module as em > > - so, even more of an abbreviation than suggested!? > - I rarely need to write a long list of import statements, so there > won't be many. > - not normally using such abbreviations in my code, they will stand-out. > - considered using upper-case, eg "EM" - it is a form of constant > after-all Just FYI, in the scientific Python community certain short abbreviations are the norm. Many modules have a ?standard? abbreviation that most people use, minimizing confusion. import numpy as np import matplotlib as mpl from matplotlib import pyplot as plt import pandas as pd import xarray as xr and so on. As long as you're consistent and use the same abbreviation across your entire codebase, and you put the imports at the top where people can find them, I think using 2?4 letter abbreviations, even without any decoration, is a fine approach. -- Thomas > - considered adding a single under(-line) suffix, eg "em_" (on the > basis of "made you think"). No, don't make me think (too much)! > - so, perhaps a two-letter abbreviation with a single under(-line), eg > "e_m", won't be confused with other naming conventions and yet > stands-out. (sadly, that is "stands-out" to me, but 'everyone else' > won't know its significance...) > > The try...except construct is a brilliant idea because it does exactly > what I asked - requires both the class (what I wanted to include) AND > the custom-exception class (what it needs included). > > If the class does not have any related error classes, then the > try...except can simply check for 'exists?'... > > > It's a habit I'm about to adopt. Many thanks!? From PythonList at DancesWithMice.info Mon Jul 22 22:27:51 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Tue, 23 Jul 2019 14:27:51 +1200 Subject: Namespaces: memory vs 'pollution' In-Reply-To: <74534bdd-8815-ee86-5a07-d9be25e6ffe1@stoneleaf.us> References: <74534bdd-8815-ee86-5a07-d9be25e6ffe1@stoneleaf.us> Message-ID: <4ad153ff-e19f-a90c-2a45-0a2da98508e1@DancesWithMice.info> On 23/07/19 11:00 AM, Ethan Furman wrote: > On 07/20/2019 05:02 PM, DL Neil wrote: > >> Upon closer inspection, I realised it didn't just fail; it failed >> badly! Some silly, little, boy had imported the PythonEnvironment >> class but failed to ALSO import PythonVersionError. So, the reported >> error was not the expected exception! > > I don't understand the significance of not importing PythonVersionError: > > - PythonEnvironment does not need it to be imported > > - when PythonEnvironment raises PythonVersionError you still get > PythonVersionError > > - if your code says `except PythonVersionError` and you haven't imported > it you'll get a NameError > > So, what's the issue? Have I correctly understood the question? NameError conveys nothing to the user. PythonVersionError is more communicative - and speaks volumes to 'us'. The mainline code is something like: p = PythonEnvironment() try: p.compatibility( ...spec... ) # eg must be Py3 not 2.n except PythonVersionError: print( more illuminating errmsg ) If I am 'the user' I'd be quite happy without the try...except; but mere mortals need/deserve something more. Accordingly, the PythonVersionError custom exception/class. Yes, we could trap NameError, but that might also catch other name-errors (unlikely in this example, but 'just sayin'). Thus the greater specificity of the custom class. NB please see alternative 'solution' proposed (for critique) as "Nesting Custom Errors in Classes" discussion topic. -- Regards =dn From PythonList at DancesWithMice.info Mon Jul 22 22:54:04 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Tue, 23 Jul 2019 14:54:04 +1200 Subject: Nesting Custom Errors in Classes Message-ID: <62dbfcca-e7bc-6e5e-6c2f-42883825d7bd@etelligence.info> Do you use nested classes? [following-on from the earlier, "Namespaces: memory vs 'pollution'" discussion thread, wherein a certain 'someone' remembered to from ... import ... as ... an 'action' class but forgot to also import the related custom error class! The original quest was for a wild-card import device. This discussion may obviate continuing the quest and/or needing to remember...] Python manages nested classes. I've NEVER seen such 'in the wild'. (but perhaps I lead a sheltered life?) What are proposed as use-cases for such a technique? - other languages may not offer decent "inheritance", and this is an alternative method/hack - the class is a 'class factory', generating/returning an object ? any others Why not use it as an "encapsulation" device? (please be gentle - reminder: I am too old to have been an OO-native!) ***** stub definitions >>> class PythonEnvironment(): ... class PythonVersionError( EnvironmentError ): ... pass ... def iscompatible( self ): ''' Ensure that the Python in-use will support all of the facilities employed by the application. ''' ... # stub to simulate failure ... raise self.PythonVersionError ... (code would require an import or from ... import ...) >>> pe = PythonEnvironment() ***** its basic application becomes:- >>> pe.iscompatible() Traceback (most recent call last): File "", line 1, in File "", line 6, in compatibility __main__.PythonVersionError ***** somewhat more realistic use:- >>> try: ... pe.iscompatible() ... except PythonEnvironment.PythonVersionError: ... print( "Trapped! -> informative errmsg" ) ... Trapped! -> informative errmsg With this construct, one only has to import the 'outer' class, rather than both the class AND its ancillary error class! Why haven't I seen it before? Can you see anything 'wrong' with this picture? -- Regards, =dn From PythonList at DancesWithMice.info Mon Jul 22 23:12:30 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Tue, 23 Jul 2019 15:12:30 +1200 Subject: Namespaces: memory vs 'pollution' In-Reply-To: References: <7da372db-b7ff-294f-e748-a4690fd891f9@DancesWithMice.info> Message-ID: <69615e96-72ab-be7a-06bc-4911898677c8@DancesWithMice.info> On 22/07/19 9:40 PM, Thomas Jollans wrote: > On 22/07/2019 07.06, DL Neil wrote: >> >> Current thoughts: >> >> ????import environment_module as em >> >> - so, even more of an abbreviation than suggested!? >> - I rarely need to write a long list of import statements, so there >> won't be many. >> - not normally using such abbreviations in my code, they will stand-out. >> - considered using upper-case, eg "EM" - it is a form of constant >> after-all > > > Just FYI, in the scientific Python community certain short abbreviations > are the norm. Many modules have a ?standard? abbreviation that most > people use, minimizing confusion. > > import numpy as np > import matplotlib as mpl > from matplotlib import pyplot as plt > import pandas as pd > import xarray as xr > > and so on. > > As long as you're consistent and use the same abbreviation across your > entire codebase, and you put the imports at the top where people can > find them, I think using 2?4 letter abbreviations, even without any > decoration, is a fine approach. +1 Thanks for this. Wow, but my little/personal utilities are far from such 'exalted' company! The important provision is that 'everyone' (affected) understands. As long as we all come from the same subject-domain, eg statisticians, then there's no problem with 'breaking the rules' because such abbreviations are easily-understood 'symbols', by definition. After all, PEP-8 does say: "names ... [should] reflect usage"! However, I'd never concede that to a group of 'commerce' trainees, for example! The use of meaningful names is a minimum professional standard (IMHO), and in my experience, a virtue that pays for itself. I am ever-so grateful that most modern text-editors will guess/read-my-mind and save the typing! -- Regards =dn From PythonList at DancesWithMice.info Tue Jul 23 05:06:07 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Tue, 23 Jul 2019 21:06:07 +1200 Subject: Hermetic environments Message-ID: Is Python going 'the right way' with virtual environments? Warning: this post may contain Python-heresy. I don't use any of the Python virtual environments! I'm not an *ops person either - unless Ops happened to be included within a wider department. However, in attempting to follow the "Proper shebang for python3" thread, the following appealed to my mind: <<< The truth is that there is no single answer to this problem until you hermetically seal an environment. Docker comes close, a freestanding VM does this most completely. But short of that, the ability to decide to use something other than system-provided tools absolutely IS a requirement in systems of any scale. >>> I do attempt to "hermetically seal" work, building a separate Virtual Box VM for each customer (and usually more than one). To be clear, this is less about Python (I use other languages), and more a method of keeping things completely separate, for privacy and security. It also allows me to conveniently separate 'dev' from 'prod'/support. I don't 'fiddle' but leave all the executables where the OpSys loads them (smarter minds than mine!), and load/pip everything as system-wide (no thinking necessary!). With the power of today's machines, I don't really notice* any performance degradation. There is no hassle about where the binaries may be found. There is no issue if one client is sticking with an earlier version but another wants the next iteration of development using some v3.7+ advancement. Similarly, there's no chance of tripping-over other clients' feet because some are web-based, some GUI, and some cmdLN! * we recently ran a test on a fairly compute-heavy statistics application: the same system on my (relatively old) portable and VBox, against a 'delivered version' installed natively on a newer and much bigger (RAM and CPU) Windows 10 box. Guess which little, old, Fedora-Linux, VBox-slowed machine 'won'!? Whilst the VM imposes an overhead 'cost', because it can be stripped-down/lean-and-mean/nothing-that's-unnecessary, the VM seems to be cost-effective. (regardless of the above tactics and criteria!) Standing on principle, I'd like to use a system other than VBox (Oracle) but it is neatly cross-platform and multi-platform. When eval-time is available, Docker is definitely on the try-out list (as mentioned in the quote, above). Normally, 'delivery' is a matter of duplicating the VM and fitting it into the client's DNS. It is similarly easy to pass work to A.N.Other, and presents no problem if there are multiple developers. (providing THE VM is not (only) on my portable...) Am I 'getting away with it', perhaps because my work-pattern doesn't touch some 'gotcha' or show-stopper? Why, if so much of 'the rest of the world' is utilising "containers", both for mobility and for growth, is the Python eco-system following its own path? Is there something about dev (and ops) using Python venvs which is a significant advantage over a language-independent (even better: an OpSys-independent) container? -- Regards, =dn From barry at barrys-emacs.org Tue Jul 23 06:02:03 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Tue, 23 Jul 2019 11:02:03 +0100 Subject: PyPi twine check does not like my README.rst files anymore In-Reply-To: <3B50624D-B7B3-4D86-A1C6-85F5698F486E@barrys-emacs.org> References: <719984C3-5731-4587-9DA3-F3D5B662C59A@barrys-emacs.org> <877e8imvw2.fsf@handshake.de> <3B50624D-B7B3-4D86-A1C6-85F5698F486E@barrys-emacs.org> Message-ID: <9C26767C-7CDA-475C-B4F1-4F719FB3CB9E@barrys-emacs.org> I tried the twine check on Fedora and the test pass. This looks looks like a bug in twine on Windows which I have reported in https://github.com/pypa/twine/issues/476 Barry From rhodri at kynesim.co.uk Tue Jul 23 07:01:15 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Tue, 23 Jul 2019 12:01:15 +0100 Subject: Namespaces: memory vs 'pollution' In-Reply-To: <4ad153ff-e19f-a90c-2a45-0a2da98508e1@DancesWithMice.info> References: <74534bdd-8815-ee86-5a07-d9be25e6ffe1@stoneleaf.us> <4ad153ff-e19f-a90c-2a45-0a2da98508e1@DancesWithMice.info> Message-ID: <9ff0ac7a-a45c-6dd0-0a16-93c82f5820ce@kynesim.co.uk> On 23/07/2019 03:27, DL Neil wrote: > On 23/07/19 11:00 AM, Ethan Furman wrote: >> On 07/20/2019 05:02 PM, DL Neil wrote: >> >>> Upon closer inspection, I realised it didn't just fail; it failed >>> badly! Some silly, little, boy had imported the PythonEnvironment >>> class but failed to ALSO import PythonVersionError. So, the reported >>> error was not the expected exception! >> >> I don't understand the significance of not importing PythonVersionError: >> >> - PythonEnvironment does not need it to be imported >> >> - when PythonEnvironment raises PythonVersionError you still get >> PythonVersionError >> >> - if your code says `except PythonVersionError` and you haven't >> imported it you'll get a NameError >> >> So, what's the issue? > > > Have I correctly understood the question? > > NameError conveys nothing to the user. > PythonVersionError is more communicative - and speaks volumes to 'us'. > > The mainline code is something like: > > ????p = PythonEnvironment() > ????try: > ??????? p.compatibility( ...spec... )??? # eg must be Py3 not 2.n > ????except PythonVersionError: > ??????? print( more illuminating errmsg ) > > If I am 'the user' I'd be quite happy without the try...except; but mere > mortals need/deserve something more. Accordingly, the PythonVersionError > custom exception/class. Ah, so you *used* PythonVersionError without importing it. Sorry, but that's on you. -- Rhodri James *-* Kynesim Ltd From tjol at tjol.eu Tue Jul 23 07:55:09 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 23 Jul 2019 13:55:09 +0200 Subject: Namespaces: memory vs 'pollution' In-Reply-To: <4ad153ff-e19f-a90c-2a45-0a2da98508e1@DancesWithMice.info> References: <74534bdd-8815-ee86-5a07-d9be25e6ffe1@stoneleaf.us> <4ad153ff-e19f-a90c-2a45-0a2da98508e1@DancesWithMice.info> Message-ID: On 23/07/2019 04.27, DL Neil wrote: > On 23/07/19 11:00 AM, Ethan Furman wrote: >> On 07/20/2019 05:02 PM, DL Neil wrote: >> >>> Upon closer inspection, I realised it didn't just fail; it failed >>> badly! Some silly, little, boy had imported the PythonEnvironment >>> class but failed to ALSO import PythonVersionError. So, the reported >>> error was not the expected exception! >> >> I don't understand the significance of not importing PythonVersionError: >> >> - PythonEnvironment does not need it to be imported >> >> - when PythonEnvironment raises PythonVersionError you still get >> PythonVersionError >> >> - if your code says `except PythonVersionError` and you haven't >> imported it you'll get a NameError >> Actually, if your code says ?except PythonVersionError? this will only raise a NameError if there is an actual exception. (this was not what I expected - I learned something new today!) If no exception is being handled, the except statements are never examined. This makes this harder to spot than you might think! % cat test.py??? try: ??? print('ok') except FakeError: ??? print('ah shit') % python3 test.py ok That makes me think... def get_exc(exc): ??? print("we're checking for exception", exc) ??? return exc try: ??? print("raising no. 1") ??? raise ValueError except get_exc(TypeError): ??? print("t'was a TypeError") except get_exc(ValueError): ??? print("t'was a ValueError") try: ??? print("raising no. 2") ??? raise TypeError except get_exc(TypeError): ??? print("t'was a TypeError") except get_exc(ValueError): ??? print("t'was a ValueError") >> So, what's the issue? > > > Have I correctly understood the question? > > NameError conveys nothing to the user. > PythonVersionError is more communicative - and speaks volumes to 'us'. > > The mainline code is something like: > > ????p = PythonEnvironment() > ????try: > ??????? p.compatibility( ...spec... )??? # eg must be Py3 not 2.n > ????except PythonVersionError: > ??????? print( more illuminating errmsg ) > > If I am 'the user' I'd be quite happy without the try...except; but > mere mortals need/deserve something more. Accordingly, the > PythonVersionError custom exception/class. > > Yes, we could trap NameError, but that might also catch other > name-errors (unlikely in this example, but 'just sayin'). Thus the > greater specificity of the custom class. > > > NB please see alternative 'solution' proposed (for critique) as > "Nesting Custom Errors in Classes" discussion topic. From cs at cskk.id.au Tue Jul 23 07:55:41 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 23 Jul 2019 21:55:41 +1000 Subject: Nesting Custom Errors in Classes In-Reply-To: <62dbfcca-e7bc-6e5e-6c2f-42883825d7bd@etelligence.info> References: <62dbfcca-e7bc-6e5e-6c2f-42883825d7bd@etelligence.info> Message-ID: <20190723115541.GA44449@cskk.homeip.net> On 23Jul2019 14:54, DL Neil wrote: >Do you use nested classes? >Python manages nested classes. > >I've NEVER seen such 'in the wild'. >(but perhaps I lead a sheltered life?) So sheltered :-) In my experience it's uncommon. I've done it a few times. "Nested" classes aren't anything very special; they're no more than yet another name defined inside a class, like a method, or a "constant". Look: class Foo: DEFAULT_FOO_MAX = 10 def __init__(...): ... class SpecialThing: ... They're all just various names in the class space. I would do it if SpecialThing wasn't meaningful outside of the class i.e. didn't really have a standalone use. Aside from not polluting the top level namespace with names that aren't useful on their own (which is kind of irrelevant since people tend to "from foo import name", so the _number_ of available names isn't very important)... One potential benefit to to associating the inner class with the outer class is the inheritance chain. Look: class Foo: class ClassyThing: ... def frob(self, it): for i in it: x = self.ClassyThing(i) x.do_something_classy(...) class Bah(Foo): class ClassyThing: ... f = Foo() f.frob([1,2,3]) b = Bah() b.frob([1,2,3]) Foo and Bah have a common method "frob", but f.frob will generate and use Foo.ClassyThing objects while b.frob will generate Bah.ClassyThing objects. Same code. Personally, I haven't done the above. But I have made subclasses where the subclass overrides a constant, like DEFAULT_FOO_MAX earlier. Then accessing self.DEFAULT_FOO_MAX finds the appropriate constant. And on thinking about it, while I haven't explicitly nested classes, I have passed in a class to the init method, for example an indexclass. I've a package which support several backend hash indices, such as gdbm, and I pass in the target index class. So a class as a parameterisable or inheritable thing isn't nonsensical. I have use nested classes when the nested class is strongly associated with the enclosing class, like your classname.exceptionname example. I'd put it inside the outer class entirely to express that association. Cheers, Cameron Simpson From blmadhavan at gmail.com Tue Jul 23 10:22:14 2019 From: blmadhavan at gmail.com (Madhavan Bomidi) Date: Tue, 23 Jul 2019 07:22:14 -0700 (PDT) Subject: How to execute shell command in Python program? In-Reply-To: <946b629e-59b6-4a61-bf1d-cf1419e31932@googlegroups.com> References: <946b629e-59b6-4a61-bf1d-cf1419e31932@googlegroups.com> Message-ID: <1dfb9002-e19d-4514-99fb-8451a3046fbe@googlegroups.com> Thanks for your suggestions. I have sorted the issue as follows: import pexpect child = pexpect.spawn('./opac') child.sendline('opac') This works!!! From ethan at stoneleaf.us Tue Jul 23 10:46:19 2019 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 23 Jul 2019 07:46:19 -0700 Subject: Namespaces: memory vs 'pollution' In-Reply-To: <4ad153ff-e19f-a90c-2a45-0a2da98508e1@DancesWithMice.info> References: <74534bdd-8815-ee86-5a07-d9be25e6ffe1@stoneleaf.us> <4ad153ff-e19f-a90c-2a45-0a2da98508e1@DancesWithMice.info> Message-ID: On 07/22/2019 07:27 PM, DL Neil wrote: > NameError conveys nothing to the user. > PythonVersionError is more communicative - and speaks volumes to 'us'. > > The mainline code is something like: > > ????p = PythonEnvironment() > ????try: > ??????? p.compatibility( ...spec... )??? # eg must be Py3 not 2.n > ????except PythonVersionError: > ??????? print( more illuminating errmsg ) This is why tests should check for the exact error raised, not that any error was raised. (That testing bug bit me a couple times before I learned that lesson.) -- ~Ethan~ From ethan at stoneleaf.us Tue Jul 23 10:46:31 2019 From: ethan at stoneleaf.us (Ethan Furman) Date: Tue, 23 Jul 2019 07:46:31 -0700 Subject: Namespaces: memory vs 'pollution' In-Reply-To: <69615e96-72ab-be7a-06bc-4911898677c8@DancesWithMice.info> References: <7da372db-b7ff-294f-e748-a4690fd891f9@DancesWithMice.info> <69615e96-72ab-be7a-06bc-4911898677c8@DancesWithMice.info> Message-ID: <62a72aee-a3af-a498-1b9b-3be000916d8e@stoneleaf.us> On 07/22/2019 08:12 PM, DL Neil wrote: > On 22/07/19 9:40 PM, Thomas Jollans wrote: >> Just FYI, in the scientific Python community certain short abbreviations >> are the norm. Many modules have a ?standard? abbreviation that most >> people use, minimizing confusion. >> >> import numpy as np >> import matplotlib as mpl >> from matplotlib import pyplot as plt >> import pandas as pd >> import xarray as xr > The use of meaningful names is a minimum professional standard (IMHO), and in my experience, a virtue that pays for itself. I am ever-so grateful that most modern text-editors will guess/read-my-mind and save the typing! It's not just typing, but reading. Long names, when used over and over, become noise. -- ~Ethan~ From pkpearson at nowhere.invalid Tue Jul 23 11:29:39 2019 From: pkpearson at nowhere.invalid (Peter Pearson) Date: 23 Jul 2019 15:29:39 GMT Subject: Creating time stamps References: Message-ID: On Mon, 22 Jul 2019 16:25:32 -0500, Michael F. Stemper wrote: > On 22/07/2019 15.58, Chris Angelico wrote: >> On Tue, Jul 23, 2019 at 6:34 AM Michael F. Stemper >> wrote: >>> [snip] >>> from datetime import datetime >>> from time import strftime >>> timestamp = datetime.now().strftime( "%Y-%m-%d %H:%M" ) [snip] >> >> What's the second import doing though? You never use strftime. > > Cleaned my contacts, cleaned my readers. Still see strftime() in the > third line. Tried to run the code without the second line. It ran > without complaint. > > Apparently, the strftime() in that last line is not the one that I > explicitly imported, but a method of datetime.now(). Did I get that > right? Yes, you got that right. The statement datetime.now().strftime("...") says that datetime.now will return an object that has a method named strftime, and calling that method is the next thing to be done. Other things that happen to be lying around named strftime just have no part in the conversation. -- To email me, substitute nowhere->runbox, invalid->com. From rgaddi at highlandtechnology.invalid Tue Jul 23 12:38:30 2019 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Tue, 23 Jul 2019 09:38:30 -0700 Subject: Nesting Custom Errors in Classes In-Reply-To: References: <62dbfcca-e7bc-6e5e-6c2f-42883825d7bd@etelligence.info> Message-ID: On 7/22/19 7:54 PM, DL Neil wrote: > Do you use nested classes? > > [following-on from the earlier, "Namespaces: memory vs 'pollution'" discussion > thread, wherein a certain 'someone' remembered to from ... import ... as ... an > 'action' class but forgot to also import the related custom error class! The > original quest was for a wild-card import device. This discussion may obviate > continuing the quest and/or needing to remember...] > > > Python manages nested classes. > > I've NEVER seen such 'in the wild'. > (but perhaps I lead a sheltered life?) > > > What are proposed as use-cases for such a technique? > - other languages may not offer decent "inheritance", and this is an alternative > method/hack > - the class is a 'class factory', generating/returning an object > ? any others > > > Why not use it as an "encapsulation" device? > (please be gentle - reminder: I am too old to have been an OO-native!) > > > ***** stub definitions > > >>> class PythonEnvironment(): > ...???? class PythonVersionError( EnvironmentError ): > ...???????????? pass > ...???? def iscompatible( self ): > ??????? ''' Ensure that the Python in-use will support > ??????? all of the facilities employed by the application. > ??????? ''' > ...???????????? # stub to simulate failure > ...???????????? raise self.PythonVersionError > ... > > (code would require an import or from ... import ...) > >>> pe = PythonEnvironment() > > > ***** its basic application becomes:- > > >>> pe.iscompatible() > Traceback (most recent call last): > ? File "", line 1, in > ? File "", line 6, in compatibility > __main__.PythonVersionError > > > ***** somewhat more realistic use:- > > >>> try: > ...???? pe.iscompatible() > ... except PythonEnvironment.PythonVersionError: > ...???? print( "Trapped! -> informative errmsg" ) > ... > Trapped! -> informative errmsg > > > With this construct, one only has to import the 'outer' class, rather than both > the class AND its ancillary error class! > > > Why haven't I seen it before? Can you see anything 'wrong' with this picture? I've used them sometimes for basic encapsulation principles without really gaining anything. I use inheritance of nested classes in my https://pypi.org/project/indexedproperty/ project, where subclasses of IndexedProperty have a _Trampoline inner class subclassed from Trampoline that can be redefined as needed, but that's a bit of an obscure use case. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From PythonList at DancesWithMice.info Tue Jul 23 15:21:28 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Wed, 24 Jul 2019 07:21:28 +1200 Subject: Nesting Custom Errors in Classes In-Reply-To: <20190723115541.GA44449@cskk.homeip.net> References: <62dbfcca-e7bc-6e5e-6c2f-42883825d7bd@etelligence.info> <20190723115541.GA44449@cskk.homeip.net> Message-ID: On 23/07/19 11:55 PM, Cameron Simpson wrote: > On 23Jul2019 14:54, DL Neil wrote: >> Do you use nested classes? >> Python manages nested classes. >> >> I've NEVER seen such 'in the wild'. >> (but perhaps I lead a sheltered life?) > > So sheltered :-) Well, over here there aren't the venomous creatures you live amongst on 'west island'... > In my experience it's uncommon. I've done it a few times. > > "Nested" classes aren't anything very special; they're no more than yet > another name defined inside a class, like a method, or a "constant". Look: Thanks for the explanation. I've absorbed the concept, but back in the ?good, old days it was common practice to alter code programmatically, ie whilst the pgm was running. The perils of not quite knowing what the code 'is' and thus what it should be doing for us, constitute a debugging nightmare. (but hey, even flicking toggle-switches on the front of an IBM System/360 to load binary data into a specified memory address, seemed like 'great power' - at the time *cue SpiderMan quote) Accordingly, the idea that there would be a ClassyThing() but it might be quite different from another ClassyThing() just makes me shudder! (sorry, I guess it's that sheltered upbringing ... again!) As you say... > Personally, I haven't done the above. But I have made subclasses where > the subclass overrides a constant, like DEFAULT_FOO_MAX earlier. Then > accessing self.DEFAULT_FOO_MAX finds the appropriate constant. > > And on thinking about it, while I haven't explicitly nested classes, I > have passed in a class to the init method, for example an indexclass. > I've a package which support several backend hash indices, such as gdbm, > and I pass in the target index class. So a class as a parameterisable or > inheritable thing isn't nonsensical. > > I have use nested classes when the nested class is strongly associated > with the enclosing class, like your classname.exceptionname example. I'd > put it inside the outer class entirely to express that association. ...and, because the (?old) boy can't be relied upon to remember to import both, nesting saves-the-day! -- Regards =dn From stefan_ml at behnel.de Tue Jul 23 16:05:18 2019 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 23 Jul 2019 22:05:18 +0200 Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <21ada95f-9f12-4b49-a2f4-f8b5f8515453@googlegroups.com> Message-ID: Jesse Ibarra schrieb am 22.07.19 um 18:12: > On Saturday, July 20, 2019 at 1:11:51 PM UTC-6, Stefan Behnel wrote: >> Jesse Ibarra schrieb am 20.07.19 um 04:12: >>> Sorry, I am not understanding. Smalltlak VW 8.3 does not support Python. >>> I can only call Pyhton code through C/Python API. >> >> Ok, but that doesn't mean you need to write code that uses the C-API of >> Python. All you need to do is: >> >> 1) Start up a CPython runtime from Smalltalk (see the embedding example I >> posted) and make it import an extension module that you write (e.g. using >> the "inittab" mechanism [1]). >> >> 2) Use Cython to implement this extension module to provide an interface >> between your Smalltalk code and your Python code. Use the Smalltalk C-API >> from your Cython code to call into Smalltalk and exchange data with it. >> >> Now you can execute Python code inside of Python and make it call back and >> forth into your Smalltalk code, through the interface module. And there is >> no need to use the Python C-API for anything beyond step 1), which is about >> 5 lines of Python C-API code if you write it yourself. Everything else can >> be implemented in Cython and Python. >> >> Stefan >> >> >> [1] >> https://docs.python.org/3/extending/embedding.html?highlight=PyImport_appendinittab#extending-embedded-python > > This cleared so much @Stefan, thank you. I just need some clarification if you don't mind. > > In (1), when you say "import an extension module that you write", do you mean the Python library that was created "import emb"? Is that gonna be written in Cython or standalone .C file? Yes. In Cython. > in (2), what do to mean when you said "Use the Smalltalk C-API from your Cython code to call into Smalltalk and exchange data with it."? Not sure what part exactly you are asking about, but you somehow have to talk to the Smalltalk runtime from your Cython/Python code if you want to interact with it. I assume that this will be done through the C API that Smalltalk provides. Just in case, did you check if there is already a bridge for your purpose? A quick web search let me find this, not sure if it helps. https://github.com/ObjectProfile/PythonBridge Stefan From cs at cskk.id.au Tue Jul 23 18:07:33 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Wed, 24 Jul 2019 08:07:33 +1000 Subject: Nesting Custom Errors in Classes In-Reply-To: References: Message-ID: <20190723220733.GA6472@cskk.homeip.net> On 24Jul2019 07:21, DL Neil wrote: >Accordingly, the idea that there would be a ClassyThing() but it might >be quite different from another ClassyThing() just makes me shudder! >(sorry, I guess it's that sheltered upbringing ... again!) It needn't be very different on the outside. The whole point of object oriented programme is to present things with the same external API but different internal implementations, or managing a different thing with common characteristics. >>I have use nested classes when the nested class is strongly associated >>with the enclosing class, like your classname.exceptionname example. >>I'd put it inside the outer class entirely to express that >>association. > >...and, because the (?old) boy can't be relied upon to remember to >import both, nesting saves-the-day! Get some linting tools. They're great for catching this kind of error. The tricky bit with dynamic language like Python is that some naming errors (missed imports) aren't apparent until the code passes through that path. Linters can cover a lot of this with static analysis. Cheers, Cameron Simpson From PythonList at DancesWithMice.info Tue Jul 23 19:47:51 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Wed, 24 Jul 2019 11:47:51 +1200 Subject: Nesting Custom Errors in Classes In-Reply-To: <20190723220733.GA6472@cskk.homeip.net> References: <20190723220733.GA6472@cskk.homeip.net> Message-ID: On 24/07/19 10:07 AM, Cameron Simpson wrote: > On 24Jul2019 07:21, DL Neil wrote: ... > Get some linting tools. They're great for catching this kind of error. SublimeText has SublimeLinter and PycodeStyle installed, but I'm still familiarising myself with ST. Nothing is reported. Any advice/correction/improvement would be most welcome... > The tricky bit with dynamic language like Python is that some naming > errors (missed imports) aren't apparent until the code passes through > that path. Linters can cover a lot of this with static analysis. Exactly! (PyTest and coverage tools were no help - they import correctly (and test adequately), but I didn't open those (separate) modules to notice the multiple import-s) Hope springs eternal ... there must be a better way??? (well yes, not wishing to 'flog a dead horse' (nor any horse for that matter) I'm pleased at the (lack of negative) response to my allied question about nested classes, which would solve the original problem) -- Regards =dn From cs at cskk.id.au Tue Jul 23 21:30:34 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Wed, 24 Jul 2019 11:30:34 +1000 Subject: Nesting Custom Errors in Classes In-Reply-To: References: Message-ID: <20190724013034.GA67972@cskk.homeip.net> On 24Jul2019 11:47, DL Neil wrote: >On 24/07/19 10:07 AM, Cameron Simpson wrote: >>On 24Jul2019 07:21, DL Neil wrote: >... >>Get some linting tools. They're great for catching this kind of error. > >SublimeText has SublimeLinter and PycodeStyle installed, but I'm still >familiarising myself with ST. Nothing is reported. > >Any advice/correction/improvement would be most welcome... Hmm. I'd expect a linter to catch this: # forgot FooException from foo import FooClass def f(): raise FooException Can you test that with a small script at your end (with real imports and names of course)? I lint from the command line with this script: https://bitbucket.org/cameron_simpson/css/src/tip/bin-cs/lint which runs a few Python linters for Python. WRT SublimeText, PycodeStyle likely only checks style (I've now shifted to an automatic formatter for this) and SublimeLinter's PythonLinter page is... silent; it looks like a base class for other actual linters. You might need to do some more digging. Maybe you need some addition lint stuff for SublimeText. >>The tricky bit with dynamic language like Python is that some naming >>errors (missed imports) aren't apparent until the code passes through >>that path. Linters can cover a lot of this with static analysis. > >Exactly! >(PyTest and coverage tools were no help - they import correctly (and >test adequately), but I didn't open those (separate) modules to notice >the multiple import-s) > >Hope springs eternal ... there must be a better way??? A good linter is usually decent at complaining about code using names that aren't defined (not imports, not otherwise defined such as a function or a variable assignment). >(well yes, not wishing to 'flog a dead horse' (nor any horse for that >matter) I'm pleased at the (lack of negative) response to my allied >question about nested classes, which would solve the original problem) There's nothing "wrong" with it. It is uncommon. But like a lot of things, the value (or cost/pitfalls) come with how things are to be used. Cheers, Cameron Simpson From jesse.ibarra.1996 at gmail.com Wed Jul 24 12:27:45 2019 From: jesse.ibarra.1996 at gmail.com (Jesse Ibarra) Date: Wed, 24 Jul 2019 09:27:45 -0700 (PDT) Subject: Embedding Python in C In-Reply-To: References: <02557d6f-5212-4a23-b63a-5f95575d9e06@googlegroups.com> <00320b4d-dc4e-4d14-8c95-56acdbc5fdca@googlegroups.com> <21ada95f-9f12-4b49-a2f4-f8b5f8515453@googlegroups.com> Message-ID: On Tuesday, July 23, 2019 at 2:20:45 PM UTC-6, Stefan Behnel wrote: > Jesse Ibarra schrieb am 22.07.19 um 18:12: > > On Saturday, July 20, 2019 at 1:11:51 PM UTC-6, Stefan Behnel wrote: > >> Jesse Ibarra schrieb am 20.07.19 um 04:12: > >>> Sorry, I am not understanding. Smalltlak VW 8.3 does not support Python. > >>> I can only call Pyhton code through C/Python API. > >> > >> Ok, but that doesn't mean you need to write code that uses the C-API of > >> Python. All you need to do is: > >> > >> 1) Start up a CPython runtime from Smalltalk (see the embedding example I > >> posted) and make it import an extension module that you write (e.g. using > >> the "inittab" mechanism [1]). > >> > >> 2) Use Cython to implement this extension module to provide an interface > >> between your Smalltalk code and your Python code. Use the Smalltalk C-API > >> from your Cython code to call into Smalltalk and exchange data with it. > >> > >> Now you can execute Python code inside of Python and make it call back and > >> forth into your Smalltalk code, through the interface module. And there is > >> no need to use the Python C-API for anything beyond step 1), which is about > >> 5 lines of Python C-API code if you write it yourself. Everything else can > >> be implemented in Cython and Python. > >> > >> Stefan > >> > >> > >> [1] > >> https://docs.python.org/3/extending/embedding.html?highlight=PyImport_appendinittab#extending-embedded-python > > > > This cleared so much @Stefan, thank you. I just need some clarification if you don't mind. > > > > In (1), when you say "import an extension module that you write", do you mean the Python library that was created "import emb"? Is that gonna be written in Cython or standalone .C file? > > Yes. In Cython. > > > > in (2), what do to mean when you said "Use the Smalltalk C-API from your Cython code to call into Smalltalk and exchange data with it."? > > Not sure what part exactly you are asking about, but you somehow have to > talk to the Smalltalk runtime from your Cython/Python code if you want to > interact with it. I assume that this will be done through the C API that > Smalltalk provides. > > Just in case, did you check if there is already a bridge for your purpose? > A quick web search let me find this, not sure if it helps. > > https://github.com/ObjectProfile/PythonBridge > > Stefan Yes, I think that can be done through the "inittab" mechanism you recommended. I will try it. Yes, I trying to implement the same thing as shown in the GitHub link but instead of Pharo I will use VisualWorks IDE Thank you From helu.careers at gmail.com Wed Jul 24 12:41:35 2019 From: helu.careers at gmail.com (helu.careers at gmail.com) Date: Wed, 24 Jul 2019 09:41:35 -0700 (PDT) Subject: Ebook for Understanding Financial Accounting, Canadian Edition by Burnley In-Reply-To: References: <08d843b4-cdb1-4aa6-9e39-7bfa02f2e1ed@googlegroups.com> Message-ID: On Friday, January 19, 2018 at 2:42:17 PM UTC-5, vaibhavb... at gmail.com wrote: > Im lookjng for the pdf file. For understanding financial canadian edition Hey buddy, have you got the PDF file for this textbook and the solutions manual of this textbook? From * at eli.users.panix.com Wed Jul 24 15:59:33 2019 From: * at eli.users.panix.com (Eli the Bearded) Date: Wed, 24 Jul 2019 19:59:33 +0000 (UTC) Subject: Hermetic environments References: Message-ID: In comp.lang.python, DL Neil wrote: > Is Python going 'the right way' with virtual environments? ... > Am I 'getting away with it', perhaps because my work-pattern doesn't > touch some 'gotcha' or show-stopper? > > Why, if so much of 'the rest of the world' is utilising "containers", > both for mobility and for growth, is the Python eco-system following its > own path? I'm going to speculate that even inside containers, some people will use multiple virtual environments. It could be that the app and the monitoring for that app are developed by different branches of the company and have different requirements. But I think a lot of the use of virtual environments is in dev environments where a developer wants to have multiple closed settings for doing work. On the dev branch, newer versions of things can be tested, but a production environment can be retained for hotfixes to deployed code. Or because the different microservices being used are each at different update levels and need their own environments. > Is there something about dev (and ops) using Python venvs which is a > significant advantage over a language-independent (even better: an > OpSys-independent) container? I'm not a big fan of language-dependent virtual environments because they only capture the needs of a particular language. Very often code works with things that are outside of that language, even if it is only system libraries. Elijah ------ interested in hearing other voices on this From barry at barrys-emacs.org Wed Jul 24 16:36:45 2019 From: barry at barrys-emacs.org (Barry Scott) Date: Wed, 24 Jul 2019 21:36:45 +0100 Subject: Proper shebang for python3 In-Reply-To: <20190722231358.GA83700@cskk.homeip.net> References: <20190722231358.GA83700@cskk.homeip.net> Message-ID: <3154FF95-BAEA-497A-AB8B-384E041A368A@barrys-emacs.org> > On 23 Jul 2019, at 00:13, Cameron Simpson wrote: > > Why _any_ modern system has anything other than /bin in the base install escapes me. In the distant past /sbin and a distinct /usr with its own bin had their values, but these days? Bah! On fedora its all in /usr these days with symlinks to the old locations. $ ls -l / | grep usr lrwxrwxrwx 1 root root 7 Feb 11 13:47 bin -> usr/bin/ lrwxrwxrwx 1 root root 7 Feb 11 13:47 lib -> usr/lib/ lrwxrwxrwx 1 root root 9 Feb 11 13:47 lib64 -> usr/lib64/ lrwxrwxrwx 1 root root 8 Feb 11 13:47 sbin -> usr/sbin/ drwxr-xr-x. 13 root root 4096 May 5 17:22 usr/ You can read about why here https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/ and also https://fedoraproject.org/wiki/Features/UsrMove Barry From cs at cskk.id.au Wed Jul 24 18:20:30 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 25 Jul 2019 08:20:30 +1000 Subject: Proper shebang for python3 In-Reply-To: <3154FF95-BAEA-497A-AB8B-384E041A368A@barrys-emacs.org> References: <3154FF95-BAEA-497A-AB8B-384E041A368A@barrys-emacs.org> Message-ID: <20190724222030.GA63158@cskk.homeip.net> On 24Jul2019 21:36, Barry Scott wrote: >> On 23 Jul 2019, at 00:13, Cameron Simpson wrote: >> Why _any_ modern system has anything other than /bin in the base >> install escapes me. In the distant past /sbin and a distinct /usr >> with its own bin had their values, but these days? Bah! > >On fedora its all in /usr these days with symlinks to the old >locations. > >$ ls -l / | grep usr >lrwxrwxrwx 1 root root 7 Feb 11 13:47 bin -> usr/bin/ >lrwxrwxrwx 1 root root 7 Feb 11 13:47 lib -> usr/lib/ >lrwxrwxrwx 1 root root 9 Feb 11 13:47 lib64 -> usr/lib64/ >lrwxrwxrwx 1 root root 8 Feb 11 13:47 sbin -> usr/sbin/ >drwxr-xr-x. 13 root root 4096 May 5 17:22 usr/ That is some progress, hooray. Then there's just sbin -> bin to go. They could merge lib and lib64 too if they embedded an architecture signature in library filenames. >You can read about why here >https://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge/ >and also https://fedoraproject.org/wiki/Features/UsrMove Thanks for these references. Cheers, Cameron Simpson From cs at cskk.id.au Wed Jul 24 18:31:17 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 25 Jul 2019 08:31:17 +1000 Subject: Hermetic environments In-Reply-To: References: Message-ID: <20190724223117.GA80263@cskk.homeip.net> On 24Jul2019 19:59, Eli the Bearded <*@eli.users.panix.com> wrote: >In comp.lang.python, DL Neil wrote: >> Is Python going 'the right way' with virtual environments? >... >> Am I 'getting away with it', perhaps because my work-pattern doesn't >> touch some 'gotcha' or show-stopper? >> >> Why, if so much of 'the rest of the world' is utilising "containers", >> both for mobility and for growth, is the Python eco-system following its >> own path? > >I'm going to speculate that even inside containers, some people will use >multiple virtual environments. It could be that the app and the >monitoring for that app are developed by different branches of the >company and have different requirements. > >But I think a lot of the use of virtual environments is in dev >environments where a developer wants to have multiple closed settings >for doing work. On the dev branch, newer versions of things can be >tested, but a production environment can be retained for hotfixes to >deployed code. > >Or because the different microservices being used are each at different >update levels and need their own environments. Yeah. In a recent former life we were maintaining some APIs with many releases (point releases every sprint, for those APIs changing that sprint). The customers could stick with older API revisions if they had special requirements (or simply lacked their own dev time to verify a successful forward version shift), so there were multiple historic versions in play in the field. >> Is there something about dev (and ops) using Python venvs which is a >> significant advantage over a language-independent (even better: an >> OpSys-independent) container? > >I'm not a big fan of language-dependent virtual environments because >they only capture the needs of a particular language. Very often code >works with things that are outside of that language, even if it is only >system libraries. The advantage of the language dependent venv is that it is self contained. You can update, say, the Python component of the project independently of some adjacent other language. This might all be contained within a larger environment which itself is snapshotted for release purposes. In my current life I'm working on a project with a python API and a JavaScript front end. A release involves building a clean versioned directory on the server machine; it contains a specific Python venv inside it; the upper layer is the encapsulation. Example: STAGING -> app/version2 app-version venv/.... webapp/javascript-here... ... app-version2 venv/.... webapp/javascript-here... ... I still want the venv because it encapsulates the Python arena's state of play. Cheers, Cameron Simpson From torriem at gmail.com Wed Jul 24 22:24:24 2019 From: torriem at gmail.com (Michael Torrie) Date: Wed, 24 Jul 2019 20:24:24 -0600 Subject: Proper shebang for python3 In-Reply-To: <20190724222030.GA63158@cskk.homeip.net> References: <3154FF95-BAEA-497A-AB8B-384E041A368A@barrys-emacs.org> <20190724222030.GA63158@cskk.homeip.net> Message-ID: On 7/24/19 4:20 PM, Cameron Simpson wrote: > That is some progress, hooray. Then there's just sbin -> bin to go. I suppose in the olden days sbin was for static binaries, usable in single user mode for recovering the system without the main drive mounted. In more recent times, binaries that are mostly applicable to the super user go there. I don't see why you would want to merge those. A normal user rarely has need of much in /sbin. Already /bin has way too much stuff in it (although I don't see any other way to practically do it without ridiculous PATHs searching all over the disk). Having said that, I note that on my CentOS 7 workstation, sbin seems to be in the path by default. So that negates my argument I suppose. Although I might have made that change myself. From cs at cskk.id.au Wed Jul 24 23:33:49 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 25 Jul 2019 13:33:49 +1000 Subject: Proper shebang for python3 In-Reply-To: References: Message-ID: <20190725033349.GA23255@cskk.homeip.net> On 24Jul2019 20:24, Michael Torrie wrote: >On 7/24/19 4:20 PM, Cameron Simpson wrote: >> That is some progress, hooray. Then there's just sbin -> bin to go. > >I suppose in the olden days sbin was for static binaries, usable in >single user mode for recovering the system without the main drive >mounted. Yep. Happy days. >In more recent times, binaries that are mostly applicable to >the super user go there. I don't see why you would want to merge those. > A normal user rarely has need of much in /sbin. I say unto to you "ifconfig". And, frankly, _any_ sbin command which can be meaningfully run as nonroot, particularly for reporting. I have always found this "oh its for root" distinction pretty vacuous, and outstandingly annoying when basic system querying stuff isn't in the default $PATH because of this. Maybe it is because I've been a sysadmin for many years, but most physical machines are personal machines these days anyway - we're our own sysadmins. > Already /bin has way>too much stuff in it (although I don't see any other way to practically >do it without ridiculous PATHs searching all over the disk). Like modules with many names, the number of things in /bin or /usr/bin is generally irrelevant. Nobody does an "ls" in there without expecting a fair amount of stuff - like imports, we invoke commands by name. Who _cares_ how many names there are? >Having said that, I note that on my CentOS 7 workstation, sbin seems to >be in the path by default. So that negates my argument I suppose. >Although I might have made that change myself. I have historically had to add it to my $PATH on most platforms. Cheers, Cameron Simpson From PythonList at DancesWithMice.info Wed Jul 24 23:45:54 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Thu, 25 Jul 2019 15:45:54 +1200 Subject: Hermetic environments In-Reply-To: <20190724223117.GA80263@cskk.homeip.net> References: <20190724223117.GA80263@cskk.homeip.net> Message-ID: On 25/07/19 10:31 AM, Cameron Simpson wrote: > On 24Jul2019 19:59, Eli the Bearded <*@eli.users.panix.com> wrote: >> In comp.lang.python, DL Neil? wrote: >>> Is Python going 'the right way' with virtual environments? >> ... >>> Am I 'getting away with it', perhaps because my work-pattern doesn't >>> touch some 'gotcha' or show-stopper? >>> >>> Why, if so much of 'the rest of the world' is utilising "containers", >>> both for mobility and for growth, is the Python eco-system following its >>> own path? >> >> I'm going to speculate that even inside containers, some people will use >> multiple virtual environments. It could be that the app and the >> monitoring for that app are developed by different branches of the >> company and have different requirements. >> >> But I think a lot of the use of virtual environments is in dev >> environments where a developer wants to have multiple closed settings >> for doing work. On the dev branch, newer versions of things can be >> tested, but a production environment can be retained for hotfixes to >> deployed code. >> >> Or because the different microservices being used are each at different >> update levels and need their own environments. > > Yeah. In a recent former life we were maintaining some APIs with many > releases (point releases every sprint, for those APIs changing that > sprint). The customers could stick with older API revisions if they had > special requirements (or simply lacked their own dev time to verify a > successful forward version shift), so there were multiple historic > versions in play in the field. > >>> Is there something about dev (and ops) using Python venvs which is a >>> significant advantage over a language-independent (even better: an >>> OpSys-independent) container? >> >> I'm not a big fan of language-dependent virtual environments because >> they only capture the needs of a particular language. Very often code >> works with things that are outside of that language, even if it is only >> system libraries. > > The advantage of the language dependent venv is that it is self > contained. You can update, say, the Python component of the project > independently of some adjacent other language. This might all be > contained within a larger environment which itself is snapshotted for > release purposes. > > In my current life I'm working on a project with a python API and a > JavaScript front end. A release involves building a clean versioned > directory on the server machine; it contains a specific Python venv > inside it; the upper layer is the encapsulation. Example: > > ?STAGING -> app/version2 > ?app-version > ?? venv/.... > ?? webapp/javascript-here... > ?? ... > ?app-version2 > ?? venv/.... > ?? webapp/javascript-here... > ?? ... > > I still want the venv because it encapsulates the Python arena's state > of play. Do you use a VCS, eg git or Subversion? Thus, have you disciplined yourself to check-in work, and subsequently NOT to work on your (old) copy, but to check-out a fresh copy? Similarly, rather than adding a second environment or updates to an existing (prod) VM, it is a 'discipline' to make a copy of the appropriate VM and work with the (new) copy! (either upgrading some component of the source, the Python eco-system, or the OpSys) Copying/backing-up a VM is a rapid operation. So, why would you prefer to set up a second and separate py-env within an existing environment? (and lose the "hermetic seal" - face the version collisions/combinations both philosophies seek to avoid) NB a problem I experienced yesterday was that VMs are differentiated by versionNR and date - but in 'client-language' the date was not when the VM was created, but when she last used it. Users! (nothing is perfect - and yes I found it by 'relative addressing' the dates) -- Regards =dn From cs at cskk.id.au Thu Jul 25 00:43:37 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 25 Jul 2019 14:43:37 +1000 Subject: Hermetic environments In-Reply-To: References: Message-ID: <20190725044337.GA72598@cskk.homeip.net> On 25Jul2019 15:45, DL Neil wrote: >>In my current life I'm working on a project with a python API and a >>JavaScript front end. A release involves building a clean versioned >>directory on the server machine; it contains a specific Python venv >>inside it; the upper layer is the encapsulation. Example: >> >> ?STAGING -> app/version2 >> ?app-version >> ?? venv/.... >> ?? webapp/javascript-here... >> ?? ... >> ?app-version2 >> ?? venv/.... >> ?? webapp/javascript-here... >> ?? ... >> >>I still want the venv because it encapsulates the Python arena's >>state of play. > > >Do you use a VCS, eg git or Subversion? Thus, have you disciplined >yourself to check-in work, and subsequently NOT to work on your (old) >copy, but to check-out a fresh copy? Well of course. Mercurial or git. That's what makes the deploy process fairly easy, one deploys from a revision with a release tag. >Similarly, rather than adding a second environment or updates to an >existing (prod) VM, it is a 'discipline' to make a copy of the >appropriate VM and work with the (new) copy! (either upgrading some >component of the source, the Python eco-system, or the OpSys) > >Copying/backing-up a VM is a rapid operation. So, why would you prefer >to set up a second and separate py-env within an existing environment? 1: it is smaller and much lower overhead. How many _live_ VMs are you keeping around? 2: no VMs in play at this end. Cheers, Cameron Simpson From PythonList at DancesWithMice.info Thu Jul 25 01:43:54 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Thu, 25 Jul 2019 17:43:54 +1200 Subject: Hermetic environments In-Reply-To: <20190725044337.GA72598@cskk.homeip.net> References: <20190725044337.GA72598@cskk.homeip.net> Message-ID: <7d9b6c87-1ee8-a10b-4402-d8cb736667f4@DancesWithMice.info> On 25/07/19 4:43 PM, Cameron Simpson wrote: > On 25Jul2019 15:45, DL Neil wrote: >>> In my current life I'm working on a project with a python API and a >>> JavaScript front end. A release involves building a clean versioned >>> directory on the server machine; it contains a specific Python venv >>> inside it; the upper layer is the encapsulation. Example: >>> >>> ?STAGING -> app/version2 >>> ?app-version >>> ?? venv/.... >>> ?? webapp/javascript-here... >>> ?? ... >>> ?app-version2 >>> ?? venv/.... >>> ?? webapp/javascript-here... >>> ?? ... >>> >>> I still want the venv because it encapsulates the Python arena's >>> state of play. >> >> >> Do you use a VCS, eg git or Subversion? Thus, have you disciplined >> yourself to check-in work, and subsequently NOT to work on your (old) >> copy, but to check-out a fresh copy? > > Well of course. Mercurial or git. That's what makes the deploy process > fairly easy, one deploys from a revision with a release tag. > >> Similarly, rather than adding a second environment or updates to an >> existing (prod) VM, it is a 'discipline' to make a copy of the >> appropriate VM and work with the (new) copy! (either upgrading some >> component of the source, the Python eco-system, or the OpSys) >> >> Copying/backing-up a VM is a rapid operation. So, why would you prefer >> to set up a second and separate py-env within an existing environment? > > 1: it is smaller and much lower overhead. How many _live_ VMs are you > keeping around? A pyenv is significantly smaller, but if we're including non-Python components in some system, 'size' increases accordingly. However, the 'saving' in either copy-time or storage-space is not significant. The important point here, is that the amount of time-taken in copying the env, might be considerably less than installing and verifying a new version of Python, pip-ing, upgrading non-Py components; and thereafter the copy and upgrade tasks are likely insignificant within the next sprint's-worth of dev-effort! A VM provides the "hermetic" insulation under discussion, with no more/less effort than any/either of the v-env-s. (and includes the same advantages for the wider environment of the application) > 2: no VMs in play at this end. Dozens - haven't counted. (until you "made you look!"...) Every production system, for every client (don't currently have any new clients who are only in 'dev'). NB Am assuming by "keeping around" you are asking about "prod" versions, ie actually running systems. However, as every 'prod' is backed by (usually) two others: 'dev' and (acceptance/user) 'test' (some of the latter are also installed in client's premises or on their networks/cloud). Then, there are some extra 'dev's which probably feature a PoC experiment or the like, completed, but yet to make it out of the client's "backlog". OTOH if the client has not 'returned' for some time, it's not as if their VMs are sitting 'spooled-up' on one of my machines! A quick 'find' shows quantities > three-digits. However, there are likely duplicates because of an active task to migrate between disks/archiving old projects 'paper work' - a task which I am assiduously finishing... Yeah right! Now you've worried me. Am I keeping track? Should I be putting VMs (which in-turn likely contain a git client and tree), into some central git tree? Do I need to take a tea-break??? NB as mentioned earlier, VMs include dates and versionNRs in their name/label (cf DNS, etc) and such appears on docs, even invoices. Oh, so that reminds me that I have an additional VM for (most of) my clients - which holds business docs, planning docs, maybe a SCRUM/Kanban board, mind-maps, etc, etc. Also, if I'm supervising and in-house or external dev-team, we might have 'minor-version' VMs (which we refer to as "snapshots", even though this means something else in some VM-speak) of 'where-we-are-up-to' part-projects (the equivalent of me taking a whole branch from vcs for code-review purposes) - these don't tend to be 'kept' long. NB I don't work exclusively in Python, eg DB and web work, so my definition of "VM" is a lot wider than Python venv-s! Also, I'm not a full-time programmer. In the Python world, back-when, while starting to embrace Py3 I became nervous of the idea that I was developing in a separate Python version to that which ran aspects of my (Fedora-Linux) machines. VirtualBox offered an escape from such; and one VM led to another... As the saying goes: what were once habits, became vices! BTW this is still a question: why the (venv) Python-only way cf 'other ideas for containerisation'; and is not intended as an exercise in self-justification or one-upmanship! -- Regards =dn From 2QdxY4RzWzUUiLuE at potatochowder.com Wed Jul 24 22:57:57 2019 From: 2QdxY4RzWzUUiLuE at potatochowder.com (Dan Sommers) Date: Wed, 24 Jul 2019 22:57:57 -0400 Subject: Proper shebang for python3 In-Reply-To: References: <3154FF95-BAEA-497A-AB8B-384E041A368A@barrys-emacs.org> <20190724222030.GA63158@cskk.homeip.net> Message-ID: <3e2ab6c4-c43f-bcdb-1c5a-9b8f44185315@potatochowder.com> On 7/24/19 10:24 PM, Michael Torrie wrote: > ... In more recent times, binaries that are mostly applicable to the > super user go there. I don't see why you would want to merge those. > A normal user rarely has need of much in /sbin. Already /bin has way > too much stuff in it (although I don't see any other way to > practically do it without ridiculous PATHs searching all over the > disk). On ancient file systems (SysV?), ISTR something like 1400 files in a directory being some sort of limit, or perhaps a major performance issue. Separate directories (including /usr/X11/bin) mitigated that, too. > Having said that, I note that on my CentOS 7 workstation, sbin seems > to be in the path by default. So that negates my argument I suppose. > Although I might have made that change myself. My .shrc file has all sorts of leftovers from the old days, but my current Linux PATH is just $HOME/bin, $HOME/local/bin, and /usr/bin. Get Off My Lawn, Dan From python at mrabarnett.plus.com Thu Jul 25 09:28:52 2019 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 25 Jul 2019 14:28:52 +0100 Subject: Proper shebang for python3 In-Reply-To: <3e2ab6c4-c43f-bcdb-1c5a-9b8f44185315@potatochowder.com> References: <3154FF95-BAEA-497A-AB8B-384E041A368A@barrys-emacs.org> <20190724222030.GA63158@cskk.homeip.net> <3e2ab6c4-c43f-bcdb-1c5a-9b8f44185315@potatochowder.com> Message-ID: On 2019-07-25 03:57, Dan Sommers wrote: > On 7/24/19 10:24 PM, Michael Torrie wrote: > > > ... In more recent times, binaries that are mostly applicable to the > > super user go there. I don't see why you would want to merge those. > > A normal user rarely has need of much in /sbin. Already /bin has way > > too much stuff in it (although I don't see any other way to > > practically do it without ridiculous PATHs searching all over the > > disk). > > On ancient file systems (SysV?), ISTR something like 1400 files in a > directory being some sort of limit, or perhaps a major performance > issue. Separate directories (including /usr/X11/bin) mitigated that, > too. > [snip] At one time I used an MS-DOS application suite whose database component stored predefined queries in individual files. There were _many_ predefined queries, so _many_ small files. This took up a lot of disk space, and running a predefined query was slow because of the need to search for the file (the FAT layout used a linked list). I put all of the predefined queries into a single file and called an external program to look up the query on demand, copy it into a file, and then performed the query. That saved a _lot_ of disk space, and, despite the overhead of calling an external program, was _much_ faster! From kirillbalunov at gmail.com Thu Jul 25 11:27:59 2019 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Thu, 25 Jul 2019 18:27:59 +0300 Subject: Difference between os.path.isdir and Path.is_dir Message-ID: Hi all! It is expected that: ``` >>> import os >>> from pathlib import Path >>> dummy = " " # or "" or " " >>> os.path.isdir(dummy) False >>> Path(dummy).is_dir() True ``` or was it overlooked? with kind regards, -gdg From rosuav at gmail.com Thu Jul 25 12:14:46 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Jul 2019 02:14:46 +1000 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: On Fri, Jul 26, 2019 at 1:30 AM Kirill Balunov wrote: > > Hi all! It is expected that: > ``` > >>> import os > >>> from pathlib import Path > >>> dummy = " " # or "" or " " > >>> os.path.isdir(dummy) > False > >>> Path(dummy).is_dir() > True > ``` > > or was it overlooked? > Was not aware of that. A bit of digging shows that asking to convert an empty path to an absolute one will give you the name of the current directory (whether you use os.path or pathlib.Path), but os.path.isdir starts by seeing if it can os.stat the given path, and if that fails, it returns False (because if something doesn't exist, it can't be a directory). As a workaround, you could simply call os.path.isdir(os.path.abspath(x)) to get consistent results. I'm not sure if this is considered an important enough bug to actually fix, or if it's merely a curiosity that occurs when you trigger undocumented behaviour. ChrisA From eryksun at gmail.com Thu Jul 25 13:26:55 2019 From: eryksun at gmail.com (eryk sun) Date: Thu, 25 Jul 2019 12:26:55 -0500 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: On 7/25/19, Kirill Balunov wrote: > >>>> import os >>>> from pathlib import Path >>>> dummy = " " # or "" or " " >>>> os.path.isdir(dummy) > False >>>> Path(dummy).is_dir() > True I can't reproduce the above result in either Linux or Windows. The results should only be different for an empty path string, since Path('') is the same as Path('.'). The results should be the same for Path(" "), depending on whether a directory named " " exists (normally not allowed in Windows, but Linux allows it). From rosuav at gmail.com Thu Jul 25 13:43:51 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Jul 2019 03:43:51 +1000 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: On Fri, Jul 26, 2019 at 3:28 AM eryk sun wrote: > > On 7/25/19, Kirill Balunov wrote: > > > >>>> import os > >>>> from pathlib import Path > >>>> dummy = " " # or "" or " " > >>>> os.path.isdir(dummy) > > False > >>>> Path(dummy).is_dir() > > True > > I can't reproduce the above result in either Linux or Windows. The > results should only be different for an empty path string, since > Path('') is the same as Path('.'). The results should be the same for > Path(" "), depending on whether a directory named " " exists (normally > not allowed in Windows, but Linux allows it). Try an empty string, no spaces. To pathlib.Path, that means the current directory. To os.path.abspath, that means the current directory. To os.stat, it doesn't exist. ChrisA From eryksun at gmail.com Thu Jul 25 13:54:38 2019 From: eryksun at gmail.com (eryk sun) Date: Thu, 25 Jul 2019 12:54:38 -0500 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: On 7/25/19, Chris Angelico wrote: > On Fri, Jul 26, 2019 at 3:28 AM eryk sun wrote: >> >> On 7/25/19, Kirill Balunov wrote: >> > >> >>>> import os >> >>>> from pathlib import Path >> >>>> dummy = " " # or "" or " " >> >>>> os.path.isdir(dummy) >> > False >> >>>> Path(dummy).is_dir() >> > True >> >> I can't reproduce the above result in either Linux or Windows. The >> results should only be different for an empty path string, since >> Path('') is the same as Path('.'). The results should be the same for >> Path(" "), depending on whether a directory named " " exists (normally >> not allowed in Windows, but Linux allows it). > > Try an empty string, no spaces. To pathlib.Path, that means the > current directory. To os.path.abspath, that means the current > directory. To os.stat, it doesn't exist. That's what I said. But the OP shows os.path.isdir(" ") == False and Path(" ").is_dir() == True, which is what I cannot reproduce and really should not be able to reproduce, unless there's a bug somewhere. From rosuav at gmail.com Thu Jul 25 13:56:24 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Jul 2019 03:56:24 +1000 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: On Fri, Jul 26, 2019 at 3:54 AM eryk sun wrote: > > On 7/25/19, Chris Angelico wrote: > > On Fri, Jul 26, 2019 at 3:28 AM eryk sun wrote: > >> > >> On 7/25/19, Kirill Balunov wrote: > >> > > >> >>>> import os > >> >>>> from pathlib import Path > >> >>>> dummy = " " # or "" or " " > >> >>>> os.path.isdir(dummy) > >> > False > >> >>>> Path(dummy).is_dir() > >> > True > >> > >> I can't reproduce the above result in either Linux or Windows. The > >> results should only be different for an empty path string, since > >> Path('') is the same as Path('.'). The results should be the same for > >> Path(" "), depending on whether a directory named " " exists (normally > >> not allowed in Windows, but Linux allows it). > > > > Try an empty string, no spaces. To pathlib.Path, that means the > > current directory. To os.path.abspath, that means the current > > directory. To os.stat, it doesn't exist. > > That's what I said. But the OP shows os.path.isdir(" ") == False and > Path(" ").is_dir() == True, which is what I cannot reproduce and > really should not be able to reproduce, unless there's a bug > somewhere. Yeah but WHY is it different for an empty string? I can well imagine that some OS/FS combinations will happily strip spaces, thus devolving the " " case to the "" one. ChrisA From eryksun at gmail.com Thu Jul 25 14:40:01 2019 From: eryksun at gmail.com (eryk sun) Date: Thu, 25 Jul 2019 13:40:01 -0500 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: On 7/25/19, Chris Angelico wrote: > On Fri, Jul 26, 2019 at 3:54 AM eryk sun wrote: > >> That's what I said. But the OP shows os.path.isdir(" ") == False and >> Path(" ").is_dir() == True, which is what I cannot reproduce and >> really should not be able to reproduce, unless there's a bug >> somewhere. > > Yeah but WHY is it different for an empty string? Path.__fspath__ returns str(self), and Path.__str__ returns "." if the path is empty, i.e. there's no drive, root, or parts. I assume this is related to os.path.normpath("") == ".", which is related to os.path.normpath("spam/..") == ".". However, an empty path string isn't consistently handled as the current directory throughout the os and os.path modules. > I can well imagine that some OS/FS combinations will happily strip > spaces, thus devolving the " " case to the "" one. Windows trims trailing spaces and dots from the final component of a path, unless we use a non-normalized \\?\ path. From mxfm at protonmail.com Thu Jul 25 13:36:47 2019 From: mxfm at protonmail.com (Maksim Fomin) Date: Thu, 25 Jul 2019 17:36:47 +0000 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: <7BLicglse8OpJ1hTmY5he7Lb2ayyGZ3dqcn52yJ4DjFZ621OOQEuENHJvtSymC-lTAPlSZd8aX78iktIwdXctBQeplWPSdwGnxXitdtpGi8=@protonmail.com> ??????? Original Message ??????? On Thursday, July 25, 2019 3:27 PM, Kirill Balunov wrote: > Hi all! It is expected that: > > >>> import os > >>> from pathlib import Path > >>> dummy = " " # or "" or " " > >>> os.path.isdir(dummy) > > False > >>> Path(dummy).is_dir() > > True > > > or was it overlooked? > > with kind regards, > -gdg > > ------------------------------------------------- > > https://mail.python.org/mailman/listinfo/python-list I also cannot reproduce it on linux (python 3.7.3). Which version do you use? From kirillbalunov at gmail.com Thu Jul 25 15:02:27 2019 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Thu, 25 Jul 2019 22:02:27 +0300 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: ??, 25 ???. 2019 ?. ? 20:28, eryk sun : > On 7/25/19, Kirill Balunov wrote: > > > >>>> import os > >>>> from pathlib import Path > >>>> dummy = " " # or "" or " " > >>>> os.path.isdir(dummy) > > False > >>>> Path(dummy).is_dir() > > True > > I can't reproduce the above result in either Linux or Windows. The > results should only be different for an empty path string, since > Path('') is the same as Path('.'). The results should be the same for > Path(" "), depending on whether a directory named " " exists (normally > not allowed in Windows, but Linux allows it). > > Heh, something fishy is going on. I also can't reproduce that behavior (with " " and " ") at home comp on Windows 10 Python 3.7.4. Tomorrow I'll check again at work and let you know. I apologize in advance. The main problem arose ???? with an empty line, but then I checked with " " and apparently made a mistake somewhere. with kind regards, -gdg From kirillbalunov at gmail.com Thu Jul 25 15:52:12 2019 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Thu, 25 Jul 2019 22:52:12 +0300 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: ??, 25 ???. 2019 ?. ? 19:16, Chris Angelico : > On Fri, Jul 26, 2019 at 1:30 AM Kirill Balunov > wrote: > > > > Hi all! It is expected that: > > ``` > > >>> import os > > >>> from pathlib import Path > > >>> dummy = " " # or "" or " " > > >>> os.path.isdir(dummy) > > False > > >>> Path(dummy).is_dir() > > True > > ``` > > > > or was it overlooked? > > > [....] > I'm not sure if this is considered an important enough bug to actually > fix, or > if it's merely a curiosity that occurs when you trigger undocumented > behaviour. > > No, it's not just because of curiosity. I will try to tell the background, and maybe I went the wrong way initially. There is a very cool project https://github.com/3b1b/manim, it allows you to visualize math (I don?t know how to describe it better you can see some examples here https://www.3blue1brown.com) and it works lovely on Linux. For some reason, I need to use it on Windows. Problems began to arise when I tried my scripts were some latex was included in the animation. So I installed TexLive, but it didn't produce any output :) In `manim` it is invoked through a system call https://github.com/3b1b/manim/blob/master/manimlib/utils/tex_file_writing.py#L40 like this: $ latex -interaction=batchmode -halt-on-error -output-directory=... input.tex > /dev/null For some reason TexLive does not understand Windows relative paths of this form -output-directory =".\Output" and ".\Input\file.tex", but it understands the absolute paths in Windows form like "C:\path\to\the\input\file.tex". I read that Windows allows also to provide paths in the usual form "./Input/file.tex" (maybe I'm wrong with my understanding what does it mean on Windows), I've tested and this worked. But the problem is that Python always inserts '\' as path separators on Windows and there is no control to set it up. I decided to rewrite all this stuff with the help of `pathlib` module and to use `Path` and `.as_posix` method everywhere. Paths are set here https://github.com/3b1b/manim/blob/c7e6d9d4742ec47098bd86a9bbb4212cc637206b/manimlib/constants.py#L10 and the author uses `MEDIA_DIR = ""` as a default unset value, and then checks `if not os.path.isdir(MEDIA_DIR)`. The documentation states that `os.path.isdir(...)` is equivalent to `Path(...).is_dir()` but it is not true. So I wrote here. with kind regards, -gdg From rosuav at gmail.com Thu Jul 25 15:57:02 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Jul 2019 05:57:02 +1000 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: On Fri, Jul 26, 2019 at 5:52 AM Kirill Balunov wrote: > > > > ??, 25 ???. 2019 ?. ? 19:16, Chris Angelico : >> >> On Fri, Jul 26, 2019 at 1:30 AM Kirill Balunov wrote: >> > >> > Hi all! It is expected that: >> > ``` >> > >>> import os >> > >>> from pathlib import Path >> > >>> dummy = " " # or "" or " " >> > >>> os.path.isdir(dummy) >> > False >> > >>> Path(dummy).is_dir() >> > True >> > ``` >> > >> > or was it overlooked? >> > >> [....] >> I'm not sure if this is considered an important enough bug to actually fix, or >> if it's merely a curiosity that occurs when you trigger undocumented >> behaviour. >> > > No, it's not just because of curiosity. I will try to tell the background, and maybe I went the wrong way initially. There is a very cool project https://github.com/3b1b/manim, it allows you to visualize math (I don?t know how to describe it better you can see some examples here https://www.3blue1brown.com) and it works lovely on Linux. For some reason, I need to use it on Windows. Problems began to arise when I tried my scripts were some latex was included in the animation. > Ah, I love 3b1b! Great videos. I toyed with manim once (wanting to create new Fourier visualizations), but the open source parts weren't enough for what I was trying to do. Very cool though. > So I installed TexLive, but it didn't produce any output :) In `manim` it is invoked through a system call https://github.com/3b1b/manim/blob/master/manimlib/utils/tex_file_writing.py#L40 like this: > > $ latex -interaction=batchmode -halt-on-error -output-directory=... input.tex > /dev/null > > For some reason TexLive does not understand Windows relative paths of this form -output-directory =".\Output" and ".\Input\file.tex", but it understands the absolute paths in Windows form like "C:\path\to\the\input\file.tex". I read that Windows allows also to provide paths in the usual form "./Input/file.tex" (maybe I'm wrong with my understanding what does it mean on Windows), > It means the same thing as your other relative path. The two types of slash should basically be interchangeable. > I've tested and this worked. But the problem is that Python always inserts '\' as path separators on Windows and there is no control to set it up. I decided to rewrite all this stuff with the help of `pathlib` module and to use `Path` and `.as_posix` method everywhere. Paths are set here https://github.com/3b1b/manim/blob/c7e6d9d4742ec47098bd86a9bbb4212cc637206b/manimlib/constants.py#L10 and the author uses `MEDIA_DIR = ""` as a default unset value, and then checks `if not os.path.isdir(MEDIA_DIR)`. The documentation states that `os.path.isdir(...)` is equivalent to `Path(...).is_dir()` but it is not true. So I wrote here. > Gotcha. It'd probably be safe to change it to "." instead; that way, it's an actual valid directory name and can be combined normally with other components. ChrisA From kirillbalunov at gmail.com Thu Jul 25 16:13:14 2019 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Thu, 25 Jul 2019 23:13:14 +0300 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: ??, 25 ???. 2019 ?. ? 22:58, Chris Angelico : > On Fri, Jul 26, 2019 at 5:52 AM Kirill Balunov > wrote: > [...] > > No, it's not just because of curiosity. I will try to tell the > background, and maybe I went the wrong way initially. There is a very cool > project https://github.com/3b1b/manim, it allows you to visualize math (I > don?t know how to describe it better you can see some examples here > https://www.3blue1brown.com) and it works lovely on Linux. For some > reason, I need to use it on Windows. Problems began to arise when I tried > my scripts were some latex was included in the animation. > > > > Ah, I love 3b1b! Great videos. I toyed with manim once (wanting to > create new Fourier visualizations), but the open source parts weren't > enough for what I was trying to do. Very cool though. > > If you can tell, what parts did you miss? with kind regards, -gdg From rosuav at gmail.com Thu Jul 25 16:21:47 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Jul 2019 06:21:47 +1000 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: On Fri, Jul 26, 2019 at 6:13 AM Kirill Balunov wrote: > > > > ??, 25 ???. 2019 ?. ? 22:58, Chris Angelico : >> >> On Fri, Jul 26, 2019 at 5:52 AM Kirill Balunov wrote: >> [...] >> > No, it's not just because of curiosity. I will try to tell the background, and maybe I went the wrong way initially. There is a very cool project https://github.com/3b1b/manim, it allows you to visualize math (I don?t know how to describe it better you can see some examples here https://www.3blue1brown.com) and it works lovely on Linux. For some reason, I need to use it on Windows. Problems began to arise when I tried my scripts were some latex was included in the animation. >> > >> >> Ah, I love 3b1b! Great videos. I toyed with manim once (wanting to >> create new Fourier visualizations), but the open source parts weren't >> enough for what I was trying to do. Very cool though. >> > > If you can tell, what parts did you miss? Basically everything to do with Fourier transforms. I wanted to hand the program a new SVG file and see it animate it, but (a) I didn't understand the code very well, and (b) a lot of the code isn't part of the freely licensed part (it's all in the same repo, but some is MIT-licensed and isn't), which means it's not documented for public use and isn't designed to be broadly usable. ChrisA From beliavsky at aol.com Thu Jul 25 18:36:29 2019 From: beliavsky at aol.com (beliavsky at aol.com) Date: Thu, 25 Jul 2019 15:36:29 -0700 (PDT) Subject: book on modern Python Message-ID: <0c13c281-e2bb-4150-a096-7d307ff9223e@googlegroups.com> I did some basic programming in Python 2 and now use Python 3. All I know about Python 3 is that print is a function requiring parentheses. What is a good book on modern Python that covers topics such as type annotations? I know of Fluent Python: Clear, Concise, and Effective Programming by Luciano Ramalho (2015) Introducing Python: Modern Computing in Simple Packages by Bill Lubanovic (2014) but wonder if the language has evolved significantly since they were published. I see that a 2nd edition of "Introducing Python" is due this year. From cs at cskk.id.au Thu Jul 25 19:19:01 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 26 Jul 2019 09:19:01 +1000 Subject: Proper shebang for python3 In-Reply-To: <3e2ab6c4-c43f-bcdb-1c5a-9b8f44185315@potatochowder.com> References: <3e2ab6c4-c43f-bcdb-1c5a-9b8f44185315@potatochowder.com> Message-ID: <20190725231901.GA97399@cskk.homeip.net> On 24Jul2019 22:57, Dan Sommers <2QdxY4RzWzUUiLuE at potatochowder.com> wrote: >On 7/24/19 10:24 PM, Michael Torrie wrote: >> ... In more recent times, binaries that are mostly applicable to the >> super user go there. I don't see why you would want to merge those. >> A normal user rarely has need of much in /sbin. Already /bin has way >> too much stuff in it (although I don't see any other way to >> practically do it without ridiculous PATHs searching all over the >> disk). > >On ancient file systems (SysV?), ISTR something like 1400 files in a >directory being some sort of limit, or perhaps a major performance >issue. Separate directories (including /usr/X11/bin) mitigated that, >too. I don't recall there being any specific hard limit though. What does happen in decent modern filesystems is that directories have a hash table in them for lookups, making a file name access constant time instead of a linear search. Also, shells like zsh keep a hash table of the available commands to speed invocation, avoiding a linear $PATH search. I don't recall stuff like /usr/X11/bin being aimed at performance (but I would not necessarily know); it has always seemed more like partitioning out various software groups - in this case the X11 Window System stuff. If nothing else, this makes separate development easier. >> Having said that, I note that on my CentOS 7 workstation, sbin seems >> to be in the path by default. So that negates my argument I suppose. >> Although I might have made that change myself. > >My .shrc file has all sorts of leftovers from the old days, but my >current Linux PATH is just $HOME/bin, $HOME/local/bin, and /usr/bin. Hmm. 21 componetent in my $PATH here :-) Cheers, Cameron Simpson From cs at cskk.id.au Thu Jul 25 19:25:37 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 26 Jul 2019 09:25:37 +1000 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: <20190725232537.GA30776@cskk.homeip.net> On 26Jul2019 03:43, Chris Angelico wrote: >On Fri, Jul 26, 2019 at 3:28 AM eryk sun wrote: >> On 7/25/19, Kirill Balunov wrote: >> >>>> import os >> >>>> from pathlib import Path >> >>>> dummy = " " # or "" or " " >> >>>> os.path.isdir(dummy) >> > False >> >>>> Path(dummy).is_dir() >> > True >> >> I can't reproduce the above result in either Linux or Windows. The >> results should only be different for an empty path string, since >> Path('') is the same as Path('.'). The results should be the same for >> Path(" "), depending on whether a directory named " " exists (normally >> not allowed in Windows, but Linux allows it). > >Try an empty string, no spaces. To pathlib.Path, that means the >current directory. To os.path.abspath, that means the current >directory. To os.stat, it doesn't exist. And for some context, on older UNIXen "" did stat successfully. Cheers, Cameron Simpson From cs at cskk.id.au Thu Jul 25 19:28:57 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 26 Jul 2019 09:28:57 +1000 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: <20190725232857.GA35178@cskk.homeip.net> On 25Jul2019 13:40, eryk sun wrote: >Windows trims trailing spaces and dots from the final component of a >path, unless we use a non-normalized \\?\ path. Hoo! Well, that explains some extremely weird NAS behaviour I encountered the other month with some paths ending in spaces. (My machine and I think also the NAS were UNIX/Linux at each end, but speaking over SMB.) Guessing that the NAS was doing only a half hearted implementation of the trailing spaces semantics. Interesting:-) Cheers, Cameron Simpson From spawnaga at gmail.com Thu Jul 25 21:26:16 2019 From: spawnaga at gmail.com (spawnaga at gmail.com) Date: Thu, 25 Jul 2019 18:26:16 -0700 (PDT) Subject: Can not execute options trade Message-ID: The placeOrder command does not send the order for options trades. I tried to submit the order for stocks and it went through but not options # -*- coding: utf-8 -*- """ Created on Wed Jul 24 21:58:32 2019 @author: Alex Oraibi """ import time from time import localtime, strftime from ib.ext.Contract import Contract from ib.ext.Order import Order from ib.opt import ibConnection, message import gspread from oauth2client.service_account import ServiceAccountCredentials def Google() : json_key = 'G:\IBridgeby\spreadsheet.json' scope = ['https://spreadsheets.google.com/feeds', 'https://www.googleapis.com/auth/drive'] credentials = ServiceAccountCredentials.from_json_keyfile_name(json_key, scope) gc = gspread.authorize(credentials) wks = gc.open("Test").sheet1 return (wks) def error_handler(msg): """Handles the capturing of error messages""" print("Server Error: %s" % msg) def reply_handler(msg): """Handles of server replies""" print("Server Response: %s, %s" % (msg.typeName, msg)) def makeStkContract(a,c): newStkContract = Contract() newStkContract.m_symbol = a newStkContract.m_secType = "OPT" newStkContract.m_strike = c newStkContract.m_exchange = "SMART" newStkContract.m_currency = "USD" return newStkContract def makeOptContract(a, b, e, c): newOptContract = Contract() newOptContract.m_symbol = a newOptContract.m_secType = "OPT" newOptContract.m_right = e newOptContract.m_expiry = b newOptContract.m_strike = float(c) newOptContract.m_exchange = "SMART" newOptContract.m_currency = "USD" # newOptContract.m_localSymbol = '' # newOptContract.m_primaryExch = '' return newOptContract def makeOptOrder(action, orderID, f): newOptOrder = Order() newOptOrder.m_orderId = orderID newOptOrder.m_permid = 0 newOptOrder.m_action = 'BUY' newOptOrder.m_lmtPrice = f newOptOrder.m_auxPrice = 0 newOptOrder.m_tif = 'DAY' newOptOrder.m_transmit = False newOptOrder.m_orderType = 'LMT' newOptOrder.m_totalQuantity = 1 newOptOrder.m_multiplier = "100" return newOptOrder if __name__ == "__main__": tws_conn = ibConnection("127.0.0.1", port=749) tws_conn.connect() order_id = 1000 Exec_time = strftime("%H", localtime()) tickID = 36 while Exec_time != '13': Exec_time = strftime("%H", localtime()) if Exec_time == '13': break else: hup = Google() tic = hup.cell(2,15).value stri = hup.cell(2,4).value exp = hup.cell(2,3).value rig = hup.cell(2,13).value lim = hup.cell(2,6).value if tic == '#VALUE!' : tic ='' stkContract = makeStkContract(tic,stri) optContract = makeOptContract(tic, exp, rig, stri) print(stkContract, optContract) tws_conn.reqMktData(1, stkContract, "", "") tws_conn.reqMktData(tickID, optContract, "", "") optOrder = makeOptOrder("BUY", order_id, lim) print(tws_conn.reqMktData(tic, optContract, "", True)) tws_conn.placeOrder(order_id, optContract, optOrder) order_id += 1 time.sleep(10) continue tws_conn.disconnect() I am loading trading suggestions from an online excel sheet and submit the order to TWS. I have successfully loaded the information from my google sheet and put the code to order the security but the execution part is not going through. I think my makeOptContract() is the problem that is holding me back from executing my order. Please provide some suggestions From dennistobias at gmail.com Thu Jul 25 22:44:03 2019 From: dennistobias at gmail.com (DT) Date: Thu, 25 Jul 2019 19:44:03 -0700 (PDT) Subject: Why am I getting duplicate values in my output? Message-ID: With the following code: import csv class AccessPoint(object): def __init__(self, name, ipv4, model, location): self.name = name self.ipv4 = ipv4 self.model = model self.location = location print('{0} born'.format(self.name)) def __del__(self): print('{0} died'.format(self.name)) def main(): ap_list = [] ap_dict = {} with open('C:\\inventory.csv', 'r') as csvfile: csv_reader = csv.DictReader(csvfile) for row in csv_reader: ap = AccessPoint(row['Host Name'], row['Static IP Address'], row['Device Model'], row['Host Name'][0:5]) ap_dict['ap_name'] = ap.name ap_dict['ap_ipv4'] = ap.ipv4 ap_dict['ap_model'] = ap.model ap_dict['ap_location'] = ap.location ap_list.append(ap_dict) print(ap_list) if __name__ == '__main__': main() When I execute print(ap_list) I get the very last row in the CSV repeated once for every row in the file. I would expect to see a list of ap_dicts, but I do not. When I add print(ap_dict) directly after the ap_list.append(ap_dict) inside the loop I see the correct values. Why is it that the print(ap_list) that happens outside of the for loop is only printing the last item on repeat? Here is some example output data: AP1 born AP2 born AP1 died AP3 born AP2 died [{'ap_name': 'AP3', 'ap_ipv4': '192.168.1.1', 'ap_model': 'Linksys', 'ap_location': 'HQ'}, {'ap_name': 'AP3', 'ap_ipv4': '192.168.1.1', 'ap_model': 'Linksys', 'ap_location': 'HQ'}, {'ap_name': 'AP3', 'ap_ipv4': '192.168.1.1', 'ap_model': 'Linksys', 'ap_location': 'HQ'} AP3 died Thanks in advance. From rosuav at gmail.com Thu Jul 25 22:57:11 2019 From: rosuav at gmail.com (Chris Angelico) Date: Fri, 26 Jul 2019 12:57:11 +1000 Subject: Why am I getting duplicate values in my output? In-Reply-To: References: Message-ID: On Fri, Jul 26, 2019 at 12:46 PM DT wrote: > def main(): > ap_list = [] > ap_dict = {} > > for row in csv_reader: > ap_dict['ap_name'] = ap.name > ap_dict['ap_ipv4'] = ap.ipv4 > ap_dict['ap_model'] = ap.model > ap_dict['ap_location'] = ap.location > > ap_list.append(ap_dict) > > print(ap_list) > > > When I execute print(ap_list) I get the very last row in the CSV repeated once for every row in the file. I would expect to see a list of ap_dicts, but I do not. When I add print(ap_dict) directly after the ap_list.append(ap_dict) inside the loop I see the correct values. Why is it that the print(ap_list) that happens outside of the for loop is only printing the last item on repeat? > You create one dictionary, and then reuse it every time. Appending it to a list doesn't change it, which means you keep overwriting the same dictionary every time you go through the loop. Try creating a new dictionary for each row; in fact, you can simplify things significantly by using a dict literal (or, more technically, "dict display") to create a new dictionary with the four fields already set. You could even do that in the same operation as appending to the list. ChrisA From nir.zairi at gmail.com Fri Jul 26 00:33:34 2019 From: nir.zairi at gmail.com (nir.zairi at gmail.com) Date: Thu, 25 Jul 2019 21:33:34 -0700 (PDT) Subject: Python 3.7 - Reading text from Terminal window Message-ID: <033ba4f3-e5d6-4cb2-9e7f-92688afd5aa6@googlegroups.com> Hi, I want it to wait for a specific text on the terminal window that it opens (putty.exe), how can i do it? from pywinauto.application import Application from pynput.keyboard import Key, Controller print('Configuring') app = Application().start(cmd_line=u'"C:\\...putty.exe" ') puttyconfigbox = app.PuTTYConfigBox puttyconfigbox.wait('ready') listbox = puttyconfigbox.ListBox listbox.select(u'COM5') button = puttyconfigbox[u'&Load'] button.click() button2 = puttyconfigbox[u'&Open'] button2.click() keyboard = Controller() keyboard.press(Key.enter) keyboard.type('password1') keyboard.press(Key.enter) keyboard.type('enable\nconf t\n') From * at eli.users.panix.com Fri Jul 26 01:19:47 2019 From: * at eli.users.panix.com (Eli the Bearded) Date: Fri, 26 Jul 2019 05:19:47 +0000 (UTC) Subject: Proper shebang for python3 References: <3154FF95-BAEA-497A-AB8B-384E041A368A@barrys-emacs.org> <12734243.RDIVbhacDa@PointedEars.de> Message-ID: In comp.lang.python, Thomas 'PointedEars' Lahn wrote: > Michael Torrie wrote: >> On 7/24/19 4:20 PM, Cameron Simpson wrote: >>> That is some progress, hooray. Then there's just sbin -> bin to go. >> I suppose in the olden days sbin was for static binaries, [?] > No, ?sbin? is short for ?*system* binaries? which in general only the > superuser should be able to execute. I think Michael is confusing "sbin" with the statically linked utilities some systems (particularly older ones, but also FreeBSD in /rescue/) have for repairing the system when things start to go bad. You'd want a shell (sh is great), a basic editor (eg, eg), and a smattering of other tools, akin to the ones listed as "must be in /sbin" in your linuxfoundation link. But more than a few utilities in /sbin are useful for non-superusers. Eg ip or ifconfig for informational purposes like identifying current IP address and getting MAC. > Which is why the above is a Very Bad Idea[tm]. Why? Programs that can *only* be usefully run by a privileged user or in a system context (eg halt or getty) already *must* prevent non privileged use. So why would it be a Very Bad Idea[tm] to have them in a common directory like /bin/? (Feel free to crosspost and set follow-ups to another group if you like. But I would suggest *not* a Linux group, since this is something general to all Unix-likes.) > Elijah ------ uses both netbsd and linux regularly From kirillbalunov at gmail.com Fri Jul 26 03:12:59 2019 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Fri, 26 Jul 2019 10:12:59 +0300 Subject: Difference between os.path.isdir and Path.is_dir In-Reply-To: References: Message-ID: ??, 25 ???. 2019 ?. ? 20:28, eryk sun : > On 7/25/19, Kirill Balunov wrote: > > > >>>> import os > >>>> from pathlib import Path > >>>> dummy = " " # or "" or " " > >>>> os.path.isdir(dummy) > > False > >>>> Path(dummy).is_dir() > > True > > I can't reproduce the above result in either Linux or Windows. The > results should only be different for an empty path string, since > Path('') is the same as Path('.'). The results should be the same for > Path(" "), depending on whether a directory named " " exists (normally > not allowed in Windows, but Linux allows it). > > I need to confirm that it was my fault and for non-empty strings (`" "` or `" "`), both `os.path.isdir(...)` and `Path(...).is_dir()` produce the same results.So sorry for the noise. Concerning the case with empty path string, I will open a ticket at the bug tracker (but for some reason it is blocked in my country :( https://isitblockedinrussia.com/?host=https%3A%2F%2Fbugs.python.org%2F). Obviously these are not equivalent forms, so either this should be noted in the documentation or corrected in the code. with kind regards, -gdg From angel.thomas1717 at gmail.com Fri Jul 26 03:20:50 2019 From: angel.thomas1717 at gmail.com (angel.thomas1717 at gmail.com) Date: Fri, 26 Jul 2019 00:20:50 -0700 (PDT) Subject: Python Certification training for Data Science Message-ID: <9e937219-4e27-4201-9038-286306b37d19@googlegroups.com> This is a comprehensive course to learn Python Programming for Data Science, Data Analysis and Data Visualization In this course we will learn: 1) Complete understanding of Python from Scratch 2) Python for Data Science and Business Analysis List of some Topics that we will cover, 1) NumPy : NumPy array, Indexing and Selection, NumPy Operations 2) Pandas : DataFrames, Series, Matrix, Working on missing data, Reading and Writing files 3) Matplotlib : Data Visualization, Plotting different graphs 4) Data Types 5) If-else statement, For loop and While loop 6) Functions and Methods 7) Object Oriented Programming Learn Python for Data Science to advance your Career in a fun and practical way !!! More click here: https://www.simpliv.com/virtual-classroom/python-certification-training-for-data-science-and-data-analysis From tjol at tjol.eu Fri Jul 26 05:09:10 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 26 Jul 2019 11:09:10 +0200 Subject: Python 3.7 - Reading text from Terminal window In-Reply-To: <033ba4f3-e5d6-4cb2-9e7f-92688afd5aa6@googlegroups.com> References: <033ba4f3-e5d6-4cb2-9e7f-92688afd5aa6@googlegroups.com> Message-ID: On 26/07/2019 06.33, nir.zairi at gmail.com wrote: > Hi, > > I want it to wait for a specific text on the terminal window that it opens (putty.exe), how can i do it? > > from pywinauto.application import Application > from pynput.keyboard import Key, Controller > > print('Configuring') > > app = Application().start(cmd_line=u'"C:\\...putty.exe" ') > puttyconfigbox = app.PuTTYConfigBox > puttyconfigbox.wait('ready') > listbox = puttyconfigbox.ListBox > listbox.select(u'COM5') > button = puttyconfigbox[u'&Load'] > button.click() > button2 = puttyconfigbox[u'&Open'] > button2.click() > > keyboard = Controller() > keyboard.press(Key.enter) > > keyboard.type('password1') > keyboard.press(Key.enter) > keyboard.type('enable\nconf t\n') > Not a clue, but why are you using PuTTY at all? Can't whatever it is you're doing be achieved much more simply using e.g. pyserial, accessing the COM port directly from Python? -- T From info at wingware.com Fri Jul 26 09:30:30 2019 From: info at wingware.com (Wingware) Date: Fri, 26 Jul 2019 09:30:30 -0400 Subject: ANN: Wing Python IDE 7.1 released Message-ID: <5D3B0076.9060701@wingware.com> Wing 7.1 is a new release of Wingware's Python IDE product line. This release adds support for Python 3.8, warns about unused symbols, improves code warnings configuration, adds new icons for the auto-completer, project, and source browser, supports Dark Mode on OS X, and makes other improvements. == Some Highlights of Wing 7.1 == * Support for Python 3.8: Wing 7.1 supports editing, testing, and debugging code written for Python 3.8, so you can take advantage of assignment expressions and other improvements introduced in this new version of Python. * Improved Code Warnings: Wing 7.1 adds unused symbol warnings for imports, variables, and arguments found in Python code. This release also improves code warnings configuration, making it easier to disable unwanted warnings. * Cosmetic Improvements: Wing 7.1 improves the auto-completer, project tool, and code browser with redesigned icons that make use of Wing's icon color configuration. This release also improves text display on some Linux systems, supports Dark Mode on macOS, and improves display of Python code and icons found in documentation. * And More: Wing 7.1 also adds support for Windows 10 native OpenSSH installations for remote development, and makes a number of other minor improvements. This release drops support for macOS 10.11. System requirements remain unchanged on Windows and Linux. For details see the change log: https://wingware.com/pub/wingpro/7.1.0.2/CHANGELOG.txt For a complete list of new features in Wing 7, see What's New in Wing 7: https://wingware.com/wingide/whatsnew == Downloads == Wing Pro: https://wingware.com/downloads/wing-pro/7.1/binaries Wing Personal: https://wingware.com/downloads/wing-personal/7.1/binaries Wing 101: https://wingware.com/downloads/wing-101/7.1/binaries Compare Products: https://wingware.com/downloads See Upgrading https://wingware.com/doc/install/upgrading for details on upgrading from Wing 6 and earlier, and Migrating from Older Versions https://wingware.com/doc/install/migrating for a list of compatibility notes. Stephan Deibel Wing Python IDE | The Intelligent Development Environment for Python From andrew.bell.ia at gmail.com Fri Jul 26 13:28:06 2019 From: andrew.bell.ia at gmail.com (Andrew Bell) Date: Fri, 26 Jul 2019 13:28:06 -0400 Subject: Multiple Interpreters? Message-ID: Hi, I'm trying to understand what might be going on with loading one Python interpreter on top of a second. I'm running on OSX with a conda python installation. I understand that python is now statically linked on conda. I'm running python, which invokes an extension module. This extension module calls some C++ code that is trying to start a second interpreter -- it loads a shared object that's linked against libpython.dylib. When I try to initialize python (Py_Initialize) in the shared object loaded by the extension module, it fails (see below). I'm not mixing anything from the original python invocation to the one I'm trying run and I'm sure that Py_Initialize in the shared library invocation are from libpython.dylib, rather than from the python interpreter (python3). Is there some reason why I can't run a second interpreter using libpython.dylib when my process is already running python3? (I know that I can solve this by not linking libpython.dylib and deferring symbol resolution to runtime, which will force resolution to symbols in python3, but I don't know at build time if I'm using a statically-linked interpreter or not. I'd like to understand why this fails when I'm not sharing context with the original interpreter.) Fatal Python error: initsite: Failed to import the site module Traceback (most recent call last): File "/Users/acbell/miniconda3/envs/pdal/lib/python3.7/site.py", line 168, in addpackage exec(line) File "", line 1, in File "/Users/acbell/miniconda3/envs/pdal/lib/python3.7/importlib/util.py", line 14, in from contextlib import contextmanager File "/Users/acbell/miniconda3/envs/pdal/lib/python3.7/contextlib.py", line 5, in from collections import deque File "/Users/acbell/miniconda3/envs/pdal/lib/python3.7/collections/__init__.py", line 24, in import heapq as _heapq File "/Users/acbell/miniconda3/envs/pdal/lib/python3.7/heapq.py", line 587, in from _heapq import * Thanks, -- Andrew Bell andrew.bell.ia at gmail.com From erik at q32.com Fri Jul 26 14:56:11 2019 From: erik at q32.com (Erik Aronesty) Date: Fri, 26 Jul 2019 14:56:11 -0400 Subject: bool(Enum) should raise ValueError Message-ID: I just spend a while tracking down and killing all "if Enum" and "if not Enum" bugs in my code. I was frankly shocked that this didn't raise a ValueError to begin with. Apparently all enums are true/false depending on whether the underlying value is truthy or falsy. Which breaks the abstraction Enum's are trying to achieve because now the user of an Enum has to know "stuff" about the underlying value and how it behaves. From ethan at stoneleaf.us Fri Jul 26 15:28:45 2019 From: ethan at stoneleaf.us (Ethan Furman) Date: Fri, 26 Jul 2019 12:28:45 -0700 Subject: bool(Enum) should raise ValueError In-Reply-To: References: Message-ID: <950de52d-5703-3f20-3203-ec9b92a1382a@stoneleaf.us> On 07/26/2019 11:56 AM, Erik Aronesty wrote: > I just spend a while tracking down and killing all "if Enum" and "if not > Enum" bugs in my code. I was frankly shocked that this didn't raise a > ValueError to begin with. Very few items, if any, raise a ValueError when asked for its True/False equivalence. > Apparently all enums are true/false depending on whether the underlying > value is truthy or falsy. That is the default. You can, of course, change that: class AlwaysFalse(Enum): def __bool__(self): return False class AlwaysTrue(Enum): def __bool__(self): return True > Which breaks the abstraction Enum's are trying to achieve because now the > user of an Enum has to know "stuff" about the underlying value and how it > behaves. You can make it always fail: class OnTheFence(Enum): def __bool__(self): raise ValueError('%s members are neither True nor False' % self.__class__.__name__) The default Enum always returns True because the default Enum starts counting at 1. Perhaps you could share your Enum definition and what you expected to happen when you used it in a boolean context? -- ~Ethan~ From rosuav at gmail.com Fri Jul 26 15:33:30 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sat, 27 Jul 2019 05:33:30 +1000 Subject: bool(Enum) should raise ValueError In-Reply-To: References: Message-ID: On Sat, Jul 27, 2019 at 5:16 AM Erik Aronesty wrote: > > I just spend a while tracking down and killing all "if Enum" and "if not > Enum" bugs in my code. I was frankly shocked that this didn't raise a > ValueError to begin with. > > Apparently all enums are true/false depending on whether the underlying > value is truthy or falsy. > > Which breaks the abstraction Enum's are trying to achieve because now the > user of an Enum has to know "stuff" about the underlying value and how it > behaves. If you want to abstract away the underlying value, just don't have one? >>> from enum import Enum, auto >>> class Color(Enum): ... red = auto() ... green = auto() ... blue = auto() ... >>> bool(Color.red) True >>> bool(Color.green) True >>> bool(Color.blue) True They happen to have the values 1, 2, and 3, but that doesn't matter. When an enum has to correspond to a real underlying value, it behaves as similarly to that value as possible: >>> http.HTTPStatus.METHOD_NOT_ALLOWED == 405 True Thus it should also inherit its truthiness from that value. ChrisA From dennistobias at gmail.com Fri Jul 26 17:19:12 2019 From: dennistobias at gmail.com (DT) Date: Fri, 26 Jul 2019 14:19:12 -0700 (PDT) Subject: Why am I getting duplicate values in my output? In-Reply-To: References: Message-ID: On Thursday, July 25, 2019 at 9:57:38 PM UTC-5, Chris Angelico wrote: > On Fri, Jul 26, 2019 at 12:46 PM DT wrote: > > def main(): > > ap_list = [] > > ap_dict = {} > > > > for row in csv_reader: > > ap_dict['ap_name'] = ap.name > > ap_dict['ap_ipv4'] = ap.ipv4 > > ap_dict['ap_model'] = ap.model > > ap_dict['ap_location'] = ap.location > > > > ap_list.append(ap_dict) > > > > print(ap_list) > > > > > > When I execute print(ap_list) I get the very last row in the CSV repeated once for every row in the file. I would expect to see a list of ap_dicts, but I do not. When I add print(ap_dict) directly after the ap_list.append(ap_dict) inside the loop I see the correct values. Why is it that the print(ap_list) that happens outside of the for loop is only printing the last item on repeat? > > > > You create one dictionary, and then reuse it every time. Appending it > to a list doesn't change it, which means you keep overwriting the same > dictionary every time you go through the loop. > > Try creating a new dictionary for each row; in fact, you can simplify > things significantly by using a dict literal (or, more technically, > "dict display") to create a new dictionary with the four fields > already set. You could even do that in the same operation as appending > to the list. > > ChrisA That is interesting to know. I guess I was confused because when I put in a print(ap_dict) in the for loop, it printed out the contents of the dict correctly for each pass. I assumed that when I then added that dict into the list that the list would be appended with that iterations values. I guess I have a bit to learn about how things are referenced in Python. Thanks for the hint. I updated my code and it works correctly now. From Richard at Damon-Family.org Fri Jul 26 17:56:05 2019 From: Richard at Damon-Family.org (Richard Damon) Date: Fri, 26 Jul 2019 17:56:05 -0400 Subject: Why am I getting duplicate values in my output? In-Reply-To: References: Message-ID: <11accb7e-2f7b-c1c5-bed4-4822809ea1e2@Damon-Family.org> On 7/26/19 5:19 PM, DT wrote: > On Thursday, July 25, 2019 at 9:57:38 PM UTC-5, Chris Angelico wrote: >> On Fri, Jul 26, 2019 at 12:46 PM DT wrote: >>> def main(): >>> ap_list = [] >>> ap_dict = {} >>> >>> for row in csv_reader: >>> ap_dict['ap_name'] = ap.name >>> ap_dict['ap_ipv4'] = ap.ipv4 >>> ap_dict['ap_model'] = ap.model >>> ap_dict['ap_location'] = ap.location >>> >>> ap_list.append(ap_dict) >>> >>> print(ap_list) >>> >>> >>> When I execute print(ap_list) I get the very last row in the CSV repeated once for every row in the file. I would expect to see a list of ap_dicts, but I do not. When I add print(ap_dict) directly after the ap_list.append(ap_dict) inside the loop I see the correct values. Why is it that the print(ap_list) that happens outside of the for loop is only printing the last item on repeat? >>> >> You create one dictionary, and then reuse it every time. Appending it >> to a list doesn't change it, which means you keep overwriting the same >> dictionary every time you go through the loop. >> >> Try creating a new dictionary for each row; in fact, you can simplify >> things significantly by using a dict literal (or, more technically, >> "dict display") to create a new dictionary with the four fields >> already set. You could even do that in the same operation as appending >> to the list. >> >> ChrisA > That is interesting to know. I guess I was confused because when I put in a print(ap_dict) in the for loop, it printed out the contents of the dict correctly for each pass. I assumed that when I then added that dict into the list that the list would be appended with that iterations values. I guess I have a bit to learn about how things are referenced in Python. > > Thanks for the hint. I updated my code and it works correctly now. dictionaries are mutable. appending the dictionary to the list adds a reference to that dictionary to the list, if you change the dictionary latter on, the list is still refering to that now changed dictionary. Either you need to copy the dictionary, and add the new copy, or create a new copy with each iteration of the loop. At the end of the loop, ap_list is a list of all the same dictionary, the same one that ap_dict is bound to, you just kept changing it every time through the loop. if ap_dict was set to a new empty dictionary in the loop, then all the copies would be distinct. -- Richard Damon From nandandor at gmail.com Fri Jul 26 18:05:12 2019 From: nandandor at gmail.com (nice guy4) Date: Fri, 26 Jul 2019 18:05:12 -0400 Subject: TypeError: 'top_block_22' object is not callable Message-ID: Hi, This is AJAX to flask server project. For more info about this project I have a couple postings about it here, please see link-1 link-2 When I run the python code in the terminal I get the TypeError: 'top_block_22' object is not callable error. The program starts running without problems, but once I start moving the slider the program will give that error. More details: This is AJAX to flask server project. I created a 'Slider' folder inside Flask directory : /home/fit-pc/my_flask_app/virtualenv/Slider. In this folder I have Templates and Static folders. Inside Templates folder I have my index.html file (see below). This index.html file had the script for the roundSlider widget that I am trying to use to control some variable value inside my python code 'top_block_22.py'. My main python code is in the main Slider folder. Static folder is just empty. Please, I need your help to solve this problem. error log: fit-pc at fitpc-fitlet2:~$ python /home/fit-pc/my_flask_app/virtualenv/Slider/ top_block_22.py * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger PIN: 269-962-008 127.0.0.1 - - [26/Jul/2019 11:20:11] "GET / HTTP/1.1" 200 - gr::log :INFO: audio source - Audio sink arch: alsa 127.0.0.1 - - [26/Jul/2019 11:20:15] "GET /valueofslider?slide_val=903 HTTP/1.1" 500 - Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1997, in __call__ return self.wsgi_app(environ, start_response) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1985, in wsgi_app response = self.handle_exception(e) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1540, in handle_exception reraise(exc_type, exc_value, tb) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app response = self.full_dispatch_request() File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1615, in full_dispatch_request return self.finalize_request(rv) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1630, in finalize_request response = self.make_response(rv) File "/usr/lib/python2.7/dist-packages/flask/app.py", line 1740, in make_response rv = self.response_class.force_type(rv, request.environ) File "/usr/lib/python2.7/dist-packages/werkzeug/wrappers.py", line 921, in force_type response = BaseResponse(*_run_wsgi_app(response, environ)) File "/usr/lib/python2.7/dist-packages/werkzeug/wrappers.py", line 59, in _run_wsgi_app return _run_wsgi_app(*args) File "/usr/lib/python2.7/dist-packages/werkzeug/test.py", line 923, in run_wsgi_app app_rv = app(environ, start_response) TypeError: 'top_block_22' object is not callable And here is the Python code: from gnuradio import analog from gnuradio import audio from gnuradio import blocks from gnuradio import eng_notation from gnuradio import gr from gnuradio.eng_option import eng_option from gnuradio.filter import firdes from optparse import OptionParser from flask import Flask, render_template, jsonify, request, redirect, url_for from random import randint class top_block_22(gr.top_block): def __init__(self, slide_val): self.slide_val = slide_val gr.top_block.__init__(self, "Top Block 22") ################################################## # Variables ################################################## self.samp_rate = samp_rate = 32000 ################################################## # Blocks ################################################## self.blocks_add_xx = blocks.add_vff(1) self.audio_sink = audio.sink(32000, '', True) self.analog_sig_source_x_1 = analog.sig_source_f(samp_rate, analog.GR_COS_WAVE, 440, 0.4, 0) self.analog_sig_source_x_0 = analog.sig_source_f(samp_rate, analog.GR_COS_WAVE, 350, 0.4, 0) self.analog_noise_source_x_0 = analog.noise_source_f(analog.GR_GAUSSIAN, 0.005, -42) ################################################## # Connections ################################################## self.connect((self.analog_noise_source_x_0, 0), (self.blocks_add_xx, 2)) self.connect((self.analog_sig_source_x_0, 0), (self.blocks_add_xx, 0)) self.connect((self.analog_sig_source_x_1, 0), (self.blocks_add_xx, 1)) self.connect((self.blocks_add_xx, 0), (self.audio_sink, 0)) app = Flask(__name__) @app.route('/') def hex_color(): return render_template("index.html") @app.route('/valueofslider') def slide(): slide_val = request.args.get('slide_val') return top_block_22(slide_val) def main(top_block_cls=top_block_22, options=None): tb = top_block_cls() tb.start() try: raw_input('Press Enter to quit: ') except EOFError: pass tb.stop() tb.wait() samp_rate = int(slide_val) + 100 print(samp_rate) return(slide_val) # Still need to return or get TypeError if __name__ == '__main__': app.run(debug=True) And this is the index.html script: jQuery roundSlider - JS Bin
From taeyeon10006 at gmail.com Sat Jul 27 07:20:33 2019 From: taeyeon10006 at gmail.com (=?utf-8?Q?=EB=AF=B8?=) Date: Sat, 27 Jul 2019 20:20:33 +0900 Subject: error Message-ID: <5d3c3380.1c69fb81.e130f.a5ab@mx.google.com> ????? ?? ??? ???? ???? ????.????? ?? ?? ??? ??? ??? ???? ? ????. Windows 10? ???? ?? From AliKeshavarzNasab at outlook.com Sat Jul 27 07:31:28 2019 From: AliKeshavarzNasab at outlook.com (Ali Keshavarz Nasab) Date: Sat, 27 Jul 2019 11:31:28 +0000 Subject: a problem in Libs installs Message-ID: Hi dear Mr/Maddam When I install some libs in CMD or Power shield the error is appeared: ?Running setup.py install for pillow ... error?. Whats the solution and what is the best way to install Libs? I use 3.8.0b2 version. What is the best editor for python? Thanks alot Sent from Mail for Windows 10 From python at mrabarnett.plus.com Sat Jul 27 12:56:03 2019 From: python at mrabarnett.plus.com (MRAB) Date: Sat, 27 Jul 2019 17:56:03 +0100 Subject: a problem in Libs installs In-Reply-To: References: Message-ID: <5fb5d88f-bfdc-15a6-6181-223ddfa36318@mrabarnett.plus.com> On 2019-07-27 12:31, Ali Keshavarz Nasab wrote: > Hi dear Mr/Maddam > When I install some libs in CMD or Power shield the error is appeared: ?Running setup.py install for pillow ... error?. Whats the solution and what is the best way to install Libs? I use 3.8.0b2 version. What is the best editor for python? > Python 3.8 is a beta version, not the final version. It's better to use Python 3.7 for now. Did you use "pip" to install Pillow? Probably the best way to install Pillow is to use the Python launcher and the pip module: py -3.7 -m pip install pillow (if you're using Python 3.7) As for the best editor, that depends on your preferences and whether you want a free one or are willing to pay. I use EditPad Pro (the paid version), but EditPad Lite is free. Or you could use Notepad++, or Visual Studio Code. Pick one that features syntax colouring, and, preferably, one that lets you run the program directly from the editor and can capture any output (very useful). From a at d-amo.re Sat Jul 27 12:28:37 2019 From: a at d-amo.re (Andrea D'Amore) Date: Sat, 27 Jul 2019 18:28:37 +0200 Subject: Why am I getting duplicate values in my output? In-Reply-To: References: Message-ID: On Fri, 26 Jul 2019 at 23:19, DT wrote: > I guess I have a bit to learn about how things are referenced in Python. Check out Ned Batchelder's excellent "Facts and myths about Python names and values" [1]. [1]: https://nedbatchelder.com/text/names.html -- Andrea From rosuav at gmail.com Sat Jul 27 17:10:41 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 28 Jul 2019 07:10:41 +1000 Subject: Definite or indefinite article for non-singletons? Message-ID: When talking about indistinguishable objects, is it correct to talk about "the " or "an "? Example: def f(s): """Frob a thing. If s is an empty string, frobs all the things. OR If s is the empty string, frobs all the things. """ It's entirely possible that a Python implementation will optimize small strings and thus have exactly one empty string, but it's also entirely possible to have multiple indistinguishable empty strings. Grammatically, is it better to think of empty strings as an entire category of object, and you were passed one from that category ("an empty string"), or to think of zero-length instances of 'str' as being implementation details referring to the one and only Platonic "empty string"? Does it make a difference to usage if the object is mutable? For instance, would you say "the empty string" but "an empty set"? ChrisA From cs at cskk.id.au Sat Jul 27 18:09:10 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 28 Jul 2019 08:09:10 +1000 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: Message-ID: <20190727220910.GA39470@cskk.homeip.net> On 28Jul2019 07:10, Chris Angelico wrote: >When talking about indistinguishable objects, is it correct to talk >about "the " or "an "? > >Example: > >def f(s): > """Frob a thing. > > If s is an empty string, frobs all the things. > OR > If s is the empty string, frobs all the things. > """ > >It's entirely possible that a Python implementation will optimize >small strings and thus have exactly one empty string, but it's also >entirely possible to have multiple indistinguishable empty strings. >Grammatically, is it better to think of empty strings as an entire >category of object, and you were passed one from that category ("an >empty string"), or to think of zero-length instances of 'str' as being >implementation details referring to the one and only Platonic "empty >string"? It depends. If the API _specifies_ that something is a singleton then I'd use its proper name (eg None or ClassName.SENTINEL or whatever) and use a definite article (eg "the queue is considered closed when the sentinel is received"). >Does it make a difference to usage if the object is mutable? For >instance, would you say "the empty string" but "an empty set"? If the implementation _may_ fold indistinguishable things together I'd speak to the case that it may not, and say "if the string is empty", which (a) sidesteps whether there's only one empty string instance and (b) talks about the criterion for the decision and not some implementation artifact. Cheers, Cameron Simpson From ethan at stoneleaf.us Sat Jul 27 18:14:52 2019 From: ethan at stoneleaf.us (Ethan Furman) Date: Sat, 27 Jul 2019 15:14:52 -0700 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: Message-ID: On 07/27/2019 02:10 PM, Chris Angelico wrote: > When talking about indistinguishable objects, is it correct to talk > about "the " or "an "? Multiple indistinguishable objects are still multiple, so "an". Implementation details should only enter the conversation when specifically discussing the implementation -- so CPython is an implementation detail while Python is the language. -- ~Ethan~ From mirkok.lists at googlemail.com Sat Jul 27 18:19:02 2019 From: mirkok.lists at googlemail.com (Mirko) Date: Sun, 28 Jul 2019 00:19:02 +0200 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: Message-ID: <5D3CCDD6.8030109@googlemail.com> Am 27.07.2019 um 23:10 schrieb Chris Angelico: > When talking about indistinguishable objects, is it correct to talk > about "the " or "an "? > > Example: > > def f(s): > """Frob a thing. > > If s is an empty string, frobs all the things. > OR > If s is the empty string, frobs all the things. > """ > > It's entirely possible that a Python implementation will optimize > small strings and thus have exactly one empty string, but it's also > entirely possible to have multiple indistinguishable empty strings. > Grammatically, is it better to think of empty strings as an entire > category of object, and you were passed one from that category ("an > empty string"), or to think of zero-length instances of 'str' as being > implementation details referring to the one and only Platonic "empty > string"? > > Does it make a difference to usage if the object is mutable? For > instance, would you say "the empty string" but "an empty set"? > > ChrisA > Quite frankly, even as a professional (german) writer and texter (but hobby-programmer), I consider this question to be an exaggeration about a very minor grammatical aspect. Yes, it makes a difference if you are allowed to eat *an* apple or *this* apple (*this* apple might be mine). But, I hobby-program since 20 years or so, but I can not remember a situation, where a documentation that confused "an" with "this" caused any troubles. Anyway: Python 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> x="" >>> y="" >>> x is y True Jython 2.5.3 (, Sep 21 2017, 03:12:48) [OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.7.0_131 Type "help", "copyright", "credits" or "license" for more information. >>> x="" >>> y="" >>> x is y False So, obviously there are multiple empty strings possible, so I'd write "an". HTH From tjreedy at udel.edu Sat Jul 27 18:26:19 2019 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 27 Jul 2019 18:26:19 -0400 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: Message-ID: On 7/27/2019 5:10 PM, Chris Angelico wrote: > When talking about indistinguishable objects, is it correct to talk > about "the " or "an "? > > Example: > > def f(s): > """Frob a thing. > > If s is an empty string, frobs all the things. > OR > If s is the empty string, frobs all the things. > """ Depends on whether one is using 'is' in the Python sense of object identify or the natural language sense of value equality. To evade, avoid the (an?) article. If string s is empty,... I think we tend to be clearer about numbers. x is an instance of 3.14. x is a float equal to 3.14. x is the number 3.14. x is the float 3.14. > It's entirely possible that a Python implementation will optimize > small strings and thus have exactly one empty string, but it's also > entirely possible to have multiple indistinguishable empty strings. > Grammatically, is it better to think of empty strings as an entire > category of object, and you were passed one from that category ("an > empty string"), or to think of zero-length instances of 'str' as being > implementation details referring to the one and only Platonic "empty > string"? > > Does it make a difference to usage if the object is mutable? For > instance, would you say "the empty string" but "an empty set"? In mathematics, *every* set is 'the'. -- Terry Jan Reedy From PythonList at DancesWithMice.info Sat Jul 27 18:34:29 2019 From: PythonList at DancesWithMice.info (DL Neil) Date: Sun, 28 Jul 2019 10:34:29 +1200 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: Message-ID: On 28/07/19 9:10 AM, Chris Angelico wrote: > When talking about indistinguishable objects, is it correct to talk > about "the " or "an "? > Example: > def f(s): > """Frob a thing. > If s is an empty string, frobs all the things. > OR > If s is the empty string, frobs all the things. > """ > It's entirely possible that a Python implementation will optimize > small strings and thus have exactly one empty string, but it's also > entirely possible to have multiple indistinguishable empty strings. > Grammatically, is it better to think of empty strings as an entire > category of object, and you were passed one from that category ("an > empty string"), or to think of zero-length instances of 'str' as being > implementation details referring to the one and only Platonic "empty > string"? > Does it make a difference to usage if the object is mutable? For > instance, would you say "the empty string" but "an empty set"? Python: Immutable: >>> NULSTRING = "" >>> A_NULSTRING = "" >>> THE_NULSTRING = "" >>> id( NULSTRING ) 139654031453296 >>> id( A_NULSTRING ) 139654031453296 <<< it's all the same, to me! >>> id( THE_NULSTRING ) 139654031453296 <<< they're all the same, to me? Mutable: >>> ONE = 1 >>> A_ONE = 1 >>> THE_ONE = 1 >>> id( ONE ) 139654256829664 >>> id( A_ONE ) 139654256829664 <<< likewise >>> id( THE_ONE ) 139654256829664 >>> ONE = 1 + 1 >>> id( ONE ) 139654256829696 <<< as you'd expect >>> id( A_ONE ) 139654256829664 >>> id( THE_ONE) 139654256829664 >>> ONE = 1 >>> id(ONE) 139654256829664 <<< did you expect this? There is only a/the ONE way. Disclaimer: I am not Dutch! English: Whilst the formalities of English grammar distinguish the specific ("the") from the general case ("a" or "an"), eg "the Python language" cf "a programming language". Spoken or colloquial English can be quite different from academically-correct text - and in this case is! (he says dangling a participle to demonstrate) Whereas the indefinite article can only be used in the singular, the definite article may be used as both singular and plural. A NULSTRING THE ONESIE THE ONESIES (a choice of 'article' which will enrage both grammarians and fashionistas, in equal measure!*) * oops, did you notice that it is "measure" and not "measures"? If you were verbally suggesting an API to someone else, eg class API(): def __init__( self, path="" ): ... Would you use either the definite or indefinite article? Trick question! The unequivocal answer is: "neither"! We would say "default id to NULSTRING". (similarly: "default [int value] to ONE") When you advise this other person, and when (s)he writes such code, do any of us care whether all NULSTRING-s have the same id() or not? When speaking, not! When typing, not! (and I venture to suggest, there'd be very few occasions when someone, suffering tight memory constraint*, went through counting NULSTRING-s seeking re-factor potential!) PS should that be "constraint" because there's only one, ie "memory"; or "constraints"? English is a ghastly language to learn because of its many irregular forms. The issue here is less about the preceding article, and more about the noun, ie NULSTRING, ONE, etc. Some nouns are positively confusing, eg "jeans". How many pairs of trousers/pants are there? (and then there is the same problem with the word "trousers", or if you prefer "pants"!) Few people wear more than one - at the same time. Yet we still say "jeans" as if the two legs are not somehow part of the single item. Other nouns are written as if plural but are only used in the singular, eg maths. If you only attend the one course, how many maths courses do you attend? Did you really think mathematics a precise science? (or should that have been "precise sciences"?) NB American English (English people reel in 'shock, horror') appears more correct in preferring the term "math" - but before anyone becomes too cocky, try "physics"! Thus George Bernard Shaw's claim that the two nations are "divided by the use of a common language", and the virtue of non-native speakers learning "Globish" or similar. To really confuse language-learners, we only need to add "collective nouns", eg "team" - they're enough to make one feel sheep-ish. (Ho ho!) Oh look, we're back to the question: is it "a team" or "the team"? Aren't most of us happy with the article-free wording? We should be concerned for "ONE" because as Harry Nilsson said, it is "the loneliest number"! I'll become concerned for you, should you start to look too closely at the concept of a NULSTRING (None, "infinite", ...). If you look into it too deeply, you will realised that you are standing on the edge of an abyss, and the roaring noise in your ears/an ear/the ears comes from the black hole, which is about to swallow you up, never to return* to us again! (*from an/the institution caring for the mentally-ill)... Yours sincerely, undoubtedly off-balance, but nevertheless well articulated. -- Regards =dn From prithwis at yantrajaal.com Sun Jul 28 00:22:30 2019 From: prithwis at yantrajaal.com (prithwis at yantrajaal.com) Date: Sat, 27 Jul 2019 21:22:30 -0700 (PDT) Subject: Swiss Ephemeris In-Reply-To: <49f767bc-8059-46bb-82b3-718dc8295b53@googlegroups.com> References: <49f767bc-8059-46bb-82b3-718dc8295b53@googlegroups.com> Message-ID: <5e4f9e79-e726-4cf1-b594-dd0e7758444b@googlegroups.com> Hello Peter I too was looking for a pythonic implementation of Swiss Ephemeris and arrived at https://pypi.python.org/pypi/pyswisseph After some fiddling around with the various parameters, I have finally made it work to calculate Ayanamsa, Planetary Body Positions & Speed and Ascendant ( which is essentially what I need for Indic Astrology ) If you want to see a working demo of the entire software please check my Google Colab Notebook located at http://bit.ly/2yhu9Cz In case you are not aware, Colab is free hosted environment for running python program on an Ubuntu platform. Cheers Prithwis On Sunday, April 9, 2017 at 11:23:45 PM UTC+5:30, Peter Henry wrote: > Hi Group > > I have a package that has been altered to imported in to python, however I tired to get is working but without success I be missing something obvious > > The Swiss Ephemeris enable planetary coordinate to be imported and used in your program > > Files access https://pypi.python.org/pypi/pyswisseph > > Many thanks in advance > > Peter From marko at pacujo.net Sun Jul 28 03:32:26 2019 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 28 Jul 2019 10:32:26 +0300 Subject: Definite or indefinite article for non-singletons? References: Message-ID: <87y30ibomt.fsf@elektro.pacujo.net> Ethan Furman : > On 07/27/2019 02:10 PM, Chris Angelico wrote: >> When talking about indistinguishable objects, is it correct to talk >> about "the " or "an "? > > Multiple indistinguishable objects are still multiple, so "an". > > Implementation details should only enter the conversation when > specifically discussing the implementation -- so CPython is an > implementation detail while Python is the language. Yes. If the API guarantees singleton-ness (reliable testability through "is"), the correct article is "the", otherwise "an". There is no guarantee that two empty strings are the same object so "an empty string" is the right expression. Now, len(string) return "the length" of the string of "a length" of the string: >>> s = "*"*2000 >>> len(s) 2000 >>> len(s) is len(s) False Here "the length" is the right answer and must be understood as a contraction of the pedantic: "an integer representing the length". So it depends on the context if the relevant equivalence is "is" or "==". Maybe the rule of thumb is that if we are talking about strings, integers and similar things, we should think about it from the point of view of Python's data model (objects; "is"). But when we talk about things like "length", "square root", "sum" or "name", the point of view is the abstractions the objects are standing for ("=="). Marko From __peter__ at web.de Sun Jul 28 04:32:56 2019 From: __peter__ at web.de (Peter Otten) Date: Sun, 28 Jul 2019 10:32:56 +0200 Subject: Definite or indefinite article for non-singletons? References: Message-ID: Chris Angelico wrote: > When talking about indistinguishable objects, is it correct to talk > about "the " or "an "? > > Example: > > def f(s): > """Frob a thing. > > If s is an empty string, frobs all the things. > OR > If s is the empty string, frobs all the things. > """ Slightly OT: While I would write "an empty string" but happily accept "the empty string" I stumbled upon "all *the* things". Shouldn't it be "all things"? Real question as I'm not a native speaker. From cs at cskk.id.au Sun Jul 28 08:30:29 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 28 Jul 2019 22:30:29 +1000 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: Message-ID: <20190728123029.GA15743@cskk.homeip.net> On 28Jul2019 10:32, Peter Otten <__peter__ at web.de> wrote: >Chris Angelico wrote: > >> When talking about indistinguishable objects, is it correct to talk >> about "the " or "an "? >> >> Example: >> >> def f(s): >> """Frob a thing. >> >> If s is an empty string, frobs all the things. >> OR >> If s is the empty string, frobs all the things. >> """ > >Slightly OT: > >While I would write "an empty string" but happily accept "the empty string" >I stumbled upon "all *the* things". Shouldn't it be "all things"? > >Real question as I'm not a native speaker. The collection is "the things". "all" qualifies it, versus, say, "some of the things" or "the first of the things" etc. You do also get "all things". It is less common. It tends to mean _all_ things, not just the ones here. So "all people" means everyone everywhere. But "all the people" usually refers to some specific group. What group depends on context. So some examples: def f(s, things): ''' Frob the things. ''' I would expect to frob the things passed as the parameter (thus, the context). Versus: class Thing: ... def f(s): ''' Frob all Things. ''' I would expect there to be some global registry of Thing instances, and to frob them all. Chris' example is a bit incomplete because there's no context to indicate what group "the things" comprise. Cheers, Cameron Simpson From michael.stemper at gmail.com Sun Jul 28 09:46:21 2019 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Sun, 28 Jul 2019 08:46:21 -0500 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: Message-ID: On 27/07/2019 17.43, Stefan Ram wrote: > Terry Reedy writes: >> In mathematics, *every* set is 'the'. This is correct, at least in ZF, where the Axiom of Extensionality says that (in English) if A and B have exactly the same elements, they are the same set. > |Example 2 Let T be an non-empty set and ... > [PDF] Real Analysis on Metric Spaces - Columbia University It's obvious that "an" is a typo, either for "any" or "a". Assume for the sake of argument that it's intended to be "a", as in "Let T be a non-empty set..." T is indeed "a" set, but it's the only set that has the same elements as T. Therefore, once you've let T be some specific non-empty set, it is *the* set T. -- Michael F. Stemper Life's too important to take seriously. From python at mrabarnett.plus.com Sun Jul 28 11:13:05 2019 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 28 Jul 2019 16:13:05 +0100 Subject: Definite or indefinite article for non-singletons? In-Reply-To: <20190728123029.GA15743@cskk.homeip.net> References: <20190728123029.GA15743@cskk.homeip.net> Message-ID: <418c3ac5-ca4b-f0a1-70e9-26637e7a7280@mrabarnett.plus.com> On 2019-07-28 13:30, Cameron Simpson wrote: > On 28Jul2019 10:32, Peter Otten <__peter__ at web.de> wrote: >>Chris Angelico wrote: >> >>> When talking about indistinguishable objects, is it correct to talk >>> about "the " or "an "? >>> >>> Example: >>> >>> def f(s): >>> """Frob a thing. >>> >>> If s is an empty string, frobs all the things. >>> OR >>> If s is the empty string, frobs all the things. >>> """ >> >>Slightly OT: >> >>While I would write "an empty string" but happily accept "the empty string" >>I stumbled upon "all *the* things". Shouldn't it be "all things"? >> >>Real question as I'm not a native speaker. > > The collection is "the things". "all" qualifies it, versus, say, "some > of the things" or "the first of the things" etc. > [snip] It's strange that "all the things" (meaning "all of the things") is OK, but otherwise it's "one of the things", "some of the things", etc. That's English for you! From jonathan-lists at lightpear.com Sun Jul 28 07:55:34 2019 From: jonathan-lists at lightpear.com (Jonathan Moules) Date: Sun, 28 Jul 2019 12:55:34 +0100 Subject: Boolean comparison & PEP8 Message-ID: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Hi List, Lets say I want to know if the value of `x` is bool(True). My preferred way to do it is: if x is True: ??? pass Because this tests both the value and the type. But this appears to be explicitly called out as being "Worse" in PEP8: """ Don't compare boolean values to True or False using ==. Yes:?? if greeting: No:??? if greeting == True: Worse: if greeting is True: """ Why? If `x` can also have a value of "1"(str) or 1(int) then in both cases this would be a false positive if I were to do the below as they'll both equate to True: if x: ??? pass The PEP for boolean type (285 - https://www.python.org/dev/peps/pep-0285/) doesn't mention the "is" comparison keyword at all as far as I can tell. What am I missing? Thanks From couldbeworse01 at protonmail.com Sun Jul 28 13:45:42 2019 From: couldbeworse01 at protonmail.com (couldbeworse01 at protonmail.com) Date: Sun, 28 Jul 2019 17:45:42 +0000 Subject: Trouble installing python Message-ID: Good afternoon, I originally started with the latest version of python (3.7), but I was unable to figure out how to get to script mode. There weren't any tabs at the top of the interactive mode window. So I decided to uninstall 3.7, and try 3.5.2 (since that is the version my online instructor is using) now i keep getting a screen asking me to modify, repair, or uninstall (screen shoot attached). I have clicked repair twice now, and modify once and I still get the same message. Am I doing something wrong in 3.7 that is maybe hiding script mode? Thank you Steven Sent from ProtonMail mobile From python at mrabarnett.plus.com Sun Jul 28 15:08:47 2019 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 28 Jul 2019 20:08:47 +0100 Subject: Trouble installing python In-Reply-To: References: Message-ID: <9ca1ea2a-4e64-be40-054b-2a40563eebe2@mrabarnett.plus.com> On 2019-07-28 18:45, Steven via Python-list wrote: > Good afternoon, > > I originally started with the latest version of python (3.7), but I was unable to figure out how to get to script mode. There weren't any tabs at the top of the interactive mode window. > What is "script mode"? Are you using IDLE? If you are, then look in the menu for File->New. > So I decided to uninstall 3.7, and try 3.5.2 (since that is the version my online instructor is using) now i keep getting a screen asking me to modify, repair, or uninstall (screen shoot attached). I have clicked repair twice now, and modify once and I still get the same message. > This list is text-only; it removes any attached pictures. > Am I doing something wrong in 3.7 that is maybe hiding script mode? > From marko at pacujo.net Sun Jul 28 15:22:30 2019 From: marko at pacujo.net (Marko Rauhamaa) Date: Sun, 28 Jul 2019 22:22:30 +0300 Subject: Boolean comparison & PEP8 References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: <87tvb6arrd.fsf@elektro.pacujo.net> Jonathan Moules : > Lets say I want to know if the value of `x` is bool(True). > My preferred way to do it is: > > if x is True: > [...] > > But this appears to be explicitly called out as being "Worse" in PEP8: > > [...] > > Why? It has primarily to do with the naturalness of expression. In English, you say: If you have a moment, I'll show you. If you had a dog, you'd understand. instead of: If your having a moment is true, I'll show you. If your having a dog were true, you'd understand. By the same vein, in Python you say: if len(students) < 7: klass.cancel() rather than: if (len(students) < 7) is True: klass.cancel() Furthermore, while True and False are singleton objects, referring to them through the "is" operator seems strikingly ontological in most contexts. You are no longer interested in the message of the letter but the fibers of the paper it was written on. I *could* imagine a special case where a positional argument's semantics would depend on the specific object. For example, >>> os.path.exists(False) True is rather funky and the os.path.exists function would probably benefit from a check such as: if path is True or path is False: raise Hell() but even in such cases, it is more customary to say: if isinstance(path, bool): raise Hell() Marko From earonesty at gmail.com Sun Jul 28 16:46:35 2019 From: earonesty at gmail.com (Erik Aronesty) Date: Sun, 28 Jul 2019 16:46:35 -0400 Subject: bool(Enum) should raise ValueError In-Reply-To: References: Message-ID: class Status: valid = 1 invalid = 2 unknown = 3 On Fri, Jul 26, 2019, 3:37 PM Chris Angelico wrote: > On Sat, Jul 27, 2019 at 5:16 AM Erik Aronesty wrote: > > > > I just spend a while tracking down and killing all "if Enum" and "if not > > Enum" bugs in my code. I was frankly shocked that this didn't raise a > > ValueError to begin with. > > > > Apparently all enums are true/false depending on whether the underlying > > value is truthy or falsy. > > > > Which breaks the abstraction Enum's are trying to achieve because now the > > user of an Enum has to know "stuff" about the underlying value and how it > > behaves. > > If you want to abstract away the underlying value, just don't have one? > > >>> from enum import Enum, auto > >>> class Color(Enum): > ... red = auto() > ... green = auto() > ... blue = auto() > ... > >>> bool(Color.red) > True > >>> bool(Color.green) > True > >>> bool(Color.blue) > True > > They happen to have the values 1, 2, and 3, but that doesn't matter. > > When an enum has to correspond to a real underlying value, it behaves > as similarly to that value as possible: > > >>> http.HTTPStatus.METHOD_NOT_ALLOWED == 405 > True > > Thus it should also inherit its truthiness from that value. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > From ethan at stoneleaf.us Sun Jul 28 18:00:52 2019 From: ethan at stoneleaf.us (Ethan Furman) Date: Sun, 28 Jul 2019 15:00:52 -0700 Subject: bool(Enum) should raise ValueError In-Reply-To: References: Message-ID: On 07/28/2019 01:46 PM, Erik Aronesty wrote: One possibility --------------- > class Status: > valid = 1 > invalid = 2 > unknown = 3 if status is Status.valid: # good status, do something elif status is Status.unknown: figure_out_status() elif status is Status.invalid: raise ValueError('this shouldn't happen!) else: raise ValueError('unhandled Status: %r' % status) -- ~Ethan~ From tjol at tjol.eu Sun Jul 28 16:35:19 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Sun, 28 Jul 2019 22:35:19 +0200 Subject: Definite or indefinite article for non-singletons? In-Reply-To: <418c3ac5-ca4b-f0a1-70e9-26637e7a7280@mrabarnett.plus.com> References: <20190728123029.GA15743@cskk.homeip.net> <418c3ac5-ca4b-f0a1-70e9-26637e7a7280@mrabarnett.plus.com> Message-ID: <853a494b-464f-7ca5-9a7a-df75bf6f438b@tjol.eu> On 28/07/2019 17:13, MRAB wrote: > [snip] > > It's strange that "all the things" (meaning "all of the things") is OK, > but otherwise it's "one of the things", "some of the things", etc. Is it? It's the same in French, Dutch and German. Can't tell if it just makes sense or if it's Common Average European doing its thing. From rosuav at gmail.com Sun Jul 28 19:04:53 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jul 2019 09:04:53 +1000 Subject: Definite or indefinite article for non-singletons? In-Reply-To: <87y30ibomt.fsf@elektro.pacujo.net> References: <87y30ibomt.fsf@elektro.pacujo.net> Message-ID: On Sun, Jul 28, 2019 at 5:36 PM Marko Rauhamaa wrote: > So it depends on the context if the relevant equivalence is "is" or > "==". Maybe the rule of thumb is that if we are talking about strings, > integers and similar things, we should think about it from the point of > view of Python's data model (objects; "is"). But when we talk about > things like "length", "square root", "sum" or "name", the point of view > is the abstractions the objects are standing for ("=="). Interesting. So in your view, a string object representing zero characters is "an", but its length is "the"? I'd have thought that both are considered value types, where identity is immaterial. Thanks for all the insights, all. This is a curious point of English and one that I've definitely seen people on both sides of. My apologies for not responding earlier or to individual posts. For a while (and I'm actually not certain that this post will make it), all Gmail addresses have been unable to post to python-list, and even contacting the list-owner wasn't possible. ChrisA From torriem at gmail.com Sun Jul 28 19:46:51 2019 From: torriem at gmail.com (Michael Torrie) Date: Sun, 28 Jul 2019 17:46:51 -0600 Subject: Boolean comparison & PEP8 In-Reply-To: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 7/28/19 5:55 AM, Jonathan Moules wrote: > But this appears to be explicitly called out as being "Worse" in PEP8: > > """ > Don't compare boolean values to True or False using ==. > > Yes:?? if greeting: > No:??? if greeting == True: > Worse: if greeting is True: > """ Yet the recommended solution to the problem of wanting a default argument of an empty list is something like this: def foo(bar=False); if bar is False: bar = [] .... Clearly in this case the expression "not bar" would be incorrect. There's a difference between looking for a particular identity or sentinel, and checking for truthiness or falsiness. So I guess it all depends on what you need to do. If you just need to check truthiness, just do "if greeting" If you need to know if the variable is some particular sentinel, use the "if greeting is whatever". From python at mrabarnett.plus.com Sun Jul 28 19:47:53 2019 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 29 Jul 2019 00:47:53 +0100 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: <87y30ibomt.fsf@elektro.pacujo.net> Message-ID: On 2019-07-29 00:04, Chris Angelico wrote: > On Sun, Jul 28, 2019 at 5:36 PM Marko Rauhamaa wrote: >> So it depends on the context if the relevant equivalence is "is" or >> "==". Maybe the rule of thumb is that if we are talking about strings, >> integers and similar things, we should think about it from the point of >> view of Python's data model (objects; "is"). But when we talk about >> things like "length", "square root", "sum" or "name", the point of view >> is the abstractions the objects are standing for ("=="). > > Interesting. So in your view, a string object representing zero > characters is "an", but its length is "the"? I'd have thought that > both are considered value types, where identity is immaterial. > There can be multiple empty strings, which the implementation _might_ optimise to a singleton, but each string has only one length. A string of length > 1 has multiple characters, each being _a_ character, but the string as a whole has only one length, so we can call it _the_ length. > Thanks for all the insights, all. This is a curious point of English > and one that I've definitely seen people on both sides of. > > My apologies for not responding earlier or to individual posts. For a > while (and I'm actually not certain that this post will make it), all > Gmail addresses have been unable to post to python-list, and even > contacting the list-owner wasn't possible. > From Richard at Damon-Family.org Sun Jul 28 20:04:36 2019 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 28 Jul 2019 20:04:36 -0400 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: <87y30ibomt.fsf@elektro.pacujo.net> Message-ID: On 7/28/19 7:04 PM, Chris Angelico wrote: > On Sun, Jul 28, 2019 at 5:36 PM Marko Rauhamaa wrote: >> So it depends on the context if the relevant equivalence is "is" or >> "==". Maybe the rule of thumb is that if we are talking about strings, >> integers and similar things, we should think about it from the point of >> view of Python's data model (objects; "is"). But when we talk about >> things like "length", "square root", "sum" or "name", the point of view >> is the abstractions the objects are standing for ("=="). > Interesting. So in your view, a string object representing zero > characters is "an", but its length is "the"? I'd have thought that > both are considered value types, where identity is immaterial. > > Thanks for all the insights, all. This is a curious point of English > and one that I've definitely seen people on both sides of. > > My apologies for not responding earlier or to individual posts. For a > while (and I'm actually not certain that this post will make it), all > Gmail addresses have been unable to post to python-list, and even > contacting the list-owner wasn't possible. > > ChrisA It is the length of a string, as a given string only has a single length, so grammatically, it wants the definite article always. When talking of empty strings, we need to look a bit at context.? "The empty string" implies that there is only one of them, and if we are talking about values, then there is only one empty string values, so "The empty string value" would be correct (and the term value might be implied by context). If we are talking about object, like with the python word "is", then the empty string is not promised to be a singleton, so grammatically, it should be "A empty string object", and again the term object might be implied by the context. English, being a natural language (and one with a complicated history), can be filled with all sorts of strangeness, and this can get compounded when it wants to be harmonized with a technical subject which can't deal with some of the naturalness of language, thus we can't try to use some sort of rule matching the type of article to the appropriateness of the python word "is". -- Richard Damon From rosuav at gmail.com Sun Jul 28 20:04:36 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jul 2019 10:04:36 +1000 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On Mon, Jul 29, 2019 at 9:48 AM Michael Torrie wrote: > > On 7/28/19 5:55 AM, Jonathan Moules wrote: > > But this appears to be explicitly called out as being "Worse" in PEP8: > > > > """ > > Don't compare boolean values to True or False using ==. > > > > Yes: if greeting: > > No: if greeting == True: > > Worse: if greeting is True: > > """ > > Yet the recommended solution to the problem of wanting a default > argument of an empty list is something like this: > > def foo(bar=False); > if bar is False: > bar = [] > > .... > > Clearly in this case the expression "not bar" would be incorrect. This is a fairly unusual case, though. More commonly, the default would be None, not False, and "if bar is None:" is extremely well known and idiomatic. > There's a difference between looking for a particular identity or > sentinel, and checking for truthiness or falsiness. > > So I guess it all depends on what you need to do. If you just need to > check truthiness, just do "if greeting" If you need to know if the > variable is some particular sentinel, use the "if greeting is whatever". This analysis is correct, but the situations where you *actually* want to know if something "is True" or "is False" are rare enough that PEP 8 doesn't need to mention them; whereas the case where you actually just need the truthiness is so common (and so often miswritten) that it's worth being clear on it. ChrisA From Richard at Damon-Family.org Sun Jul 28 20:11:40 2019 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 28 Jul 2019 20:11:40 -0400 Subject: Definite or indefinite article for non-singletons? In-Reply-To: <418c3ac5-ca4b-f0a1-70e9-26637e7a7280@mrabarnett.plus.com> References: <20190728123029.GA15743@cskk.homeip.net> <418c3ac5-ca4b-f0a1-70e9-26637e7a7280@mrabarnett.plus.com> Message-ID: <78be19a5-3375-5f1e-855a-691a364cb0a5@Damon-Family.org> On 7/28/19 11:13 AM, MRAB wrote: > On 2019-07-28 13:30, Cameron Simpson wrote: >> >> The collection is "the things". "all" qualifies it, versus, say, "some >> of the things" or "the first of the things" etc. >> > [snip] > > It's strange that "all the things" (meaning "all of the things") is > OK, but otherwise it's "one of the things", "some of the things", etc. > > That's English for you! My thoughts is that "of" implies membership in some category, some part, but the phrase "all the things", we aren't looking at any category, but literally ALL the things. "All of the things" implies all of the things within some group, possibly implied by context. The category "thing" is implied by the word thing, so doesn't call for the preposition. Saying "All of the things" to mean every single one of them implies creating some category that just happens to include every one of them. -- Richard Damon From Richard at Damon-Family.org Sun Jul 28 20:14:27 2019 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 28 Jul 2019 20:14:27 -0400 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 7/28/19 7:46 PM, Michael Torrie wrote: > On 7/28/19 5:55 AM, Jonathan Moules wrote: >> But this appears to be explicitly called out as being "Worse" in PEP8: >> >> """ >> Don't compare boolean values to True or False using ==. >> >> Yes:?? if greeting: >> No:??? if greeting == True: >> Worse: if greeting is True: >> """ > Yet the recommended solution to the problem of wanting a default > argument of an empty list is something like this: > > def foo(bar=False); > if bar is False: > bar = [] > > .... > > Clearly in this case the expression "not bar" would be incorrect. > > There's a difference between looking for a particular identity or > sentinel, and checking for truthiness or falsiness. > > So I guess it all depends on what you need to do. If you just need to > check truthiness, just do "if greeting" If you need to know if the > variable is some particular sentinel, use the "if greeting is whatever". I thought the recommended value was None, not False (so you don't need to ask why not True). I suppose if None is a valid value for the input you would need something else, and False is one possibility. -- Richard Damon From rosuav at gmail.com Sun Jul 28 20:15:16 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jul 2019 10:15:16 +1000 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: <87y30ibomt.fsf@elektro.pacujo.net> Message-ID: On Mon, Jul 29, 2019 at 10:06 AM Richard Damon wrote: > When talking of empty strings, we need to look a bit at context. "The > empty string" implies that there is only one of them, and if we are > talking about values, then there is only one empty string values, so > "The empty string value" would be correct (and the term value might be > implied by context). If we are talking about object, like with the > python word "is", then the empty string is not promised to be a > singleton, so grammatically, it should be "A empty string object", and > again the term object might be implied by the context. So I guess the original question can be reworded as: When you're describing indistinguishable objects, are you really talking about objects, or are you talking about values? With numbers, it's pretty obvious that you talk about values. You can logically say "if the spamminess is zero, blah blah blah" even though you'd actually say "if spam == 0:" in the code. With strings, is it therefore logical to say "is the empty string" even though you'd actually be comparing to see if it "==" another empty string? ChrisA From rosuav at gmail.com Sun Jul 28 20:25:23 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jul 2019 10:25:23 +1000 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On Mon, Jul 29, 2019 at 10:15 AM Richard Damon wrote: > > On 7/28/19 7:46 PM, Michael Torrie wrote: > > On 7/28/19 5:55 AM, Jonathan Moules wrote: > >> But this appears to be explicitly called out as being "Worse" in PEP8: > >> > >> """ > >> Don't compare boolean values to True or False using ==. > >> > >> Yes: if greeting: > >> No: if greeting == True: > >> Worse: if greeting is True: > >> """ > > Yet the recommended solution to the problem of wanting a default > > argument of an empty list is something like this: > > > > def foo(bar=False); > > if bar is False: > > bar = [] > > > > .... > > > > Clearly in this case the expression "not bar" would be incorrect. > > > > There's a difference between looking for a particular identity or > > sentinel, and checking for truthiness or falsiness. > > > > So I guess it all depends on what you need to do. If you just need to > > check truthiness, just do "if greeting" If you need to know if the > > variable is some particular sentinel, use the "if greeting is whatever". > > I thought the recommended value was None, not False (so you don't need > to ask why not True). I suppose if None is a valid value for the input > you would need something else, and False is one possibility. > In that situation, you'd be more likely to use a dedicated sentinel object: _SENTINEL = object() def foo(bar=_SENTINEL): if bar is _SENTINEL: bar = [] But there is a not-too-rare situation where you'd want to check if something "is False", and that's some form of three-valued logic, eg "true, false, and unspecified". For instance: def foo(verbose=None): if verbose is False: # caller specifically wanted quiet mode if verbose is True: # caller specifically wanted verbose mode Of course, if the third value can be simplified away (eg None means "use the global default"), then you can still just use "if verbose is None:" and then reassign it. But this is a legit reason to use "is False". ChrisA From Richard at Damon-Family.org Sun Jul 28 20:42:25 2019 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 28 Jul 2019 20:42:25 -0400 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: <3acf2dfe-ef77-1459-036b-625e2c7e582c@Damon-Family.org> On 7/28/19 8:25 PM, Chris Angelico wrote: > On Mon, Jul 29, 2019 at 10:15 AM Richard Damon wrote: >> On 7/28/19 7:46 PM, Michael Torrie wrote: >>> On 7/28/19 5:55 AM, Jonathan Moules wrote: >>>> But this appears to be explicitly called out as being "Worse" in PEP8: >>>> >>>> """ >>>> Don't compare boolean values to True or False using ==. >>>> >>>> Yes: if greeting: >>>> No: if greeting == True: >>>> Worse: if greeting is True: >>>> """ >>> Yet the recommended solution to the problem of wanting a default >>> argument of an empty list is something like this: >>> >>> def foo(bar=False); >>> if bar is False: >>> bar = [] >>> >>> .... >>> >>> Clearly in this case the expression "not bar" would be incorrect. >>> >>> There's a difference between looking for a particular identity or >>> sentinel, and checking for truthiness or falsiness. >>> >>> So I guess it all depends on what you need to do. If you just need to >>> check truthiness, just do "if greeting" If you need to know if the >>> variable is some particular sentinel, use the "if greeting is whatever". >> I thought the recommended value was None, not False (so you don't need >> to ask why not True). I suppose if None is a valid value for the input >> you would need something else, and False is one possibility. >> > In that situation, you'd be more likely to use a dedicated sentinel object: > > _SENTINEL = object() > def foo(bar=_SENTINEL): > if bar is _SENTINEL: bar = [] > > But there is a not-too-rare situation where you'd want to check if > something "is False", and that's some form of three-valued logic, eg > "true, false, and unspecified". For instance: > > def foo(verbose=None): > if verbose is False: > # caller specifically wanted quiet mode > > if verbose is True: > # caller specifically wanted verbose mode > > Of course, if the third value can be simplified away (eg None means > "use the global default"), then you can still just use "if verbose is > None:" and then reassign it. But this is a legit reason to use "is > False". > > ChrisA The more common way to handle the 3 value case in my experience is to first handle the 'special case' (check for None in your example) and then use the normal truthiness test. Then you example would be: def foo(verbose=None): ??? if verbose is None: ??????? # Didn't specify verbosity ??? elif verbose: ????? # Specified Verbose case ?? else: ????? # Specified none Verbose case Do you REALLY mean that foo(0) is to be different than foo(false)? -- Richard Damon From rosuav at gmail.com Sun Jul 28 20:46:10 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jul 2019 10:46:10 +1000 Subject: Boolean comparison & PEP8 In-Reply-To: <3acf2dfe-ef77-1459-036b-625e2c7e582c@Damon-Family.org> References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> <3acf2dfe-ef77-1459-036b-625e2c7e582c@Damon-Family.org> Message-ID: On Mon, Jul 29, 2019 at 10:43 AM Richard Damon wrote: > > On 7/28/19 8:25 PM, Chris Angelico wrote: > > Of course, if the third value can be simplified away (eg None means > > "use the global default"), then you can still just use "if verbose is > > None:" and then reassign it. But this is a legit reason to use "is > > False". > > > > ChrisA > > The more common way to handle the 3 value case in my experience is to > first handle the 'special case' (check for None in your example) and > then use the normal truthiness test. > That works fine if you can actually handle the special case first and dispose of it, but that doesn't always work, and when it doesn't, you may indeed want to specify "is False" or "is True". Chris From grant.b.edwards at gmail.com Sun Jul 28 21:17:23 2019 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 29 Jul 2019 01:17:23 -0000 (UTC) Subject: Boolean comparison & PEP8 References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 2019-07-29, Richard Damon wrote: > On 7/28/19 7:46 PM, Michael Torrie wrote: >> Yet the recommended solution to the problem of wanting a default >> argument of an empty list is something like this: >> >> def foo(bar=False); >> if bar is False: >> bar = [] >> >> .... > > I thought the recommended value was None, not False (so you don't need > to ask why not True). I suppose if None is a valid value for the input > you would need something else, and False is one possibility. Yes 'None' is definitely the normal idiom, and everybody will know what you're doing. If you use False, people are going to be confused. def foo(bar=None) if bar is None bar = [] ... From Richard at Damon-Family.org Sun Jul 28 21:57:07 2019 From: Richard at Damon-Family.org (Richard Damon) Date: Sun, 28 Jul 2019 21:57:07 -0400 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> <3acf2dfe-ef77-1459-036b-625e2c7e582c@Damon-Family.org> Message-ID: <9fec2c84-f62b-1246-2eb5-d354295a6c6c@Damon-Family.org> On 7/28/19 8:46 PM, Chris Angelico wrote: > On Mon, Jul 29, 2019 at 10:43 AM Richard Damon wrote: >> On 7/28/19 8:25 PM, Chris Angelico wrote: >>> Of course, if the third value can be simplified away (eg None means >>> "use the global default"), then you can still just use "if verbose is >>> None:" and then reassign it. But this is a legit reason to use "is >>> False". >>> >>> ChrisA >> The more common way to handle the 3 value case in my experience is to >> first handle the 'special case' (check for None in your example) and >> then use the normal truthiness test. >> > That works fine if you can actually handle the special case first and > dispose of it, but that doesn't always work, and when it doesn't, you > may indeed want to specify "is False" or "is True". > > Chris If you can't handle it first (or really at the same time) def foo(verbose=None): ??? if verbose is not None: ??? if verbose: # Specified Verbose case ?? else: ????? # Specified none Verbose case which is really the same as before if you left the Not Specified case empty. Again, the question is do really need to force the giving of a bool, or can you deal with a truthy/falsey/None value? (which may just require a bool call at the call site, with some code to possibly handle the None case. -- Richard Damon From torriem at gmail.com Sun Jul 28 22:01:38 2019 From: torriem at gmail.com (Michael Torrie) Date: Sun, 28 Jul 2019 20:01:38 -0600 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 7/28/19 6:04 PM, Chris Angelico wrote: > This is a fairly unusual case, though. More commonly, the default > would be None, not False, and "if bar is None:" is extremely well > known and idiomatic. Ahh yes, true. > This analysis is correct, but the situations where you *actually* want > to know if something "is True" or "is False" are rare enough that PEP > 8 doesn't need to mention them; whereas the case where you actually > just need the truthiness is so common (and so often miswritten) that > it's worth being clear on it. It may well be that a distinct sentinel object is preferred to True or False in the majority of cases. From tjreedy at udel.edu Sun Jul 28 22:36:42 2019 From: tjreedy at udel.edu (Terry Reedy) Date: Sun, 28 Jul 2019 22:36:42 -0400 Subject: Boolean comparison & PEP8 In-Reply-To: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 7/28/2019 7:55 AM, Jonathan Moules wrote: > Lets say I want to know if the value of `x` is bool(True). > My preferred way to do it is: > > if x is True: > ??? pass If you know that expression x is boolean, and one usually knows or should know whether is it or is not, '= True' and 'is True' and similarly for False are redundant. Why not 'if x is True is True' and so on. > Because this tests both the value and the type. See below. Newbies *have* written things like 'if (x == 3) is True' and this is what prompted this entry in PEP 8. > """ > Don't compare boolean values to True or False using ==. > > Yes:?? if greeting: > No:??? if greeting == True: > Worse: if greeting is True: > """ > If `x` can also have a value of "1"(str) or 1(int) Have you run across any stdlib function that returns such a mixture of types? Remember that PEP 8 is specifically a style guide for stdlib code. -- Terry Jan Reedy From cs at cskk.id.au Sun Jul 28 22:54:09 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Mon, 29 Jul 2019 12:54:09 +1000 Subject: Definite or indefinite article for non-singletons? In-Reply-To: <78be19a5-3375-5f1e-855a-691a364cb0a5@Damon-Family.org> References: <78be19a5-3375-5f1e-855a-691a364cb0a5@Damon-Family.org> Message-ID: <20190729025409.GA57318@cskk.homeip.net> On 28Jul2019 20:11, Richard Damon wrote: >On 7/28/19 11:13 AM, MRAB wrote: >> On 2019-07-28 13:30, Cameron Simpson wrote: >>> The collection is "the things". "all" qualifies it, versus, say, >>> "some of the things" or "the first of the things" etc. >>> >> [snip] >> It's strange that "all the things" (meaning "all of the things") is >> OK, but otherwise it's "one of the things", "some of the things", etc. >> >> That's English for you! Well, we also say "one thing", synonymous with "a thing" and "some things". Also "some thing" synonymous with "a thing" and "something" meaning an arbitrary context dependent thing, but not necessarily one of the "thing"s we might otherwise have been talking about. On reflection, "the things" is a definite group, and "some of the things" or "one of the things" is a subset/member of that group. Whereas "something" or "some things" is usually vaguer, where you don't have a defined group of things but your are talking about things in a context. >My thoughts is that "of" implies membership in some category, some part, >but the phrase "all the things", we aren't looking at any category, but >literally ALL the things. "All of the things" implies all of the things >within some group, possibly implied by context. The category "thing" is >implied by the word thing, so doesn't call for the preposition. Saying >"All of the things" to mean every single one of them implies creating >some category that just happens to include every one of them. I think I'm saying what you're saying here. Cheers, Cameron Simpson From cs at cskk.id.au Sun Jul 28 22:59:30 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Mon, 29 Jul 2019 12:59:30 +1000 Subject: Definite or indefinite article for non-singletons? In-Reply-To: References: Message-ID: <20190729025930.GA73820@cskk.homeip.net> On 29Jul2019 10:15, Chris Angelico wrote: >On Mon, Jul 29, 2019 at 10:06 AM Richard Damon wrote: >> When talking of empty strings, we need to look a bit at context. "The >> empty string" implies that there is only one of them, and if we are >> talking about values, then there is only one empty string values, so >> "The empty string value" would be correct (and the term value might be >> implied by context). If we are talking about object, like with the >> python word "is", then the empty string is not promised to be a >> singleton, so grammatically, it should be "A empty string object", and >> again the term object might be implied by the context. > >So I guess the original question can be reworded as: > >When you're describing indistinguishable objects, are you really >talking about objects, or are you talking about values? > >With numbers, it's pretty obvious that you talk about values. You can >logically say "if the spamminess is zero, blah blah blah" even though >you'd actually say "if spam == 0:" in the code. With strings, is it >therefore logical to say "is the empty string" even though you'd >actually be comparing to see if it "==" another empty string? I wouldn't be saying "the empty string". I'd being describing the string: "if the string is empty", "if s is an empty string", etc. So values again. I think the wish to use the phrasing "the empty string" either comes from an implied metaphorthat assumes deduped immutable objects or from a mathematical background where eg in set theory one talks about "the empty set". Someone else posted a better description of the methematical context, where "the empty set" is effectively naming a category of sets (those of size 0) and considering that a single thing in that context. I'm for running with "values" in the example you started with. I think the "the empty string" is mathematical context specific terminology leaking into the wrong domain. Cheers, Cameron Simpson From robin at reportlab.com Mon Jul 29 06:48:30 2019 From: robin at reportlab.com (Robin Becker) Date: Mon, 29 Jul 2019 11:48:30 +0100 Subject: ubuntu python 2.7 variations Message-ID: <19173259-332c-ef7d-d722-d0bcdc340a32@everest.reportlab.co.uk> I help maintain apps on a number of machines and recently a bug surfaced which affected some of them using the preppy template module; the bug ended up being the way tokenize worked as in the following ############################# from StringIO import StringIO import tokenize, token L = [] s='i' tokenize.tokenize(StringIO(s.strip()).readline,lambda *a: L.append(a)) print('L=%s' % repr(L)) ############################# Older working machines Python 2.7.15rc1 (default, Nov 12 2018, 14:31:15)\n[GCC 7.3.0] on linux2 L=[(1, 'i', (1, 0), (1, 1), 'i'), (0, '', (2, 0), (2, 0), '')] upgraded machines showing a bug Python 2.7.15+ (default, Nov 27 2018, 23:36:35)\n[GCC 7.3.0] on linux2 L=[(1, 'i', (1, 0), (1, 1), 'i'), (4, '', (1, 1), (1, 2), ''), (0, '', (2, 0), (2, 0), '')] The extra token represents a linefeed; however StringIO('i').readline() doesn't contain a linefeed. so the problem is that parsing a simple string 'i' results in two different parses. On my Arch machines running 2.7.16 I see the original parse. The fix was simple and involved using more robust code. My question is that since the package builders feel able to modify and affect behaviour in such a simple case will they do it randomly elsewhere perhaps leading to other less obvious changes. Should I always be using self build python versions? It seems that ubuntu feels able to provide packages which are rc versions or have a + indicating they're modified. They'll probably argue that this improves things and I shouldn't be using such low level code ....... :( -- Robin Becker From rosuav at gmail.com Mon Jul 29 07:33:05 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 29 Jul 2019 21:33:05 +1000 Subject: ubuntu python 2.7 variations In-Reply-To: <19173259-332c-ef7d-d722-d0bcdc340a32@everest.reportlab.co.uk> References: <19173259-332c-ef7d-d722-d0bcdc340a32@everest.reportlab.co.uk> Message-ID: On Mon, Jul 29, 2019 at 8:50 PM Robin Becker wrote: > My question is that since the package builders feel able to modify and affect > behaviour in such a simple case will they do it randomly elsewhere perhaps > leading to other less obvious changes. > > Should I always be using self build python versions? If you want to maintain your own Python, then by all means, go ahead. I don't maintain my own Python 2.7, but I have a number of Python 3.x builds, since Debian Stretch doesn't ship with anything newer than 3.5. > It seems that ubuntu feels able to provide packages which are rc versions or > have a + indicating they're modified. They'll probably argue that this improves > things and I shouldn't be using such low level code ....... :( The plus sign does indeed mean that it's modified, but often that just means they backported some (but not all) of the changes in newer 2.7.x builds. (Which, at the moment, would only be 2.7.16.) I can't speak specifically about tokenize, but if you're using it for anything that isn't actually Python code, you're vulnerable to this kind of change. I wouldn't normally expect it in a point release, though. ChrisA From David.Raymond at tomtom.com Mon Jul 29 10:02:46 2019 From: David.Raymond at tomtom.com (David Raymond) Date: Mon, 29 Jul 2019 14:02:46 +0000 Subject: Boolean comparison & PEP8 In-Reply-To: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: I think the other part of the discussion to be had here is: how do you name your booleans? As with other things, "x", and "greeting" aren't really the best names. "greeting" sounds like it should hold what the greeting _is_, not whether or not there _should be_ a greeting. If it were something like "greetingRequired" or "greetingRequested" then it gets to be more readable as to what you're storing and looking for with the variable. if greeting: something... vs. if greetingRequired: something... I think the "greetingRequired" version is much more readable. To me the name of a boolean variable should be obvious that it's a boolean, be it as a statement or as an instruction. Looking at the arguments for subprocess.popen for example, lets put some of the boolean arguments behind an if and see how readable they are: if start_new_session: #instruction, I need to "start (a) new session" if close_fds: #instruction, I need to "close file descriptors" if restore_signals: #instruction, I need to "restore signals" if shell: #wait, "shell" is not really a statement or an instruction. "If shell"... what? "I need to shell"? Is this whether we want to use a shell, or if we discovered that we're already in one? Or is "shell" not really a boolean, and is holding which shell we want, and we're just using the "truthyness" to make sure it's not None? What's going on here? Dangit, ok, where's the documentation? (subprocess is common enough that people know this, but imagine a much less frequently used module/function) Do I have a point or am I just smoking crack? Does anyone else have examples? -----Original Message----- From: Python-list On Behalf Of Jonathan Moules Sent: Sunday, July 28, 2019 7:56 AM To: python-list at python.org Subject: Boolean comparison & PEP8 Hi List, Lets say I want to know if the value of `x` is bool(True). My preferred way to do it is: if x is True: ??? pass Because this tests both the value and the type. But this appears to be explicitly called out as being "Worse" in PEP8: """ Don't compare boolean values to True or False using ==. Yes:?? if greeting: No:??? if greeting == True: Worse: if greeting is True: """ Why? If `x` can also have a value of "1"(str) or 1(int) then in both cases this would be a false positive if I were to do the below as they'll both equate to True: if x: ??? pass The PEP for boolean type (285 - https://www.python.org/dev/peps/pep-0285/) doesn't mention the "is" comparison keyword at all as far as I can tell. What am I missing? Thanks -- https://mail.python.org/mailman/listinfo/python-list From robin at reportlab.com Mon Jul 29 10:11:38 2019 From: robin at reportlab.com (Robin Becker) Date: Mon, 29 Jul 2019 15:11:38 +0100 Subject: ubuntu python 2.7 variations In-Reply-To: References: <19173259-332c-ef7d-d722-d0bcdc340a32@everest.reportlab.co.uk> Message-ID: On 29/07/2019 12:33, Chris Angelico wrote: ......... >> >> Should I always be using self build python versions? > > If you want to maintain your own Python, then by all means, go ahead. > I don't maintain my own Python 2.7, but I have a number of Python 3.x > builds, since Debian Stretch doesn't ship with anything newer than > 3.5. > I have all the pythons except 2.7 as user builds > > The plus sign does indeed mean that it's modified, but often that just > means they backported some (but not all) of the changes in newer 2.7.x > builds. (Which, at the moment, would only be 2.7.16.) > I have just tried a standard make configure dance with 2.7.16 and that python seems to behave like the original python 2.7.15rc1 version. So it seems the current system 2.7.15+ is patched in some way differently. I can't simply switch pythons as I used the system python to create virtual environments and all of those need rebuilding. > I can't speak specifically about tokenize, but if you're using it for > anything that isn't actually Python code, you're vulnerable to this > kind of change. I wouldn't normally expect it in a point release, > though. well preppy is using the ast/tokenizer etc etc to put a lot of python into a template form with {{ }} expressions etc etc > > ChrisA > From robin at reportlab.com Mon Jul 29 10:11:38 2019 From: robin at reportlab.com (Robin Becker) Date: Mon, 29 Jul 2019 15:11:38 +0100 Subject: ubuntu python 2.7 variations In-Reply-To: References: <19173259-332c-ef7d-d722-d0bcdc340a32@everest.reportlab.co.uk> Message-ID: On 29/07/2019 12:33, Chris Angelico wrote: ......... >> >> Should I always be using self build python versions? > > If you want to maintain your own Python, then by all means, go ahead. > I don't maintain my own Python 2.7, but I have a number of Python 3.x > builds, since Debian Stretch doesn't ship with anything newer than > 3.5. > I have all the pythons except 2.7 as user builds > > The plus sign does indeed mean that it's modified, but often that just > means they backported some (but not all) of the changes in newer 2.7.x > builds. (Which, at the moment, would only be 2.7.16.) > I have just tried a standard make configure dance with 2.7.16 and that python seems to behave like the original python 2.7.15rc1 version. So it seems the current system 2.7.15+ is patched in some way differently. I can't simply switch pythons as I used the system python to create virtual environments and all of those need rebuilding. > I can't speak specifically about tokenize, but if you're using it for > anything that isn't actually Python code, you're vulnerable to this > kind of change. I wouldn't normally expect it in a point release, > though. well preppy is using the ast/tokenizer etc etc to put a lot of python into a template form with {{ }} expressions etc etc > > ChrisA > From grant.b.edwards at gmail.com Mon Jul 29 10:16:30 2019 From: grant.b.edwards at gmail.com (Grant Edwards) Date: Mon, 29 Jul 2019 14:16:30 -0000 (UTC) Subject: Boolean comparison & PEP8 References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 2019-07-29, Michael Torrie wrote: > On 7/28/19 6:04 PM, Chris Angelico wrote: >> This is a fairly unusual case, though. More commonly, the default >> would be None, not False, and "if bar is None:" is extremely well >> known and idiomatic. > > Ahh yes, true. No... None. ;) -- Grant Edwards grant.b.edwards Yow! -- I love KATRINKA at because she drives a gmail.com PONTIAC. We're going away now. I fed the cat. From 2QdxY4RzWzUUiLuE at potatochowder.com Mon Jul 29 10:41:12 2019 From: 2QdxY4RzWzUUiLuE at potatochowder.com (Dan Sommers) Date: Mon, 29 Jul 2019 10:41:12 -0400 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: <388aa176-c731-2214-6606-fa246f8c4fda@potatochowder.com> On 7/29/19 10:02 AM, David Raymond wrote: > I think the other part of the discussion to be had here is: how do you > name your booleans? Yep. > ... To me the name of a boolean variable should be obvious that it's a > boolean ... Well, yeah, maybe. If it's really only a boolean, and its value is always either True or False, then I agree. > if shell: > #wait, "shell" is not really a statement or an instruction. "If > shell"... what? "I need to shell"? Is this whether we want to use a > shell, or if we discovered that we're already in one? Or is "shell" > not really a boolean, and is holding which shell we want, and we're > just using the "truthyness" to make sure it's not None? What's going > on here? Dangit, ok, where's the documentation? (subprocess is common > enough that people know this, but imagine a much less frequently used > module/function) > Do I have a point or am I just smoking crack? Does anyone else have > examples? Consider an algorithm like this to determine which shell to use: shell = (get_shell_from_command_line() or get_shell_from_project_preferences() or get_shell_from_personal_preferences() or get_shell_from_system_preferences()) where each get_shell_from_XXX() function returns a /path/to/shell string or None. At least to me, Python's "or" operator and its notion of truthiness make this algorithm natural. Perhaps a better name is path_to_shell, but I'd still end up with: if path_to_shell: execute_program(path_to_shell, ...) else: raise NoShellException() Dan From rosuav at gmail.com Mon Jul 29 10:50:40 2019 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Jul 2019 00:50:40 +1000 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On Tue, Jul 30, 2019 at 12:04 AM David Raymond wrote: > if shell: > #wait, "shell" is not really a statement or an instruction. "If shell"... what? "I need to shell"? Is this whether we want to use a shell, or if we discovered that we're already in one? Or is "shell" not really a boolean, and is holding which shell we want, and we're just using the "truthyness" to make sure it's not None? What's going on here? Dangit, ok, where's the documentation? > (subprocess is common enough that people know this, but imagine a much less frequently used module/function) > I suppose "use_shell" would be clearer, but a lot of verbs become implicit with the right choice of noun, as some nouns can become implicit with the right verb. ChrisA From python at mrabarnett.plus.com Mon Jul 29 12:01:19 2019 From: python at mrabarnett.plus.com (MRAB) Date: Mon, 29 Jul 2019 17:01:19 +0100 Subject: Definite or indefinite article for non-singletons? In-Reply-To: <20190729025930.GA73820@cskk.homeip.net> References: <20190729025930.GA73820@cskk.homeip.net> Message-ID: On 2019-07-29 03:59, Cameron Simpson wrote: > On 29Jul2019 10:15, Chris Angelico wrote: >>On Mon, Jul 29, 2019 at 10:06 AM Richard Damon wrote: >>> When talking of empty strings, we need to look a bit at context. "The >>> empty string" implies that there is only one of them, and if we are >>> talking about values, then there is only one empty string values, so >>> "The empty string value" would be correct (and the term value might be >>> implied by context). If we are talking about object, like with the >>> python word "is", then the empty string is not promised to be a >>> singleton, so grammatically, it should be "A empty string object", and >>> again the term object might be implied by the context. >> >>So I guess the original question can be reworded as: >> >>When you're describing indistinguishable objects, are you really >>talking about objects, or are you talking about values? >> >>With numbers, it's pretty obvious that you talk about values. You can >>logically say "if the spamminess is zero, blah blah blah" even though >>you'd actually say "if spam == 0:" in the code. With strings, is it >>therefore logical to say "is the empty string" even though you'd >>actually be comparing to see if it "==" another empty string? > > I wouldn't be saying "the empty string". I'd being describing the > string: "if the string is empty", "if s is an empty string", etc. So > values again. > > I think the wish to use the phrasing "the empty string" either comes > from an implied metaphorthat assumes deduped immutable objects or from a > mathematical background where eg in set theory one talks about "the > empty set". Someone else posted a better description of the methematical > context, where "the empty set" is effectively naming a category of sets > (those of size 0) and considering that a single thing in that context. > > I'm for running with "values" in the example you started with. I think > the "the empty string" is mathematical context specific terminology > leaking into the wrong domain. > There's also the use of the definite article when referring to the members of a class as whole, e.g. "the elephant is a large mammal". From michael.stemper at gmail.com Mon Jul 29 13:44:40 2019 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 29 Jul 2019 12:44:40 -0500 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 28/07/2019 19.04, Chris Angelico wrote: > On Mon, Jul 29, 2019 at 9:48 AM Michael Torrie wrote: >> >> On 7/28/19 5:55 AM, Jonathan Moules wrote: >>> But this appears to be explicitly called out as being "Worse" in PEP8: >>> >>> """ >>> Don't compare boolean values to True or False using ==. >>> >>> Yes: if greeting: >>> No: if greeting == True: >>> Worse: if greeting is True: >>> """ >> >> Yet the recommended solution to the problem of wanting a default >> argument of an empty list is something like this: >> >> def foo(bar=False); >> if bar is False: >> bar = [] >> >> .... >> >> Clearly in this case the expression "not bar" would be incorrect. > > This is a fairly unusual case, though. More commonly, the default > would be None, not False, and "if bar is None:" is extremely well > known and idiomatic. That's certainly how I would have done it until I read your post. But reading it immediately raised the question of why not: def foo( bar=[] ): if len(bar)==0: print( "Pretty short" ) else: print( bar ) return Seems to work just fine. -- Michael F. Stemper Deuteronomy 24:17 From rosuav at gmail.com Mon Jul 29 13:58:15 2019 From: rosuav at gmail.com (Chris Angelico) Date: Tue, 30 Jul 2019 03:58:15 +1000 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On Tue, Jul 30, 2019 at 3:46 AM Michael F. Stemper wrote: > > On 28/07/2019 19.04, Chris Angelico wrote: > > On Mon, Jul 29, 2019 at 9:48 AM Michael Torrie wrote: > >> > >> On 7/28/19 5:55 AM, Jonathan Moules wrote: > >>> But this appears to be explicitly called out as being "Worse" in PEP8: > >>> > >>> """ > >>> Don't compare boolean values to True or False using ==. > >>> > >>> Yes: if greeting: > >>> No: if greeting == True: > >>> Worse: if greeting is True: > >>> """ > >> > >> Yet the recommended solution to the problem of wanting a default > >> argument of an empty list is something like this: > >> > >> def foo(bar=False); > >> if bar is False: > >> bar = [] > >> > >> .... > >> > >> Clearly in this case the expression "not bar" would be incorrect. > > > > This is a fairly unusual case, though. More commonly, the default > > would be None, not False, and "if bar is None:" is extremely well > > known and idiomatic. > > That's certainly how I would have done it until I read your post. But > reading it immediately raised the question of why not: > > def foo( bar=[] ): > if len(bar)==0: > print( "Pretty short" ) > else: > print( bar ) > return > > Seems to work just fine. > If you're not mutating it, then fine. But if you're going to append to the list, this is falling into the classic trap of mutable default arguments. ChrisA From rgaddi at highlandtechnology.invalid Mon Jul 29 13:56:21 2019 From: rgaddi at highlandtechnology.invalid (Rob Gaddi) Date: Mon, 29 Jul 2019 10:56:21 -0700 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 7/29/19 10:44 AM, Michael F. Stemper wrote: > On 28/07/2019 19.04, Chris Angelico wrote: >> On Mon, Jul 29, 2019 at 9:48 AM Michael Torrie wrote: >>> >>> On 7/28/19 5:55 AM, Jonathan Moules wrote: >>>> But this appears to be explicitly called out as being "Worse" in PEP8: >>>> >>>> """ >>>> Don't compare boolean values to True or False using ==. >>>> >>>> Yes: if greeting: >>>> No: if greeting == True: >>>> Worse: if greeting is True: >>>> """ >>> >>> Yet the recommended solution to the problem of wanting a default >>> argument of an empty list is something like this: >>> >>> def foo(bar=False); >>> if bar is False: >>> bar = [] >>> >>> .... >>> >>> Clearly in this case the expression "not bar" would be incorrect. >> >> This is a fairly unusual case, though. More commonly, the default >> would be None, not False, and "if bar is None:" is extremely well >> known and idiomatic. > > That's certainly how I would have done it until I read your post. But > reading it immediately raised the question of why not: > > def foo( bar=[] ): > if len(bar)==0: > print( "Pretty short" ) > else: > print( bar ) > return > > Seems to work just fine. > > Works find right up until you do anything that modifies bar, and find that bar always points not to a new empty list each time but to the same empty list on each call. >>> def foo(bar=[]): ... bar.append(5) ... return bar ... >>> foo() [5] >>> foo() [5, 5] >>> foo() [5, 5, 5] As far as ways to shoot one's own foot in Python, this is one of the most common. -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. From michael.stemper at gmail.com Mon Jul 29 15:10:31 2019 From: michael.stemper at gmail.com (Michael F. Stemper) Date: Mon, 29 Jul 2019 14:10:31 -0500 Subject: Boolean comparison & PEP8 In-Reply-To: References: <04ba8e4f-3fa4-64fb-b0af-591047c6e578@lightpear.com> Message-ID: On 29/07/2019 12.56, Rob Gaddi wrote: > On 7/29/19 10:44 AM, Michael F. Stemper wrote: >> On 28/07/2019 19.04, Chris Angelico wrote: >>> On Mon, Jul 29, 2019 at 9:48 AM Michael Torrie >>> wrote: >>>> Yet the recommended solution to the problem of wanting a default >>>> argument of an empty list is something like this: >>>> >>>> def foo(bar=False); >>>> ???? if bar is False: >>>> ???????? bar = [] >>>> >>>> ???? .... >>>> >>>> Clearly in this case the expression "not bar" would be incorrect. >>> >>> This is a fairly unusual case, though. More commonly, the default >>> would be None, not False, and "if bar is None:" is extremely well >>> known and idiomatic. >> >> That's certainly how I would have done it until I read your post. But >> reading it immediately raised the question of why not: >> >> ? def foo( bar=[] ): >> ??? if len(bar)==0: >> ????? print( "Pretty short" ) >> ??? else: >> ????? print( bar ) >> ??? return > Works fine right up until you do anything that modifies bar, and find > that bar always points not to a new empty list each time but to the same > empty list on each call. > >>>> def foo(bar=[]): > ...???? bar.append(5) > ...???? return bar > ... >>>> foo() > [5] >>>> foo() > [5, 5] >>>> foo() > [5, 5, 5] > > As far as ways to shoot one's own foot in Python, this is one of the > most common. I knew there had to be a good reason. -- Michael F. Stemper Always use apostrophe's and "quotation marks" properly. From harry.grey1694 at gmail.com Mon Jul 29 17:06:51 2019 From: harry.grey1694 at gmail.com (Harry Grey) Date: Mon, 29 Jul 2019 23:06:51 +0200 Subject: .python_history file Message-ID: <1564434411.2556.0.camel@gmail.com> Hy to Everyone first of all : Sorry for my english what do you think about introducing a feature that allows you to group by date and after to filter the istruction that are written in the .python_history file: maybe with a new structure for .python_history:? """ ###DATE-OF-SOME-DAY-START### ###DATE-OF-SOME-DAY-END### """ and maybe developing a new tool, or a core functionality of python interpreter callable as an argument in the cmd, that allow you to parse the .python_history file in given date: With a tool: PhYstory -d 16 -m 03 -y 2019 or like python Functionality: python -hist 16-03-2019/17-03-2019 and why not? why not have a the chance to enable and disable, at will, writing in the .python_history: "" >>> skip_hist >>> >>> # Make some unusefull instruction >>> # Ignored in the history >>> class TryClass: >>> # Some Code >>> >>> >>> tc = TryClass() >>> keep_hist >>> # make some interesting code >>> # and keep it in the history file. """ Thanks DG. From lukasz at langa.pl Mon Jul 29 17:48:44 2019 From: lukasz at langa.pl (=?utf-8?Q?=C5=81ukasz_Langa?=) Date: Mon, 29 Jul 2019 23:48:44 +0200 Subject: [RELEASE] Python 3.8.0b3 is now available for testing Message-ID: <767F3537-ADCC-4F30-A9CF-9B0ECF5F99B6@langa.pl> This time without delays, I present you Python 3.8.0b3: https://www.python.org/downloads/release/python-380b3/ This release is the third of four planned beta release previews. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. The next pre-release of Python 3.8 will be 3.8.0b4, the last beta release, currently scheduled for 2019-08-26. Call to action We strongly encourage maintainers of third-party Python projects to test with 3.8 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2019-09-30). Our goal is have no ABI changes after beta 3 and no code changes after 3.8.0rc1, the release candidate. To achieve that, it will be extremely important to get as much exposure for 3.8 as possible during the beta phase. Please keep in mind that this is a preview release and its use is not recommended for production environments. Last beta coming Beta 4 can only be released if all ?Release blocker? and ?Deferred blocker? issues on bugs.python.org for 3.8.0 are resolved. Please prioritize those for the next four weeks. Acknowledgements Thanks to our binary builders, Ned and Steve, who were very quick today to get the macOS and Windows installers ready. The Windows story in particular got pretty magical, it?s now really fully automatic end-to-end. Thanks to Victor for vastly improving the reliability of multiprocessing tests since Beta 2. Thanks to Pablo for keeping the buildbots green. - ? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From tjol at tjol.eu Mon Jul 29 19:18:32 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 30 Jul 2019 01:18:32 +0200 Subject: .python_history file In-Reply-To: <1564434411.2556.0.camel@gmail.com> References: <1564434411.2556.0.camel@gmail.com> Message-ID: <76c288be-233b-e0a4-2bab-2b4453cc9930@tjol.eu> On 29/07/2019 23:06, Harry Grey wrote: > Hy to Everyone > first of all : Sorry for my english > > what do you think about introducing a feature that allows you to group > by date > and after to filter the istruction that are written in the > .python_history file: > maybe with a new structure for .python_history:? > > """ > ###DATE-OF-SOME-DAY-START### > > > ###DATE-OF-SOME-DAY-END### > > """ > > and maybe developing a new tool, or a core functionality of python > interpreter callable as an argument in the cmd, that allow you to parse > the .python_history file in given date: > > With a tool: > > PhYstory -d 16 -m 03 -y 2019 Hi! IMHO, this is a rather niche use case. And I suppose you could relatively easily implement it yourself with a site hook that adds magic comments to ~/.pyhistory as you suggest. If you want this functionality, there's no need for it to be part of Python itself. FWIW, IPython, as far as I can tell, already saves some timestamp information in its history file. -- Thomas > > or like python Functionality: > > python -hist 16-03-2019/17-03-2019 > > and why not? why not have a the chance to enable and disable, at will, > writing in the .python_history: > > "" > >>> skip_hist > >>> > >>> # Make some unusefull instruction > >>> # Ignored in the history > >>> class TryClass: > >>> # Some Code > >>> > >>> > >>> tc = TryClass() > >>> keep_hist > >>> # make some interesting code > >>> # and keep it in the history file. > """ > > Thanks > > DG. > From barry at python.org Mon Jul 29 19:42:03 2019 From: barry at python.org (Barry Warsaw) Date: Mon, 29 Jul 2019 16:42:03 -0700 Subject: [python-committers] [RELEASE] Python 3.8.0b3 is now available for testing In-Reply-To: <767F3537-ADCC-4F30-A9CF-9B0ECF5F99B6@langa.pl> References: <767F3537-ADCC-4F30-A9CF-9B0ECF5F99B6@langa.pl> Message-ID: <51A59053-973C-41DD-9EF1-7FEF7587D9B2@python.org> I have updated the official docker images with 3.8b3: https://gitlab.com/python-devs/ci-images/tree/master -Barry > On Jul 29, 2019, at 14:48, ?ukasz Langa wrote: > > Signed PGP part > This time without delays, I present you Python 3.8.0b3: > > https://www.python.org/downloads/release/python-380b3/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From dmitrey15 at gmail.com Tue Jul 30 04:06:34 2019 From: dmitrey15 at gmail.com (dmitrey15 at gmail.com) Date: Tue, 30 Jul 2019 01:06:34 -0700 (PDT) Subject: how to generate a standard email form by default OS mail client? Message-ID: <1ee0799e-c7a2-495e-aa16-53f4a6f61c84@googlegroups.com> Hello, is Python capable of generating a email form (from standard data - address, topic, string message) with raising it by a default OS email client, e.g. Thunderbird? User would like to have a possibility to review/modify email content inside the OS email client window before sending it. Raising gmail.com GUI form also could be a solution. Thank you in advance, D. From tjol at tjol.eu Tue Jul 30 06:31:00 2019 From: tjol at tjol.eu (Thomas Jollans) Date: Tue, 30 Jul 2019 12:31:00 +0200 Subject: how to generate a standard email form by default OS mail client? In-Reply-To: <1ee0799e-c7a2-495e-aa16-53f4a6f61c84@googlegroups.com> References: <1ee0799e-c7a2-495e-aa16-53f4a6f61c84@googlegroups.com> Message-ID: <111b8320-7832-4e7e-7127-507731a20c9e@tjol.eu> On 30/07/2019 10.06, dmitrey15 at gmail.com wrote: > Hello, > > is Python capable of generating a email form (from standard data - address, topic, string message) with raising it by a default OS email client, e.g. Thunderbird? User would like to have a possibility to review/modify email content inside the OS email client window before sending it. > > Raising gmail.com GUI form also could be a solution. > > Thank you in advance, D. You could construct a mailto URI https://en.wikipedia.org/wiki/Mailto >>>> subject = 'This is the subject' >>> body = 'This is the text' >>> to = 'god at heaven.example.com' >>> cc = 'friedrich.nietzsche at unibas.ch' >> mailto_url = urllib.parse.quote(f'mailto:{to}?') + urllib.parse.urlencode({'cc': cc, 'subject': subject, 'body': body}, quote_via=urllib.parse.quote) If you open that URI with the webbrowser module, it should work. Well, it might. On my system, Chrome refuses to open a mailto URI like this, but Firefox plays along. The better option would be to call the mail program directly (such as using the subprocess module), but how you find out what to call will depend on your OS. If this is just for one PC and you use Thunderbird, then you might as well hard-code the Thunderbird executable, of course... From dipangi.shah at gmail.com Tue Jul 30 03:20:05 2019 From: dipangi.shah at gmail.com (Dipangi Shah) Date: Tue, 30 Jul 2019 16:20:05 +0900 Subject: Xml File Error Message-ID: Hi, import os from xml.etree import ElementTree file_name = 'Users.xml' full_file = os.path.abspath(os.path.join('data', file_name)) print(full_file) with above code, path is successfully printed as "C:\Users\Evosys\PycharmProjects\Python Level1\data\Users.xml" but when I add below 2 lines, there is an error saying "FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Evosys\\PycharmProjects\\Python Level1\\data\\Users.xml' dom = ElementTree.parse(full_file,None) print(dom) Can you suggest to me why I am facing the error? -- Regards, Dipangi Shah From David.Raymond at tomtom.com Tue Jul 30 10:48:05 2019 From: David.Raymond at tomtom.com (David Raymond) Date: Tue, 30 Jul 2019 14:48:05 +0000 Subject: Xml File Error In-Reply-To: References: Message-ID: Is that the correct path to the file, without any typos? os.path.abspath and os.path.join don't do any checking on whether their resulting path exists. So if there is a typo or error in your path it doesn't get reported until you actually try and open it by running ElementTree.parse You can check if your path is ok using either os.path.exists() or os.path.isfile() before trying to actually open it to see if it is already there. -----Original Message----- From: Python-list On Behalf Of Dipangi Shah Sent: Tuesday, July 30, 2019 3:20 AM To: python-list at python.org Subject: Xml File Error Hi, import os from xml.etree import ElementTree file_name = 'Users.xml' full_file = os.path.abspath(os.path.join('data', file_name)) print(full_file) with above code, path is successfully printed as "C:\Users\Evosys\PycharmProjects\Python Level1\data\Users.xml" but when I add below 2 lines, there is an error saying "FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\Evosys\\PycharmProjects\\Python Level1\\data\\Users.xml' dom = ElementTree.parse(full_file,None) print(dom) Can you suggest to me why I am facing the error? -- Regards, Dipangi Shah -- https://mail.python.org/mailman/listinfo/python-list From couldbeworse01 at protonmail.com Tue Jul 30 13:20:24 2019 From: couldbeworse01 at protonmail.com (couldbeworse01 at protonmail.com) Date: Tue, 30 Jul 2019 17:20:24 +0000 Subject: Modify setup Message-ID: Good morning, I have uninstalled and reinstalled different versions of Python, and continue to get the "modify setup" box when I try to open. Is there something I'm doing wrong? Sent from ProtonMail mobile From python at mrabarnett.plus.com Tue Jul 30 14:02:18 2019 From: python at mrabarnett.plus.com (MRAB) Date: Tue, 30 Jul 2019 19:02:18 +0100 Subject: Modify setup In-Reply-To: References: Message-ID: <4dd7b2a0-6afc-6532-49cc-18d4c942e887@mrabarnett.plus.com> On 2019-07-30 18:20, Steven via Python-list wrote: > Good morning, > > I have uninstalled and reinstalled different versions of Python, and continue to get the "modify setup" box when I try to open. > > Is there something I'm doing wrong? > Try to open what? It sounds to me like you're just running the installer again. The installer is just that, the _installer_. From harry.grey1694 at gmail.com Tue Jul 30 16:15:37 2019 From: harry.grey1694 at gmail.com (teXture) Date: Tue, 30 Jul 2019 22:15:37 +0200 Subject: .python_history file In-Reply-To: <76c288be-233b-e0a4-2bab-2b4453cc9930@tjol.eu> References: <1564434411.2556.0.camel@gmail.com> <76c288be-233b-e0a4-2bab-2b4453cc9930@tjol.eu> Message-ID: <1564517737.1889.1.camel@gmail.com> > IMHO, this is a rather niche use case. And I suppose you could > relatively easily implement it yourself with a site hook that adds > magic > comments to ~/.pyhistory as you suggest. If you want this > functionality, > there's no need for it to be part of Python itself. > > FWIW, IPython, as far as I can tell, already saves some timestamp > information in its history file. > > -- Thomas Hy Thomas, where i can find info on how python writes te history file? Thanks. From phoenix1987 at gmail.com Tue Jul 30 17:14:35 2019 From: phoenix1987 at gmail.com (Gabriele) Date: Tue, 30 Jul 2019 21:14:35 GMT Subject: ANN: Austin -- CPython frame stack sampler 0.7.0 Message-ID: <%q20F.1953439$Ud5.784769@fx12.am4> I am delighted to announce the release 0.7.0 of Austin. If you haven't heard of Austin before, it is a frame stack sampler for CPython. It can be used to obtain statistical profiling data out of a running Python application without a single line of instrumentation. This means that you can start profiling a Python application straightaway, even while it's running on a production environment, with minimal impact on performance. The simplest way of using Austin is by piping its output to FlameGraph for a quick and detailed representation of the collected samples. The latest release introduces a memory profiling mode which allows you to profile memory usage. Austin is a pure C application that has no other dependencies other than the C standard library. Its source code is hosted on GitHub at https://github.com/P403n1x87/austin The README contains installation and usage details, as well as some examples of Austin in action. Details on how to contribute to Austin's development can be found at the bottom of the page. I hope that you can find Austin useful! All the best, Gabriele From jfong at ms4.hinet.net Wed Jul 31 22:16:05 2019 From: jfong at ms4.hinet.net (jfong at ms4.hinet.net) Date: Wed, 31 Jul 2019 19:16:05 -0700 (PDT) Subject: Is there a simple way to wrap a built-in function for the whole package? Message-ID: <188ce0c3-ff6c-4f57-864d-b807da37e186@googlegroups.com> I get a package from Pypi. The package has many modules using built-in open() function. I like to redefine all the open() there with the default encoding 'utf-8', but not for code outside the package. Maybe I can put my def statement at the beginning of every module of this package, but just wondering is there a simple way of doing it? Best Regards, Jach From jsalsman at gmail.com Wed Jul 31 23:19:58 2019 From: jsalsman at gmail.com (jsalsman at gmail.com) Date: Wed, 31 Jul 2019 20:19:58 -0700 (PDT) Subject: Why won't the run-time error reporter point to the error position with a caret like the syntax error reporter does? It knows exactly where the error is. Message-ID: <1d65ba31-842b-40cb-9100-0b719207938b@googlegroups.com> Honestly this is the only thing in over half a decade of daily python use which has disappointed me enough to want to ask the devs: >>> print(1/) File "", line 1 print(1/) ^ SyntaxError: invalid syntax >>> print(1/1, 1/0, 1/1) Traceback (most recent call last): File "", line 1, in ZeroDivisionError: division by zero >>> print( ... 1 ... / ... 1 ... , ... 1 ... / ... 0 # line 8 ... , ... 1 ... / ... 1 ... ) Traceback (most recent call last): File "", line 8, in ZeroDivisionError: division by zero Why not print the caret?